A scatter plot rendering method and system based on non-sampled big data

Abstract

The present invention discloses a scatter plot rendering method and system based on non-sampling big data, and the method comprises: a front-end browser initiates a rendering request to an RPC gateway according to a scatter plot type selected by a user; the RPC gateway routes the rendering request to a corresponding business system; the business system optimizes data of a relational database, records optimized filtering conditions and optimized data; the business system loads a data analysis strategy, obtains a script of a database query language in combination with configuration parameters of a set of data obtained after optimization, and transmits the script to the front-end browser via the RPC gateway after processing; the front-end browser grammar analyzes the processed script, and forwards the processed script to a proxy gateway for verification; the proxy gateway splices the processed script and converts it into a new HTTP request and sends it to a ClickHouse database; an image rendering component stores the data of the ClickHouse database returned by the proxy gateway; and a rendered image is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of data information processing, and particularly relates to a scatter plot rendering method and system based on non-sampled big data. Background Art

[0002] In the field of new energy wind power generation, data analysis and operation and maintenance personnel need to analyze historical data and continuously adjust the fan equipment and operating parameters to maximize production capacity while reducing wind curtailment and equipment failure rates. The operation steps of traditional scatter plot rendering methods are roughly as follows: making data samples (up to 65,535 rows) through an excel table, then loading formulas, matching data dictionaries, and forming images using the pivot chart function. Then, data modeling is done using Matlab software, data processing scripts are written, parameters are modified, and the number of points in the interval is counted step by step. This is repeated to form relatively complete images and data. However, traditional scatter plot rendering methods have complex and cumbersome steps, rely on the expertise and experience of analysts, have high labor costs, and require high performance of the client computer. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a scatter plot rendering method and system based on non-sampled big data to automatically render scatter plots for non-sampled big data, without the need for manual writing of data processing scripts and modification of parameters, reducing labor costs and the requirements for the performance of the client computer.

[0004] In a first aspect, an embodiment of the present invention provides a scatter plot rendering method based on non-sampled big data, which is applied to a scatter plot rendering system based on non-sampled big data. The scatter plot rendering system based on non-sampled big data includes a front-end browser, an RPC gateway, multiple business systems, a relational database, a ClickHouse database, a proxy gateway, and an image rendering component. The scatter plot rendering method based on non-sampled big data includes the following steps:

[0005] Initiate a request: The front-end browser initiates a rendering request to the RPC gateway according to the scatter plot type selected by the user.

[0006] Route and distribute the request: The RPC gateway routes and distributes the rendering request to the corresponding business system according to the scatter plot type corresponding to the received rendering request.

[0007] Optimize data: The business system performs query optimization on the data stored in the relational database, records the optimization filtering and screening conditions and the set of data after optimization filtering and screening.

[0008] Obtain Script: The business system loads the corresponding data analysis strategy, calculates by combining the configuration parameters in the set of data obtained after optimization filtering and screening recorded after query optimization, obtains the script of the database query language, processes the script of the database query language and then feeds it back to the RPC gateway and transmits it to the front-end browser;

[0009] Verify Script: The front-end browser performs syntax analysis on the processed database query language script transmitted by the PRC gateway received, and forwards the processed database query language script to the proxy gateway for verification;

[0010] Convert Request: The proxy gateway concatenates the proxy gateway address, session status, and authentication information at the beginning of the received processed database query language script, converts the concatenated processed database query language script into a new HTTP request, generates an HTTP request message and sends it to the ClickHouse database; among them, the converted HTTP request type is obtain;

[0011] Data Reception and Storage: The image rendering component stores the data received from the ClickHouse database returned by the proxy gateway using a floating-point 32-bit binary type array;

[0012] Obtain Rendered Image: The image rendering component renders according to the received data using a scatter plot component library, and feeds back the rendered image to the front-end browser.

[0013] Its further technical solution is: The steps of verifying the script are specifically: The front-end browser performs syntax analysis on the processed database query language script transmitted by the PRC gateway received; The front-end browser forwards the received processed database query language script to the proxy gateway; The proxy gateway verifies the received processed database query language script; Among them, when the index in the script is incomplete or the verification analysis duration exceeds the preset duration, an error occurrence message is fed back to the front-end browser to prompt an error and end the rendering operation.

[0014] Its further technical solution is: After the step that the front-end browser forwards the received processed database query language script to the proxy gateway, it further includes: When the response is abnormal, the front-end browser prompts an error and ends the rendering operation.

[0015] Its further technical solution is: Specifically before the step of optimizing the data includes:

[0016] The business system queries whether the data corresponding to the scatter plot type selected by the user in the relational database is empty;

[0017] If so, the business system feeds back error occurrence information and sends it to the front-end browser through the RPC gateway to prompt an error, and ends the rendering operation;

[0018] If not, the business system queries whether the length of the data corresponding to the scatter plot type selected by the user in the relational database exceeds a preset data length threshold;

[0019] If so, the business system feeds back error occurrence information and sends it to the front-end browser through the RPC gateway to prompt an error, and ends the rendering operation;

[0020] If not, execute the step of optimizing the data.

[0021] Its further technical solution is: The step of obtaining the script specifically includes:

[0022] The business system loads the corresponding data analysis strategy, combines the configuration parameters in the set of optimized filtered and screened data obtained by query optimization records for calculation, and obtains the script of the database query language;

[0023] The business system concatenates a timestamp, a version number, a message digest, and a specified binary line format at the end of the script of the database query language.

[0024] Its further technical solution is: Before the step of routing and distributing requests, it further includes: When the PRC gateway learns from the received rendering request that the time dimension selected by the user exceeds a preset time dimension threshold or the total number of fans exceeds a preset total number of fans threshold, the RPC gateway feeds back error occurrence information to the front-end browser to prompt an error, and ends the rendering operation.

[0025] Its further technical solution is: After the step of obtaining the rendered image, it further includes: The front-end browser displays the rendered image.

[0026] Second aspect, the embodiments of the present invention further provide a scatter plot rendering system based on unsampled big data. The scatter plot rendering system based on unsampled big data includes a front-end browser, an RPC gateway, multiple business systems, a relational database, a ClickHouse database, a proxy gateway, and an image rendering component. Among them, the front-end browser is communicatively connected to the RPC gateway, the proxy gateway, and the image rendering component respectively, and is used to interact with the user, allowing the user to select a scatter plot type, and initiate a rendering request to the RPC gateway according to the selected scatter plot type; and is used to perform syntax analysis on the processed database query language script transmitted by the PRC gateway and forward the processed database query language script to the proxy gateway. The RPC gateway is communicatively connected to the relational database through the business system. The relational database is used to store data related to the scatter plot type, including configuration parameters and authentication data. The RPC gateway is used to route and distribute the rendering request to the corresponding business system. The business system is used to perform query optimization on the data stored in the relational database, record the optimization filtering and screening conditions and the set of data after optimization filtering and screening, load the data analysis strategy, calculate in combination with the configuration parameters in the set of data after optimization filtering and screening obtained by the query optimization, obtain the database query language script, process the database query language script and then feedback it to the RPC gateway and transmit it to the front-end browser. The proxy gateway is communicatively connected to the image rendering component and the ClickHouse database respectively. The ClickHouse database is used to store past experimental data. The proxy gateway is used to splice the proxy gateway address, session status, and authentication information at the beginning of the processed database query language script received from the front-end browser, convert the spliced database query language script into a new HTTP request, generate an HTTP request message and send it to the ClickHouse database; obtain and return the data searched in the ClickHouse database according to the HTTP request message to the image rendering component. The image rendering component is used to store the received data, perform rendering using the scatter plot component library according to the received data, and feedback the rendered image to the front-end browser.

[0027] The beneficial technical effects of the present invention are as follows: A scatter plot rendering method based on non-sampled big data of the present invention routes and distributes the rendering requests initiated by the front-end browser according to the selected scatter plot type to the corresponding business systems through the RPC gateway. The business systems optimize the data stored in the relational database, calculate the optimized data in combination with the corresponding data analysis strategies of the business systems and the configuration parameters in the set of the optimized data, obtain the script of the database query language, and after processing the script of the database query language, feedback it to the RPC gateway and then transmit it to the front-end browser for syntax analysis. Moreover, the front-end browser forwards the processed script of the database query language to the proxy gateway for re-verification. The proxy gateway converts the verified processed script of the database query language into an HTTP request and sends it to the ClickHouse database for data search, and returns the searched data to the image rendering component for storage and rendering to obtain the rendered image; the browser cooperates with the gateway, business systems, and relational database to automatically optimize and filter relational data, generate a script according to the corresponding data analysis strategies of the business systems and the configuration parameters in the set of the optimized and filtered data, convert the request to obtain the experimental data to be rendered stored in the ClickHouse database, and store and render the obtained experimental data through the graphic rendering component, realizing automatic scatter plot rendering of non-sampled big data, without the need for manual writing of data processing scripts and modification of parameters, with low labor costs, improved rendering efficiency, and low requirements for the performance of the client computer. Moreover, the graphic rendering component stores the data using a floating-point 32-bit binary type array, making the occupied space small and avoiding browser jams. A scatter plot rendering system based on non-sampled big data of the present invention also has the above functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0029] Figure 1 It is a schematic flowchart of the scatter plot rendering method based on non-sampled big data provided by the embodiment of the present invention;

[0030] Figure 2 It is a schematic flowchart of the first sub-process of the scatter plot rendering method based on non-sampled big data provided by the embodiment of the present invention;

[0031] Figure 3 It is a schematic flowchart of the second sub-process of the scatter plot rendering method based on non-sampled big data provided by the embodiment of the present invention;

[0032] Figure 4 It is a schematic block diagram of a scatter plot rendering system based on unsampled big data provided by an embodiment of the present invention. Detailed implementation manners

[0033] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0034] It should be understood that when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0035] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in this specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0036] It should be further understood that the term "and / or" used in this specification of the present invention and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0037] Please refer to Figure 1 as shown Figure 1 It is a schematic flowchart of a scatter plot rendering method based on unsampled big data. The scatter plot rendering method based on unsampled big data is applied to a scatter plot rendering system based on unsampled big data. The scatter plot rendering system based on unsampled big data includes a front-end browser, an RPC gateway, multiple business systems, a relational database, a ClickHouse database, a proxy gateway, and an image rendering component. As shown in the figure, the scatter plot rendering method based on unsampled big data includes the following steps S11-S18:

[0038] Step S11, initiate a request: The front-end browser initiates a rendering request to the RPC gateway according to the scatter plot type selected by the user. Here, the scatter plot type refers to the business type of the scatter plot. In this embodiment, the scatter plot rendering method based on non-sampled big data is applied to render the scatter plot of wind turbine power generation data. Correspondingly, the scatter plot types may include: wind speed-power scatter plot, wind speed-torque scatter plot, wind speed-rotation speed scatter plot, wind speed-pitch angle scatter plot, power-torque scatter plot, power-rotation speed scatter plot, power-pitch angle scatter plot, torque-rotation speed scatter plot, and torque-pitch angle scatter plot.

[0039] Step S12, route and distribute the request: The RPC gateway routes and distributes the rendering request to the corresponding business system according to the corresponding scatter plot type in the received rendering request; where different business systems correspond to different data analysis strategies. The data analysis strategies of the business systems correspond one-to-one with the scatter plot types to analyze the data related to the scatter plot types. The rendering request contains the information of the scatter plot type selected by the user, so that the RPC gateway can route and distribute the rendering request to the corresponding business system. Moreover, the RPC gateway can also obtain the corresponding pre-data according to the received rendering request to verify the rendering request.

[0040] Step S13, optimize the data: The business system performs query optimization on the data stored in the relational database, and records the optimization filtering and screening conditions and the set of data after optimization filtering and screening. Among them, the business system performs query optimization on the data stored in the relational database according to the data analysis strategy and the preset optimization filtering and screening conditions to prevent the set of obtained data from being too large or having boundary value problems, effectively reducing the query time and improving the query efficiency. The query optimization can adopt the database query optimization method of the existing technology, such as the database query optimization method disclosed in Chinese Patent CN106919678A, a database query optimization system and method, which will not be elaborated here. The relational database is used to store the data related to the scatter plot type, including configuration parameters and authentication data.

[0041] Step S14, obtain the script: The business system loads the corresponding data analysis strategy, combines the configuration parameters in the set of data after optimization filtering and screening obtained by the query optimization for calculation, obtains the script of the database query language, processes the script of the database query language and then feeds it back to the RPC gateway and transmits it to the front-end browser; where the configuration parameters are the parameter information related to the configuration of the scatter plot type. In this embodiment, the scatter plot rendering method based on non-sampled big data is applied to render the scatter plot of wind turbine power generation data. Correspondingly, the configuration parameters may include the wind farm information where the wind turbine is located, the wind turbine type, the wind turbine unit number, and the wind turbine number information, etc., which are the parameter information related to the configuration of the wind turbine and the wind farm.

[0042] Step S15, script verification: The front-end browser performs syntax analysis on the processed database query language script transmitted by the PRC gateway and forwards the processed database query language script to the proxy gateway for verification. The processed database query language script is verified and analyzed by the front-end browser and the proxy gateway respectively to check whether the script is incorrect.

[0043] Step S16, request conversion: The proxy gateway concatenates the proxy gateway address, session status, and authentication information at the beginning of the received processed database query language script, converts the concatenated processed database query language script into a new HTTP request, generates an HTTP request message, and sends it to the ClickHouse database to obtain corresponding data from the ClickHouse database; wherein, the request type of the converted HTTP request is GET. The ClickHouse database is used to store past experimental data, that is, historical experimental data, so the data obtained from the ClickHouse database is specifically historical experimental data related to the corresponding scatter plot type.

[0044] Step S17, data reception and storage: The image rendering component stores the data obtained by the ClickHouse database searching according to the HTTP request message returned by the proxy gateway using a floating-point 32-bit binary type array; wherein, by using a floating-point 32-bit binary type array to save data, that is, the data is stored in byte form, ensuring small resource consumption.

[0045] Step S18, obtaining the rendered image: The image rendering component renders according to the received data using a scatter plot component library and feeds the rendered image back to the front-end browser. Among them, the scatter plot component library supports binary, and the scatter plot component library is a library composed of rendering components created when the image rendering component is initialized for directly performing scatter plot rendering according to different data.

[0046] Specifically, in some embodiments, after the step S18, it further includes: The front-end browser displays the rendered image.

[0047] Among them, the scatter plot rendering method based on non-sampled big data routes and distributes the rendering requests initiated by the front-end browser according to the selected scatter plot type to the corresponding business systems through the RPC gateway. The business systems optimize the data stored in the relational database, calculate the optimized data in combination with the corresponding data analysis strategies of the business systems, and cooperate with the configuration parameters in the set of optimized data to obtain the script of the database query language. After processing the script of the database query language, it is fed back to the RPC gateway and then transmitted to the front-end browser for syntax analysis. The front-end browser forwards the processed script of the database query language to the proxy gateway for re-verification. The proxy gateway converts the verified processed script of the database query language into an HTTP request and sends it to the ClickHouse database for data search, and returns the searched data to the image rendering component for storage and rendering to obtain the rendered image. The browser cooperates with the gateway, business systems, and relational database to automatically optimize and filter relational data, automatically generate a script according to the corresponding data analysis strategy of the business system and the configuration parameters in the set of optimized and filtered data, convert the request to obtain the experimental data to be rendered stored in the ClickHouse database, and store and render the obtained experimental data through the graphics rendering component, realizing automatic scatter plot rendering of non-sampled big data, without the need for manual writing of data processing scripts and modification of parameters, with low labor costs, high rendering efficiency, low requirements for the performance of the client computer. Moreover, the graphics rendering component stores the data in a floating-point 32-bit binary type array, making the occupied space small and avoiding browser lag.

[0048] Specifically, in this embodiment, before step S13, it specifically includes:

[0049] The business system queries whether the data corresponding to the scatter plot type selected by the user in the relational database is empty;

[0050] If so, the business system feeds back an error occurrence message and sends it to the front-end browser through the RPC gateway to prompt an error, and ends the rendering operation;

[0051] If not, the business system queries whether the length of the data corresponding to the scatter plot type selected by the user in the relational database exceeds a preset data length threshold;

[0052] If so, the business system feeds back an error occurrence message and sends it to the front-end browser through the RPC gateway to prompt an error, and ends the rendering operation;

[0053] If not, execute step S13.

[0054] Combined with Figure 2, specifically, in this embodiment, step S14 specifically includes:

[0055] Step S141: The business system loads the corresponding data analysis strategy, calculates in combination with the configuration parameters in the set of data obtained after optimized filtering and screening of the query-optimized records, and obtains the script of the database query language.

[0056] Step S142: The business system concatenates a timestamp, a version number, a message digest, and a specified binary line format at the end of the script of the database query language; wherein, by concatenating the binary line format at the end of the script of the database query language, the script can be converted into a binary format file, which occupies less space and can improve the processing efficiency.

[0057] Specifically, the scatter plot rendering method based on non-sampled big data is applied to render the scatter plot of the wind turbine power generation data. The pre-data of the rendering request may include the time dimension and the total number of wind turbines that the user selects and sets for rendering. The PRC gateway can perform verification according to the obtained pre-data of the rendering request to compare and judge with the predetermined conditions of the business system. Then, before step S12, it may further include:

[0058] When the PRC gateway learns from the received rendering request that the time dimension selected by the user exceeds the preset time dimension threshold or the total number of wind turbines exceeds the preset wind turbine total threshold, the RPC gateway feeds back an error occurrence message to the front-end browser to prompt an error and ends the rendering operation. Among them, when the PRC gateway learns from the received rendering request that the time dimension selected by the user is too long or the total number of wind turbines in multiple wind farms is too large and does not meet the predetermined conditions of the business system, the RPC gateway determines that the rendering request is an abnormal request and intercepts it, feeds back the error occurrence message to the front-end browser for display to prompt the user that an error has occurred, and ends the rendering operation. The error occurrence message refers to the message used to prompt that an error has occurred. The business system can pre-define a time dimension threshold, and the corresponding predetermined condition can be: not exceeding the time dimension threshold. The time dimension threshold can be 1 year, that is, the time dimension that the user selects and sets for rendering should not exceed 1 year, and it can be 1 month, 3 months, or 1 year, etc. Otherwise, the rendering request is regarded as an abnormal request because it does not meet the predetermined conditions of the business system. The business system also pre-defines a wind turbine total threshold, and the corresponding predetermined condition is: not exceeding the wind turbine total threshold, that is, the total number of wind turbines that the user selects and sets for rendering should not exceed the preset wind turbine total threshold. Otherwise, the rendering request is regarded as an abnormal request because it does not meet the predetermined conditions of the business system. The abnormal request will be intercepted during the verification of the PRC gateway.

[0059] Preferably, before the step S12, the following steps may further be included: When the PRC gateway learns from the received rendering request that the time dimension selected by the user does not exceed a preset time dimension threshold and the total number of wind turbines does not exceed a preset total number of wind turbines threshold, step S12 is executed. That is, when the time dimension selected by the user meets the predetermined conditions of the business system and the total number of wind turbines in multiple wind farms also meets the predetermined conditions of the business system, step S12 is executed.

[0060] Combined with Figure 3 , the step S15 may specifically be:

[0061] Step S151, the front-end browser performs syntax analysis on the processed database query language script transmitted by the PRC gateway; among them, syntax analysis is a prior art. For example, in Chinese Patent CN1379358 Information Processing Device and Method, Recording Medium and Program, syntax analysis of the extracted script is mentioned, which will not be elaborated here.

[0062] Step S152, the front-end browser forwards the received processed database query language script to the proxy gateway; among them, the front-end browser sends a cross-domain preflight request and the processed database query language script to the proxy gateway to control the proxy gateway to perform a verification operation. A network connection is established by sending a cross-domain preflight request from the front-end browser to the proxy gateway to achieve data transmission. The cross-domain preflight request is an OPTIONS type request in the HTTP request.

[0063] Step S153, the proxy gateway verifies the received processed database query language script. Among them, when the index in the script is incomplete or the verification analysis duration exceeds a preset duration, error occurrence information is fed back to the front-end browser to prompt an error, and the rendering operation ends. By feeding back the error occurrence information to the front-end browser for display to prompt the user that an error has occurred, the rendering operation ends. The error occurrence information refers to the information used to prompt that an error has occurred.

[0064] Among them, after the step S152, the following steps may further be included: When a response exception occurs, the front-end browser prompts that an error has occurred, and the rendering operation ends. Response exceptions include network jitter and / or traffic control restrictions, etc., and may specifically be manifested as the cross-domain preflight request being rejected, the cross-domain preflight request being an unsupported request type, the cross-domain preflight request being reset, and / or a connection timeout between the front-end browser and the proxy gateway.

[0065] Figure 4 is a schematic block diagram of a scatter plot rendering system based on non-sampled big data provided by an embodiment of the present invention. As Figure 4As shown in the figure, the present invention further provides a scatter plot rendering system based on unsampled big data. The scatter plot rendering system 10 based on unsampled big data includes a front-end browser 11, an RPC gateway 12, multiple business systems 13, a relational database 14, a ClickHouse database 15, a proxy gateway 16, and an image rendering component 17. Among them, the front-end browser 11 is communicatively connected to the RPC gateway 12, the proxy gateway 16, and the image rendering component 17 respectively, and is used to interact with users, allowing users to select the type of scatter plot, and initiate a rendering request to the RPC gateway 12 according to the selected scatter plot type; it is used to perform syntax analysis on the processed database query language script transmitted by the PRC gateway 12 and forward the processed database query language script to the proxy gateway 16; the RPC gateway 12 is communicatively connected to the relational database 14 through the business system 13, and the relational database 14 is used to store data related to the scatter plot type, including configuration parameters and authentication data. The RPC gateway 12 is used to route and distribute the rendering request to the corresponding business system 13. The business system 13 is used to perform query optimization on the data stored in the relational database 14, record the optimization filtering and screening conditions and the set of data after optimization filtering and screening, load the data analysis strategy, and calculate in combination with the configuration parameters in the set of data after optimization filtering and screening obtained by the query optimization, obtain the database query language script, process the database query language script and then feedback it to the RPC gateway 12 and transmit it to the front-end browser 11; the proxy gateway 16 is communicatively connected to the image rendering component 17 and the ClickHouse database 15 respectively, and the ClickHouse database 15 is used to store past experimental data; the proxy gateway 16 is used to splice the proxy gateway address, session status, and authentication information at the beginning of the processed database query language script received from the front-end browser 11, convert the spliced database query language script into a new HTTP request, generate an HTTP request message and send it to the ClickHouse database 15; obtain and return the data searched in the ClickHouse database 15 according to the HTTP request message to the image rendering component 17; the image rendering component 17 is used to store the received data, render it using the scatter plot component library according to the received data, and feedback the rendered image to the front-end browser 11.

[0066] Among them, different business systems 13 correspond to different data analysis strategies. The data analysis strategies of business systems 13 are in one-to-one correspondence with the scatter plot types to analyze data related to the scatter plot types. The rendering request contains information about the scatter plot type selected by the user, so that the RPC gateway 12 can route and distribute the rendering request to the corresponding business system 13. Moreover, the RPC gateway 12 can also obtain the corresponding pre-data based on the received rendering request to verify the rendering request. The relational database 14 is a database based on the relational model to manage data in the ordinary data relational model. The relational database 14 can adopt a MySQL database or an Oracle database. The configuration parameters are parameter information related to the configuration for the scatter plot type. In this embodiment, the scatter plot rendering system based on non-sampled big data is applied to render scatter plots for wind turbine power generation data. Correspondingly, the configuration parameters may include parameters related to the configuration of the wind farm where the wind turbine is located, the wind turbine type, the wind turbine unit number, and the wind turbine number information, etc. The image rendering component 17 includes a scatter plot component library for rendering according to the received data to obtain a rendered image of the corresponding data. The image rendering component 17 uses an array buffer and 32-bit floating-point binary to receive and store data, and renders the image through a scatter plot component library that supports binary. The proxy gateway 16 can perform protocol conversion, convert and generate an HTTP request, and distribute the HTTP request to the ClickHouse database 15, supporting binary format output.

[0067] The scatter plot rendering system 10 based on unsampled big data routes and distributes the rendering requests initiated by the front-end browser 11 according to the selected scatter plot type through the RPC gateway 12 to the corresponding business system 13. The business system 13 optimizes the data stored in the relational database 14, calculates the optimized data in combination with the corresponding data analysis strategy of the business system and the configuration parameters in the set of the optimized data, obtains the script of the database query language, processes the script of the database query language and then feeds it back to the RPC gateway 12 and transmits it to the front-end browser 11 for syntax analysis. The front-end browser 11 forwards the processed script of the database query language to the proxy gateway 16 for re-verification. The proxy gateway 16 converts the verified processed script of the database query language into an HTTP request and sends it to the ClickHouse database 15 for data search, and returns the searched data to the image rendering component 17 for storage and rendering to obtain the rendered image; the browser cooperates with the gateway, the business system 13 and the relational database 14 to automatically optimize and screen the relational data, generates a script according to the corresponding data analysis strategy of the business system 13 and the configuration parameters in the set of the optimized and screened data, converts the request to obtain the experimental data to be rendered stored in the ClickHouse database 15, and stores and renders the obtained experimental data through the graphic rendering component 17, realizing automatic scatter plot rendering of unsampled big data, without the need for manual writing of data processing scripts and modification of parameters, with low labor costs, high rendering efficiency, and low requirements for the performance of the client computer. Moreover, the graphic rendering component stores the data using a floating-point 32-bit binary type array, making the occupied space small and avoiding browser lag.

[0068] As described above, the above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A scatter plot rendering method based on non-sampled big data, characterized in that, Applied to a scatter plot rendering system based on unsampled big data, the scatter plot rendering system based on unsampled big data includes a front-end browser, an RPC gateway, multiple business systems, a relational database, a ClickHouse database, a proxy gateway, and an image rendering component; The scatter plot rendering method based on unsampled big data includes the following steps: Initiate a request: The front-end browser initiates a rendering request to the RPC gateway according to the scatter plot type selected by the user; Route and distribute the request: The RPC gateway routes and distributes the rendering request to the corresponding business system according to the corresponding scatter plot type in the received rendering request; Optimize data: The business system performs query optimization on the data stored in the relational database, and records the optimization filtering and screening conditions and the set of data after optimization filtering and screening; Obtain a script: The business system loads the corresponding data analysis strategy, calculates by combining the configuration parameters in the set of data after optimization filtering and screening obtained by recording after query optimization, obtains a script of the database query language, processes the script of the database query language and then feeds it back to the RPC gateway and transmits it to the front-end browser; Verify the script: The front-end browser performs syntax analysis on the processed database query language script transmitted by the RPC gateway received, and forwards the processed database query language script to the proxy gateway for verification; Convert the request: The proxy gateway concatenates the proxy gateway address, session status, and authentication information at the beginning of the received processed database query language script, converts the concatenated processed database query language script into a new HTTP request, generates an HTTP request message and sends it to the ClickHouse database; among them, the converted HTTP request type is get; Data reception and storage: The image rendering component stores the data obtained by the ClickHouse database searching according to the HTTP request message returned by the proxy gateway received using a floating-point 32-bit binary type array; Obtain the rendered image: The image rendering component renders according to the received data using the scatter plot component library, and feeds back the rendered image to the front-end browser.

2. The scatter plot rendering method based on non-sampled big data according to claim 1, wherein The specific steps of the verifying the script are: The front-end browser performs syntax analysis on the processed database query language script transmitted by the RPC gateway received; The front-end browser forwards the received processed database query language script to the proxy gateway; The proxy gateway verifies the received processed database query language script; among them, when the index in the script is incomplete or the verification analysis duration exceeds the preset duration, an error occurrence message is fed back to the front-end browser to prompt an error, and the rendering operation ends.

3. The scatter plot rendering method based on non-sampled big data according to claim 2, characterized in that, After the step of the front-end browser forwarding the received processed database query language script to the proxy gateway, it further includes: When the response is abnormal, the front-end browser prompts an error and ends the rendering operation.

4. The scatter plot rendering method based on non-sampled big data according to claim 1, wherein Before the step of optimizing the data, it specifically includes: The business system queries whether the data corresponding to the scatter plot type selected by the user in the relational database is empty; If so, the business system feeds back error occurrence information and sends it to the front-end browser through the RPC gateway to prompt an error, and ends the rendering operation; If not, the business system queries whether the length of the data corresponding to the scatter plot type selected by the user in the relational database exceeds a preset data length threshold; If so, the business system feeds back error occurrence information and sends it to the front-end browser through the RPC gateway to prompt an error, and ends the rendering operation; If not, perform the step of optimizing the data.

5. The scatter plot rendering method based on non-sampled big data according to claim 1, wherein The step of obtaining the script specifically includes: The business system loads the corresponding data analysis strategy, calculates in combination with the configuration parameters in the set of optimized filtered data obtained by querying and optimizing the records, and obtains the script of the database query language; The business system concatenates a timestamp, a version number, a message digest, and a specified binary line format at the end of the script of the database query language.

6. The scatter plot rendering method based on non-sampled big data according to claim 1, wherein Before the step of routing and distributing requests, it also includes: When the RPC gateway learns from the received rendering request that the time dimension selected and set by the user exceeds the preset time dimension threshold or the total number of fans exceeds the preset total number of fans threshold, the RPC gateway feeds back error occurrence information to the front-end browser to prompt an error, and ends the rendering operation.

7. The scatter plot rendering method based on non-sampled big data according to claim 1, wherein After the step of obtaining the rendered image, it also includes: The front-end browser displays the rendered image.

8. A scatter plot rendering system based on non-sampled big data, characterized in that, The scatter plot rendering system based on non-sampled big data includes a front-end browser, an RPC gateway, multiple business systems, a relational database, a ClickHouse database, a proxy gateway, and an image rendering component; among them, The front-end browser is respectively communicatively connected to the RPC gateway, the proxy gateway, and the image rendering component, and is used to interact with the user, allowing the user to select a scatter plot type, and initiate a rendering request to the RPC gateway according to the selected scatter plot type; and is used to perform syntax analysis on the processed database query language script transmitted by the received RPC gateway and forward the processed database query language script to the proxy gateway; The RPC gateway is communicatively connected to the relational database through the business system. The relational database is used to store data related to the scatter plot type, including configuration parameters and authentication data. The RPC gateway is used to route and distribute rendering requests to the corresponding business systems. The business systems are used to perform query optimization on the data stored in the relational database, record the optimization filtering conditions and the set of optimized filtered data, load the data analysis strategy, calculate in combination with the configuration parameters in the set of optimized filtered data obtained by querying and optimizing the records, obtain the script of the database query language, process the script of the database query language and then feedback it to the RPC gateway and transmit it to the front-end browser; The proxy gateway is communicatively connected to the image rendering component and the ClickHouse database respectively. The ClickHouse database is used to store previous experimental data. The proxy gateway is used to splice the proxy gateway address, session status and authentication information at the beginning of the processed database query language script forwarded by the front-end browser it receives, convert the spliced database query language script into a new HTTP request, generate an HTTP request message and send it to the ClickHouse database; obtain and return the data searched in the ClickHouse database according to the HTTP request message to the image rendering component; the image rendering component is used to store the received data, render it using the scatter plot component library according to the received data, and feedback the rendered image to the front-end browser.

Images