Data access and modularization presentation method based on interface

Abstract

The invention relates to the technical field of interface data management, in particular to an interface-based data access and modularization presentation method, which comprises the following steps of: connecting different data sources through interfaces to obtain original data, and storing the original data; the front-end component triggers a component event and judges whether a corresponding component exists or not, if yes, the corresponding component is called, and if not, a request is sent to the API gateway; the API gateway verifies the legality of the request, and if the request is legal, the request is forwarded to the corresponding micro-service according to a routing rule; the micro-service analyzes the parameters of the request to obtain corresponding data: the micro-service encapsulates the obtained corresponding data, and feeds the encapsulated data back to the front-end component through the API gateway; and the front-end component dynamically renders the corresponding component according to the type of the data. According to the scheme, the data request efficiency can be improved, data visualization is realized, and the same data request is prevented from being repeated.

Description

Technical Field

[0001] The present invention relates to the technical field of interface data management, and specifically provides a method for data access and componentized presentation based on an interface. Background Art

[0002] In the current era of digital transformation, enterprises are facing an unprecedented data explosion. Whether it is traditional business systems, emerging Internet of Things devices, or social media and third-party APIs, data sources are becoming increasingly diverse and complex. How to efficiently integrate these multi-source heterogeneous data and extract valuable information from them has become the key for enterprises to enhance their competitiveness. Traditional enterprise information systems usually rely on single or limited data sources, such as internal databases or file systems. However, with business expansion and technological progress, enterprises need to process data from multiple different sources. These data sources may include, but are not limited to: structured data, semi-structured data, and unstructured data. Each data source has its unique access method and format, which poses a huge challenge to data integration. For example, data obtained from a REST API needs to be fetched via an HTTP request and parsed in JSON format, while data extracted from a database requires the use of SQL queries. In addition, different data sources may also have requirements in terms of security, performance, and consistency, further increasing the integration difficulty.

[0003] To solve the above problems, interface-driven technology emerged. An interface, as an abstract layer defining the interaction rules between modules, can effectively decouple each functional module, making the system more flexible and scalable. Different data sources are connected to the system through API interfaces, enabling the system to retrieve and utilize data from different data sources. However, each time data is obtained, it is necessary to call the data source through the interface. Moreover, for some duplicate data, requests need to be sent again, and the obtained data is only encapsulated and fed back, which is not conducive to user query and viewing. Summary of the Invention

[0004] The present invention aims to provide a method for data access and componentized presentation based on an interface, which can improve data request efficiency, realize data visualization, and avoid repeated identical data requests.

[0005] The present invention provides the following basic solution: A method for data access and componentized presentation based on an interface, including the following steps: Connect to different data sources through an interface, obtain the original data, and store it; When a front-end component triggers a component event, determine whether it corresponds to a component. If so, call the corresponding component; if not, send a request to the API gateway; The API gateway verifies the legitimacy of the request. If the request is legitimate, forward the request to the corresponding microservice according to the routing rules; The microservice parses the parameters of the request and obtains the corresponding data: If the data is from a single data source, the corresponding original data stored is obtained as the corresponding data obtained; If the data is cross-data-source data, data aggregation is performed to generate an aggregation result as the corresponding data obtained; Among them, for data aggregation, the corresponding original data stored is obtained through a batch processing engine, a window is established through a stream processing engine, and aggregation calculations are performed on the obtained original data to generate an aggregation result as the corresponding data obtained; The microservice encapsulates the corresponding data obtained and feeds it back to the front-end component through the API gateway; The front-end component dynamically renders the corresponding component according to the type of data.

[0006] Furthermore, it also includes: caching the aggregation result.

[0007] Furthermore, before data aggregation, it is judged whether there is data corresponding to the current request parameters in the cached aggregation result. If so, the cached aggregation result is retrieved, combined with the corresponding original data obtained through the batch processing engine, a window is established through the stream processing engine, and aggregation calculations are performed on the obtained aggregation result and the original data to generate a new aggregation result as the corresponding data obtained.

[0008] Furthermore, it also includes: counting the number of calls of each aggregation result in the cache. If the number of calls exceeds the first preset number of calls, the aggregation result is statically stored, and the corresponding aggregation result in the cache is cleared.

[0009] Furthermore, it also includes: if the cached aggregation result exceeds the preset cache time, the corresponding aggregation result is deleted.

[0010] Furthermore, when any aggregation result in the cache is statically stored, correlation analysis is performed between the statically stored aggregation result and other aggregation results, and according to the correlation, the preset cache time of the aggregation result with a correlation greater than the preset correlation is increased.

[0011] Furthermore, when any aggregation result in the cache is statically stored, correlation analysis is performed between the statically stored aggregation result and other aggregation results; Mark the aggregation results with a correlation greater than the preset minimum correlation; Determine whether the time difference between the preset cache time of the aggregation result of the judgment mark and the current time is less than the preset time difference. If so, determine whether the number of calls of the aggregation result of the judgment mark is greater than the second preset number of calls. If so, increase the preset cache time of the aggregation result with an association degree greater than the preset association degree according to the association degree; if not, set that when the time difference between the current time and the preset cache time of the aggregation result of the judgment mark is equal to the preset time difference, determine whether the number of calls of the aggregation result of the judgment mark is greater than the second preset number of calls.

[0012] Further, for the association degree analysis, a neural network model is used to analyze the association degree between two aggregation results, including: an input layer, a hidden layer, and an output layer; Among them, the input layer is used to input the aggregation results as variables, and 2 neurons are set; several neurons are set in the hidden layer; 1 neuron is set in the output layer to predict the association degree between two aggregation results; the ReLU activation function is used in the hidden layer.

[0013] Further, the neural network model adopts a multi-layer perceptron model to analyze the association degree between two aggregation results: For the two aggregation results respectively, according to the aggregation calculation adopted by the aggregation result, obtain different original data of the same data source for several times, perform aggregation calculation, generate several aggregation results, and form the observed values of variables, X = [x1, x2,..., xn]) and Y = [y1, y2,..., yn], where n is the number of samples; Input the two observed values into the constructed multi-layer perceptron model to analyze the association degree and obtain a scalar value representing the association degree score between the two variables.

[0014] Further, the step of increasing the preset cache time of the aggregation result with an association degree greater than the preset association degree according to the association degree includes: if the association degree is greater than the preset association degree, calculate the difference between the association degree and the preset association degree, and according to the mapping relationship between the preset difference and the preset cache time increment value, obtain the corresponding preset cache time increment value and add it to the preset cache time.

[0015] Beneficial effects: This solution connects different data sources through an interface, obtains and stores the original data, so that the original data can be directly called; Specifically, during the calling process, when the front-end component triggers a component event, it first checks whether the corresponding component exists. If so, it directly calls the corresponding component without performing processes such as data request calculation, thus improving the data calling efficiency. If not, it sends a request to the API gateway. After the API gateway verifies that the request is legal, according to the routing rules, it forwards the request to the corresponding microservice. The microservice parses the parameters of the request, obtains the corresponding data, encapsulates it, and then feeds it back to the front-end component through the API gateway. The front-end component dynamically renders the corresponding component according to the type of data, thereby completing the calling of different data sources; the formed components can be called later, and data visualization is achieved. Users can directly see and select the components for use, avoiding repeated identical data requests.

[0016] Among them, when the microservice parses the parameters of the request and obtains the corresponding data, it will perform different processing according to the number of data sources of the data. Specifically, if the data is single-source data, it obtains the corresponding stored original data as the corresponding data obtained; if the data is cross-source data, it performs data aggregation to generate an aggregation result as the corresponding data obtained; in data aggregation, the batch processing engine is used to obtain the corresponding stored original data, and the stream processing engine is used to establish a window to perform aggregation calculation on the obtained original data to generate an aggregation result as the corresponding data obtained; this ensures the calling of single-source data and realizes the integration of multi-source data.

[0017] This solution can avoid repeated identical data requests, improve the data request efficiency, and realize data visualization. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic flowchart of an embodiment of a method for interface-based data access and componentized presentation according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The following is a further detailed description through specific embodiments: Embodiment 1 The embodiment is basically as shown in the attached Figure 1 figures: A method for interface-based data access and componentized presentation includes the following contents: Connect to different data sources through an interface, obtain the original data, and store it; The front-end component triggers a component event and checks whether there is a corresponding component. If so, it calls the corresponding component. If not, it sends a request to the API gateway; The API gateway verifies the legality of the request. If the request is legal, it forwards the request to the corresponding microservice according to the routing rules; if the request is illegal, it feeds back that the request is illegal through the front-end component; The microservice parses the parameters of the request and obtains the corresponding data; If the data is from a single data source, obtain the corresponding stored original data as the acquired data; If the data is cross - data - source data, perform data aggregation to generate an aggregation result as the acquired data; Among them, for data aggregation, obtain the corresponding stored original data through a batch processing engine, establish a window through a stream processing engine, perform aggregation calculations on the acquired original data to generate an aggregation result as the corresponding acquired data, and cache the aggregation result; The microservice encapsulates the acquired data and feeds it back to the front - end component through the API gateway; The front - end component dynamically renders the corresponding component according to the type of data. In the actual use process, the component is displayed through a display screen. The user can directly input component selection information through the display screen to call the corresponding component, greatly improving the data request efficiency.

[0020] In addition, before data aggregation, determine whether there is data corresponding to the current request parameters in the cached aggregation result. If so, retrieve the cached aggregation result, combine it with the corresponding stored original data obtained through the batch processing engine, establish a window through the stream processing engine, and perform aggregation calculations on the retrieved aggregation result and the original data to generate a new aggregation result as the corresponding acquired data; For the cached aggregation result, count the number of calls for each aggregation result in the cache. If the number of calls exceeds the first preset number of calls, statically store the aggregation result and clear the corresponding aggregation result in the cache; If the cached aggregation result exceeds the preset cache time, delete the corresponding aggregation result; When statically storing any aggregation result in the cache, perform an association analysis between the statically stored aggregation result and other aggregation results, and according to the association degree, increase the preset cache time of the aggregation result with an association degree greater than the preset association degree; specifically, in this embodiment, if the association degree is greater than the preset association degree, calculate the difference between the association degree and the preset association degree, and according to the mapping relationship between the preset difference and the preset cache time increment value, obtain the corresponding preset cache time increment value and add it to the preset cache time; Among them, for the association analysis, a neural network model is used to analyze the association degree between two aggregation results; in this embodiment, a multi - layer perceptron model (MLP model) is used to analyze the association degree between two aggregation results, including: an input layer, a hidden layer, and an output layer; Among them, the input layer is used to input the features of the aggregation result, and 2 neurons are set; several neurons are set in the hidden layer. In this embodiment, 10 neurons are set, which can be set according to specific requirements; 1 neuron is set in the output layer to predict the correlation degree between the two aggregation results; the ReLU activation function is adopted in the hidden layer, which helps to alleviate the problem of gradient disappearance and accelerate the training process.

[0021] For the two aggregation results respectively, according to the aggregation calculation adopted by the aggregation result, several different original data of the same data source are obtained for aggregation calculation, and several aggregation results are generated to form the observed values of the variable, X = [x1, x2, ..., xn]) and Y = [y1, y2, ..., yn], where n is the number of samples; The two observed values are input into the constructed multi-layer perceptron model to analyze the correlation degree, and a scalar value is obtained, indicating the correlation degree score between the two variables.

[0022] This solution caches the aggregation results, and when the number of calls reaches a certain amount, it will be statically stored, that is, permanently stored, and the cache will be cleared regularly, so as to clear the aggregation results whose number of calls has not reached the preset first preset number of calls within the preset cache time. Compared with the existing data cache cleaning, this solution also analyzes the correlation degree between the aggregation result during static storage and other aggregation results, and increases the preset cache time of the aggregation results with a correlation degree greater than the preset correlation degree according to the correlation degree. The higher the correlation degree, the more likely the number of subsequent calls will increase. Compared with cleaning all cached data according to the preset cache time, this solution can dynamically optimize the cached aggregation results, avoid data being cleared too quickly, and prevent all data from accumulating.

[0023] Embodiment 2 This embodiment is basically the same as the above embodiment, the difference is: When any aggregation result in the cache is statically stored, analyze the correlation degree between the aggregation result for static storage and other aggregation results; Mark the aggregation results with a correlation degree greater than the preset minimum correlation degree; Judge whether the time difference between the preset cache time of the marked aggregation result and the current time is less than the preset time difference. If so, judge whether the number of calls of the marked aggregation result is greater than the second preset number of calls. If so, increase the preset cache time of the aggregation results with a correlation degree greater than the preset correlation degree according to the correlation degree; if not, set that when the time difference between the current time and the preset cache time of the marked aggregation result is equal to the preset time difference, judge whether the number of calls of the marked aggregation result is greater than the second preset number of calls; For the increase in the preset cache time of the aggregation result after correlation analysis, it does not solely consider the correlation metric. Instead, it takes into account the problem of low accuracy of the uniqueness metric. If the preset minimum correlation is set too high, there may be no aggregating results that can be marked. If the preset minimum correlation is set too low, a large amount of aggregating data may be marked. If the preset cache time is increased in both cases, it will cause a large amount of cache space to be occupied. Therefore, after marking the aggregating results with a correlation greater than the preset minimum correlation, this solution first determines whether the time difference between the preset cache time of the marked aggregating results and the current time is less than the preset time difference to analyze whether the marked aggregating results are due to be cleared. If so, it further analyzes whether the number of calls is greater than the second preset number of calls. If so, it means that the marked aggregating results are frequently called within a certain period of their cache, but do not meet the requirement of the first preset number of calls, so they cannot be statically stored. However, their frequent calls are highly correlated with the aggregating results that can be statically stored. Therefore, the preset cache time can be increased to prevent the need to perform aggregation calculations again after direct deletion. During the increased cache time, if there are still calls and the number of calls reaches the first preset number of calls, static storage can be performed, ensuring that the cached data can be statically stored according to actual call requirements, and preventing excessive data from being statically stored or cached for too long, resulting in the occupation and waste of storage resources.

[0024] Embodiment Three This embodiment is basically the same as the above embodiment, with the difference that: It further includes: presetting a number of components, defining corresponding data standards and connected data sources for different components, obtaining the original data of the data sources, and performing transformation and integration processing on the original data according to the data standards; Obtaining a component style selection signal and updating the style of the component according to the selected component style; Obtaining a component movement signal and moving the component according to the component movement signal to achieve different layouts.

[0025] This solution realizes the access and integration of data from multiple different data sources, and at the same time supports the adjustment of the display layout and style of components. By this means, the display logic of the data is separated from the data itself, making the display of the data more flexible and personalized.

[0026] The above are only embodiments of the present invention. Specific structures and common knowledge such as characteristics well-known in the art are not described in detail herein. Those of ordinary skill in the art know all the common general technical knowledge in the technical field to which the invention pertains before the filing date or the priority date, are able to learn all the prior art in this field, and have the ability to apply conventional experimental means before this date. Those of ordinary skill in the art can, under the inspiration given in this application, complete and implement this solution in combination with their own abilities. Some typical well-known structures or well-known methods should not become an obstacle for those of ordinary skill in the art to implement this application. It should be noted that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can also be made, and these should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope claimed in this application should be based on the content of its claims, and the specific implementation manners and the like recorded in the specification can be used to interpret the content of the claims.

Claims

1. A method for data access and component presentation based on an interface, characterized in that: It includes the following: Connect different data sources through interfaces, obtain raw data, and store them; The front-end component triggers a component event and determines whether it corresponds to a component. If so, it calls the corresponding component. If not, it sends a request to the API gateway. The API gateway verifies the legitimacy of the request. If the request is legitimate, it forwards the request to the corresponding microservice according to the routing rules; The microservice parses the request parameters and obtains the corresponding data: If the data is from a single data source, the corresponding original data stored is obtained as the corresponding data to be obtained; If the data is from different data sources, data aggregation is performed to generate an aggregation result as the corresponding data obtained; The data aggregation is to obtain the corresponding original data stored through the batch processing engine, establish a window through the stream processing engine, perform aggregation calculation on the obtained original data, and generate the aggregation result as the corresponding data obtained; The microservice encapsulates the corresponding data obtained and feeds it back to the front-end component through the API gateway; The front-end component dynamically renders the corresponding component according to the type of data.

2. The method for interface-based data access and componentized presentation according to claim 1, characterized in that: Also includes: Cache the aggregation results.

3. The method for interface-based data access and componentized presentation according to claim 2, characterized in that: Before data aggregation, determine whether the cached aggregation results contain the data that needs to be obtained corresponding to the parameters of the current request. If so, call the cached aggregation results, combine them with the corresponding original data stored through the batch processing engine, establish a window through the stream processing engine, and perform aggregation calculations on the obtained aggregation results and original data to generate new aggregation results as the corresponding data obtained.

4. The method for interface-based data access and componentized presentation according to claim 3, characterized in that: Also includes: The number of calls of each aggregation result in the cache is counted. If the number of calls exceeds a first preset number of calls, the aggregation result is statically stored and the corresponding aggregation result in the cache is cleared.

5. The method for interface-based data access and componentized presentation according to claim 3, characterized in that: Also includes: If the cached aggregation results exceed the preset cache time, the corresponding aggregation results will be deleted.

6. The method for interface-based data access and componentized presentation according to claim 4, characterized in that: When any aggregation result in the cache is statically stored, a correlation analysis is performed between the statically stored aggregation result and other aggregation results, and based on the correlation, a preset cache time of the aggregation result with a correlation greater than a preset correlation is increased.

7. The method for interface-based data access and componentized presentation according to claim 4, characterized in that: When any aggregation result in the cache is statically stored, a correlation analysis is performed between the statically stored aggregation result and other aggregation results; Mark the aggregation results whose correlation is greater than the preset minimum correlation; Determine whether the time difference between the preset cache time of the marked aggregation result and the current time is less than the preset time difference, if so, determine whether the number of calls of the marked aggregation result is greater than a second preset number of calls, if so, increase the preset cache time of the aggregation result whose correlation degree is greater than the preset correlation degree according to the correlation degree; If not, it is set that when the time difference between the current time of the marked aggregation result and the preset cache time is equal to the preset time difference, it is determined whether the calling number of the marked aggregation result is greater than the second preset calling number.

8. The method for interface-based data access and componentized presentation according to any one of claims 6 and 7, characterized in that: Correlation analysis, using a neural network model to analyze the correlation between two converged results, including: input layer, hidden layer and output layer; The input layer is used to input the aggregation results as variables, and 2 neurons are set; the hidden layer is set with several neurons; the output layer is set with 1 neuron to predict the correlation between the two aggregation results; the hidden layer uses the ReLU activation function.

9. The method for interface-based data access and componentized presentation according to claim 8, characterized in that: The neural network model adopts a multi-layer perceptron model to analyze the correlation between two convergence results: For the two converged results, obtain several different original data from the same data source according to the converged calculation used in the converged results, perform converged calculations, generate several converged results, and form the observed values ​​of the variables, X = [x1, x2, ..., xn]) and Y = [y1, y2, ..., yn], where n is the number of samples; The two observations are input into the constructed multilayer perceptron model, the association is analyzed, and a scalar value is obtained, which represents the association score between the two variables.

10. The method for interface-based data access and componentized presentation according to any one of claims 6 and 7, characterized in that: The method of increasing the preset cache time of the aggregation result whose correlation degree is greater than the preset correlation degree according to the correlation degree includes: if the correlation degree is greater than the preset correlation degree, calculating the difference between the correlation degree and the preset correlation degree, and obtaining the corresponding preset cache time increase value according to the mapping relationship between the preset difference and the preset cache time increase value, and adding it to the preset cache time.

Images