A method for data access and component-based presentation based on interfaces

Abstract

The application relates to the technical field of interface data management, in particular to a data access and componentized presentation method based on an interface, which comprises the following steps: connecting different data sources through an interface, obtaining original data, and storing the original data; triggering a component event by a front-end component, judging whether the component event corresponds to a component, if yes, calling the corresponding component, and if not, sending a request to an API gateway; verifying the legality of the request by the API gateway, if the request is legal, forwarding the request to corresponding micro services according to routing rules; analyzing parameters of the request by the micro services, obtaining corresponding data; encapsulating the obtained corresponding data by the micro services, and feeding back the encapsulated data to the front-end component through the API gateway; and dynamically rendering corresponding components according to the type of the data by the front-end component. The scheme can improve data request efficiency, realize data visualization, and avoid repeated same data requests.

Description

Technical Field

[0001] This invention relates to the field of interface data management technology, specifically a method for data access and component-based presentation based on interfaces. Background Technology

[0002] In today's era of digital transformation, enterprises face an unprecedented data explosion. Whether from traditional business systems, emerging IoT devices, or social media and third-party APIs, data sources are becoming increasingly diverse and complex. Efficiently integrating this multi-source, heterogeneous data and extracting valuable information has become crucial for enterprises to enhance their competitiveness. Traditional enterprise information systems typically rely on a single or limited data source, such as an internal database or file system. However, with business expansion and technological advancements, enterprises need to handle data from multiple different sources. These data sources may include, but are not limited to, structured, semi-structured, and unstructured data. Each data source has its unique access methods and formats, posing significant challenges to data integration. For example, data obtained from a REST API requires HTTP requests and parsing of JSON format, while data extracted from a database requires SQL queries. Furthermore, different data sources may have requirements regarding security, performance, and consistency, further increasing the difficulty of integration.

[0003] To address these issues, interface-driven architecture emerged. An interface, as an abstraction layer defining the interaction rules between modules, effectively decouples various functional modules, giving the system greater flexibility and scalability. Different data sources connect to the system through API interfaces, allowing the system to retrieve and utilize data from different data sources. However, each data retrieval requires calling the data source from the interface, and for some duplicate data, requests need to be sent again. Furthermore, the retrieved data is merely a packaged feedback, which is not conducive to user querying and viewing. Summary of the Invention

[0004] The present invention aims to provide a method for data access and component-based presentation based on interfaces, which can improve data request efficiency, realize data visualization, and avoid repeated data requests.

[0005] This invention provides the following basic solution: a method for data access and component-based presentation based on interfaces, comprising the following:

[0006] Connect to different data sources via interfaces to obtain raw data and store it;

[0007] When a front-end component triggers a component event, it checks if the event corresponds to a specific component. If it does, the corresponding component is invoked; otherwise, a request is sent to the API gateway.

[0008] The API gateway verifies the validity of the request. If the request is valid, it forwards the request to the corresponding microservice according to the routing rules.

[0009] The microservice parses the request parameters and retrieves the corresponding data:

[0010] If the data is from a single data source, then the corresponding original data stored in the database is retrieved as the corresponding data to be retrieved.

[0011] If the data comes from multiple data sources, then the data is aggregated to generate an aggregated result, which is used as the corresponding data to be obtained.

[0012] Data aggregation involves using a batch processing engine to retrieve the corresponding raw data from storage, and then using a stream processing engine to create a window to perform aggregation calculations on the retrieved raw data, generating an aggregation result that serves as the corresponding retrieved data.

[0013] The microservice encapsulates the acquired data and feeds it back to the front-end component through the API gateway;

[0014] The front-end component dynamically renders the corresponding component based on the data type.

[0015] Furthermore, it also includes caching the aggregation results.

[0016] Furthermore, before data aggregation, it is determined whether the cached aggregation result contains the data that the current request parameter corresponds to. If so, the cached aggregation result is retrieved, combined with the corresponding original data obtained from the batch processing engine, and a window is established through the stream processing engine to perform aggregation calculations on the obtained aggregation result and the original data to generate a new aggregation result, which is used as the corresponding data to be obtained.

[0017] Furthermore, it also includes: counting the number of times each aggregation result is called in the cache; if the number of calls exceeds the first preset number of calls, the aggregation result is statically stored and the corresponding aggregation result in the cache is cleared.

[0018] Furthermore, it also includes: if the cached aggregation result exceeds the preset cache time, then the corresponding aggregation result will be deleted.

[0019] Furthermore, when any aggregation result in the cache is statically stored, the correlation between the statically stored aggregation result and other aggregation results is analyzed, and based on the correlation, the preset cache time for aggregation results with a correlation greater than the preset correlation is increased.

[0020] Furthermore, when any aggregation result in the cache is statically stored, a correlation analysis is performed between the statically stored aggregation result and other aggregation results.

[0021] Aggregation results with a correlation degree greater than the preset minimum correlation degree are marked;

[0022] Determine whether the time difference between the preset cache time of the aggregated result of the marked item and the current time is less than the preset time difference. If so, determine whether the number of times the aggregated result of the marked item has been called is greater than the second preset number of calls. If so, increase the preset cache time of the aggregated result with a correlation greater than the preset correlation based on the correlation. If not, set the time difference between the current time of the aggregated result of the marked item and the preset cache time to be equal to the preset time difference, and then determine whether the number of times the aggregated result of the marked item has been called is greater than the second preset number of calls.

[0023] Furthermore, correlation analysis is performed using a neural network model to analyze the correlation between the two convergence results, including the input layer, hidden layer, and output layer.

[0024] The input layer, which takes the convergence result as input variable, has 2 neurons; the hidden layer has several neurons; the output layer has 1 neuron to predict the correlation between the two convergence results; and the hidden layer uses the ReLU activation function.

[0025] Furthermore, the neural network model employs a multilayer perceptron model to analyze the correlation between the two convergence results:

[0026] For each of the two aggregation results, based on the aggregation calculation used in the aggregation results, several different original data from the same data source are obtained and aggregated to generate several aggregation results, forming the observed values ​​of the variables, X = [x1,x2, ..., xn] and Y = [y1, y2, ..., yn], where n is the number of samples;

[0027] Two observations are input into the constructed multilayer perceptron model to analyze the correlation and obtain a scalar value representing the correlation score between the two variables.

[0028] Furthermore, the step of increasing the preset cache time for aggregation results with a correlation degree greater than a preset correlation degree according to the correlation degree includes: if the correlation degree is greater than the preset correlation degree, calculating the difference between the correlation degree and the preset correlation degree, and obtaining the corresponding preset cache time increase value according to the mapping relationship between the preset difference value and the preset cache time increase value, and adding it to the preset cache time.

[0029] Beneficial effects: This solution connects to different data sources through interfaces, obtains and stores raw data, and allows direct access to the raw data;

[0030] Specifically, during the invocation process, when a front-end component triggers a component event, it first checks if a corresponding component exists. If so, it directly invokes the corresponding component without performing data request calculations, thus improving data retrieval efficiency. If not, it sends a request to the API gateway. After verifying the validity of the request, the API gateway forwards the request to the corresponding microservice according to routing rules. The microservice parses the request parameters, obtains the corresponding data, encapsulates it, and then feeds it back to the front-end component through the API gateway. The front-end component dynamically renders the corresponding component based on the data type, thereby completing the invocation of different data sources. The resulting component can be invoked later and data visualization is implemented, allowing users to directly see which component to select and use, avoiding repeated requests for the same data.

[0031] The microservice parses the request parameters and retrieves the corresponding data. The processing varies depending on the number of data sources. Specifically, if the data comes from a single data source, the corresponding original data stored in the database is retrieved as the retrieved data. If the data comes from multiple data sources, data aggregation is performed to generate an aggregation result, which is then used as the retrieved data. Data aggregation involves using a batch processing engine to retrieve the corresponding original data from the database, and a stream processing engine to create a window and perform aggregation calculations on the retrieved original data to generate the aggregation result, which is then used as the retrieved data. This ensures data retrieval from a single data source while also enabling the fusion of data from multiple data sources.

[0032] This solution avoids making the same data requests repeatedly, improves data request efficiency, and enables data visualization. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating an embodiment of an interface-based data access and component-based presentation method according to the present invention. Detailed Implementation

[0034] The following detailed description illustrates the specific implementation method:

[0035] Example 1

[0036] The basic implementation examples are as follows: Figure 1 As shown: A method for data access and component-based presentation based on interfaces, including the following:

[0037] Connect to different data sources via interfaces to obtain raw data and store it;

[0038] When a front-end component triggers a component event, it checks if the event corresponds to a specific component. If it does, the corresponding component is invoked; otherwise, a request is sent to the API gateway.

[0039] The API gateway verifies the validity of the request. If the request is valid, it forwards the request to the corresponding microservice according to the routing rules; if the request is invalid, it reports the invalidity of the request through the front-end component.

[0040] The microservice parses the request parameters and retrieves the corresponding data;

[0041] If the data is from a single data source, then the corresponding original data stored in the database is retrieved as the retrieved data.

[0042] If the data comes from multiple data sources, then the data is aggregated to generate an aggregated result, which is then used as the acquired data.

[0043] The data aggregation process involves using a batch processing engine to retrieve the corresponding raw data from storage, then using a stream processing engine to create a window to perform aggregation calculations on the retrieved raw data, generating aggregation results which are then used as the corresponding retrieved data, and finally caching the aggregation results.

[0044] The microservice encapsulates the acquired data and feeds it back to the front-end component through the API gateway;

[0045] The front-end component dynamically renders the corresponding component based on the data type. In actual use, the component is displayed on the screen, and users can directly input component selection information on the screen to call the corresponding component, which greatly improves the efficiency of data request.

[0046] In addition, before data aggregation, it is determined whether the cached aggregation result contains the data that the current request parameter corresponds to. If so, the cached aggregation result is retrieved, combined with the corresponding original data obtained from the batch processing engine, and a window is established through the stream processing engine to perform aggregation calculation on the obtained aggregation result and the original data to generate a new aggregation result as the corresponding data to be obtained.

[0047] For cached aggregation results, the number of calls for each aggregation result in the cache will be counted. If the number of calls exceeds the first preset number of calls, the aggregation result will be statically stored and the corresponding aggregation result in the cache will be cleared.

[0048] If the cached aggregation results exceed the preset cache time, the corresponding aggregation results will be deleted.

[0049] When any aggregation result in the cache is statically stored, the correlation between the statically stored aggregation result and other aggregation results is analyzed, and the preset cache time is increased for aggregation results with a correlation greater than a preset correlation based on the correlation. Specifically, in this embodiment, if the correlation is greater than the preset correlation, the difference between the correlation and the preset correlation is calculated, and the corresponding preset cache time increase value is obtained and added to the preset cache time according to the mapping relationship between the preset difference and the preset cache time increase value.

[0050] The correlation analysis employs a neural network model to analyze the correlation between two converged results. In this embodiment, a multilayer perceptron (MLP) model is used to analyze the correlation between two converged results, including an input layer, a hidden layer, and an output layer.

[0051] The input layer, which takes the features of the convergence result as input, has two neurons; the hidden layer has several neurons, 10 in this embodiment, but the number can be adjusted according to specific needs; the output layer has one neuron to predict the correlation between the two convergence results; the hidden layer uses the ReLU activation function, which helps to alleviate the gradient vanishing problem and accelerate the training process.

[0052] For each of the two aggregation results, based on the aggregation calculation used in the aggregation results, several different original data from the same data source are obtained and aggregated to generate several aggregation results, forming the observed values ​​of the variables, X = [x1,x2, ..., xn] and Y = [y1, y2, ..., yn], where n is the number of samples;

[0053] Two observations are input into the constructed multilayer perceptron model to analyze the correlation and obtain a scalar value representing the correlation score between the two variables.

[0054] This solution caches the aggregation results and, once a certain number of calls are made, statically stores them permanently. It also periodically cleans the cache, removing aggregation results that haven't reached a preset first number of calls within a pre-defined cache period. Compared to existing data cache cleanup methods, this solution analyzes the correlation between aggregation results and other aggregation results during static storage. Based on this correlation, it increases the pre-defined cache time for aggregation results with a correlation greater than a preset correlation. Higher correlation may lead to more subsequent calls. Instead of cleaning all cached data according to a preset cache time, this solution dynamically optimizes cached aggregation results, preventing data from being cleaned up too quickly and avoiding data backlog.

[0055] Example 2

[0056] This embodiment is basically the same as the above embodiments, except that:

[0057] When any aggregation result in the cache is statically stored, perform a correlation analysis between the statically stored aggregation result and other aggregation results;

[0058] Aggregation results with a correlation degree greater than the preset minimum correlation degree are marked;

[0059] Determine whether the time difference between the preset cache time of the aggregated result of the tag and the current time is less than the preset time difference. If so, determine whether the number of calls to the aggregated result of the tag is greater than the second preset number of calls. If so, increase the preset cache time of the aggregated result with a correlation greater than the preset correlation based on the correlation. If not, set the time difference between the current time of the aggregated result of the tag and the preset cache time to be equal to the preset time difference, and then perform the determination whether the number of calls to the aggregated result of the tag is greater than the second preset number of calls.

[0060] The increase in the preset caching time for aggregation results after correlation analysis is not solely based on correlation as a metric, but also addresses the issue of low accuracy of a single metric. If the preset minimum correlation is set too high, there might be no aggregation results to label; conversely, if the preset minimum correlation is set too low, a large amount of aggregation data might be labeled. Increasing the preset caching time for both would result in excessive cache space usage. Therefore, this solution, after labeling aggregation results with a correlation greater than the preset minimum correlation, first checks if the time difference between the preset caching time of the labeled aggregation result and the current time is less than the preset time difference. This is used to analyze whether the labeled aggregation result is due for deletion. If so, further analysis is performed. If the number of calls exceeds the second preset number of calls, it means that the aggregated result marked is frequently called within a certain period of its cache, but it has not reached the first preset number of calls, so it cannot be statically stored. However, its frequent calls are highly correlated with the aggregated result that is statically stored. Therefore, its preset cache time can be increased to prevent the need to perform aggregated calculation again after direct deletion. If there are still calls within the increased cache time, and the number of calls reaches the first preset number of calls, then static storage can be performed. This ensures that cached data can be statically stored according to actual call requirements, while preventing too much data from being statically stored or cached for too long, which would cause storage resource occupation and waste.

[0061] Example 3

[0062] This embodiment is basically the same as the above embodiments, except that:

[0063] It also includes: pre-setting several components, defining corresponding data standards and data sources for different components, obtaining the raw data from the data sources, and transforming and integrating the raw data according to the data standards;

[0064] Get the component style selection signal, and update the component style according to the selected component style;

[0065] Obtain component movement signals and move components according to the movement signals to achieve different layouts.

[0066] This solution enables the access and integration of data from multiple different data sources, while also supporting adjustments to the display layout and style of components. This approach separates the data display logic from the data itself, making the data display more flexible and personalized.

[0067] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A method for interface-based data access and componentized presentation, characterized in that, The application relates to a data processing method and device. Different data sources are connected through an interface to obtain original data and store the original data; A front-end component triggers a component event to determine whether the component event corresponds to a component, if yes, the corresponding component is called, and if not, a request is sent to an API gateway; The API gateway verifies the legality of the request, and if the request is legal, the request is forwarded to a corresponding microservice according to a routing rule; The microservice parses parameters of the request to obtain corresponding data: If the data is single-source data, the corresponding original data stored is obtained as the corresponding data; If the data is cross-source data, data aggregation is performed to generate an aggregation result as the corresponding data; The aggregation result is cached; Before data aggregation, it is determined whether the data required by the parameters of the current request exists in the cached aggregation result, if yes, the cached aggregation result is called, the corresponding original data stored is obtained through a batch processing engine, a window is established through a stream processing engine, and the aggregation result and the original data are aggregated to generate a new aggregation result as the corresponding data; The number of times of calling each aggregation result in the cache is counted, if the number of times of calling exceeds a first preset number of times, the aggregation result is statically stored, and the corresponding aggregation result in the cache is cleared; If the cached aggregation result exceeds a preset cache time, the corresponding aggregation result is deleted; When any aggregation result in the cache is statically stored, the correlation degree between the statically stored aggregation result and other aggregation results is analyzed, and the preset cache time of the aggregation result with a correlation degree greater than a preset correlation degree is increased according to the correlation degree; The microservice encapsulates the corresponding data obtained and feeds back the data to the front-end component through the API gateway; The front-end component dynamically renders a corresponding component according to the type of the data. When any aggregation result in the cache is statically stored, the correlation degree between the statically stored aggregation result and other aggregation results is analyzed; 2. The method for interface-based data access and componentized presentation of claim 1, wherein, The aggregation result with a correlation degree greater than a preset minimum correlation degree is marked; If the time difference between the preset cache time of the marked aggregation result and the current time is less than a preset time difference, it is determined whether the number of times of calling the marked aggregation result is greater than a second preset number of times, if yes, the preset cache time of the aggregation result with a correlation degree greater than a preset correlation degree is increased according to the correlation degree; If not, it is set that when the time difference between the current time and the preset cache time of the marked aggregation result is equal to the preset time difference, it is determined whether the number of times of calling the marked aggregation result is greater than the second preset number of times. The correlation degree analysis adopts a neural network model to analyze the correlation degree between two aggregation results, including an input layer, a hidden layer and an output layer; 3. The method for interface-based data access and componentized presentation according to any one of claims 1 and 2, characterized in that, The input layer is used for inputting the aggregation result as a variable, and two neurons are set in the input layer; a plurality of neurons are set in the hidden layer; one neuron is set in the output layer to predict the correlation degree between the two aggregation results; and a ReLU activation function is used in the hidden layer. ​ 4. The method for interface-based data access and componentized presentation according to any one of claims 1 and 2, characterized in that, The method comprises the following steps: if the correlation degree is greater than the preset correlation degree, calculating the difference between the correlation degree and the preset correlation degree, and according to the mapping relationship between the preset difference and the preset cache time increase value, obtaining the corresponding preset cache time increase value and adding it to the preset cache time.

Images