Big Data Platform API Data Service Visible Domain Processing Method, Device and Storage Medium

Abstract

The present application discloses a method, device, and storage medium for processing the visible domain of API data services in a big data platform, relating to the technical field of data processing. The above method realizes the access range control of data by different APPs by defining a visible domain object. When a data request is triggered at the user layer, the data request is sent to the adaptation and conversion layer through the gateway layer. If the adaptation and conversion layer receives the data request, the adaptation and conversion layer determines the value range of the visible domain object corresponding to the APP that triggers the data request according to the APP identifier associated with the data request, and after generating a query statement based on the value range of the visible domain object and the data request, the query statement is sent to the calculation layer, where the value range is used to determine the data access range of the triggering APP. If the calculation layer receives the query statement, the calculation layer generates a query result according to the query statement and sends the query result to the triggering APP, reducing the management and maintenance costs of API data services.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a method, device, and storage medium for processing the visible domain of API data services in a big data platform. Background Art

[0002] In the current big data platform, data access control usually creates a new API (Application Programming Interface) for each different access scope of an APP (Application). This control method associates the API with specific data in a specified physical table in the database to limit the user's access scope. However, with the increase in big data service application scenarios, the number of APPs and the demand for data access have increased significantly. Using the above control method requires frequent creation of a large number of APIs, resulting in a high maintenance cost for API data services. Summary of the Invention

[0003] The main purpose of this application is to provide a method, device, and storage medium for processing the visible domain of API data services in a big data platform, aiming to solve the technical problem of high maintenance cost of API data services in current data access control.

[0004] To achieve the above purpose, this application proposes a method for processing the visible domain of API data services in a big data platform. The method includes:

[0005] When a data request is triggered at the user layer, the data request is sent to the adaptation and conversion layer through the gateway layer;

[0006] If the adaptation and conversion layer receives the data request, the adaptation and conversion layer determines the value range of the visible domain object corresponding to the triggering APP of the data request according to the APP identifier associated with the data request, and after generating a query statement based on the value range of the visible domain object and the data request, the query statement is sent to the computing layer, where the value range is used to determine the data access scope of the triggering APP;

[0007] If the computing layer receives the query statement, the computing layer generates a query result according to the query statement and sends the query result to the triggering APP.

[0008] In an embodiment, the adaptation and conversion layer includes an application visible domain authorizer. Before the step of sending the data request to the adaptation and conversion layer through the gateway layer when a data request is triggered at the user layer, it further includes:

[0009] Based on the table fields input by the user, a visible domain object is created in the application visible domain authorizer;

[0010] By applying a visible domain authorizer, based on the attribute value of the table field selected by the user, determine the value domain of the visible domain object, and establish a mapping relationship between the APP identifier and the visible domain object.

[0011] In one embodiment, the gateway layer includes at least one API, and the API corresponds to a database. The step of sending the data request to the adaptation and transformation layer through the gateway layer includes:

[0012] Based on the target API selected by the user, the gateway layer sends the data request to the adaptation and transformation layer through the target API.

[0013] In one embodiment, the adaptation and transformation layer further includes a task engine adapter and an API data manager. After the step of sending the data request to the adaptation and transformation layer through the gateway layer, it further includes:

[0014] If the adaptation and transformation layer receives the data request sent by the target API, the API data manager determines the type of the database corresponding to the target API according to the correspondence between the target API and the database;

[0015] Through the task engine adapter, determine the corresponding database adapter based on the type of the database. The database adapter is connected to the database and is used to execute the step of sending the query statement to the computing layer.

[0016] In one embodiment, the adaptation and transformation layer further includes a data visible domain processing device. The step of generating a query statement based on the value domain of the visible domain object and the data request includes:

[0017] Through the data visible domain processing device, generate an initial query statement based on the request field corresponding to the data request and the preset query field corresponding to the target API, and use the value domain of the visible domain object as a query condition and insert it into the initial query statement to obtain the query statement.

[0018] In one embodiment, before the step of using the value domain of the visible domain object as a query condition and inserting it into the initial query statement to obtain the query statement, it further includes:

[0019] Through the data visible domain processing device, parse the initial query statement into an abstract syntax tree, and determine the insertion position of the value domain according to the position of the preset query field in the abstract syntax tree.

[0020] In one embodiment, the computing layer includes at least one database. The step that when the computing layer receives the query statement, the computing layer generates a query result according to the query statement and sends the query result to the triggering APP includes:

[0021] If the computing layer receives the query statement sent by the database adapter, the computing layer determines the target database corresponding to the database adapter, and after executing the query statement, obtains the query result from the target database and sends the query result to the adaptation and conversion layer;

[0022] The adaptation and conversion layer sends the query result to the triggering APP through the gateway layer.

[0023] In one embodiment, the adaptation and conversion layer further includes a result desensitization processor. The step that the adaptation and conversion layer sends the query result to the triggering APP through the gateway layer includes:

[0024] The result desensitization processor obtains the visible field range and field desensitization rules of the triggering APP from the application visible domain authorizer, and after desensitizing the query result according to the visible field range and the field desensitization rules, sends the desensitized query result to the gateway layer;

[0025] The gateway layer sends the desensitized query result to the triggering APP in the user layer.

[0026] In addition, to achieve the above object, the present application further provides a big data platform API data service visible domain processing device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the big data platform API data service visible domain processing method as described above.

[0027] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the big data platform API data service visible domain processing method as described above are implemented.

[0028] The present application provides a method for processing the visible domain of API data services in a big data platform. By defining a visible domain object, it realizes the access range control of data for different APPs. When a data request is triggered at the user layer, the data request is sent to the adaptation and conversion layer through the gateway layer. If the adaptation and conversion layer receives the data request, the adaptation and conversion layer determines the value range of the visible domain object corresponding to the APP that triggers the data request according to the APP identifier associated with the data request. After generating a query statement based on the value range of the visible domain object and the data request, the query statement is sent to the computing layer, where the value range is used to determine the data access range of the triggering APP. If the computing layer receives the query statement, the computing layer generates a query result according to the query statement and sends the query result to the triggering APP.

[0029] The above method adds an adaptation and conversion layer between the gateway layer and the computing layer, and uses the value range of the visible domain object in the adaptation and conversion layer to limit the user's data access range. When a new APP or a new data access requirement appears, this method does not need to recreate and deploy new APIs at the gateway layer, but can quickly adapt to the new data access service requirements by modifying the value range of the visible domain object, reducing the time and workload of API data services, and achieving the effect of reducing the maintenance cost of API data services. Brief Description of the Drawings

[0030] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0031] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0032] Figure 1 It is a schematic diagram of the architecture of the device for processing the visible domain of API data services in the big data platform related to the method for processing the visible domain of API data services in the embodiments of the present application;

[0033] Figure 2 It is a schematic flowchart provided by Embodiment 1 of the method for processing the visible domain of API data services in the big data platform of the present application;

[0034] Figure 3 It is a schematic flowchart provided by Embodiment 2 of the method for processing the visible domain of API data services in the big data platform of the present application;

[0035] Figure 4 It is a schematic diagram of the method provided by Embodiment 2 of the method for processing the visible domain of API data services in the big data platform of the present application;

[0036] Figure 5 It is a schematic flow chart provided for the third embodiment of the method for processing the visible domain of API data service in the big data platform of the present application;

[0037] Figure 6 It is a schematic structural diagram provided for the fourth embodiment of the method for processing the visible domain of API data service in the big data platform of the present application;

[0038] Figure 7 It is a schematic flow chart provided for the fifth embodiment of the method for processing the visible domain of API data service in the big data platform of the present application;

[0039] Figure 8 It is a schematic structural diagram of the device of the hardware operating environment involved in the method for processing the visible domain of API data service in the big data platform in the embodiments of the present application.

[0040] The realization of the purpose, functional characteristics and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0041] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0042] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings of the specification and specific implementation manners.

[0043] In the current big data platform, data access control usually creates a new API separately for the different access scopes of each APP. This control method associates the API with specific data in a specified physical table in the database, thereby limiting the access scope of users. However, with the increase in the application scenarios of big data services, the number of APPs and the demand for data access have increased significantly. Adopting the above control method requires frequent creation of a large number of APIs, resulting in a relatively high maintenance cost for API data services.

[0044] In view of the above problems, the present application proposes a method for processing the visible domain of API data service in a big data platform. By defining a visible domain object, the access scope control of different APPs to data is realized. When a data request is triggered at the user layer, the data request is sent to the adaptation and conversion layer through the gateway layer. If the adaptation and conversion layer receives the data request, the adaptation and conversion layer determines the value range of the visible domain object corresponding to the APP that triggers the data request according to the APP identifier associated with the data request, and after generating a query statement based on the value range of the visible domain object and the data request, the query statement is sent to the computing layer, where the value range is used to determine the data access scope of the triggering APP. If the computing layer receives the query statement, the computing layer generates a query result according to the query statement and sends the query result to the triggering APP.

[0045] By adding an adaptation and conversion layer between the gateway layer and the computing layer, the above method uses the value range of the visible domain object in the adaptation and conversion layer to limit the user's data access scope. When a new APP or new data access requirement appears, instead of re-creating and deploying new APIs in the gateway layer, this method can quickly adapt to the new data access service requirements by modifying the value range of the visible domain object, reducing the time and workload of API data services, and achieving the effect of reducing the maintenance cost of API data services.

[0046] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, etc., or an electronic device, a big data platform API data service visible domain processing device, etc. that can implement the above functions. Hereinafter, taking the big data platform API data service visible domain processing device as an example, this embodiment and the following embodiments will be described.

[0047] To facilitate understanding of the implementation process of the big data platform API data service visible domain processing method in the embodiments of the present application, refer to Figure 1 , Figure 1 A schematic diagram of the architecture of a big data platform API data service visible domain processing device is provided. Specifically:

[0048] The big data platform API data service visible domain processing device is mainly divided into four parts: the user layer, the gateway layer, the adaptation and conversion layer, and the computing layer.

[0049] The big data platform API data service visible domain processing device allows an APP to call an API. Each APP has a unique APP identifier and can bind multiple published library table query Restful API services. When a service request is submitted through the gateway layer, a task will be submitted to the adaptation and conversion layer according to the APP identifier in the request path and the request header. In the adaptation and conversion layer, the big data platform API data service visible domain processing device will identify the database type of the query task through the database information associated with the API and automatically load the required database adapter. Then, according to the authorization configuration of the visible domain object, a query statement will be dynamically generated based on the value range of the visible domain object and the data request, and finally the query statement will be submitted to the computing layer for processing.

[0050] Among them, the user layer includes at least one APP, and the gateway layer includes at least one API, which is managed by the API data gateway center. The gateway layer receives the task request from the user layer, authenticates, performs security verification, and service validity verification on the APP identifier of the request. After the verification passes, the data request is sent to the adaptation and conversion layer.

[0051] The adaptation and conversion layer includes an API data manager, an application visible domain authorizer, a task engine adapter, a data visible domain processing device, a log processor, and a result desensitization processor. Among them, the API data manager is used to manage API data information, including: API parameter names, database information associated with the API, and preset query field information associated with the API. The application visible domain authorizer is used to manage the authorization of the visible domain object for the APP, limit the data access scope by accessing the value domain of the visible domain object, and at the same time limit the data reception scope of the APP and set the visible field scope and field desensitization rules for the query results. The task engine adapter is used to process and adapt all task requests supported by this device, and coordinate appropriate plugins such as database adapters for data logic processing. The data visible domain processing device uses ANTRL4 (Another Tool for Language Recognition 4, a parser generator for reading, processing, executing, or translating structured text or binary files) to parse the initial query statement according to different database types, and dynamically splice the value domain in the visible domain object as the query visible domain field information to generate physical query statements for different database engines. The log processor is used to monitor the logs of all running tasks and report them uniformly to the specified log service center. The result desensitization and encryption processor is used to perform data desensitization operations on the returned query results according to the field desensitization rules.

[0052] The computing layer is an API engine resource pool composed of various types of databases such as MySQL, Oracle, Trino, PostgreSQL, and Neo4j (Neo4j Graph Database), responsible for querying and obtaining API task data, and providing interfaces such as query job logs to the outside.

[0053] Based on this, the first embodiment proposed in this application provides a method for processing the visible domain of API data services in a big data platform. Referring to Figure 2 , in this embodiment, the method for processing the visible domain of API data services in the big data platform includes steps S10 to S30:

[0054] Step S10, when a data request is triggered at the user layer, the data request is sent to the adaptation and conversion layer through the gateway layer.

[0055] Among them, the gateway layer includes at least one API, and the API corresponds to the database. When a data request is triggered at the user layer, based on the target API selected by the user, the gateway layer sends the data request to the adaptation and conversion layer through the target API.

[0056] Specifically, each API within the gateway layer designates one or more databases as data sources, and the correspondence between the APIs and the databases is stored in the API data manager within the adaptation and transformation layer. The gateway layer forwards data requests to the adaptation and transformation layer via the target API to ensure that the adaptation and transformation layer can locate the corresponding database according to the correspondence between the target API and the database stored in the API data manager.

[0057] Step S20: If the adaptation and transformation layer receives the data request, the adaptation and transformation layer determines the value range of the visible domain object corresponding to the triggering APP for the data request based on the APP identifier associated with the data request, and after generating a query statement based on the value range of the visible domain object and the data request, sends the query statement to the computing layer, where the value range is used to determine the data access scope of the triggering APP.

[0058] It should be noted that a visible domain object refers to an object that defines the data scope accessible by one or more APPs. The value range defined by the visible domain object is used to determine which data items are visible to the APP. Exemplarily, a visible domain object can be a database table, a section of configuration in a configuration file, or an object instance in a program, storing information about which data resources can be accessed by a specific user. For example, for an APP of an e-commerce platform, its visible domain object may define which product information, order records, etc. the APP can view. Among them, a visible domain object can be authorized to one or more APPs. The triggering APP refers to the APP in the user layer that triggers a data request and has established a mapping relationship with the visible domain object.

[0059] Exemplarily, when the triggering APP in the user layer sends a data request to the database via the API in the gateway layer, it will first send valid authentication information such as application identifiers, usernames, user tokens, etc. to the gateway layer. After the triggering APP passes the authentication, the gateway layer sends the data request to the adaptation and transformation layer, and the adaptation and transformation layer locates the corresponding visible domain object according to the authentication information of the triggering APP and obtains the value range of the visible domain object for use in subsequent data access control logic.

[0060] Among them, the value range of the visible domain object refers to the range of the data set covered by the data access rules defined in the visible domain object. This value range can be specific database table records, file paths, data fields returned by APIs, etc. For example, if a user can only view the orders they created, then the value range of their visible domain object can be the application identifier of the user, used to limit the user's data access scope.

[0061] Exemplarily, after the big data platform API data service visible domain processing device determines the visible domain object corresponding to the triggered APP, it obtains the value range of the visible domain object, and based on this value range, determines the data range that the triggered APP can access, such as order data within a specific date range, transaction data within a specific amount range, etc.

[0062] It can be understood that in the conventional data API service, the control of the data access range is mainly filtered based on the inherent attributes of the APP user, such as information like username, organizational structure, job role, and geographical location. By associating the API with the above-mentioned inherent attributes, the user's data access range is restricted. However, in actual business scenarios, sometimes it is necessary to set the data visible range according to other dimensions that have nothing to do with the user's own attributes, such as information like the unit of goods inventory, pricing, and discount rate in the order table. At this time, it is necessary to create and deploy a new API to associate with the information of other dimensions to achieve the limitation of the user's data visible range. However, this embodiment proposes to define through the value range of the visible domain object, and further perform conditional screening on the query fields associated with the data request API to achieve data visible range control. By dynamically adjusting the value range of the visible domain object, it can easily adapt to business changes and adjustments of user requirements without creating and deploying a new API, reducing the time cost and workload cost.

[0063] Step S30, if the computing layer receives the query statement, the computing layer generates a query result according to the query statement and sends the query result to the triggered APP.

[0064] The query statement includes specific requirements for the data that the user hopes to retrieve or analyze, such as query fields, filtering conditions, sorting methods, etc. After receiving the query statement, the computing layer will parse it, understand the intention and structure of the query statement, access the underlying data storage such as databases, data warehouses, etc. according to the parsing result, and execute the actual query operation. After the query execution is completed, the computing layer will organize and generate the final query result and send the generated query result back to the APP that triggered the query.

[0065] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as the above-mentioned embodiment one can be referred to the above introduction and will not be repeated hereinafter. On this basis, the adaptation conversion layer includes an application visible domain authorizer. Please refer to Figure 3 , before step S10, the big data platform API data service visible domain processing method further includes steps S40 - S50:

[0066] Step S40, based on the table fields input by the user, create a visible domain object in the application visible domain authorizer;

[0067] Exemplarily, the user selects, through the management interface, the table fields that are desired to be used to define the visible domain object, and the table fields are the fields in the database table in the computing layer. As Figure 4 shown, it is assumed that the computing layer stores the "Exam Score Table" in the MySQL database, and its fields include "Name", "City", and "Exam Score", and the "City GDP Table" in the Oracle database, and its fields include "Quarter", "City", and "GDP". When wanting to perform regional access restrictions on the APP, the table field "City" can be selected as the visible domain object to limit the city range.

[0068] Step S50: Based on the attribute values of the table fields selected by the user, the application visible domain authorizer determines the value range of the visible domain object, and establishes a mapping relationship between the APP identifier and the visible domain object.

[0069] Exemplarily, referring to Figure 4 , the detailed attribute values of the table field "City" are used as the value range of the visible domain object. Among them, the identification value corresponding to the identification code "1" is "City A"; the identification value corresponding to the identification code "2" is "City B". By establishing a mapping relationship between the APP identifier and the visible domain object, authorization of the APP by the visible domain object is realized. When the visible domain object authorizes the APP, the access range of the APP can be restricted through the value range of the visible domain object. For example Figure 4 in, when the mapping relationship is established between the visible domain object and the APP identifier, if the value range identification code of the visible domain object authorized to the APP is set to "1", it means that the APP can only access the relevant data of City A.

[0070] It should be noted that the setting form and specific values of the value range of the visible domain object can be adaptively adjusted according to the actual situation. For example Figure 4 , other value ranges can also be added: the identification value corresponding to the identification code "3" is "﹩NO_DATA", indicating that no data is allowed to be accessed; the identification value corresponding to the identification code "4" is "﹩ALL_DATA", indicating that all data is allowed to be accessed.

[0071] After establishing the mapping relationship between the APP identifier and the visible domain object, when the adaptation and conversion layer receives a data request that triggers the APP, it can determine the value range of the visible domain corresponding to the triggered APP according to the above mapping relationship, and use the value range to limit the data retrieval range when performing the data request task subsequently.

[0072] In this embodiment, through the above-mentioned visible domain processing method for big data platform API data services, visible domain objects can be flexibly defined according to business requirements. By modifying the value range of the visible domain objects, fine control over the data acquisition scope for different applications can be achieved. Moreover, the visible domain objects can be applied to multiple APIs and databases, no longer limited to the control method of associating a single API with a specified physical table in a specific database, improving the reusability of data access scope control and reducing the management and maintenance costs of API data services.

[0073] Based on the above embodiments of the present application, in the third embodiment of the present application, for the same or similar content as the above embodiments, reference can be made to the above introduction and will not be repeated hereinafter. On this basis, the adaptation conversion layer further includes a task engine adapter and an API data manager. Please refer to Figure 5 , after step S10, the visible domain processing method for big data platform API data services further includes steps S60 to S70:

[0074] Step S60, if the adaptation conversion layer receives the data request sent by the target API, the API data manager determines the type of the database corresponding to the target API according to the corresponding relationship between the target API and the database.

[0075] Step S70, the task engine adapter determines the corresponding database adapter based on the type of the database, where the database adapter is connected to the database and is used to execute the step of sending the query statement to the computing layer.

[0076] It should be noted that the database adapter is a component used to convert the database operation requests of an application program into a format that a specific database system can understand, realizing the compatibility and communication between the APP and the database. Since there are differences in the syntax and processing logics of different types of databases, it is necessary to determine the corresponding database adapter according to the type of the database.

[0077] The API data manager is used to manage API data information, including: API parameter names, database information associated with the API, and preset query fields associated with the API. Optionally, a certain number of APIs are preset in the gateway layer, and these APIs are associated with the databases in the computing layer. When the APP triggers a data request and needs to obtain data from a specific database, the data request is sent to the adaptation conversion layer through the target API associated with the specific database. The adaptation conversion layer can determine the database corresponding to the target API through the corresponding relationship between the target API and the database stored in the API data manager, and then determine the corresponding database adapter through the task engine adapter. The database adapter sends the query statement corresponding to the data request to the corresponding database in the computing layer.

[0078] In this embodiment, the adaptation and conversion layer introduces a task engine adapter and an API data manager. When the adaptation and conversion layer receives a data request sent by a target API, the API data manager accurately determines the database type corresponding to the target API based on the corresponding relationship between the target API and the database. Subsequently, the task engine adapter matches the corresponding database adapter according to the database type, and this adapter is connected to the database and is responsible for passing the query statement to the computing layer. As a key component, the database adapter can convert the database operation requests of the application program into a format recognizable by a specific database system, ensuring the compatibility and smooth communication between the APP and different types of databases. By presetting the API in the gateway layer and associating it with the computing layer database, when the APP triggers a data request to obtain data from a specific database, the target API sends the request to the adaptation and conversion layer. The adaptation and conversion layer uses the corresponding relationship stored in the API data manager to determine the database and adapter corresponding to the target API, and then the database adapter sends the query statement to the corresponding database in the computing layer, making the data request processing more efficient and accurate, being able to better adapt to different database environments, and improving the accuracy and efficiency of data acquisition.

[0079] Based on the above embodiments of the present application, in the fourth embodiment of the present application, the adaptation and conversion layer further includes a data visible domain processing device. The step of generating a query statement based on the value range of the visible domain object and the data request in step S20 includes: The data visible domain processing device generates an initial query statement based on the request fields corresponding to the data request and the preset query fields corresponding to the target API, and uses the value range of the visible domain object as a query condition to insert it into the initial query statement to obtain the query statement.

[0080] It should be noted that in traditional data access control methods, usually the API is associated with specific data in a specified physical table in the database to limit the user's access scope. However, in this embodiment, the API is only associated with the database, and preset query fields can be set in advance for the API. When the API receives a data request, an initial query statement is formed based on the request fields corresponding to the request data and the preset query fields, and then the user's access scope is further limited according to the value range of the visible domain object.

[0081] Exemplarily, a preset query field "SELECT %FIELDS% FROM %TABLE% WHERE%CONDITIONS%" is set for each API, where "%" represents a placeholder. When the adaptation and conversion layer receives a data request, the request fields specified by the user are extracted from the data request and filled into the placeholder positions in the preset query field. For example Figure 2As shown in the figure, assume that the APP triggers a data request to query all exam score information from the MySQL database. Then, the request field "exam score table" is filled into the placeholder position of the preset query field in the exam score API to generate an initial query statement "SELECT * FROM exam score table". Then, according to the value range of the visible domain object "City A", the access range of the user is further restricted to generate a query statement "SELECT * FROM exam score table WHERE city = 'City A'".

[0082] To better understand the solution given in this example, the example is further described in combination with a specific application scenario.

[0083] Refer to Figure 6 , define the basic information of the visible domain object, such as the object name, object code, etc., and then define the value range of the visible domain object, such as the date range, amount range, sequence range, or define specific dictionary values such as city names, department names, educational backgrounds, etc. A certain APP calls multiple APIs, including "order API", "customer API", and "spatial coordinate API". Among them, the database name associated with the "order API" is "Mysql-x", the table name is "order_t", and the field name is "order_city"; the database name associated with the "customer API" is "MySQL-x", the table name is "cust_t", and the field name is "c_city"; the database name associated with the "spatial coordinate API" is "PostGIS-x", the table name is "city_gis_t", and the field name is "g_cc". In the conventional data access range control method, when the APP calls the above APIs to query city data, if the accessible city ranges of each API are inconsistent, then each API needs to be specified and associated with a specific city in the table to limit the data access range of the user side. However, through the big data platform API data service visible domain processing method proposed in this embodiment, the defined visible domain object can be mapped to the APP identifier, and the value range of the visible domain object corresponding to the APP can be obtained through the mapping relationship, so as to limit the accessible city range of the APP. Among them, a visible domain object can be mapped to multiple APPs, and the access range of multiple APIs can be controlled and applied to multiple databases. Among them, MySQL, Oracle, Trino, PostgreSQL, and Neo4j are five different types of databases. In other embodiments, the database engine includes but is not limited to the above five types.

[0084] It can be understood that since a mapping relationship is established between the visible domain object and the APP identifier, when the query permission of the APP changes, that is, when the data access range of the APP changes, the relevant API request processing logic can be automatically updated by modifying the value range of the visible domain object or specifying the specific table field attribute values authorized for the APP in the value range of the visible domain object, without manually modifying each API.

[0085] Optionally, before the step of obtaining the query statement by inserting the value range of the visible domain object as a query condition into the initial query statement, the data visible domain processing device parses the initial query statement into an abstract syntax tree and determines the insertion position of the value range according to the position of the preset query field in the abstract syntax tree.

[0086] It should be noted that the abstract syntax tree is a tree-like representation of the abstract syntax structure of the source code, and each node in the abstract syntax tree represents a structure in the source code such as an expression, a statement, a declaration, etc.

[0087] Exemplarily, based on ANTRL4, different types of query statements are parsed into an abstract syntax tree through a lexical rule file such as Lexer.g4 and a syntax rule file such as Parser.g4. Then, traverse the abstract syntax tree, access the custom Visitor and Listener classes of the nodes in the abstract syntax tree, and identify the semantics and preset query fields in the query statement such as SELECT, FROM, JOIN, WHERE, etc. After determining the position of the preset query field, the value range of the visible domain object is assigned and concatenated to dynamically add a filtering condition to the query statement and generate the final query statement. Among them, ANTRL4 is a grammar generator tool, and the Visitor class contains methods for accessing each node in the abstract syntax tree. Then, by calling the visit method of the Visitor and passing in the root node of the abstract syntax tree, all nodes in the abstract syntax tree can be recursively accessed. The Listener class contains event handling methods for entering and exiting each node in the abstract syntax tree, and ANTLR4 can automatically call these methods to traverse the abstract syntax tree.

[0088] By traversing the nodes of the abstract syntax tree and determining where to insert the value range condition according to the type and position of the query field in the node, it is possible to dynamically append a filtering field to the query statement, making the modification and extension of the query statement more intuitive and easy to manage.

[0089] Based on the above embodiments of the present application, in the fifth embodiment of the present application, the same or similar content as the above embodiments can be referred to the above introduction and will not be repeated hereinafter. On this basis, the computing layer includes at least one database. Please refer to Figure 7, after step S30, the big data platform API data service visible domain processing method further includes steps S80 to S90:

[0090] Step S80, if the computing layer receives the query statement sent by the database adapter, the computing layer determines the target database corresponding to the database adapter, and after executing the query statement, obtains the query result from the target database and sends the query result to the adaptation and conversion layer.

[0091] It should be noted that different database adapters correspond to different types of databases. The database adapter will convert the query statement into a format that a specific database can understand, and then send the query statement to the computing layer. The computing layer first receives the query statement sent by the database adapter, then executes the query statement to obtain the query result from the corresponding target database, and then sends the query result back to the adaptation and conversion layer. The target database is the database corresponding to the database adapter.

[0092] Step S90, the adaptation and conversion layer sends the query result to the trigger APP through the gateway layer.

[0093] Optionally, the adaptation and conversion layer further includes a result desensitization processor, and step S90 includes steps S91 to S92:

[0094] Step S91, the result desensitization processor obtains the visible field range and field desensitization rules of the trigger APP from the application visible domain authorizer, and after desensitizing the query result according to the visible field range and the field desensitization rules, sends the desensitized query result to the gateway layer.

[0095] It should be noted that the result desensitization processor is a component or module for processing data desensitization, which is used to process sensitive data according to preset field desensitization rules to protect data privacy, such as replacing personal address information with an anonymous number.

[0096] The application visible domain authorizer stores the visible field range and field desensitization rules of each APP. When the query result returns to the adaptation and conversion layer, the result desensitization processor first obtains the visible field range and field desensitization rules of the trigger APP from the application visible domain authorizer. The field desensitization rules can be set according to the data type of the query result, such as encoding desensitization for numerical data and encryption desensitization for text data.

[0097] Step S92, the gateway layer sends the desensitized query result to the trigger APP in the user layer.

[0098] In this embodiment, the database adapter can convert the query statement into a format that can be understood by a specific database, ensuring that different types of databases can correctly parse and execute the query operation. Further, after receiving the query result, the result desensitization processor in the adaptation and conversion layer obtains the visible field range and field desensitization rules for triggering the APP from the application visible domain authorizer. According to these rules, the result desensitization processor performs desensitization processing on the query result to prevent the leakage of sensitive information and ensure the privacy protection of data during transmission and use.

[0099] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the method for processing the visible domain of the big data platform API data service in this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0100] This application provides a device for processing the visible domain of the big data platform API data service. The device for processing the visible domain of the big data platform API data service includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method for processing the visible domain of the big data platform API data service in the first embodiment above.

[0101] Next, refer to Figure 8 , which shows a schematic structural diagram of a device for processing the visible domain of the big data platform API data service suitable for implementing the embodiments of this application. The device for processing the visible domain of the big data platform API data service in the embodiments of this application may include, but is not limited to, mobile terminals such as laptop computers, PADs (Portable Application Description, tablet computers), and fixed terminals such as desktop computers. Figure 8 The device for processing the visible domain of the big data platform API data service shown is only an example and should not bring any limitations to the functions and usage scopes of the embodiments of this application.

[0102] As Figure 8As shown, the big data platform API data service visible domain processing device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM, Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM, Random Access Memory) 1004. In the random access memory 1004, various programs and data required for the operation of the big data platform API data service visible domain processing device are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD, Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the big data platform API data service visible domain processing device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a big data platform API data service visible domain processing device with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems can be implemented or had.

[0103] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above functions defined in the method of the embodiments disclosed in the present application are executed.

[0104] The big data platform API data service visible domain processing device provided by this application, which adopts the big data platform API data service visible domain processing method in the above-mentioned embodiment, can solve the technical problem of high maintenance cost of API data service in current data access control. Compared with the prior art, the beneficial effects of the big data platform API data service visible domain processing device provided by this application are the same as those of the big data platform API data service visible domain processing method provided by the above-mentioned embodiment, and other technical features in this big data platform API data service visible domain processing device are the same as the features disclosed in the method of the previous embodiment, which will not be elaborated here.

[0105] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0106] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all of them should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0107] This application provides a computer-readable storage medium with computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the big data platform API data service visible domain processing method in the above-mentioned embodiment.

[0108] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memories, read-only memories, erasable programmable read-only memories (EPROMs), or flash memories, optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination of the above.

[0109] The above computer-readable storage medium can be included in the big data platform API data service visible domain processing device; or it can exist independently and not be assembled into the big data platform API data service visible domain processing device.

[0110] The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by the big data platform API data service visible domain processing device, the big data platform API data service visible domain processing device can write computer program code for performing the operations of this application in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, or executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0112] The modules described in the embodiments of the present application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0113] The readable storage medium provided by the present application is a computer-readable storage medium, and the computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned big data platform API data service visible domain processing method, which can solve the technical problem of high maintenance cost of API data services in current data access control. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the big data platform API data service visible domain processing method provided by the above embodiments, and will not be elaborated here.

[0114] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made using the specification and drawings of the present application under the technical concept of the present application, or any direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A method for processing the visible domain of API data services in a big data platform, characterized in that, The method includes: When a data request is triggered at the user layer, based on the target API selected by the user, the gateway layer sends the data request to the adaptation and transformation layer through the target API. The gateway layer includes at least one API, and the API corresponds to a database. If the adaptation and transformation layer receives the data request, the adaptation and transformation layer determines the value range of the visible domain object corresponding to the triggering APP according to the APP identifier associated with the data request, and generates an initial query statement through a data visible domain processing device based on the request fields corresponding to the data request and the preset query fields corresponding to the target API. Then, taking the value range of the visible domain object as a query condition and inserting it into the initial query statement, after obtaining the query statement, the query statement is sent to the computing layer, where the value range is used to determine the data access range of the triggering APP. If the adaptation and transformation layer receives the data request sent by the target API, the API data manager determines the type of the database corresponding to the target API according to the correspondence between the target API and the database, and determines the corresponding database adapter based on the type of the database through a task engine adapter. The database adapter is connected to the database and is used to execute the step of sending the query statement to the computing layer. If the computing layer receives the query statement, the computing layer generates a query result according to the query statement and sends the query result to the triggering APP.

2. The method for processing the visible domain of API data service in the big data platform according to claim 1, wherein, The adaptation and transformation layer includes an application visible domain authorizer. Before the step of sending the data request to the adaptation and transformation layer through the gateway layer when a data request is triggered at the user layer, it further includes: Creating a visible domain object in the application visible domain authorizer based on the table fields input by the user. Determining the value range of the visible domain object by the application visible domain authorizer based on the table field attribute values selected by the user, and establishing a mapping relationship between the APP identifier and the visible domain object.

3. The method for processing the visible domain of big data platform API data services according to claim 1, wherein, Before the step of taking the value range of the visible domain object as a query condition and inserting it into the initial query statement to obtain the query statement, it further includes: Parsing the initial query statement into an abstract syntax tree by the data visible domain processing device, and determining the insertion position of the value range according to the position of the preset query field in the abstract syntax tree.

4. The method for processing the visible domain of big data platform API data services according to claim 1, wherein, The computing layer includes at least one database. The step of the computing layer generating a query result according to the query statement and sending the query result to the triggering APP when the computing layer receives the query statement includes: If the computing layer receives the query statement sent by the database adapter, the computing layer determines the target database corresponding to the database adapter, and after executing the query statement, obtains the query result from the target database and sends the query result to the adaptation and transformation layer. Sending the query result to the triggering APP through the gateway layer by the adaptation and transformation layer.

5. The method for processing the visible domain of API data service in a big data platform according to claim 4, characterized in that, The adaptation and transformation layer further includes a result desensitization processor. The step of sending the query result to the triggering APP through the gateway layer by the adaptation and transformation layer includes: Obtaining the visible field range and field desensitization rules of the triggering APP from the application visible domain authorizer through the result desensitization processor, and after desensitizing the query result according to the visible field range and the field desensitization rules, sending the desensitized query result to the gateway layer; Sending the desensitized query result to the triggering APP in the user layer through the gateway layer.

6. A big data platform API data service visible domain processing device, characterized in that, The device includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the method for processing the visible domain of the big data platform API data service according to any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for processing the visible domain of the big data platform API data service according to any one of claims 1 to 5 are implemented.

Images