Big data platform API data service visible domain processing method and device and storage medium

Abstract

The invention discloses a big data platform API data service visible domain processing method and device and a storage medium, and relates to the technical field of data processing. According to the method, access range control of different APPs to data is realized by defining a visible domain object, when a user layer triggers a data request, the data request is sent to an adaptation conversion layer through a gateway layer, and if the adaptation conversion layer receives the data request, the adaptation conversion layer sends the data request to the user layer according to an APP identifier associated with the data request. The value domain of a visible domain object corresponding to a trigger APP of the data request is determined, after a query statement is generated based on the value domain of the visible domain object and the data request, the query statement is sent to a calculation layer, if the calculation layer receives the query statement, the calculation layer generates a query result according to the query statement, and the query result is sent to the calculation layer according to the query result. And the query result is sent to the triggering APP, so that the management and maintenance cost of the API data service is reduced.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and storage medium for processing visible domains of API data services on a big data platform. Background Art

[0002] In current big data platforms, data access control usually involves creating a new API (Application Programming Interface) for each APP (Application) with different access scopes. This control method limits the user's access scope by associating the API with specific data in a specified physical table in the database. However, with the increase in big data service application scenarios, the number of APPs and the demand for data access have increased significantly. The above control method requires frequent creation of a large number of APIs, resulting in high maintenance costs for API data services. Summary of the invention

[0003] The main purpose of this application is to provide a method, device and storage medium for processing the visible domain of API data services on a big data platform, aiming to solve the technical problem of high maintenance cost of API data services in current data access control.

[0004] To achieve the above objectives, the present application proposes a method for processing visible domains of API data services on a big data platform, the method comprising: When the user layer triggers a data request, the data request is sent to the adaptation and conversion layer through the gateway layer; If the adaptation and conversion layer receives the data request, the adaptation and conversion layer determines the value range of the visible domain object corresponding to the triggering APP of the data request according to the APP identifier associated with the data request, and after generating a query statement based on the value range of the visible domain object and the data request, sends the query statement to the computing layer, wherein the value range is used to determine the data access scope of the triggering APP; If the computing layer receives the query statement, the computing layer generates a query result according to the query statement, and sends the query result to the triggering APP.

[0005] In one embodiment, the adaptation and conversion layer includes an application-visible domain authorizer, and before the step of sending the data request to the adaptation and conversion layer through the gateway layer when the user layer triggers the data request, it also includes: Based on the table fields input by the user, create a visible domain object in the application visible domain authorizer; The visible domain authorizer is applied to determine the value domain of the visible domain object based on the table field attribute value selected by the user, and to establish a mapping relationship between the APP identifier and the visible domain object.

[0006] In one embodiment, the gateway layer includes at least one API, the API corresponds to a database, and the step of sending the data request to the adaptation conversion layer through the gateway layer includes: Based on the target API selected by the user, the gateway layer sends the data request to the adaptation and conversion layer through the target API.

[0007] In one embodiment, the adaptation and conversion layer further includes a task engine adapter and an API data manager. After the step of sending the data request to the adaptation and conversion layer through the gateway layer, the following steps are further included: If the adaptation and conversion layer receives the data request sent by the target API, the API data manager determines the type of database corresponding to the target API according to the corresponding relationship between the target API and the database; The task engine adapter determines the corresponding database adapter based on the type of the database, wherein the database adapter is connected to the database to execute the step of sending the query statement to the computing layer.

[0008] In one embodiment, the adaptation and conversion layer further includes a data visible domain processing device, and the step of generating a query statement based on the value range of the visible domain object and the data request includes: The data visible domain processing device generates an initial query statement based on the request field corresponding to the data request and the preset query field corresponding to the target API, and uses the value domain of the visible domain object as a query condition and inserts it into the initial query statement to obtain the query statement.

[0009] In one embodiment, before the step of inserting the value range of the visible domain object as a query condition into the initial query statement to obtain the query statement, the step further includes: The data visible domain processing device parses the initial query statement into an abstract syntax tree, and determines the insertion position of the value domain according to the position of the preset query field in the abstract syntax tree.

[0010] In one embodiment, the computing layer includes at least one database, and if the computing layer receives the query statement, the computing layer generates a query result according to the query statement, and sends the query result to the triggering APP, the step includes: If the computing layer receives the query statement sent by the database adapter, the computing layer determines the target database corresponding to the database adapter, and after executing the query statement, obtains the query result from the target database, and sends the query result to the adaptation and conversion layer; The query result is sent to the trigger APP through the gateway layer via the adaptation and conversion layer.

[0011] In one embodiment, the adaptation and conversion layer further includes a result desensitization processor, and the step of sending the query result to the triggering APP through the gateway layer through the adaptation and conversion layer includes: Obtain the visible field range and field desensitization rules of the triggered APP from the application visible domain authorizer through the result desensitization processor, desensitize the query result according to the visible field range and the field desensitization rules, and send the desensitized query result to the gateway layer; The desensitized query result is sent to the trigger APP in the user layer through the gateway layer.

[0012] In addition, to achieve the above-mentioned objectives, the present application also proposes a big data platform API data service visible domain processing device, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the big data platform API data service visible domain processing method as described above.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the big data platform API data service visible domain processing method as described above are implemented.

[0014] The present application provides a method for processing the visible domain of a big data platform API data service. By defining a visible domain object, different APPs can control the access scope of data. When the user layer triggers a data request, the data request is sent to the adaptation and conversion layer through the gateway layer. If the adaptation and conversion layer receives the data request, the adaptation and conversion layer determines the value domain of the visible domain object corresponding to the triggering APP of the data request according to the APP identifier associated with the data request, and after generating a query statement based on the value domain of the visible domain object and the data request, the query statement is sent to the computing layer, wherein the value domain is used to determine the data access scope of the triggering APP. If the computing layer receives the query statement, the computing layer generates a query result according to the query statement and sends the query result to the triggering APP.

[0015] The above method adds an adaptation conversion layer between the gateway layer and the computing layer, and uses the value range of the visible domain object in the adaptation conversion layer to limit the user's data access scope. When a new APP or new data access demand appears, this method does not need to recreate and deploy a new API at the gateway layer, but can quickly adapt to the new data access service demand by modifying the value range of the visible domain object, reducing the time and workload of API data services, and achieving the effect of reducing the maintenance cost of API data services. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0018] Figure 1 It is a schematic diagram of the architecture of a big data platform API data service visible domain processing device involved in a big data platform API data service visible domain processing method in an embodiment of the present application; Figure 2 A flowchart diagram of the first embodiment of the method for processing the visible domain of the API data service of the big data platform of this application; Figure 3 A flowchart diagram of the second embodiment of the method for processing the visible domain of the API data service of the big data platform of this application; Figure 4 A method diagram for providing a second embodiment of the method for processing the visible domain of the API data service of the big data platform of this application; Figure 5 A flowchart diagram of the third embodiment of the method for processing the visible domain of the API data service of the big data platform of this application; Figure 6 A structural diagram of a fourth embodiment of a method for processing a visible domain of an API data service on a big data platform of this application; Figure 7 A flowchart diagram of Embodiment 5 of the method for processing the visible domain of the API data service of the big data platform of this application; Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the big data platform API data service visible domain processing method in the embodiment of the present application.

[0019] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] In current big data platforms, data access control usually creates a new API for each APP's different access scope. This control method limits the user's access scope by associating the API with specific data in a specified physical table in the database. However, with the increase in big data service application scenarios, the number of APPs and the demand for data access have increased significantly. The above control method requires the frequent creation of a large number of APIs, resulting in high maintenance costs for API data services.

[0023] In view of the above problems, the present application proposes a method for processing the visible domain of API data services on a big data platform. By defining a visible domain object, different APPs can control the access scope of data. When the user layer triggers a data request, the data request is sent to the adaptation and conversion layer through the gateway layer. If the adaptation and conversion layer receives the data request, the adaptation and conversion layer determines the value domain of the visible domain object corresponding to the triggering APP of the data request based on the APP identifier associated with the data request, and after generating a query statement based on the value domain of the visible domain object and the data request, the query statement is sent to the computing layer, wherein the value domain is used to determine the data access scope of the triggering APP. If the computing layer receives the query statement, the computing layer generates a query result based on the query statement and sends the query result to the triggering APP.

[0024] The above method adds an adaptation conversion layer between the gateway layer and the computing layer, and uses the value range of the visible domain object in the adaptation conversion layer to limit the user's data access scope. When a new APP or new data access demand appears, this method does not need to recreate and deploy a new API at the gateway layer, but can quickly adapt to the new data access service demand by modifying the value range of the visible domain object, reducing the time and workload of API data services, and achieving the effect of reducing the maintenance cost of API data services.

[0025] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, etc., or an electronic device capable of realizing the above functions, a big data platform API data service visible domain processing device, etc. The following takes the big data platform API data service visible domain processing device as an example to illustrate this embodiment and the following embodiments.

[0026] In order to help understand the implementation process of the big data platform API data service visible domain processing method in the embodiment of the present application, refer to Figure 1 , Figure 1 A schematic diagram of the architecture of a visible domain processing device for a big data platform API data service is provided, specifically: The visible domain processing device of the big data platform API data service is mainly divided into four parts: user layer, gateway layer, adaptation and conversion layer, and computing layer.

[0027] The visible domain processing device of the big data platform API data service allows APP to call the API. Each APP has a unique APP identifier and can be bound to multiple published libraries and tables to query the Restful API service. When the service request is submitted through the gateway layer, the task will be submitted to the adaptation and conversion layer according to the request path and the APP identifier in the request header. In the adaptation and conversion layer, the visible domain processing device of the big data platform API data service will identify the database type of the query task through the database information associated with the API, and automatically load the required database adapter. Then, according to the authorization configuration of the visible domain object, the query statement is dynamically generated according to the value domain and data request of the visible domain object, and finally the query statement is submitted to the computing layer for processing.

[0028] The user layer includes at least one APP, and the gateway layer includes at least one API, which are managed by the API data gateway center. The gateway layer receives task requests from the user layer, authenticates the requested APP ID, performs security verification, and service validity verification. After verification, the data request is sent to the adaptation and conversion layer.

[0029] The adaptation and conversion layer includes an API data manager, an application visible domain authorizer, a task engine adapter, a data visible domain processing device, a log processor, and a result desensitization processor. Among them, the API data manager is used to manage API data information, including: API parameter name, API-related database information, and API-related preset query field information. The application visible domain authorizer is used to manage the authorization of the visible domain object to the APP, and limit the data access range by accessing the value domain of the visible domain object. At the same time, it limits the data receiving range of the APP and sets the visible field range and field desensitization rules for the query results. The task engine adapter is used to process and adapt all task requests supported by this device, and coordinate appropriate plug-ins such as database adapters for data logic processing. The data visible domain processing device uses ANTRL4 (Another Tool for Language Recognition 4, a parser generator for reading, processing, executing or translating structured text or binary files) to parse the initial query statement according to different database types, and dynamically splices the value domain in the visible domain object as the query visible domain field information to generate physical query statements for different database engines. The log processor is used to monitor the logs of all running tasks and report them to the designated log service center. The result desensitization and encryption processor is used to perform data desensitization operations on the returned query results according to the field desensitization rules.

[0030] The computing layer is an API engine resource pool composed of various types of databases such as MySQL, Oracle, Trino, PostgreSQL, and Neo4j (Neo4j Graph Database). It is responsible for querying and acquiring API task data and providing external interfaces such as query job logs.

[0031] Based on this, the first embodiment proposed in this application provides a method for processing the visible domain of a big data platform API data service, referring to Figure 2 In this embodiment, the big data platform API data service visible domain processing method includes steps S10 to S30: Step S10: When the user layer triggers a data request, the data request is sent to the adaptation and conversion layer through the gateway layer.

[0032] Among them, the gateway layer includes at least one API, and the API corresponds to the database. When the user layer triggers a data request, based on the target API selected by the user, the gateway layer sends the data request to the adaptation and conversion layer through the target API.

[0033] Specifically, the APIs in the gateway layer specify one or more databases as data sources, and the correspondence between the APIs and the databases is stored in the API data manager in the adaptation and conversion layer. The gateway layer forwards the data request to the adaptation and conversion layer through the target API to ensure that the adaptation and conversion layer can find the corresponding database based on the correspondence between the target API and the database stored in the API data manager.

[0034] Step S20, if the adaptation and conversion layer receives the data request, the adaptation and conversion layer determines the value range of the visible domain object corresponding to the triggering APP of the data request according to the APP identifier associated with the data request, and after generating a query statement based on the value range of the visible domain object and the data request, sends the query statement to the computing layer, wherein the value range is used to determine the data access scope of the triggering APP.

[0035] It should be noted that a visible domain object refers to an object that defines the range of data that can be accessed by one or more APPs, and the value domain defined by the visible domain object is used to determine which data items are visible to the APP. Exemplarily, a visible domain object can be a database table, a configuration in a configuration file, or an object instance in a program, which stores information about which data resources can be accessed by specific users. For example, an APP on an e-commerce platform may have a visible domain object that defines which product information, order records, etc. the APP can view. Among them, a visible domain object can be authorized to one or more APPs. A triggering APP refers to an APP that triggers a data request in the user layer and has established a mapping relationship with a visible domain object.

[0036] For example, when the triggering APP in the user layer sends a data request to the database through the API in the gateway layer, it will first send valid authentication information such as application identifier, user name, user token, etc. to the gateway layer. After the triggering APP passes the authentication, the gateway layer sends the data request to the adaptation and conversion layer. The adaptation and conversion layer searches for the corresponding visible domain object according to the authentication information of the triggering APP, and obtains the value domain of the visible domain object for subsequent data access control logic.

[0037] The value domain of the visible domain object refers to the range of data sets covered by the data access rules defined in the visible domain object. The value domain can be a specific database table record, file path, data field returned by an API, etc. For example, if a user can only view orders created by himself, the value domain of his visible domain object can be the application identifier of the user, which is used to limit the data access scope of the user.

[0038] Exemplarily, after determining the visible domain object corresponding to the trigger APP, the big data platform API data service visible domain processing device obtains the value domain of the visible domain object, and determines the data range that the trigger APP can access based on the value domain, such as order data within a specific date range, transaction data within a specific amount range, etc.

[0039] It is understandable that in conventional data API services, the access scope of data is mainly controlled based on the inherent attributes of APP users, such as user name, organizational structure, job role, and geographic location. The user's data access scope is limited by associating the API with the above inherent attributes. However, in actual business scenarios, it is sometimes necessary to set the data visibility range based on other dimensions that are not related to the user's own attributes, such as commodity inventory units, pricing, discount rates, and other information in the order table. At this time, it is necessary to create and deploy new APIs and associate them with information in other dimensions to achieve the limitation of the user data visibility range. However, this embodiment proposes to define the value domain of the visible domain object, and further perform conditional screening on the query field associated with the data request API to achieve data visibility range control. By dynamically adjusting the value domain of the visible domain object, it is possible to easily adapt to business changes and adjustments to user needs without creating and deploying new APIs, reducing time costs and workload costs.

[0040] Step S30: If the computing layer receives the query statement, the computing layer generates a query result according to the query statement and sends the query result to the triggering APP.

[0041] The query statement includes the specific requirements of the data that the user wants to retrieve or analyze, such as query fields, filtering conditions, sorting methods, etc. After receiving the query statement, the computing layer will parse it, understand the intent and structure of the query statement, access the underlying data storage such as databases, data warehouses, etc. according to the parsing results, and perform the actual query operation. After the query is executed, the computing layer will organize and generate the final query results, and send the generated query results back to the APP that triggered the query.

[0042] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and no further description will be given later. On this basis, the adaptation conversion layer includes an application visible domain authorizer, please refer to Figure 3 Before step S10, the big data platform API data service visible domain processing method further includes steps S40 to S50: Step S40, creating a visible domain object in the application visible domain authorizer based on the table fields input by the user; Exemplarily, the user selects a table field that he wishes to use to define a visible domain object through the management interface. The table field is a field in a database table in the computing layer. Figure 4 As shown in the figure, it is assumed that the computing layer stores the "Exam Score Table" in the MySQL database, whose fields include "Name", "City", and "Exam Score", and the "City GPD Table" in the Oracle database, whose fields include "Quarter", "City", and "GPD". When you want to restrict the access to the APP by region, you can select the table field "City" as the visible domain object to limit the city range.

[0043] Step S50: determining the value domain of the visible domain object based on the table field attribute value selected by the user through the application visible domain authorizer, and establishing a mapping relationship between the APP identifier and the visible domain object.

[0044] For example, refer to Figure 4 , the detailed attribute value of the table field "city" is used as the value range of the visible domain object. Among them, the identification code "1" corresponds to the identification value "city A"; the identification code "2" corresponds to the identification value "city B". By establishing a mapping relationship between the APP identifier and the visible domain object, the authorization of the visible domain object to the APP is realized. When the visible domain object is authorized to the APP, the access scope of the APP can be limited by the value range of the visible domain object. For example Figure 4 In the example, after a mapping relationship is established between the visible domain object and the APP identifier, if the value domain identifier of the visible domain object authorized to the APP is set to "1", it means that the APP can only access the relevant data of city A.

[0045] It should be noted that the setting form and specific value of the value range of the visible domain object can be adaptively adjusted according to actual conditions, for example Figure 4 Other value domains can also be added: the identification code "3" corresponds to the identification value "﹩NO_DATA", indicating that access to any data is not allowed; the identification code "4" corresponds to the identification value "﹩ALL_DATA", indicating that access to all data is allowed.

[0046] After establishing the mapping relationship between the APP identifier and the visible domain object, when the adaptation conversion layer receives a data request that triggers the APP, it can determine the value range of the visible domain corresponding to the triggered APP based on the above mapping relationship, and use the value range to limit the data retrieval scope when executing the data request task subsequently.

[0047] In this embodiment, through the above-mentioned big data platform API data service visible domain processing method, visible domain objects can be flexibly defined according to business needs, and fine control over the data range obtained by different applications can be achieved by modifying the value range of the visible domain objects. The visible domain objects can be applied to multiple APIs and databases, and are no longer limited to the control method of associating a single API with a specified physical table in a specific database. This improves the reusability of data access range control and reduces the management and maintenance costs of API data services.

[0048] Based on the above embodiments of the present application, in the third embodiment of the present application, the same or similar contents as the above embodiments can be referred to the above introduction, and no further description will be given later. On this basis, the adaptation conversion layer also includes a task engine adapter and an API data manager, please refer to Figure 5 After step S10, the big data platform API data service visible domain processing method further includes steps S60 to S70: Step S60: If the adaptation and conversion layer receives the data request sent by the target API, the API data manager determines the type of database corresponding to the target API according to the corresponding relationship between the target API and the database.

[0049] Step S70, determining a corresponding database adapter based on the type of the database through the task engine adapter, wherein the database adapter is connected to the database to execute the step of sending the query statement to the computing layer.

[0050] It should be noted that the database adapter is a component used to convert the application's database operation request into a format that a specific database system can understand, to achieve compatibility and communication between the APP and the database. Different types of databases have different syntax and processing logic, so it is necessary to determine the corresponding database adapter based on the type of database.

[0051] The API data manager is used to manage API data information, including: API parameter names, API-associated database information, and API-associated preset query fields. Optionally, a certain number of APIs are preset in the gateway layer, and these APIs are associated with the database in the computing layer. When the APP triggers a data request that needs to obtain data from a specific database, the data request is sent to the adaptation and conversion layer through the target API associated with the specific database. The adaptation and conversion layer can determine the database corresponding to the target API through the correspondence between the target API and the database stored in the API data manager, and then determine the corresponding database adapter through the task engine adapter, and the database adapter sends the query statement corresponding to the data request to the corresponding database in the computing layer.

[0052] In this embodiment, the adaptation and conversion layer introduces a task engine adapter and an API data manager. When the adaptation and conversion layer receives the data request sent by the target API, the API data manager accurately determines the database type corresponding to the target API based on the correspondence between the target API and the database. Subsequently, the task engine adapter matches the corresponding database adapter according to the database type. The adapter is connected to the database and is responsible for passing the query statement to the computing layer. As a key component, the database adapter can convert the database operation request of the application into a format that can be recognized by a specific database system, ensuring the compatibility and smooth communication between the APP and different types of databases. By presetting the API at the gateway layer and associating it with the computing layer database, when the APP triggers a data request to obtain data from a specific database, the target API sends the request to the adaptation and conversion layer. The adaptation and conversion layer uses the correspondence stored in the API data manager to determine the database and adapter corresponding to the target API, and then the database adapter sends the query statement to the corresponding database of the computing layer, making the data request processing more efficient and accurate, and can better adapt to different database environments, and improve the accuracy and efficiency of data acquisition.

[0053] Based on the above embodiments of the present application, in the fourth embodiment of the present application, the adaptation conversion layer also includes a data visible domain processing device, and the step of generating a query statement based on the value domain of the visible domain object and the data request in step S20 includes: the data visible domain processing device generates an initial query statement based on the request field corresponding to the data request and the preset query field corresponding to the target API, and uses the value domain of the visible domain object as a query condition, inserts it into the initial query statement, and obtains a query statement.

[0054] It should be noted that in traditional data access control methods, the API is usually associated with specific data in a specified physical table in the database, thereby limiting the user's access scope. However, in this embodiment, the API is only associated with the database, and a preset query field can be set for the API in advance. When the API receives a data request, an initial query statement is formed based on the request field corresponding to the request data and the preset query field, and then the user's access scope is further limited based on the value range of the visible domain object.

[0055] For example, a preset query field "SELECT %FIELDS% FROM %TABLE% WHERE %CONDITIONS%" is set for each API, where "%" represents a placeholder. When the adaptation conversion layer receives a data request, it extracts the request field specified by the user from the data request and fills the extracted request field into the placeholder position in the preset query field. Figure 2As shown, assuming that the APP triggers a data request to query all test score information from the MySQL database, the request field "Test Score Table" is filled into the placeholder position of the preset query field in the test score API, and the initial query statement "SELECT * FROM Test Score Table" is generated. Then, the user's access scope is further limited according to the value domain "City A" of the visible domain object, and the query statement "SELECT * FROM Test Score Table WHERE City = 'City A'" is generated.

[0056] To better understand the solution provided in this example, this example is further explained in combination with specific application scenarios.

[0057] Reference Figure 6 , define the basic information of the visible domain object, such as object name, object code, etc., and then define the value range of the visible domain object, such as date range, amount range, sequence range, or define specific dictionary values ​​such as city name, department name, education level, etc. A certain APP calls multiple APIs, including "order API", "customer API", and "spatial coordinate API". Among them, the database name associated with the "order API" is "Mysql-x", the table name is "order_t", and the field name is "order_city"; the database name associated with the "customer API" is "MySQL-x", the table name is "cust_t", and the field name is "c_city"; the database name associated with the "spatial coordinate API" is "PostGIS-x", the table name is "city_gis_t", and the field name is "g_cc". In the conventional data access range control method, when the APP calls the above API to query city data, if the city range that each API can access is inconsistent, each API needs to be specified to be associated with a specific city in the table, so as to limit the data access range of the user end. However, through the big data platform API data service visible domain processing method proposed in this embodiment, a mapping relationship can be established between the defined visible domain object and the APP identifier, and the value range of the visible domain object corresponding to the APP can be obtained through the mapping relationship, thereby limiting the city range that the APP can access. Among them, a visible domain object can be mapped with multiple APPs, and the access range of multiple APIs can be controlled and applied to multiple databases. Among them, MySQL, Oracle, Trino, PostgreSQL and Neo4j are five different types of databases. In other embodiments, the database engine includes but is not limited to the above five.

[0058] It can be understood that since a mapping relationship is established between the visible domain object and the APP identifier, when the query permission of the APP changes, that is, the data access scope of the APP changes, the relevant API request processing logic can be automatically updated by modifying the value domain of the visible domain object, or specifying the specific table field attribute value authorized for the APP in the value domain of the visible domain object, without the need to manually modify each API.

[0059] Optionally, the value domain of the visible domain object is used as a query condition and inserted into the initial query statement. Before obtaining the query statement, the data visible domain processing device parses the initial query statement into an abstract syntax tree and determines the insertion position of the value domain based on the position of the preset query field in the abstract syntax tree.

[0060] It should be noted that the abstract syntax tree is a tree representation of the abstract syntax structure of the source code. Each node in the abstract syntax tree represents a structure in the source code, such as an expression, statement, declaration, etc.

[0061] Exemplarily, based on ANTRL4, different types of query statements are parsed into abstract syntax trees through lexical rule files such as Lexer.g4 files and grammar rule files such as Parser.g4 files. Afterwards, the abstract syntax tree is traversed, and the custom Visitor and Listener classes of the nodes in the abstract syntax tree are accessed to identify the semantics and preset query fields such as SELECT, FROM, JOIN, WHERE, etc. in the query statement. After determining the position of the preset query field, the value domain of the visible domain object is assigned and spliced ​​to dynamically add filter conditions in the query statement and generate the final query statement. Among them, ANTRL4 is a grammar generator tool, and the Visitor class contains methods for accessing each node in the abstract syntax tree. Then, by calling the visit method of the Visitor and passing in the root node of the abstract syntax tree, all nodes in the abstract syntax tree can be recursively accessed. The Listener class contains event processing methods for each node in the abstract syntax tree when entering and exiting, and ANTLR4 can automatically call these methods to traverse the abstract syntax tree.

[0062] By traversing the nodes of the abstract syntax tree and determining where to insert the value range condition according to the type and position of the query field in the node, it is possible to dynamically append filter fields to the query statement, making the modification and extension of the query statement more intuitive and easy to manage.

[0063] Based on the above embodiments of the present application, in the fifth embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and no further description will be given later. On this basis, the computing layer includes at least one database, please refer to Figure 7After step S30, the big data platform API data service visible domain processing method further includes steps S80 to S90: Step S80, if the computing layer receives the query statement sent by the database adapter, the computing layer determines the target database corresponding to the database adapter, and after executing the query statement, obtains the query result from the target database, and sends the query result to the adaptation conversion layer.

[0064] It should be noted that different database adapters correspond to different types of databases. The database adapter converts the query statement into a format that a specific database can understand, and then sends the query statement to the computing layer. The computing layer first receives the query statement sent by the database adapter, then executes the query statement to obtain the query result from the corresponding target database, and then sends the query result back to the adapter conversion layer. The target database is the database corresponding to the database adapter.

[0065] Step S90: Send the query result to the trigger APP through the gateway layer via the adaptation and conversion layer.

[0066] Optionally, the adaptation conversion layer further includes a result desensitization processor, and step S90 includes steps S91-S92: Step S91, obtain the visible field range and field desensitization rules of the triggered APP from the application visible domain authorizer through the result desensitization processor, and desensitize the query result according to the visible field range and the field desensitization rules, and send the desensitized query result to the gateway layer.

[0067] It should be noted that the result desensitization processor is a component or module used to process data desensitization, which is used to process sensitive data according to preset field desensitization rules to protect data privacy, such as replacing personal address information with anonymous numbers.

[0068] The application visible domain authorizer stores the visible field range and field desensitization rules of each APP. When the query results are returned to the adaptation conversion layer, the result desensitization processor first obtains the visible field range and field desensitization rules of the triggered APP from the application visible domain authorizer. The field desensitization rules can be set according to the data type of the query results, such as encoding and desensitizing numeric data, encrypting and desensitizing text data, etc.

[0069] Step S92: Send the desensitized query result to the trigger APP in the user layer through the gateway layer.

[0070] In this embodiment, the database adapter can convert the query statement into a format that can be understood by a specific database, ensuring that different types of databases can correctly parse and execute query operations. Furthermore, after receiving the query results, the result desensitization processor in the adaptation and conversion layer will obtain the visible field range and field desensitization rules that trigger the APP from the application visible domain authorizer. According to these rules, the result desensitization processor desensitizes the query results to prevent the leakage of sensitive information and ensure the privacy protection of data during transmission and use.

[0071] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the visible domain processing method of the API data service of the big data platform of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0072] The present application provides a big data platform API data service visible domain processing device, and the big data platform API data service visible domain processing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the big data platform API data service visible domain processing method in the above-mentioned embodiment one.

[0073] Reference below Figure 8 , which shows a schematic diagram of the structure of a big data platform API data service visible domain processing device suitable for implementing the embodiment of the present application. The big data platform API data service visible domain processing device in the embodiment of the present application may include but is not limited to mobile terminals such as laptops, PADs (Portable Application Description, tablet computers), etc., and fixed terminals such as desktop computers. Figure 8 The big data platform API data service visible domain processing device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0074] like Figure 8As shown, the big data platform API data service visible domain processing device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM) 1004. In the random access memory 1004, various programs and data required for the operation of the big data platform API data service visible domain processing device are also stored. The processing device 1001, the read-only memory 1002 and the random access memory 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the big data platform API data service visible domain processing device to communicate wirelessly or wired with other devices to exchange data. Although the big data platform API data service visible domain processing device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0075] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0076] The big data platform API data service visible domain processing device provided by the present application adopts the big data platform API data service visible domain processing method in the above embodiment, which can solve the technical problem of high maintenance cost of API data service in current data access control. Compared with the prior art, the beneficial effects of the big data platform API data service visible domain processing device provided by the present application are the same as the beneficial effects of the big data platform API data service visible domain processing method provided by the above embodiment, and the other technical features in the big data platform API data service visible domain processing device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0077] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0078] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0079] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the big data platform API data service visible domain processing method in the above-mentioned embodiment.

[0080] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM, CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, radio frequencies (RF, Radio Frequency), etc., or any suitable combination of the above.

[0081] The above-mentioned computer-readable storage medium may be included in the big data platform API data service visible domain processing device; or it may exist independently without being assembled into the big data platform API data service visible domain processing device.

[0082] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the big data platform API data service visible domain processing device, the big data platform API data service visible domain processing device can write computer program codes for performing the operations of the present application in one or more programming languages ​​or a combination thereof. The above-mentioned programming languages ​​include object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, or as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0083] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0084] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0085] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned big data platform API data service visible domain processing method, which can solve the technical problem of high maintenance cost of API data services in current data access control. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the big data platform API data service visible domain processing method provided by the above-mentioned embodiment, and will not be repeated here.

[0086] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for processing visible domain of API data service on a big data platform, characterized in that: The method comprises: When the user layer triggers a data request, the data request is sent to the adaptation and conversion layer through the gateway layer; If the adaptation and conversion layer receives the data request, the adaptation and conversion layer determines the value range of the visible domain object corresponding to the triggering APP of the data request according to the APP identifier associated with the data request, and after generating a query statement based on the value range of the visible domain object and the data request, sends the query statement to the computing layer, wherein the value range is used to determine the data access scope of the triggering APP; If the computing layer receives the query statement, the computing layer generates a query result according to the query statement, and sends the query result to the triggering APP.

2. The method for processing visible domain of a big data platform API data service according to claim 1, characterized in that: The adaptation and conversion layer includes an application-visible domain authorizer. When the user layer triggers a data request, before the step of sending the data request to the adaptation and conversion layer through the gateway layer, the method further includes: Based on the table fields input by the user, create a visible domain object in the application visible domain authorizer; The application visible domain authorizer determines the value domain of the visible domain object based on the table field attribute value selected by the user, and establishes a mapping relationship between the APP identifier and the visible domain object.

3. The big data platform API data service visible domain processing method according to claim 1, characterized in that: The gateway layer includes at least one API, and the API corresponds to a database. The step of sending the data request to the adaptation conversion layer through the gateway layer includes: Based on the target API selected by the user, the gateway layer sends the data request to the adaptation and conversion layer through the target API.

4. The method for processing visible domain of a big data platform API data service according to claim 3, characterized in that: The adaptation and conversion layer also includes a task engine adapter and an API data manager. After the step of sending the data request to the adaptation and conversion layer through the gateway layer, the method further includes: If the adaptation and conversion layer receives the data request sent by the target API, the API data manager determines the type of database corresponding to the target API according to the corresponding relationship between the target API and the database; The task engine adapter determines a corresponding database adapter based on the type of the database, wherein the database adapter is connected to the database to execute the step of sending the query statement to the computing layer.

5. The method for processing visible domain of a big data platform API data service according to claim 4, characterized in that: The adaptation conversion layer also includes a data visible domain processing device, and the step of generating a query statement based on the value range of the visible domain object and the data request includes: The data visible domain processing device generates an initial query statement based on the request field corresponding to the data request and the preset query field corresponding to the target API, and uses the value domain of the visible domain object as a query condition and inserts it into the initial query statement to obtain the query statement.

6. The method for processing visible domain of a big data platform API data service according to claim 5, characterized in that: Before the step of using the value range of the visible domain object as a query condition and inserting it into the initial query statement to obtain the query statement, the step further includes: The data visible domain processing device parses the initial query statement into an abstract syntax tree, and determines the insertion position of the value domain according to the position of the preset query field in the abstract syntax tree.

7. The method for processing visible domain of a big data platform API data service according to claim 4, characterized in that: The computing layer includes at least one database. If the computing layer receives the query statement, the computing layer generates a query result according to the query statement, and sends the query result to the triggering APP. The steps include: If the computing layer receives the query statement sent by the database adapter, the computing layer determines the target database corresponding to the database adapter, and after executing the query statement, obtains the query result from the target database, and sends the query result to the adaptation and conversion layer; The query result is sent to the trigger APP through the gateway layer via the adaptation and conversion layer.

8. The method for processing visible domain of a big data platform API data service according to claim 7, characterized in that: The adaptation and conversion layer also includes a result desensitization processor, and the step of sending the query result to the trigger APP through the gateway layer through the adaptation and conversion layer includes: Obtain the visible field range and field desensitization rules of the triggered APP from the application visible domain authorizer through the result desensitization processor, desensitize the query result according to the visible field range and the field desensitization rules, and send the desensitized query result to the gateway layer; The desensitized query result is sent to the trigger APP in the user layer through the gateway layer.

9. A big data platform API data service visible domain processing device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the big data platform API data service visible domain processing method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the big data platform API data service visible domain processing method as described in any one of claims 1 to 8 are implemented.

Images