Data desensitization processing method and device, data desensitization access method and device, equipment and medium

Abstract

The invention relates to the technical field of data security and the field of financial science and technology, and discloses a data desensitization processing and access method, which comprises the following steps: deploying a data query service in a production data processing platform, receiving a request and returning desensitized data; pre-defined mark annotation is applied in the development test platform, and a dynamic agent mechanism is configured; intercepting data access interface calling, and constructing and sending a hypertext transfer protocol request; after receiving the request, the production data processing platform executes query and returns desensitization data, and the test environment receives and processes the desensitization data. According to the method, the process of accessing the production environment data by the test environment is simplified through a dynamic agent mechanism, automatic data calling and desensitization processing are realized, and the real-time performance and efficiency of data acquisition are improved. The data privacy is guaranteed, the requirement of rapid iterative development is met, and the working efficiency of development and testing is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the fields of data security technology and financial technology, and in particular to a data desensitization processing and access method, device, equipment and storage medium. Background Art

[0002] In the financial sector, the agility of system development and testing is crucial to quickly respond to market demands and improve competitiveness. Typically, the production environment of financial systems stores a large amount of sensitive data, including users' personal information, transaction data, and financial records. In order to ensure data security, production data is often required to be desensitized in development and testing environments, so that developers and testers can use secure desensitized data to avoid sensitive data leakage when debugging and verifying system functions.

[0003] However, in the existing technology, there are many difficulties in obtaining production data in the test environment and desensitizing it. First, the traditional production environment launch process needs to go through multiple complex approval links and strict compliance requirements, which makes it difficult to use the data in the production environment directly for testing. This lengthy and cumbersome review process conflicts with the agile development model of the current project and cannot meet the needs of rapid iteration and demand changes. Development teams in financial institutions often face the problem of long system iteration launch time and difficulty in flexibly responding to market changes.

[0004] In addition, in the current development and testing process, proxy classes are usually used to simulate data calls in the production environment in the test environment. However, as the scale of the system expands and the complexity of the business increases, a large amount of proxy class code needs to be written and maintained in the test environment to implement the call to the production data. This excessive accumulation of proxy classes not only increases the difficulty of code maintenance, but may also cause problems such as low system efficiency and code redundancy, further reducing the flexibility and iteration efficiency of development and testing.

[0005] At the same time, existing production data desensitization processing methods usually lack real-time and flexibility. Most desensitization operations need to be performed after the data is exported in batches, which is difficult to meet the demand for real-time data during the testing process. Moreover, existing desensitization methods cannot fully guarantee the security of sensitive information throughout the development and testing process, increasing the risk of data leakage. Summary of the invention

[0006] The main purpose of the present invention is to provide a data desensitization processing and access method, device, equipment and storage medium, aiming to solve the technical problem in the prior art that the test environment cannot safely and efficiently access and process the desensitized data in the production environment in an automated manner.

[0007] To achieve the above object, the present invention provides a data desensitization processing and access method, comprising:

[0008] Deploy a data query service in the production data processing platform, wherein the data query service is used to receive a request based on the hypertext transfer protocol and return a query result after desensitization processing;

[0009] In the development test platform, apply predefined marker annotations to the data access interfaces that need to be proxied;

[0010] During the service container startup process of the development and testing platform, a postprocessor is registered, and a dynamic proxy mechanism in the development and testing platform is configured through the postprocessor;

[0011] Intercepting calls to data access interfaces annotated with predefined tags through the dynamic proxy mechanism;

[0012] Building a request based on the hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service;

[0013] The data query service performs a data query operation according to the request and obtains query result data;

[0014] The data query service performs desensitization processing on the sensitive data in the query result data to obtain desensitized result data;

[0015] Returning the desensitized result data to the development and testing platform through a response based on the Hypertext Transfer Protocol;

[0016] The dynamic proxy mechanism in the development and testing platform receives the desensitized result data and transmits the desensitized result data to the caller of the data access interface for business logic processing.

[0017] Furthermore, to achieve the above-mentioned purpose, the present invention provides a data desensitization processing and access device, comprising:

[0018] A data query service module is used to deploy a data query service in a production data processing platform, wherein the data query service is used to receive a request based on the hypertext transfer protocol and return a query result after desensitization processing;

[0019] The data access interface module is used to apply predefined tag annotations to the data access interfaces that need to be proxied in the development and testing platform;

[0020] A post-processor module is used to register a post-processor during the startup of the service container of the development and testing platform, and configure the dynamic proxy mechanism in the development and testing platform through the post-processor;

[0021] A dynamic proxy module, used to intercept calls to data access interfaces with predefined tag annotations through the dynamic proxy mechanism;

[0022] A request building module, used for building a request based on the hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service;

[0023] A data query module, used for the data query service to perform a data query operation according to the request and obtain query result data;

[0024] A data desensitization module, used for the data query service to perform desensitization processing on sensitive data in the query result data to obtain desensitized result data;

[0025] A response generation and transmission module, used to return the desensitization result data to the development and testing platform through a response based on the hypertext transfer protocol;

[0026] The result processing module is used to develop a dynamic proxy mechanism in the test platform to receive the desensitized result data, and pass the desensitized result data to the caller of the data access interface for business logic processing.

[0027] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer device, which includes a memory, a processor, and a data desensitizing processing and access program stored in the memory and executable on the processor, wherein the data desensitizing processing and access program, when executed by the processor, implements the steps of the data desensitizing processing and access method as described above.

[0028] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a data desensitizing processing and access program is stored, and when the data desensitizing processing and access program is executed by a processor, the steps of the data desensitizing processing and access method as described above are implemented.

[0029] Beneficial effects: The present invention relates to the fields of data security technology and financial technology, and discloses a data desensitization processing and access method, which deploys a data query service in a production data processing platform, receives requests and returns desensitized data; applies predefined tag annotations in a development and testing platform, configures a dynamic proxy mechanism; intercepts data access interface calls, builds and sends hypertext transfer protocol requests; the production data processing platform executes queries and returns desensitized data after receiving the request, and the test environment receives and processes the desensitized data. The present invention simplifies the process of the test environment accessing the production environment data through a dynamic proxy mechanism, realizes automated data calling and desensitization processing, and improves the real-time and efficiency of data acquisition. It ensures data privacy, meets the needs of rapid iterative development, and significantly improves the work efficiency of development and testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0031] Figure 1 A schematic diagram of an application environment of a data desensitization processing and access method in an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of a flow chart of an embodiment of a data desensitization processing and access method of the present invention;

[0033] Figure 3 A schematic diagram of functional modules of a preferred embodiment of the data desensitization processing and access device of the present invention;

[0034] Figure 4 A schematic diagram of the structure of a computer device in one embodiment of the present invention;

[0035] Figure 5 FIG. 4 is another schematic diagram of the structure of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0036] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0037] The data desensitization processing and access method provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In an application environment, the user end communicates with the server end through a network. The server end can deploy a data query service in a production data processing platform through the user end, receive requests and return desensitized data; apply predefined tag annotations in the development and testing platform, configure a dynamic proxy mechanism; intercept data access interface calls, build and send hypertext transfer protocol requests; the production data processing platform executes queries and returns desensitized data after receiving the request, and the test environment receives and processes the desensitized data. The present invention simplifies the process of the test environment accessing the production environment data through a dynamic proxy mechanism, realizes automated data calling and desensitization processing, and improves the real-time and efficiency of data acquisition. It guarantees data privacy, meets the needs of rapid iterative development, and significantly improves the work efficiency of development and testing. Among them, the user end can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The server end can be implemented with an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0038] See also Figure 2 , Figure 2 This is a flow chart of an embodiment of a data desensitization processing and access method provided by the present invention. It should be noted that although a logical sequence is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0039] like Figure 2 As shown, the data desensitization processing and access method proposed by the present invention includes the following steps:

[0040] S10, deploying a data query service in the production data processing platform, wherein the data query service is used to receive a request based on the hypertext transfer protocol and return a query result after desensitization processing;

[0041] In this embodiment, the production data processing platform is part of the core business system deployed in financial institutions and other enterprises, and is dedicated to processing data requests from other environments (such as test environments). The data query service is a module on the platform that is responsible for obtaining business data from the database based on query conditions. This service needs to have high concurrent processing capabilities, be able to respond quickly to a large number of requests, and be able to access data sources in the production environment. It also needs to cooperate with the desensitization processing module to ensure that sensitive data is desensitized before being returned.

[0042] Data query services can be developed using mainstream Web frameworks, such as Java's Spring framework. Typical implementations include: using the Spring Boot framework to quickly build services; exposing data query interfaces through RESTful APIs; using MyBatis or JPA to perform database operations and query data in production databases; listening to specific ports after the service is started, waiting for query requests to arrive; the service also needs to configure a connection pool (such as HikariCP) to handle high-concurrency requests and ensure data query performance.

[0043] The data query service needs to support communication based on the HTTP protocol, which means that external systems (such as the test environment) can send query requests through the HTTP protocol. The HTTP protocol is the most commonly used application layer protocol on the Internet, which allows clients to communicate with servers through request methods such as GET and POST. In this technical architecture, the test environment sends query requests through HTTP, and the production data processing platform receives and processes these requests through HTTP.

[0044] Use Spring MVC's @RequestMapping annotation to configure the service interface to receive HTTP requests. The request can be a GET or POST request, depending on the type of data and the complexity of the request; parse the HTTP request header, query parameters, and data in the request body; when processing HTTP requests, you can use the HttpServletRequest object to obtain request details and extract query conditions; the service needs to support authentication and authorization (such as through JWT tokens or OAuth) to ensure that only authorized requests can query data; when the service responds, build an HTTP response object and return data in JSON or XML format based on the query results.

[0045] After the production data processing platform obtains the original data from the database, it must desensitize the sensitive information (such as personal identity information, financial data, etc.). The purpose of desensitization is to protect user privacy and prevent sensitive information leakage when the data is transferred to the test environment. Desensitization methods can include data masking (such as partial character replacement), encryption, or obfuscation.

[0046] Implement a desensitization module. You can select different desensitization methods (such as regular expression matching, data masking algorithm, etc.) through the strategy mode. This module is responsible for processing sensitive fields (such as ID number, bank card number). After obtaining the query results through ORM frameworks such as MyBatis or JPA, the service will process the sensitive fields in the data; the processed data is returned in JSON or XML format. The Spring framework supports automatic serialization of Java objects into JSON responses; the service returns the desensitized data as a response body to the test environment via HTTP, and usually uses the ResponseEntity object to set the HTTP status code, response header, and response body.

[0047] Example description: In financial institutions, data in the production environment usually involves sensitive data such as customers' personal information and transaction records. In order to protect data privacy, developers in the test environment cannot directly access this data. By deploying a data query service in the production data processing platform and receiving HTTP requests from the test environment, the desensitized data is securely transmitted to the test environment. For example, when the testing team of a financial institution develops a new risk control model, it can access desensitized historical transaction data in real time through this query service without worrying about the leakage of sensitive customer information. This not only improves the efficiency of development and testing, but also ensures data security.

[0048] By deploying data query services in the production data processing platform, we can efficiently process query requests from the test environment and automatically desensitize the query results, ensuring data privacy while improving the real-time and security of data acquisition. The request and response mechanism based on the Hypertext Transfer Protocol ensures system compatibility and data transmission stability, avoids complex manual operations, and simplifies the development and testing process.

[0049] S20, in the development test platform, applying a predefined tag annotation to the data access interface that requires a proxy;

[0050] In this embodiment, in the development and testing platform, some data access interfaces need to access the data query service in the production environment through a proxy. In order to implement automatic proxy, predefined marker annotations are used. This annotation is used to identify which interfaces need to call external services through a proxy instead of directly accessing the database. Marker annotations are a declarative way to mark data access interfaces or methods without the need for developers to manually modify the code logic. The annotation itself does not change the implementation logic of the interface, but provides information for the subsequent dynamic proxy mechanism so that the proxy can access production data.

[0051] Use a predefined annotation mechanism to mark the data access interface that needs to be proxied in the development test platform. Annotations are usually defined as metadata to identify methods or interfaces that require special processing. The actual implementation will rely on runtime processing, using frameworks (such as Spring) to automatically scan and identify interfaces or methods marked with this annotation. The main function of the annotation is to serve as an identifier, indicating that the method does not directly access the data source, but calls the data query service of the production environment through a proxy mechanism. In the specific implementation, the service dynamically scans these marking annotations at runtime, identifies the interface that needs to be proxied, and then redirects the call to the proxy module for processing. The proxy module is responsible for communicating with the services in the production environment, instead of the test environment directly accessing the data.

[0052] Predefined annotations are used to clearly indicate that the marked data access interface needs to access the data query service in the production data processing platform through a cross-domain proxy. The core function of a cross-domain proxy is to solve data access problems between different network domains. For example, the test environment and the production environment are usually located in different network areas, and direct access is limited. The proxy can act as an intermediate layer to securely forward requests in the test environment to the production environment. Through the annotation mechanism, developers can instruct specific data access operations to be completed through the proxy. The cross-domain proxy is not only a bridge for data transmission, but also increases security to ensure that sensitive data is protected and controlled when transmitted across the network. At the same time, this annotation helps automatically identify which interfaces require proxy access and which can be processed directly, making the development and testing process more flexible and automated.

[0053] When implemented, the system automatically scans all interfaces or methods marked with the annotation at startup and generates the corresponding proxy logic. These annotated methods will not communicate directly with the database, but forward the request to the service in the production environment through the proxy module. The proxy module uses the HTTP protocol to interact with the data query service in the production environment through a cross-domain mechanism. The cross-domain proxy can be implemented by configuring a reverse proxy server or by calling the back-end service layer (such as HTTPClient). After receiving the request from the development and testing platform, the proxy module packages the request data and sends it to the data query service in the production environment. The proxy module can also perform security operations such as identity authentication and request signing when necessary to ensure the security and reliability of data transmission.

[0054] For example, in the development and testing platform, testers only need to call the annotated data access interface, such as getUserTransactionData(), without having to understand the complex proxy mechanism behind it. The system will automatically identify the annotations and use the cross-domain proxy mechanism to securely forward the query request to the production environment. The data query service in the production environment returns the user transaction data after desensitization (such as hiding some account information or blurring the amount), and finally returns it to the tester.

[0055] By applying predefined tag annotations in the development and testing platform, data access operations are made more automated and flexible. Developers do not need to manually modify the code or configure complex proxy logic. They can implement the proxy mechanism through annotations, reducing the development workload. The introduction of cross-domain proxy ensures secure data access between the test environment and the production environment, improves testing efficiency, and ensures data privacy and security.

[0056] S30, during the service container startup process of the development and testing platform, registering a postprocessor, and configuring a dynamic proxy mechanism in the development and testing platform through the postprocessor;

[0057] In this embodiment, in the development and testing platform, when the service container starts, all components (or "Beans") in the system will be loaded and initialized. The purpose of registering the postprocessor is to add an additional processing step in this initialization process. The postprocessor is an extension mechanism that allows developers to further process the Bean after it is initialized. Through the postprocessor, the system can automatically scan all data access interfaces in the system when the service container starts, and apply a proxy mechanism to them without manual intervention or modification of the original code logic. The postprocessor is the basis for implementing dynamic proxies, which ensures that all interfaces marked with annotations can be proxied.

[0058] The post-processor is automatically registered when the service container starts and is triggered after each bean is loaded. During this process, it scans each data access interface and checks whether there is a marker annotation (such as @ProxySqlId) that needs to be accessed through a proxy. If an interface needs to be proxied, the post-processor will record the interface and configure the proxy logic for it in subsequent steps. Through this mechanism, the system can dynamically proxy the required interface without modifying the actual business code or manually configuring the proxy logic.

[0059] The dynamic proxy mechanism allows the system to dynamically generate proxy classes at runtime to intercept calls to data access interfaces marked with annotations. Through the automatic registration of post-processors, the system can configure dynamic proxies after Bean initialization to implement proxy processing of data access interfaces. The core function of dynamic proxy is to intercept requests and execute customized logic (such as forwarding requests to the production data processing platform) when the interface is called. In this way, when developers call certain data access interfaces in the test environment, the system will automatically intercept the call and send the request to the production environment through the dynamic proxy mechanism instead of directly calling the local service.

[0060] After the dynamic proxy mechanism is configured, any call to the annotated data access interface will be intercepted by the dynamic proxy. The proxy mechanism will automatically replace the original interface call logic and forward the call request to the data query service in the production data processing platform. The proxy mechanism can package the request data into a remote request format through remote call technology (such as HTTP) and send it to the data service in the production environment. Subsequently, the production environment will process the request and return the data, and the proxy mechanism is responsible for returning the data to the caller in the test environment.

[0061] By automatically registering the postprocessor during the service container startup process, it is possible to automatically scan and process data access interfaces marked with specific annotations, and dynamically configure proxy mechanisms for these interfaces. The introduction of this dynamic proxy mechanism avoids the complex operation of manually configuring proxy classes and realizes the automatic access of the test environment to the production environment data. The system automatically configures the proxy logic at startup, which simplifies the development and testing process and improves the flexibility and efficiency of the system.

[0062] S40, intercepting the call to the data access interface with the predefined tag annotation through the dynamic proxy mechanism;

[0063] In this embodiment, the core of the dynamic proxy mechanism is to dynamically generate a proxy object at runtime to replace the original data access interface implementation. When the system recognizes that there is a predefined mark annotation on the data access interface (for example, used to indicate that the interface needs to access production data through a proxy), the proxy mechanism will intercept all calls to the interface. Interception means that when the caller attempts to execute a method in the marked annotated interface, the call will not directly execute the logic of the original method, but will execute the alternative logic through the proxy mechanism. The task of the proxy mechanism is to first capture the method call, and then process the call according to the predefined logic, such as forwarding it to the production data query service.

[0064] The dynamic proxy mechanism is usually implemented in combination with Java's reflection mechanism or bytecode generation technology. When the system starts, the dynamic proxy scans and identifies the data access interface marked with predefined annotations. When the interface method marked with the annotation is called, the proxy mechanism intercepts the call and processes the logic in the proxy object. This mechanism can be implemented through AOP (aspect-oriented programming) or a dynamic proxy library (such as Java's Proxy class or a third-party CGLIB library). In the actual call, the proxy class generated by the dynamic proxy mechanism replaces the original interface implementation, and when the interface is called, the request is redirected to the proxy logic.

[0065] When a user calls an annotated data access interface, the dynamic proxy mechanism is responsible for intercepting the call. After the proxy class intercepts the call, it can choose to execute predefined business logic, such as forwarding the request to the production environment via HTTP, or performing some processing locally. After the proxy logic is executed, the result is returned to the caller, making the entire process transparent to developers.

[0066] By intercepting calls to data access interfaces marked with predefined annotations through a dynamic proxy mechanism, the system can automatically process interface requests and redirect them to the production environment or other services. This mechanism simplifies the operation of calling production data from the test environment, without the need to manually configure each interface or modify the original code, ensuring the flexibility and efficiency of the development process. At the same time, by intercepting calls, the proxy mechanism can add additional logic, such as security verification, logging, etc., improving the scalability and security of the system.

[0067] S50, constructing a request based on the hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service;

[0068] In this embodiment, when the system calls a data access interface marked with a predefined annotation, the caller usually passes a series of parameters, which may include query conditions, filter conditions, paging information, etc. The proxy mechanism captures the parameters passed during these calls and uses them for subsequent remote request construction. These parameters are an important basis for building requests and directly affect the response results of the data query service. Therefore, the system needs to obtain and retain these parameters during the proxy process and ensure that they are correctly encapsulated in the HTTP request.

[0069] When the dynamic proxy mechanism intercepts the call, it obtains the parameters passed during the call through the reflection mechanism, including the parameter list and specific values ​​of the method. The proxy mechanism captures these parameters and builds the query part or request body of the HTTP request based on their content. For example, when querying a user's transaction information, the parameters may include the user ID and the query time period. After capturing these parameters, the proxy mechanism converts them into query parameters of the HTTP request, or encapsulates them in JSON format as the request body of the POST request.

[0070] Based on the captured interface call parameters, the system needs to build a request based on the Hypertext Transfer Protocol (HTTP). HTTP is a protocol used for communication between clients and servers. When building an HTTP request, the system needs to encapsulate the parameters passed by the interface into the request URL (GET request) or the request body (POST request). In addition, the HTTP request may also need to be accompanied by authentication information, request header information (such as Content-Type), etc., to ensure that the request can be correctly processed by the data query service in the production environment.

[0071] When building an HTTP request, the system can decide to use a GET or POST request based on the parameters passed:

[0072] GET request: Add the passed parameters to the query string portion of the URL (e.g., / query?userId=123&startDate=2023-01-01).

[0073] POST request: Encapsulate the passed parameters as JSON or XML format in the request body (for example, {"userId":123,"startDate":"2023-01-01"}).

[0074] During the request construction process, the system also needs to set appropriate HTTP header information, such as Content-Type (indicating the data format) and Authorization (if authentication is required).

[0075] The constructed HTTP request needs to be sent to the data query service in the production environment. The data query service receives the request and executes the query according to the passed parameters, and finally returns the data to the caller. In this step, the proxy mechanism sends the encapsulated request to the server in the production environment through the HTTP Client (such as Java's HttpClient or RestTemplate) to complete the remote access to the production data.

[0076] When sending an HTTP request, the system uses a network transmission tool (such as HttpClient or RestTemplate in Java) to send the constructed request to the specified production data query service URL. Usually, the service address of the production environment is defined through configuration files or environment variables, and the proxy mechanism sends requests based on this address. The request is transmitted to the production environment via HTTP, and the server executes the query based on the parameters in the request, generates response data, and returns the query results to the caller via HTTP response.

[0077] By capturing the call parameters of the annotated data access interface, the proxy mechanism can automatically build HTTP-based requests and send them to the production data query service, greatly simplifying the data access process for developers in the test environment. The process of building and sending HTTP requests is automated, ensuring the integrity and security of the parameters and improving the efficiency of remote calls to production data. At the same time, this proxy mechanism reduces the burden on developers to handle the underlying communication details, making testing and development more efficient.

[0078] S60, the data query service performs a data query operation according to the request and obtains query result data;

[0079] In this embodiment, after receiving the HTTP request from the development and testing platform, the data query service performs a specific query operation according to the parameters carried in the request. The query operation usually involves accessing the database system of the production environment and performing data retrieval based on the incoming query conditions (such as user ID, time range, etc.). The key to this step is that the data query service needs to accurately parse the parameters in the HTTP request and convert them into SQL queries or other database query languages ​​to finally obtain the matching business data.

[0080] After receiving the HTTP request, the data query service will parse the request parameters, such as the query string in the URL or the JSON field in the request body. After parsing, the service will convert these parameters into SQL query statements or generate query operations through ORM (such as MyBatis or JPA). Based on the query conditions, the service will access the production database and execute the corresponding query. To improve efficiency, the query service may use a connection pool to manage database connections to avoid creating a new connection for each query. The service may also perform paging queries to ensure the response speed and efficiency of queries in the case of large data volumes.

[0081] For example, if the user ID and time range are passed in the request, the data query service will generate the following SQL query: SELECT * FROM transactions WHERE user_id = ? AND transaction_date BETWEEN ? AND ? .

[0082] After the query operation is completed, the service extracts the results from the database and prepares them for subsequent processing.

[0083] After the data query service executes the query operation, it encapsulates the business data obtained from the database into query results. These data may be multiple data records, a single record, or aggregated statistical information, depending on the type of query. The query result data may include sensitive fields, such as user personal information, transaction records, etc., and need to be further processed (such as desensitization) before being returned.

[0084] Obtaining query result data is the process of extracting the original results from the database and saving them in a structured form after the query operation is completed. The query results are usually returned in the form of a list or object for subsequent desensitization and processing.

[0085] The query result data can be obtained in the following ways: directly use SQL to query the database and encapsulate the results as Java objects or lists; or use an ORM framework (such as MyBatis or JPA) to simplify database interaction. ORM will automatically map the query results to Java objects to facilitate subsequent operations.

[0086] The service stores the queried data in memory for subsequent processing (such as desensitization). Query results may need to be returned in pages, especially when processing large amounts of data. The data query service can perform preliminary data sorting when obtaining results, such as sorting and filtering the results, or formatting some fields.

[0087] By executing data query operations according to HTTP requests through the data query service, accurate data retrieval can be achieved. Through parameterized query methods, the system can dynamically generate query conditions and flexibly respond to various query requirements. After acquisition, the query result data can be used for desensitization processing or other business logic, realizing the automated data processing link from request to result. This not only improves the efficiency of queries, but also lays the foundation for subsequent data security and processing operations.

[0088] S70, the data query service performs desensitization processing on the sensitive data in the query result data to obtain desensitized result data;

[0089] In this embodiment, in a production environment, the query results may contain sensitive data fields (such as user's personal identity information, financial data, etc.). In order to prevent sensitive data leakage, the system needs to desensitize the sensitive fields in the results before the data query service returns the query results. Desensitization refers to covering, encrypting or replacing sensitive information through technical means, so that even if the data is obtained by the test environment or other users, the original sensitive information cannot be inferred.

[0090] The target fields for desensitization may include user ID, bank account number, transaction amount, etc. These fields need to remove or cover sensitive parts without affecting business functions to ensure that the data used in the test environment is safe.

[0091] Desensitization can be done using a variety of techniques, including:

[0092] Data masking: mask or replace some sensitive data, such as only showing the first six digits and the last four digits of a bank card number and hiding the middle part (e.g. 123456******7890).

[0093] Encryption processing: Sensitive information is encrypted using an encryption algorithm so that even if the data is transmitted or stored, unauthorized persons cannot decrypt the original information.

[0094] Fuzzification: Fuzzify specific values, for example, display the actual transaction amount as a range (such as "200-300 yuan").

[0095] Random replacement: Use randomly generated virtual data to replace the original sensitive data, such as randomly generated user IDs or addresses, to ensure that the data structure is intact but the data content is false.

[0096] After obtaining the query results, the data query service will automatically perform desensitization operations based on the type of sensitive data fields and predefined desensitization rules. The desensitized results are used for subsequent data transmission to ensure that the data returned to the test environment is protected and does not contain any actual sensitive information.

[0097] After the desensitization process is completed, the original sensitive data in the query results is replaced or hidden, forming the desensitized result data. These desensitized result data have the same structure and format as the original data, but the sensitive information has been processed into an unrecognizable form. The desensitized result data ensures data security, which can meet the test requirements without leaking sensitive information. The desensitized result data will eventually be returned to the caller (such as the system in the test environment) to ensure that secure data is used in the development and testing process and prevent sensitive data from being leaked in non-production environments.

[0098] The system can use a predefined rule base to decide which fields to desensitize and what desensitization method to use. For example, a masking strategy is used for ID card numbers and bank card numbers, while fuzzification is used for transaction amounts. Desensitization can be dynamically selected based on different business scenarios. For example, in a test environment, the requirements for data privacy are higher, so a stronger desensitization strategy can be implemented before returning.

[0099] The masked result data can maintain the structure and format of the original data. For example, if the query result is a JSON object, the returned JSON object after masking is still in the same format, except that the sensitive fields have been masked. The returned data can be directly used for business processing in the test environment.

[0100] By desensitizing the sensitive fields in the query result data, it is ensured that sensitive information will not be leaked when the production environment data is transferred to the test environment. Desensitization can dynamically adapt to different types of data and use appropriate strategies to process them securely, while ensuring that the returned data format and structure are consistent with the original data, so that the test process can proceed smoothly without destroying the business logic. The result data after desensitization meets the needs of development and testing while ensuring security, reducing the risk of data leakage.

[0101] S80, returning the desensitization result data to the development and testing platform through a response based on the hypertext transfer protocol;

[0102] In the present embodiment, desensitization result data is the query result data after processing, comprising the non-sensitive information in all original data and the sensitive information after desensitization processing. The core task of this step is to return these desensitized data to the development and test platform in a safe and standardized manner. In network communication, HyperText Transfer Protocol (HTTP, HyperText Transfer Protocol) is usually used to transmit data. This protocol is widely used in Web service communication, and supports data in various formats such as text, JSON, XML, etc. transmitted from the server to the client. Desensitization result data is returned to the development and test platform by HTTP response, ensuring that data is safely and reliably transmitted. In this process, the server needs to build an HTTP response object, using the desensitization result data as the main content of the response, and setting the response header information according to the protocol standard to ensure that data can be correctly transmitted and parsed.

[0103] After executing the data query and desensitization, the data query service encapsulates the result data into an HTTP response. The specific steps include:

[0104] Response formatting: Encapsulate the masked result data into a structured data format, usually JSON or XML. This formatting operation ensures that the data can be correctly parsed and used in the client (development and testing platform).

[0105] Set HTTP response header: To ensure the format and transmission of data, the system needs to set the correct header information for the HTTP response. For example, Content-Type is set to application / json (if JSON data is returned) to tell the client how to parse the response content.

[0106] Return response data: The desensitized result data is returned to the development and testing platform through network transmission as part of the HTTP response body. The server can also use a secure protocol (such as HTTPS) to encrypt data during transmission to prevent data from being intercepted or tampered with during transmission.

[0107] For example, the desensitized user transaction data will be returned in JSON format, where the user's sensitive information (such as transaction amount, user ID) has been desensitized. After the development and testing platform receives this data, it can be directly used for business logic processing.

[0108] HTTP response is the basic way for Web services to communicate with clients. When building a response, the system needs to ensure that the response content is structured and formatted correctly. The response header is used to pass meta information such as the response format and status code, and the response body contains the actual data content. In this scenario, the response body is the query result data after desensitization processing. The development and testing platform obtains data by parsing the response body and performs the next step of processing.

[0109] Set the appropriate response header according to the type of data returned. For example, when returning JSON format data, the header should be set to Content-Type:application / json so that the development and test platform can recognize and parse the data. The desensitized result data will be set as the HTTP response body. Java objects can be converted to JSON string format using the JSON serialization tool for easy transmission. Similarly, if the XML format is used, the data needs to be converted to an XML string. The system transmits the response content to the development and test platform via the HTTP protocol. In order to ensure the integrity and security of data transmission, the HTTPS protocol can be used for encrypted transmission.

[0110] Through the response mechanism based on Hypertext Transfer Protocol (HTTP), the desensitized result data in the production data query service can be safely and effectively returned to the development and testing platform. HTTP response standardizes the data transmission process to ensure the standardization of data structure and the security of transmission. After desensitization, the development and testing platform can obtain safe and protected data, avoiding the risk of sensitive information leakage while ensuring the compatibility and scalability of the system. This data transmission mechanism simplifies the development and testing process and improves data security.

[0111] S90, developing a dynamic proxy mechanism in the test platform to receive the desensitized result data, and passing the desensitized result data to the caller of the data access interface for business logic processing.

[0112] In this embodiment, the task of the dynamic proxy mechanism is to intercept the call of the data access interface, forward the request to the production data processing platform, and receive the returned desensitized result data after the query is completed. The desensitized result data is the processed security data returned by the production environment data query service, and the proxy mechanism needs to receive this data from the HTTP response. The dynamic proxy mechanism is not only responsible for forwarding requests, but also for passing the data returned by the production environment to the caller. Receiving the desensitized result data means that the proxy mechanism parses the desensitized query result data from the response of the data query service of the production environment. This process usually includes parsing the HTTP response body and extracting the desensitized result data (such as business data in JSON format).

[0113] The dynamic proxy mechanism sends a request through an HTTP Client (such as Java's HttpClient or RestTemplate) and waits for a response. After the response is returned, the proxy mechanism extracts the anonymized data in the HTTP response body. This data is usually in JSON or XML format, and the system can convert it into a Java object or other data structure suitable for processing through a serialization tool. The process includes: receiving the HTTP response; parsing the anonymized data (usually in JSON or XML format) from the response body; storing the parsed data as an internal data structure for passing to the caller.

[0114] After receiving the masked data, the dynamic proxy mechanism needs to pass the data to the caller, that is, the business code in the development and testing platform that initiates the data access request. The key to this step is to pass the processed masked data back to the calling interface, so that the business logic can transparently process the returned data without having to worry about how the data is obtained from the production environment through the proxy.

[0115] The caller may be a service or module in the application, which obtains the required data by calling the data access interface. The proxy mechanism passes the desensitized data to the caller to ensure that the business logic can use the data for further processing, such as analysis, display or storage.

[0116] After intercepting the call, the dynamic proxy mechanism will handle data transmission and response. When the desensitized result data is received, the proxy mechanism passes the data to the caller of the data access interface as the return value of the method. This transmission process is usually automated. The caller only needs to call the interface like a normal local method. The proxy mechanism will handle the remote call and data transmission in the background, making the entire process transparent to the caller. The caller will eventually obtain desensitized data, which fully meets business needs and does not require additional processing of data privacy issues.

[0117] By receiving the desensitized result data through a dynamic proxy mechanism and passing it to the caller for business logic processing, the development and testing platform achieves secure access to production data without exposing sensitive information. This mechanism simplifies the development process, the caller does not need to understand the complexity of underlying data acquisition, and the entire process is transparent to developers, improving the efficiency of development and testing. In addition, desensitization processing and secure transmission ensure the privacy and security of production data and reduce the potential risk of data leakage.

[0118] The present invention relates to the fields of data security technology and financial technology, and discloses a data desensitization processing and access method, which comprises deploying a data query service in a production data processing platform, receiving a request and returning desensitized data; applying predefined tag annotations in a development and testing platform, configuring a dynamic proxy mechanism; intercepting a data access interface call, building and sending a hypertext transfer protocol request; and executing a query and returning desensitized data after the production data processing platform receives the request, and a test environment receives and processes the desensitized data. The present invention simplifies the process of a test environment accessing production environment data through a dynamic proxy mechanism, realizes automated data calling and desensitization processing, and improves the real-time and efficiency of data acquisition. Data privacy is guaranteed, the needs of rapid iterative development are met, and the work efficiency of development and testing is significantly improved.

[0119] In one embodiment, the above S50 includes:

[0120] S501, generating an authentication token in the development test platform;

[0121] S502, constructing a request conforming to the hypertext transfer protocol format based on the parameters passed when calling the data access interface with the predefined tag annotation;

[0122] S503, during the request construction process, adding the identity authentication token to the header information of the request;

[0123] S504, sending the constructed request to the data query service;

[0124] S505, the data query service extracts and verifies the identity authentication token from the request;

[0125] S506: When the identity authentication token is verified, the data query service performs a data query operation.

[0126] In this embodiment, the authentication token is used to ensure the legitimacy of the identity of the request initiator and prevent unauthorized requests from accessing the production data query service. In the development and testing platform, the system generates a unique authentication token for each request as a credential for the request. This token is usually generated based on a security algorithm and contains information such as a signature and expiration date to ensure the legitimacy and validity of the request. The authentication token can be generated through OAuth2.0, JWT (JSON Web Token) or other encryption algorithms. The token is generated before the request is initiated and is securely stored in the header information of the request and sent to the production data query service along with the request. After the token is generated, it is valid for a limited time, and the system will encrypt the token to prevent tampering.

[0127] The dynamic proxy mechanism captures and parses the passed parameters when intercepting the data access interface call. These parameters determine the conditions for data query, such as user ID, query time period, etc. The system needs to build a request that complies with the Hypertext Transfer Protocol (HTTP) based on these parameters. The request can be a GET request (parameters are in the URL) or a POST request (parameters are in the request body), depending on business needs and data volume. When the request is constructed, the system first determines the type of passed parameters and embeds these parameters into the URL or request body of the request in accordance with the HTTP specification. For GET requests, the parameters are attached to the query string part of the URL; for POST requests, the parameters are encapsulated as a data body in JSON or XML format. Regardless of the request method, the system needs to ensure that the parameters are correctly parsed and transmitted securely.

[0128] The authentication token must be passed through the HTTP header information of the request to ensure that the production data query service can identify and verify the legitimacy of the request. When building a request, the system needs to add the generated authentication token (such as JWT or OAuth token) to the HTTP header, usually in the Authorization field, so that the production environment can verify the token when the request arrives. When building a request, the system sets the Authorization field in the HTTP header with a value in the "Bearer{token}" format (that is, the authentication token with a prefix). This token ensures that the request can be identified and verified when it is sent to the production data query service. The token is generated by an encryption algorithm, is tamper-proof and time-valid, and ensures the security of data requests.

[0129] The constructed HTTP request needs to be sent to the production data query service through network transmission. The system uses the HTTP client tool to send the request to the specified production environment URL. This step ensures that the data access request can reach the production data processing platform, and the server can perform corresponding operations based on the parameters in the request. Use HTTP Client (such as HttpClient or RestTemplate in Java) to send the constructed request. The request contains all necessary parameters, header information (including authentication tokens), and is transmitted through the network to the specified interface address of the production data query service. This process is usually performed through HTTP POST or GET, depending on specific business needs.

[0130] After receiving the request, the production data query service first needs to extract and verify the authentication token to ensure that the request is sent by a legitimate initiator. The verification process usually involves steps such as decrypting the token, checking the signature, and verifying the validity period of the token. If the token verification fails, the service will reject the request to prevent unauthorized access. The production data query service extracts the authentication token by parsing the Authorization field in the HTTP header. When verifying the token, the system uses a predefined key or algorithm for decryption and signature verification, and checks the validity period of the token. If the verification passes, the system allows the request to continue processing; if the verification fails, the system returns a corresponding error response.

[0131] The data query service will only perform specific database query operations after the authentication token is verified. At this point, the service will perform the corresponding data query based on the parameters carried in the request to obtain business data in the production environment. This step ensures that data access operations are performed only after verification, ensuring the security of production data. When the authentication is passed, the data query service will perform database query operations based on the query conditions in the request. This step usually involves SQL queries or accessing the database through an ORM framework (such as MyBatis or JPA). The query results will be used for subsequent desensitization processing and then returned to the development and testing platform.

[0132] By adding an authentication token to the request, this embodiment ensures the security and legitimacy of the request and prevents unauthorized requests from accessing production environment data. The process of generating and verifying authentication tokens is automated, which increases the security of data transmission and ensures that only legitimate requests can trigger data query operations. This mechanism protects sensitive data, enhances the security of the system, and provides a flexible cross-environment data access solution.

[0133] In one embodiment, the above S30 includes:

[0134] S301, during the service container startup process of the development test platform, registering a postprocessor;

[0135] S302, the post-processor scans all data access interfaces in the development test platform and identifies data access interfaces marked with predefined tag annotations;

[0136] S303, when a data access interface marked with a predefined tag annotation is identified, a dynamic proxy mechanism is configured through the post-processor, and the dynamic proxy mechanism is used to generate a corresponding proxy class to intercept the call of the data access interface when the data access interface marked with the predefined tag annotation is called.

[0137] In the present embodiment, when the service container of the development and testing platform is started, the system will load and initialize all service components. The postprocessor (BeanPostProcessor) is a special component in the service container, which allows the addition of custom processing logic after all Beans (service components) are registered and initialized. The purpose of registering the postprocessor is to expand the startup process of the service container and add a dynamic proxy mechanism to the data access interface in the development and testing platform. During the startup of the development and testing platform, the system will automatically load and register the custom postprocessor. The postprocessor is usually implemented through the BeanPostProcessor interface in the Spring framework, allowing the system to execute additional logic after the Bean is initialized. Through the postprocessor, the system can scan all data access interfaces and perform further processing according to specific tag annotations.

[0138] After the post-processor is registered and activated, its task is to scan all data access interfaces in the development and testing platform. The goal of the scan is to identify data access interfaces that are marked with specific predefined annotations. These annotations (such as @ProxySqlId) are used to indicate which interfaces need to be proxied to call the data query service of the production environment. The post-processor will automatically recognize these annotations and mark these interfaces as interfaces that need to be proxied. The post-processor traverses all registered beans in the development and testing platform through reflection mechanisms or other scanning tools to check whether their methods or interfaces have specific annotation marks. The marked data access interfaces will be recorded for dynamic proxy configuration in subsequent steps. The process is transparent to developers, and the system automatically completes identification and processing in the background.

[0139] When the post-processor identifies that a data access interface is marked with a predefined annotation, it triggers the configuration of the dynamic proxy mechanism. The role of the dynamic proxy is to generate proxy classes for these interfaces at runtime. The main task of the proxy class is to intercept calls to the data access interface and process remote calls, data transmission and other operations in the background. Through dynamic proxy, the system can automatically forward the call request of the data access interface to the production data query service without modifying the business code. When the post-processor identifies the interface that needs to be proxied, it will dynamically generate proxy classes and replace the implementation of the original interface with these proxy classes. Dynamic proxy can be implemented using Java's Proxy class or bytecode enhancement libraries such as CGLIB. The proxy class is responsible for executing additional logic when the interface is called, such as capturing method parameters, generating HTTP requests, etc. Finally, the proxy class will forward the data query request to the data query service in the production environment.

[0140] The core function of the dynamic proxy mechanism is to intercept these calls and perform proxy operations when calling data access interfaces marked with specific annotations. The generated proxy class captures calls to the interface and executes according to the predefined proxy logic instead of directly calling the implementation of the original interface. The proxy class can build an HTTP request based on the call parameters, send the request to the production data query service, and process the returned response data. The dynamic proxy class is dynamically generated at runtime based on the marked annotations. The proxy class replaces the implementation of the original data access interface. When the method is called, the proxy class intercepts the call and executes custom logic, such as building an HTTP request or other remote call operations. The intercepted call result will be returned to the caller through the proxy class. The whole process is transparent to the caller, that is, the caller does not need to care about the underlying proxy implementation.

[0141] This embodiment realizes automatic proxy configuration and call interception by registering a postprocessor when the service container of the development test platform is started, and configuring a dynamic proxy mechanism through the postprocessor, thereby simplifying the operation of the test environment to access production data. The postprocessor can automatically scan and identify the data access interface that needs to be proxied, and the dynamic proxy mechanism ensures transparent processing of data requests. The entire process is automated, reducing the workload of developers and improving the security and efficiency of data access.

[0142] In one embodiment, the above S40 includes:

[0143] S401, generating a proxy class for a data access interface marked with a predefined tag annotation through a dynamic proxy mechanism;

[0144] S402, registering the proxy class into the service container of the development and testing platform to replace the implementation of the corresponding data access interface;

[0145] S403: In the development test platform, the proxy class intercepts the call to the data access interface marked with the predefined marker annotation.

[0146] In this embodiment, the task of the dynamic proxy mechanism is to generate a proxy class for a data access interface marked with a specific predefined tag annotation at runtime. The proxy class is used to replace the actual implementation of the original data access interface and is responsible for intercepting all calls to the interface. The proxy class can capture the call information of the interface (such as the parameters passed) and execute customized logic, such as forwarding the interface call to a remote data query service. The dynamic proxy class is generated based on the Java reflection mechanism or bytecode enhancement technology. When the system is running, the proxy mechanism identifies the interface marked with a predefined annotation (such as @ProxySqlId) and dynamically generates a corresponding proxy class based on the interface. The responsibility of the proxy class is to intercept calls to interface methods, capture parameters, execute proxy logic, and return the call results to the caller. Commonly used technologies in Java include java.lang.reflect.Proxy and CGLIB.

[0147] After the proxy class is generated, the system needs to register the proxy class with the service container of the development and testing platform to replace the original data access interface implementation. In this way, when the business code in the development and testing platform calls these interfaces, the proxy class will actually be called instead of the original interface implementation. The proxy class performs all necessary proxy operations in the background, and the developer does not need to modify the business logic. When the proxy class is generated, the system will register the proxy class as a service component in the service container of the development and testing platform. The registration process can be implemented through the Bean registration mechanism in the Spring container. The proxy class will overwrite the original interface implementation, thereby implementing the proxy operation when called. The registered proxy class will replace the original interface to execute the customized logic, and the caller does not need to know the existence of the proxy.

[0148] Once the proxy class is registered and replaces the implementation of the original interface, it will intercept all calls to the data access interface marked with predefined annotations when called. The intercepted calls will not directly execute the original implementation logic of the interface, but will execute the proxy logic through the proxy class. The proxy class can perform operations such as remote calls and data queries based on the call parameters and return the results to the caller. The proxy class intercepts the call by implementing the same method of the original interface. When the caller calls the interface marked with predefined annotations, the proxy class automatically takes over the call and executes the customized proxy logic. The proxy class can capture the call parameters, execute the remote service call, process the response data, and pass the final result back to the caller. Developers do not need to modify the business code, and the proxy logic is executed transparently in the background.

[0149] This embodiment generates a proxy class through a dynamic proxy mechanism and registers it in the service container, which can transparently intercept calls to the annotated data access interface to achieve automated remote data access. The proxy class registration and call interception process is transparent to developers, which simplifies the proxy configuration and avoids the complexity of manually creating proxy classes. The proxy mechanism improves the efficiency of the development and testing platform's access to production environment data while ensuring the security and maintainability of the system.

[0150] In one embodiment, the above S50 includes:

[0151] S507, based on the proxy class corresponding to the data access interface with the predefined tag annotation, obtaining the calling parameters passed when the data access interface is called;

[0152] S508, the proxy class encapsulates the call parameters as part of the hypertext transfer protocol request;

[0153] S509, the proxy class constructs a complete remote service request based on the partial content encapsulating the call parameters;

[0154] S510: The proxy class sends the remote service request to the data query service.

[0155] In this embodiment, when the dynamic proxy class intercepts the call to the data access interface, it will capture all the parameters passed during the call. The call parameters are the core of the business request, usually including query conditions, user ID, time range, etc. After being captured by the proxy class, these parameters need to be used to build a remote service request so as to forward it to the production data query service. The proxy class obtains the passed parameters through the Java reflection mechanism. Through the interceptor or proxy mode, the proxy class can dynamically capture the call information of the interface method, including the actual parameters passed. The reflection mechanism enables the proxy class to traverse the parameter list and obtain the name and value of each parameter for subsequent use in the request.

[0156] After capturing the call parameters, the proxy class needs to encapsulate these parameters as part of the HTTP request. The formatting of the HTTP request involves converting the call parameters into a standardized request format. Specifically, the parameters of the GET request are usually encapsulated as the query string of the URL, and the parameters of the POST request are encapsulated as JSON or XML format in the request body.

[0157] The proxy class will choose to encapsulate the call parameters into different parts of the HTTP request based on the content of the call parameters:

[0158] For GET requests, the parameters are appended to the URL in a format similar to / query?userId=123&startDate=2023-01-01.

[0159] For POST requests, the parameters are encapsulated as a JSON object in the request body, for example: {"userId":123,"startDate":"2023-01-01"}.

[0160] Building a complete HTTP request is one of the key tasks of the proxy class. In this step, the proxy class not only encapsulates the passed call parameters as the content of the request, but also needs to add necessary HTTP header information (for example, Content-Type, Authorization, etc.) to ensure that the request can be correctly processed by the remote service. A complete request includes the URL, request method, request header, and request body. Using HTTP client tools (such as HttpClient or RestTemplate in Java), the proxy class builds a complete HTTP request. The specific steps include:

[0161] Set the request URL: determine the address of the remote service;

[0162] Add request header: add authentication token, content type and other information according to service requirements;

[0163] Set the request method: select a suitable request method such as GET or POST;

[0164] Set the request body: For POST requests, encapsulate the parameters as the request body content.

[0165] The fully constructed HTTP request needs to be sent to the data query service of the production environment through network transmission. The process of sending the request is the core part completed by the proxy class. The proxy class is responsible for forwarding the constructed remote service request to the specified production data service interface through network communication. After the request is sent, the proxy class waits for the response of the service and processes the returned data. The proxy class sends the constructed request to the URL of the production data query service through the HTTP client tool. HttpClient, RestTemplate and other tool libraries are usually used to implement the request sending operation. The proxy class can also handle the synchronous and asynchronous modes of the request, and ensure the integrity and security of the data during the request sending process (by using encryption protocols such as HTTPS).

[0166] In this embodiment, the proxy class captures the call parameters and encapsulates them into HTTP requests, and the system implements transparent cross-environment data access. The proxy class is responsible for automatically building and sending requests, and developers do not need to manually build complex remote requests, which greatly improves development efficiency. The encapsulation and transmission method of the request ensures the security of the data. The proxy class enables the test environment to safely and quickly access the data of the production environment, meeting the development and testing requirements.

[0167] In one embodiment, the above S70 includes:

[0168] S701, the data query service identifies sensitive data fields contained in the query result data based on preset field information;

[0169] S702, the data query service determines a corresponding desensitization method according to the type of the sensitive data field;

[0170] S703, the data query service performs a desensitization operation on the sensitive data field according to the desensitization method to generate a corresponding desensitized data field;

[0171] S704: Use the desensitized data field to replace the corresponding sensitive data field in the query result data to obtain desensitized result data.

[0172] In this embodiment, when processing the query results, the data query service first needs to identify which fields contain sensitive data. Sensitive data fields may include users' personal identity information, financial information, transaction records, etc. In order to automatically and accurately identify sensitive data fields, the system predefines a field information list, which lists which fields belong to sensitive data. In the data query service, the system maintains a field metadata table or configuration file to define which fields need to be desensitized. After obtaining the query results, the service matches the field name with the sensitive field information in the metadata table to identify the fields that need to be desensitized. For example, if the query results contain fields such as user_id and credit_card_number, these fields will be identified as sensitive data fields and marked as requiring desensitization.

[0173] Different types of sensitive data may require different desensitization methods. For example, the user's ID number may need to be partially masked, while the transaction amount may need to be blurred. Therefore, the data query service determines the specific desensitization method to be used based on the type of sensitive field to ensure that each type of sensitive data can be protected in the most appropriate way. In the preset field information, the system not only identifies which fields are sensitive fields, but also includes the type of each field (for example, string, number, date, etc.). The service selects the appropriate desensitization method based on the field type. Common desensitization methods include:

[0174] Partial masking: For example, for an ID card number or bank card number, only some characters are displayed, and the other characters are replaced by “*”.

[0175] Fuzzy processing: Fuzzy processing is performed on numerical data, such as changing a specific transaction amount into a range.

[0176] Random replacement: Use randomly generated data to replace the original sensitive data, ensuring the same structure but different content.

[0177] Through the previous steps, the system has determined the sensitive fields and their corresponding desensitization methods. Next, the data query service needs to perform desensitization operations on these fields. The desensitization operation will convert or replace the original data of the sensitive fields to generate the corresponding desensitized data. The service processes each sensitive field by calling different desensitization algorithms. For example:

[0178] For the ID number field, the system will use a masking algorithm to retain only the first two digits and the last two digits, and replace the remaining characters with "*"; for the transaction amount field, the system will convert the actual amount into a range, such as converting "1,000 yuan" to "500-1,500 yuan"; the desensitization result of each sensitive field will generate a new data field as the desensitization result.

[0179] After the desensitization process is completed, the data query service needs to replace the sensitive data fields in the original query results with the processed desensitized data fields. In this way, the fields that originally contained sensitive information are replaced by safe desensitized data, thus forming the final desensitized result data.

[0180] The data query service will traverse the query result data, find the sensitive fields that need to be replaced, and replace them with the corresponding desensitized data fields. The replaced result data will maintain the original data structure and format, but the sensitive information has been desensitized. These desensitized result data can be safely returned to the development and testing platform for business logic processing.

[0181] This embodiment effectively protects the privacy of data by selecting an appropriate desensitization method based on the field type and replacing the sensitive data field in the query result after processing. While ensuring that the data structure remains unchanged, sensitive data is properly desensitized, making the use of data in the testing and development process safer and avoiding the risk of data leakage. The desensitization operation is automated, which improves the security of the system and the efficiency of data processing.

[0182] In one embodiment, the above S80 includes:

[0183] S801, the production data processing platform encapsulates the desensitization result data into a structured format;

[0184] S802, the production data processing platform constructs a result response according to the hypertext transfer protocol, and configures the response header information of the result response;

[0185] S803, setting the desensitization result data encapsulated in a structured format as the response body of the result response;

[0186] S804, the production data processing platform sends a result response including the desensitized result data to the development and testing platform.

[0187] In the present embodiment, the result data after desensitization needs to be returned to the development and testing platform in a standard, easy-to-parse format. Structured formats (such as JSON, XML) ensure that the data can be effectively read and processed by the platform. By using a structured format, the production data processing platform can maintain the integrity and consistency of the data. The production data processing platform encapsulates the desensitized data according to a predetermined structured format, usually selecting JSON or XML format. These formats support nested structures and multi-field data representation, and are suitable for cross-system transmission and parsing. This encapsulation process ensures that the data content can be completely packaged into a unified format during transmission.

[0188] The Hypertext Transfer Protocol (HTTP) is used to transfer data between different platforms. When building an HTTP response, the platform must not only prepare the response content, but also set the response header to specify information such as the content type and data length. This information in the response header helps the receiver (i.e., the development and testing platform) to correctly parse the response content. In the HTTP response, set the Content-Type (such as application / json or application / xml) to specify the format of the response data. Configuring the response header information (such as length, status code, etc.) ensures that the receiving platform can accurately interpret and use the data. When configuring the response header, you may also need to include additional security authentication information or other metadata.

[0189] The main body of the HTTP response (response body) carries the actual data content. In this step, the production data processing platform sets the structured masked data as part of the HTTP response body. The response body is the main carrier of transmitted data and contains the final processed data content. The masked data will be placed in the main body of the HTTP response. This data can be a JSON object, XML document, or other standardized format. The production data processing platform will ensure that the data content is complete and correct, and organize it in the agreed format for easy parsing and use by the development test platform.

[0190] The production data processing platform sends the complete response to the development and test platform via the HTTP protocol. This process ensures that the data can be transmitted from the production environment to the test environment over the network. During the transmission process, the data maintains its structured format and content. Use the HTTP or HTTPS protocol to send the constructed response to the designated receiving interface of the development and test platform. This step involves network transmission, and the production data processing platform needs to ensure the security and integrity of data transmission. Encryption protocols (such as HTTPS) can be used to protect the privacy of data during transmission.

[0191] This embodiment can ensure the security and standardization of data transmission between the production environment and the test environment by encapsulating the desensitization result data into a structured format and transmitting it using the HTTP protocol. The structured response format helps the development test platform to accurately parse the data, and the response header information ensures the integrity and compatibility of data transmission. In this way, the system can effectively protect sensitive data and ensure the security of data transmission.

[0192] In one embodiment, a data desensitization processing and access device is provided, and the data desensitization processing and access device corresponds one-to-one with the data desensitization processing and access method in the above embodiment. Figure 3 , Figure 3 This is a functional module diagram of a preferred embodiment of the data desensitization processing and access device of the present invention. Data query service module 10, data access interface module 20, post-processor module 30, dynamic proxy module 40, request construction module 50, data query module 60, data desensitization module 70, response generation and transmission module 80 and result processing module 90. The functional modules are described in detail as follows:

[0193] The data query service module 10 is used to deploy a data query service in the production data processing platform, wherein the data query service is used to receive a request based on the hypertext transfer protocol and return a query result after desensitization processing;

[0194] The data access interface module 20 is used to apply predefined tag annotations to the data access interface that needs to be proxied in the development test platform;

[0195] The post-processor module 30 is used to register the post-processor during the service container startup process of the development and testing platform, and configure the dynamic proxy mechanism in the development and testing platform through the post-processor;

[0196] A dynamic proxy module 40, used to intercept calls to data access interfaces with predefined tag annotations through the dynamic proxy mechanism;

[0197] A request building module 50, for building a request based on the hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service;

[0198] A data query module 60, used for the data query service to perform a data query operation according to the request and obtain query result data;

[0199] A data desensitization module 70, used for the data query service to perform desensitization processing on sensitive data in the query result data to obtain desensitized result data;

[0200] A response generation and transmission module 80 is used to return the desensitization result data to the development and testing platform through a response based on the hypertext transfer protocol;

[0201] The result processing module 90 is used to develop a dynamic proxy mechanism in the test platform to receive the desensitized result data, and pass the desensitized result data to the caller of the data access interface for business logic processing.

[0202] In one embodiment, the request construction module 50 is specifically configured to:

[0203] Generate an authentication token in the development test platform;

[0204] Constructing a request conforming to the hypertext transfer protocol format based on the parameters passed when calling the data access interface with the predefined tag annotation;

[0205] During the request construction process, the authentication token is added to the header information of the request;

[0206] Sending the constructed request to the data query service;

[0207] The data query service extracts and verifies the authentication token from the request;

[0208] When the authentication token is verified, the data query service performs a data query operation.

[0209] In one embodiment, the post-processor module 30 is specifically configured to:

[0210] During the service container startup process of the development test platform, register the postprocessor;

[0211] The post-processor scans all data access interfaces in the development test platform and identifies data access interfaces marked with predefined tag annotations;

[0212] When a data access interface marked with a predefined tag annotation is identified, a dynamic proxy mechanism is configured through the postprocessor, and the dynamic proxy mechanism is used to generate a corresponding proxy class to intercept the call of the data access interface when the data access interface marked with the predefined tag annotation is called.

[0213] In one embodiment, the dynamic proxy module 40 is specifically used for:

[0214] Generate proxy classes for data access interfaces marked with predefined annotations through dynamic proxy mechanism;

[0215] Registering the proxy class in the service container of the development and testing platform to replace the implementation of the corresponding data access interface;

[0216] In the development and testing platform, the proxy class intercepts the call to the data access interface marked with the predefined marker annotation.

[0217] In one embodiment, the request construction module 50 is specifically configured to:

[0218] Based on the proxy class corresponding to the data access interface with the predefined tag annotation, obtaining the calling parameters passed when the data access interface is called;

[0219] The proxy class encapsulates the call parameters as part of the content in the Hypertext Transfer Protocol request;

[0220] The proxy class constructs a complete remote service request based on the partial content encapsulating the call parameters;

[0221] The proxy class sends the remote service request to the data query service.

[0222] In one embodiment, the data desensitization module 70 is specifically used to:

[0223] The data query service identifies the sensitive data fields contained in the query result data based on the preset field information;

[0224] The data query service determines a corresponding desensitization method according to the type of the sensitive data field;

[0225] The data query service performs a desensitization operation on the sensitive data field according to the desensitization method to generate a corresponding desensitized data field;

[0226] The desensitized data field is used to replace the corresponding sensitive data field in the query result data to obtain desensitized result data.

[0227] In one embodiment, the response generation and transmission module 80 is specifically configured to:

[0228] The production data processing platform encapsulates the desensitization result data into a structured format;

[0229] The production data processing platform constructs a result response according to the hypertext transfer protocol, and configures response header information of the result response;

[0230] Setting the desensitized result data encapsulated in a structured format as the response body of the result response;

[0231] The production data processing platform sends a result response including the desensitized result data to the development and testing platform.

[0232] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external user terminal through a network connection. When the computer program is executed by the processor, it realizes a function or step on the server side of a data desensitization processing and access method.

[0233] In one embodiment, a computer device is provided. The computer device may be a user terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the user side of a data desensitization processing and access method

[0234] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0235] Deploy a data query service in the production data processing platform, wherein the data query service is used to receive a request based on the hypertext transfer protocol and return a query result after desensitization processing;

[0236] In the development test platform, apply predefined marker annotations to the data access interfaces that need to be proxied;

[0237] During the service container startup process of the development and testing platform, a postprocessor is registered, and a dynamic proxy mechanism in the development and testing platform is configured through the postprocessor;

[0238] Intercepting calls to data access interfaces annotated with predefined tags through the dynamic proxy mechanism;

[0239] Building a request based on the hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service;

[0240] The data query service performs a data query operation according to the request and obtains query result data;

[0241] The data query service performs desensitization processing on the sensitive data in the query result data to obtain desensitized result data;

[0242] Returning the desensitized result data to the development and testing platform through a response based on the Hypertext Transfer Protocol;

[0243] The dynamic proxy mechanism in the development and testing platform receives the desensitized result data and transmits the desensitized result data to the caller of the data access interface for business logic processing.

[0244] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0245] Deploy a data query service in the production data processing platform, wherein the data query service is used to receive a request based on the hypertext transfer protocol and return a query result after desensitization processing;

[0246] In the development test platform, apply predefined marker annotations to the data access interfaces that need to be proxied;

[0247] During the service container startup process of the development and testing platform, a postprocessor is registered, and a dynamic proxy mechanism in the development and testing platform is configured through the postprocessor;

[0248] Intercepting calls to data access interfaces annotated with predefined tags through the dynamic proxy mechanism;

[0249] Building a request based on the hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service;

[0250] The data query service performs a data query operation according to the request and obtains query result data;

[0251] The data query service performs desensitization processing on the sensitive data in the query result data to obtain desensitized result data;

[0252] Returning the desensitized result data to the development and testing platform through a response based on the Hypertext Transfer Protocol;

[0253] The dynamic proxy mechanism in the development and testing platform receives the desensitized result data and transmits the desensitized result data to the caller of the data access interface for business logic processing.

[0254] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the user side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0255] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0256] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0257] It should be noted that if software tools or components other than those of the Company appear in the embodiments of the present application, they are only used for illustration and do not represent actual use. The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the above-mentioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A data desensitization processing and access method, characterized in that: The following steps are involved: Deploy a data query service in the production data processing platform, wherein the data query service is used to receive a request based on the hypertext transfer protocol and return a query result after desensitization processing; In the development test platform, apply predefined marker annotations to the data access interfaces that need to be proxied; During the service container startup process of the development and testing platform, a postprocessor is registered, and a dynamic proxy mechanism in the development and testing platform is configured through the postprocessor; Intercepting calls to data access interfaces annotated with predefined tags through the dynamic proxy mechanism; Building a request based on the hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service; The data query service performs a data query operation according to the request and obtains query result data; The data query service performs desensitization processing on the sensitive data in the query result data to obtain desensitized result data; Returning the desensitized result data to the development and testing platform through a response based on the Hypertext Transfer Protocol; The dynamic proxy mechanism in the development and testing platform receives the desensitized result data and transmits the desensitized result data to the caller of the data access interface for business logic processing.

2. The data desensitization processing and access method according to claim 1, characterized in that: Constructing a request based on a hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service, including: Generate an authentication token in the development test platform; Constructing a request conforming to the hypertext transfer protocol format based on the parameters passed when calling the data access interface with the predefined tag annotation; During the request construction process, the authentication token is added to the header information of the request; Sending the constructed request to the data query service; The data query service extracts and verifies the authentication token from the request; When the authentication token is verified, the data query service performs a data query operation.

3. The data desensitization processing and access method according to claim 1, characterized in that: During the service container startup process of the development and testing platform, a postprocessor is registered, and the dynamic proxy mechanism in the development and testing platform is configured through the postprocessor, including: During the service container startup process of the development test platform, register the postprocessor; The post-processor scans all data access interfaces in the development test platform and identifies data access interfaces marked with predefined tag annotations; When a data access interface marked with a predefined tag annotation is identified, a dynamic proxy mechanism is configured through the postprocessor, and the dynamic proxy mechanism is used to generate a corresponding proxy class to intercept the call of the data access interface when the data access interface marked with the predefined tag annotation is called.

4. The data desensitization processing and access method as claimed in claim 1, characterized in that: Through the dynamic proxy mechanism, the call to the data access interface with the predefined tag annotation is intercepted, including: Generate proxy classes for data access interfaces marked with predefined annotations through dynamic proxy mechanism; Registering the proxy class in the service container of the development and testing platform to replace the implementation of the corresponding data access interface; In the development and testing platform, the proxy class intercepts the call to the data access interface marked with the predefined marker annotation.

5. The data desensitization processing and access method according to claim 1, characterized in that: Constructing a request based on a hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service, including: Based on the proxy class corresponding to the data access interface with the predefined tag annotation, obtaining the calling parameters passed when the data access interface is called; The proxy class encapsulates the call parameters as part of the content in the Hypertext Transfer Protocol request; The proxy class constructs a complete remote service request based on the partial content encapsulating the call parameters; The proxy class sends the remote service request to the data query service.

6. The data desensitization processing and access method according to claim 1, characterized in that: The data query service performs desensitization processing on the sensitive data in the query result data to obtain desensitized result data, including: The data query service identifies the sensitive data fields contained in the query result data based on the preset field information; The data query service determines a corresponding desensitization method according to the type of the sensitive data field; The data query service performs a desensitization operation on the sensitive data field according to the desensitization method to generate a corresponding desensitized data field; The desensitized data field is used to replace the corresponding sensitive data field in the query result data to obtain desensitized result data.

7. The data desensitization processing and access method according to claim 1, characterized in that: The desensitized result data is returned to the development test platform through a response based on the hypertext transfer protocol, including: The production data processing platform encapsulates the desensitization result data into a structured format; The production data processing platform constructs a result response according to the hypertext transfer protocol, and configures response header information of the result response; Setting the desensitized result data encapsulated in a structured format as the response body of the result response; The production data processing platform sends a result response including the desensitized result data to the development and testing platform.

8. A data desensitization processing and access device, characterized in that: The data desensitization processing and access device comprises: A data query service module is used to deploy a data query service in a production data processing platform, wherein the data query service is used to receive a request based on the hypertext transfer protocol and return a query result after desensitization processing; The data access interface module is used to apply predefined tag annotations to the data access interfaces that need to be proxied in the development and testing platform; A post-processor module is used to register a post-processor during the startup of the service container of the development and testing platform, and configure the dynamic proxy mechanism in the development and testing platform through the post-processor; A dynamic proxy module, used to intercept calls to data access interfaces with predefined tag annotations through the dynamic proxy mechanism; A request building module, used for building a request based on the hypertext transfer protocol based on the parameters passed when calling the data access interface with the predefined tag annotation, and sending the request to the data query service; A data query module, used for the data query service to perform a data query operation according to the request and obtain query result data; A data desensitization module, used for the data query service to perform desensitization processing on sensitive data in the query result data to obtain desensitized result data; A response generation and transmission module, used to return the desensitization result data to the development and testing platform through a response based on the hypertext transfer protocol; The result processing module is used to develop a dynamic proxy mechanism in the test platform to receive the desensitized result data, and pass the desensitized result data to the caller of the data access interface for business logic processing.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a data desensitization processing and access program stored in the memory and executable on the processor. When the data desensitization processing and access program is executed by the processor, the steps of the data desensitization processing and access method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores a data desensitization processing and access program, which, when executed by a processor, implements the steps of the data desensitization processing and access method as described in any one of claims 1 to 7.

Images