A self-service process data processing method of field dimension

Abstract

The application provides a field dimension self-service process data processing method, comprising: in response to a process approval request, analyzing a business type, a field set and a channel identifier, generating a permission control matrix and storing it in a database; dynamically generating an SQL query statement with a field level filtering condition based on the permission matrix, and limiting the query result according to user institutions and channel parameters; performing a desensitization mapping function on sensitive fields, distinguishing between character type and numerical type for desensitization, and generating a one-time data access token with delayed effect and time control; directly exporting the desensitized data result to a cloud storage space without passing through the front end landing. Through the field level permission vector, the classification desensitization function and the dynamic token, the application realizes fine-grained, safe and self-service data processing, and is suitable for data service scenarios in the financial, insurance and other industries.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and in particular to a self-service workflow-based data processing method based on field dimensions. Background Technology

[0002] In existing technologies, data extraction in the traditional insurance and financial industries mainly relies on manual submission of work orders, technical personnel writing SQL scripts, and returning result files. However, existing solutions have the following problems: 1. Extraction efficiency is limited by manpower. The workload varies at different times, resulting in unstable efficiency. During peak periods, there is a serious backlog of demand, and data acquisition is often delayed by 1-2 days.

[0003] 2. Mixing anonymized data with original data carries the risk of not anonymizing the data or processing it improperly, which could easily lead to the leakage of sensitive information.

[0004] 3. The data sources are obtained in a inconsistent manner and the logic is not fixed. The query logic written by different technical personnel varies from person to person, making it difficult to guarantee the accuracy of the data results.

[0005] 4. Inaccurate descriptions of requirements and discrepancies between them and common language lead to repeated rollbacks of the same requirement, extending the delivery cycle.

[0006] Therefore, there is an urgent need for a self-service data processing method that can achieve automation, fine-grained processing, and secure control. Summary of the Invention

[0007] This invention proposes a self-service, process-oriented data processing method based on field dimensions to address the problems of low data acquisition efficiency, poor security, and coarse access control in existing technologies.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: a self-service workflow-based data processing method based on field dimensions, comprising the following steps: S1: Obtain the data requirements input by the first user and parse it to obtain the set of application fields. Business type tag b and organizational levels O, as the first intermediate result; S11: Receive natural language text and use a natural language processing model to extract business entities, time ranges, and indicator fields; S12: Based on the historical high-frequency field co-occurrence matrix, when the co-occurrence frequency of the extracted field and the unapplication field exceeds the threshold θ, the field is automatically recommended; S13: Merge the extracted fields with the recommended fields to generate the application field set F, which is then confirmed by the user. After confirmation, F, business type b, and organizational level org are passed to subsequent steps.

[0009] S2: Collect the historical behavior data of the first user, including the application frequency (freq) and the number of fields in this application. And the requirement complexity coefficient C, calculate the risk score: The risk level is determined based on R's percentile ranking among all users. L is used as the second intermediate result; S21: Collect the first user's historical behavior data, including the application frequency (freq) and the number of fields in this application. The requirement complexity coefficient C is calculated by comparing the requirement text corresponding to business type b with a preset sensitive word library. If the number of matched keywords is m, then C = 1.0 + 0.2 × min(m, 5), with a value range of [value missing]. ; S22: Calculate the risk score R: ; in, , , These are the preset weighting coefficients; freq is a natural number. Either the natural logarithm or the common logarithm can be used; n is the number of fields in this application; C is the requirement complexity coefficient.

[0010] S23: Calculate the percentile ranking p of the current user's R among all applicant users to determine the risk level: L ; The system dynamically adjusts the approval process hierarchy based on L: If L=high, the approval node will be automatically promoted to the level of the supervising general manager; If L=, then the approval node will be moved up one level; If L = low, then maintain the original approval level; S3 to S5 can only proceed after all approval nodes have passed.

[0011] S3: Based on the business type b and the organizational level org, retrieve the field visibility vector from the pre-configured permission table Π. ,in It also binds the user's identity identifier (uid) to generate an access control matrix; it dynamically generates a structured query statement (Q) based on P, and executes the query to obtain the result dataset (D); S31: Based on the business type b and the organizational level org, retrieve the field visibility vector from the pre-configured permission table Π. ,in 1 indicates that access to the i-th field is allowed; the permission table Π is pre-configured by the system administrator and loaded into the memory hash table at runtime; S32: bind the field visibility vector V with the user identity identifier uid, business type b, and organization level org to generate the permission control matrix. And write it to the permission mapping table in the database; T=SHA256(uid||nonce| ), where nonce is a 16-byte random number generated by the operating system's secure random number generator; The Unix timestamp of the approval completion time; S33: Dynamically generate structured query statements Q based on P, including: Extract the basic query template Qtemplate from the data dictionary based on the report topic; Use V to trim the SELECT clause: ; Dynamically add mechanism constraints org∈ in the WHERE clause The channel constraint is `channel=req.channel`. Here, `channel` is a recursive query function that queries the organization relationship table (containing fields: organization code, parent organization code) and returns the set of codes for all subordinate organizations under the current user's jurisdiction. Finally, executing Q yields the resulting dataset D.

[0012] S4: Select the corresponding desensitization strength parameter k according to the risk level L, and execute the adaptive desensitization mapping function on each sensitive field value x in the result dataset D. We obtain the anonymized dataset D′.

[0013] Furthermore, the k: Among them, when and When k=0, at this time; An adaptive desensitization mapping function is executed on each sensitive field value x in the result dataset D to obtain the desensitized dataset D′. The adaptive desensitization mapping function: If x is a character type: Specifically, for high-risk and low-risk levels, retain the first and last digits, and for medium-risk levels, retain the first and last digits, and replace all digits in the middle with '*'. If x is a numeric type: .

[0014] S5: Generate a one-time data access token The de-identified dataset D′ is then directly written to the user-exclusive storage space of the third platform via HTTPS protocol, and an authorized access link is generated. Return only the stated Serves as the access point for the de-identified dataset D′; in, It is a 16-byte random number, generated by the operating system's secure random number generator, ensuring that each generated token is unique and unpredictable. The Unix timestamp (in seconds) at the time the approval was completed.

[0015] The validity control of the token T satisfies: ; in, The Unix timestamp of the approval completion time; =600 seconds, =604800 seconds; only when When =true, the authorized link URL can be accessed; The anonymized dataset D′ is written directly to the user's dedicated storage space on a third-party platform via HTTPS. The data results are not cached by the front-end browser during the generation process, nor are they saved locally on the client. At the same time, a header is set in the returned HTTP response to prevent the browser from caching the content of the link.

[0016] Generate authorized access link: , where base is the preset base link address, token=" is the parameter name, and T is the token value.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: By using field-level permission vector V and de-identification function M(x), the problems of coarse traditional permission control and non-standard de-identification are solved, and fine-grained secure data processing is achieved. By using dynamic token T and delayed activation + time-limit control, the risk of authorized link abuse is resolved, and the security of data delivery is improved; By exporting data to a cloud drive that does not physically reside on the ground, the possibility of data leakage during intermediate processes is avoided. By implementing process rollback and automatic dispatch, the fault tolerance and automation level of the process have been improved. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the implementation of the present invention. Detailed Implementation

[0019] To provide a further understanding of the purpose, structure, features, and functions of the present invention, detailed descriptions are provided below with reference to specific embodiments.

[0020] A self-service, workflow-based data processing method based on field dimensions includes the following steps: S1: Obtain the data requirements input by the first user and parse it to obtain the set of application fields. Business type tag b and organizational levels O, as the first intermediate result; S11: Receive natural language text and use a natural language processing model to extract business entities, time ranges, and indicator fields. In this embodiment, the user inputs "Query the top ten medical insurance policies in Jiangsu Province last week in terms of claim amount". The model extracts: business entity = "medical insurance", time range = "last week" (mapped to the specific date range from 2025-04-14 to 2025-04-20), indicator field = "claim amount", filter condition = "Jiangsu Province", and sort condition = "top ten".

[0021] S12: Based on the historical high-frequency field co-occurrence matrix, when the co-occurrence frequency of the extracted field and the unapplied field exceeds the threshold θ, the field is automatically recommended; preferably, the threshold θ is 60%; in this embodiment, the proportion of users who apply for "claim amount" and "policy number" in the historical data is 85%, then "policy number" is automatically recommended.

[0022] S13: Merge the extracted fields with the recommended fields to generate the application field set F, which is then confirmed by the user. After confirmation, F, the business type b (e.g., "claims"), and the organizational level org (e.g., "Jiangsu Branch") are passed to subsequent steps.

[0023] This step reduces the average communication time for needs assessment from 2 hours to less than 2 minutes, lowering the barrier to entry.

[0024] S2: Collect the historical behavior data of the first user, including the application frequency (freq) and the number of fields in this application. And the requirement complexity coefficient C, calculate the risk score: The risk level is determined based on R's percentile ranking among all users. L is used as the second intermediate result; S21: Collect the first user's historical behavior data, including the application frequency (freq) and the number of fields in this application. The requirement complexity coefficient C is calculated by comparing the requirement text corresponding to business type b with a preset sensitive word library. If the number of matched keywords is m, then C = 1.0 + 0.2 × min(m, 5), with a value range of [value missing]. ; Furthermore, the application frequency freq refers to the number of applications made by the user in the past 30 days, which is obtained from system log statistics; S22: Calculate the risk score R: ; in, , , In this embodiment, the preset weighting coefficients are used. freq is a natural number. Either the natural logarithm or the common logarithm can be used; this embodiment uses the natural logarithm; n is the number of fields in this application; C is the requirement complexity coefficient.

[0025] S23: Calculate the percentile ranking p of the current user's R among all users who have applied for data (i.e., all users who have initiated data applications within a certain period in the past) to determine the risk level: L ; The system dynamically adjusts the approval process hierarchy based on L: If L=high, the approval node will be automatically promoted to the supervising level (i.e., the highest level). If L=, then the approval node will be promoted one level (e.g., from department head to general manager). If L=low, then the original approval level (such as department head) will be maintained. S3 to S5 can only proceed after all approval nodes have passed.

[0026] This step avoids numerical overflow through logarithmic transformation, and all parameters are automatically collected by the system, enabling objective quantitative risk assessment of user behavior and solving the problem that static permissions cannot detect changes in behavior.

[0027] S3: Based on the business type b and the organizational level org, retrieve the field visibility vector from the pre-configured permission table Π. ,in And bind it with the user's identity identifier (uid) to generate an access control matrix. A structured query statement Q is dynamically generated based on P, and the query is executed to obtain the result dataset D. S31: Based on the business type b and the organizational level org, retrieve the field visibility vector from the pre-configured permission table Π. ,in 1 indicates that access to the i-th field is allowed. The permission table Π is pre-configured by the system administrator and loaded into an in-memory hash table at runtime; S32: Bind the field visibility vector V to the user identifier uid, business type b, and organization level org to generate an access control matrix. And write it to the permission mapping table in the database; T=SHA256(uid||nonce| ), where nonce is a 16-byte random number generated by the operating system's secure random number generator; The Unix timestamp of the approval completion time; S33: Dynamically generate structured query statements Q based on P, including: Extract basic query templates from the data dictionary based on the report topic. ; Use V to trim the SELECT clause: ; Dynamically add mechanism constraints org∈ in the WHERE clause And the channel constraint `channel=req.channel`. Wherein, It is a recursive query function that returns a set of codes for all subordinate organizations under the current user's jurisdiction by querying the organization relationship table (which contains fields: organization code and parent organization code). Finally, executing Q yields the resulting dataset D.

[0028] This step implements dual access control at the column and row levels, ensuring that users can only query data within their authorized fields and the scope of their governing bodies. S4: Based on the risk level L, select the corresponding desensitization strength parameter k, and execute an adaptive desensitization mapping function on each sensitive field value x in the result dataset D. We obtain the anonymized dataset D′.

[0029] Furthermore, the k: Among them, when and When k=0, at this time ; An adaptive desensitization mapping function is executed on each sensitive field value x in the result dataset D to obtain the desensitized dataset D′. The adaptive desensitization mapping function: If x is a character type (such as name, ID number, policy number): Specifically, for high-risk and low-risk levels, retain the first and last digits, and for medium-risk levels, retain the first and last digits, and replace all digits in the middle with '*'. If x is a numerical value (such as claim amount, premium): .

[0030] By adaptively linking risk level and desensitization intensity, stronger desensitization is implemented for high-risk users (e.g., only the thousands digit and above of the amount are retained), while more details are retained for low-risk users (e.g., the hundreds digit of the amount is retained), achieving a dynamic balance between data availability and privacy protection.

[0031] S5: Generate a one-time data access token The de-identified dataset D′ is then directly written to the user-exclusive storage space of the third platform via HTTPS protocol, and an authorized access link is generated. Return only the stated Serves as the access point for the de-identified dataset D′; in, It is a 16-byte random number generated by the operating system's secure random number generator (such as Linux's getrandom), ensuring that each generated token is unique and unpredictable; The Unix timestamp (in seconds) at the time the approval was completed.

[0032] The validity control of the token T satisfies: ; in, The Unix timestamp of the approval completion time; =600 seconds, =604800 seconds; only when When =true, the authorized link URL can be accessed; The anonymized dataset D′ is written directly to the user's dedicated storage space on a third-party platform via HTTPS. The data results are not cached by the front-end browser during the generation process, nor are they saved locally on the client (the browser download dialog is not triggered, and the data is not written to the temporary directory). At the same time, a header is set in the returned HTTP response to prevent the browser from caching the content of the link.

[0033] Generate authorized access link: , where base is the preset base link address, token=" is the parameter name, and T is the token value.

[0034] This step prevents token abuse through delayed activation and time-limited control; and completely eliminates the risk of leakage in intermediate links by writing to cloud storage without the token being stored locally.

[0035] After S5 completes the data export, the system performs the following logging operations: records the user identifier (uid); records the export time (texport, a Unix timestamp); records the row count of the de-identified dataset D′; records the file storage path (path); and calculates the hash fingerprint. ,in This is a string representation (or its hash value) of the anonymized dataset. This fingerprint is used for subsequent auditing and leak tracing.

[0036] If a rejection instruction is detected at any intermediate node in the approval process (such as department head approval, data security approval, etc.), then: the current process status is redirected back to the initial application node; the rejection node identifier is recorded. Record the return time. The reason for the return (to be filled in by the approver) should also be included. All return records should be stored in the process log table to support subsequent auditing and process optimization.

[0037] The present invention has been described in the above-described embodiments; however, these embodiments are merely examples for implementing the present invention. It must be noted that the disclosed embodiments do not limit the scope of the present invention. Conversely, any modifications and refinements made without departing from the spirit and scope of the present invention are within the scope of patent protection of the present invention.

Claims

1. A self-service workflow-based data processing method based on field dimensions, characterized in that, include: S1: Obtain the data requirements input by the first user and parse it to obtain the set of application fields. Business type tag b and organizational levels O; S2: Collect the first user's historical behavior data, including the application frequency (freq) and the number of fields in this application. And the requirement complexity coefficient C, calculate the risk score: The risk level is determined based on R's percentile ranking among all users. ; S3: Based on the business type b and the organizational level org, retrieve the field visibility vector from the pre-configured permission table Π. ,in And bind it with the user's identity identifier (uid) to generate an access control matrix. ; A structured query statement Q is dynamically generated based on P, and the query is executed to obtain the result dataset D; S4: Select the corresponding desensitization strength parameter k according to the risk level L, and execute the adaptive desensitization mapping function on each sensitive field value x in the result dataset D. We obtain the desensitized dataset D′; S5: Generate a one-time data access token ,in The Unix timestamp of the approval completion time is used, with a nonce of a 16-byte random number. The anonymized dataset D′ is directly written to the user-specific storage space of a third-party platform via HTTPS protocol, and an authorized access link is generated, returning only the specified data. This serves as the access point for the de-identified dataset D′.

2. The self-service workflow-based data processing method based on field dimensions as described in claim 1, characterized in that, S1 includes the following sub-steps: S11: Receive natural language text and use a natural language processing model to extract business entities, time ranges, and indicator fields; S12: Based on the historical high-frequency field co-occurrence matrix, when the co-occurrence frequency of the extracted field and the unapplication field exceeds the threshold θ, the field is automatically recommended; S13: Merge the extracted fields with the recommended fields to generate the application field set F, which is then confirmed by the user.

3. The self-service workflow-based data processing method based on field dimensions as described in claim 1, characterized in that, The calculation method for the requirement complexity coefficient C is as follows: Compare the requirement text corresponding to business type b with the preset sensitive word library. If the number of matched keywords is m, then... The range of values ​​is .

4. The self-service workflow-based data processing method based on field dimensions as described in claim 1, characterized in that: The risk level L is determined as follows: Calculate the percentile ranking p of the current user's R among all registered users. Furthermore, the relationship between the desensitization intensity parameter k and L is as follows: .

5. The self-service workflow-based data processing method based on field dimensions as described in claim 1, characterized in that, The dynamic generation of the structured query statement Q in step S3 includes: S31: Extract basic query templates from the data dictionary based on the report topic. ; S32: Trimming the SELECT clause using the field visibility vector V: ; S33: Dynamically add mechanism constraints in the WHERE clause and channel constraints ,in The system recursively queries the organizational relationship table to return the set of codes for all subordinate organizations under the current user's jurisdiction.

6. The self-service workflow-based data processing method based on field dimensions as described in claim 1, characterized in that: The validity control of the token T satisfies: ； in, The Unix timestamp of the approval completion time; =600 seconds, =604800 seconds; only when The authorized link URL can only be accessed when `=true`.

7. The self-service workflow-based data processing method based on field dimensions as described in claim 1, characterized in that: In step S5, when writing to the user-exclusive storage space of the third platform, the data result is not cached by the front-end browser, and the HTTP response header Cache-Control="no-store,no-cache" is set.

8. The self-service workflow-based data processing method for field dimensions as described in claim 1, characterized in that: The approval level is dynamically adjusted according to the risk level L. High-risk users are automatically promoted to the general manager level for approval, medium-risk users are promoted one level for approval, and low-risk users maintain the original approval level. Steps S2 to S5 will only be executed after approval.

9. The self-service workflow-based data processing method for field dimensions as described in claim 1, characterized in that: Step S5 is followed by log recording: recording the user identifier (uid), export time (export), row count of the de-identified dataset D′, file storage path (path), and calculating the hash fingerprint. For auditing purposes.

10. The self-service workflow-based data processing method for field dimensions as described in claim 1, characterized in that: It also includes a process rollback mechanism: when a rollback instruction is detected at any intermediate node in the approval process, the process status is switched back to the initial application node, and the rollback node identifier is recorded. Return time The reason for the return should be recorded in the log table.

Images