A method of managing data access in a data management system

Abstract

The application provides a method for managing data access in a data management system, the system comprising a data user execution environment, the environment being a data analysis environment for authorizing a user to use data of a data owner for data analysis in a supervised manner, the method comprising: in the data user execution environment, determining a user-defined variable in a user-defined data analysis process file, and generating a corresponding fingerprint set for the variable, wherein the fingerprint set indicates the data of the data owner associated with the corresponding variable; in response to a request of the user for exporting a value of the corresponding user-defined variable, determining whether the variable whose value is applied to export can export the data user execution environment according to a preset variable export rule table for protecting data security and the fingerprint set corresponding to the variable; and thereby, limiting the export behavior of the user-defined variable to protect the data property of the data owner.

Description

Technical Field

[0001] This invention relates to the field of big data, specifically to access control of big data, and more specifically, to a method for managing data access in a data management system. Background Technology

[0002] As the value of big data becomes increasingly apparent, people are becoming more aware of data protection. In some scenarios, individuals or organizations will spend a significant amount of time, effort, and money collecting data. This collected data can be used to develop, train, or fine-tune predictive models to achieve predictive tasks in specific scenarios.

[0003] However, collected data, as an intangible asset, can be freely copied and distributed to any user without control, compromising the data owner's data security (also known as data asset security). Therefore, data owners typically provide a data analysis environment for data users. The primary purpose of this environment is to prevent data dissemination. In practice, this environment might be a host machine that does not provide external data storage (e.g., without floppy disks or USB drives), or a software system that does not support data storage.

[0004] Current data analytics environments designed to prevent data proliferation not only prevent users from exporting data, but also from exporting derived information obtained from data-driven calculations (such as the parameters and results of trained predictive models). Completely preventing data users from exporting any variables hinders their ability to deeply develop predictive models, making it difficult for certain tasks in various fields to be efficiently assisted by artificial intelligence. However, allowing data users to freely export variables could lead to the original data being deduced, causing data security issues and jeopardizing the rights of data owners. Therefore, it is necessary to improve existing technologies. Summary of the Invention

[0005] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a method for managing data access in a data management system.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] According to a first aspect of the present invention, a method for managing data access in a data management system is provided, the data management system including a data user execution environment, the data user execution environment being a data analysis environment that authorizes users to use data owner's data for data analysis in a regulated manner, the method comprising: in the data user execution environment, determining user-defined variables in a user-defined data analysis process file, generating a corresponding fingerprint set for the variables, wherein the fingerprint set indicates the data of the data owner associated with the corresponding variable; in response to a user's request to export the value of a corresponding user-defined variable, determining whether the value of the requested variable can be exported from the data user execution environment according to a preset variable export rule table for protecting data security and the fingerprint set corresponding to the variable, wherein the variable export rule table records historical data of one or more variables historically defined by the user, the fingerprint set corresponding to each variable, and the exportability flag of each variable.

[0008] Optionally, the determination of whether the value of the variable to be exported can be exported to the data user execution environment is made in the following way: based on the fingerprint set corresponding to the variable currently applied for export by the user and the fingerprint set corresponding to the variables already exported recorded in the historical data, it is determined whether the value of the variable currently applied for export can be exported. That is, the fingerprint set (or fingerprint features) of the variable currently applied for export determines whether the value of the variable currently applied for export can be exported. Here, the variable currently applied for export by the user is the variable corresponding to the value of the variable currently applied for export by the user.

[0009] Optionally, if the data indicated by the relevant fingerprint set can be deduced based on the value of the variable currently being exported or based on the combination of the value of the variable currently being exported and the values ​​of the variables already exported, it is determined that the value of the variable currently being exported cannot be exported.

[0010] Optionally, a variable cannot be exported if any of the following conditions exist: the fingerprint set corresponding to the variable being exported contains only one element; the fingerprint set corresponding to the variable being exported is the smallest parent set or the largest subset of the fingerprint sets corresponding to a single variable that has been exported in the historical data; or the fingerprint set corresponding to the variable being exported can be combined with the fingerprint sets corresponding to two or more variables that have been exported in the historical data through a union operation to form a smallest parent set or the largest subset.

[0011] Optionally, when determining user-defined variables in a user-defined data analysis process file, a unique variable internal code is assigned to each user-defined variable to identify its identity, and a corresponding fingerprint set is maintained for each variable internal code. In different fingerprint sets, a description method is set to identify the identity of the same data.

[0012] Optionally, the data management system further includes a data owner or a business manager authorized by the data owner, and the method further includes: synchronizing a summary of the variable export rule table determined based on a preset hash operation to the data owner or the business manager authorized by the data owner, so as to prevent abnormal users from using the data by checking the continuity of the summary.

[0013] Optionally, the exportable flag includes: pending, exportable, and non-exportable.

[0014] Optionally, if the fingerprint set of any variable is not empty, it means that the value of the variable is calculated based on the data in the fingerprint set corresponding to the variable.

[0015] Optionally, a user-defined data analysis process file is used to train and / or predict the model in the data user execution environment. The data analysis process file stores the model's parameter values, intermediate values ​​obtained based on some of the model's parameters, and the model's output values ​​in the form of variables.

[0016] According to a second aspect of the present invention, a data management system is provided, comprising: a data user data processing device for performing the method described in the first aspect; and a data owner data management device for operating a data owner data authorization service to provide data of the data owner to users for use based on authorization.

[0017] According to a third aspect of the present invention, a data management system is provided, comprising: a data user data processing device for performing the method described in the first aspect; a data owner data management device for running a data owner data authorization service to allow users to use the data of the data owner based on authorization; and a business manager management device for running a management business service, the management business service being used to manage the registration and login of users and data owners using the data, and to provide a platform for establishing information interaction between the users and the data owners.

[0018] Optionally, the management service is used to provide user registration for data users, provide data users with the network service entry address of the data owner; and provide data owners with service name registration and network service entry address registration; wherein, after the data user execution environment obtains the network service entry address of the data owner through the data user data processing device, the data user execution environment initiates a service request to the data owner data authorization service through the data user data processing device, and the two parties exchange information to determine whether to authorize the user to obtain and use the data owner's data based on the data user execution environment.

[0019] Optionally, the data user execution environment includes: an export rule storage area for storing variable export rule tables; wherein, when the data user execution environment is closed, the information in the export rule storage area is stored in a variable export rule file, and a summary of the variable export rule table is calculated based on the variable export rule file and synchronized to the data owner or a business manager authorized by the data owner; when the data user execution environment is started, a summary of the variable export rule table is calculated based on the local variable export rule file and sent to the data owner or a business manager authorized by the data owner to verify the continuity of the summary.

[0020] Optionally, if the digest sent when the data user execution environment is currently started is inconsistent with the digest synchronized when the previous shutdown was performed, the continuity of the digest is determined to be impaired and the abnormal user corresponding to the data user execution environment is prevented from using the data.

[0021] Optionally, the data owner's data is recorded in tabular form, and the access path for each piece of data is defined by the table name, the item name in the table, and the element number in the table, or a combination thereof, and users are authorized to access the required data through this definition.

[0022] Optionally, user-defined variables can access all data under the corresponding table name and project name in batches through the access path defined by the table name and project name, and indicate the data of the data owner associated with the variable in the fingerprint set using the access path defined by the table name and project name.

[0023] According to a fourth aspect of the present invention, an electronic device is provided, comprising: one or more processors; and a memory for storing executable instructions; wherein the one or more processors are configured to implement the steps of the method of the first aspect by executing the executable instructions. Attached Figure Description

[0024] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0025] Figure 1 This is a flowchart illustrating a method for limiting the export behavior of user-defined variables according to an embodiment of the present invention.

[0026] Figure 2 a is a schematic diagram of system entity relationships according to an embodiment of the present invention;

[0027] Figure 2 b is a schematic diagram illustrating how various entities in the system provide and obtain services according to an embodiment of the present invention;

[0028] Figure 3This is a schematic diagram illustrating the sequence of service provision and acquisition methods according to embodiments of the present invention;

[0029] Figure 4 This is a schematic diagram of the logical structure of the data user execution environment according to an embodiment of the present invention;

[0030] Figure 5 This is a schematic diagram of the original data storage area structure according to an embodiment of the present invention;

[0031] Figure 6 This is a schematic diagram of the storage structure of the currently defined variable area according to an embodiment of the present invention;

[0032] Figure 7 This is a schematic diagram of the storage structure of the export rule storage area according to an embodiment of the present invention;

[0033] Figure 8 This is a schematic diagram of the main control flowchart according to an embodiment of the present invention;

[0034] Figure 9 This is a schematic diagram of the initialization sub-process according to an embodiment of the present invention;

[0035] Figure 10 A schematic diagram of the variable derivation rule initialization sub-process according to an embodiment of the present invention;

[0036] Figure 11 This is a schematic diagram of the interactive process for obtaining data usage rights according to an embodiment of the present invention;

[0037] Figure 12 This is a schematic diagram of the data owner authorization management process according to an embodiment of the present invention;

[0038] Figure 13 This is a schematic diagram illustrating the data user request authorization process according to an embodiment of the present invention;

[0039] Figure 14 This is a schematic diagram of a file preprocessing sub-flow according to an embodiment of the present invention;

[0040] Figure 15 This is a schematic diagram of the variable definition preprocessing sub-process according to an embodiment of the present invention;

[0041] Figure 16 This is a schematic diagram of the assignment operation preprocessing sub-process according to an embodiment of the present invention;

[0042] Figure 17 This is a schematic diagram of a sub-process for generating an associated original data set according to an embodiment of the present invention;

[0043] Figure 18 This is a schematic diagram of the sub-process for establishing variable derivation rules according to an embodiment of the present invention;

[0044] Figure 19 This is a schematic diagram of a variable derivation sub-process according to an embodiment of the present invention;

[0045] Figure 20 This is a schematic diagram of the union processing sub-process according to an embodiment of the present invention;

[0046] Figure 21 This is a schematic diagram of a sub-process for calculating the difference between two sets according to an embodiment of the present invention;

[0047] Figure 22 This is a schematic diagram of a sub-process for reading external files according to an embodiment of the present invention;

[0048] Figure 23 This is a schematic diagram of the interactive process of summary updating according to an embodiment of the present invention;

[0049] Figure 24 This is a schematic diagram of the sub-process for deriving rules from stored variables according to an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0051] As mentioned in the background section, existing data analysis environments that prevent data diffusion not only refuse users from exporting data, but also refuse users from exporting derived information obtained from calculations based on the data (such as parameter values ​​and calculation results of trained prediction models). Regarding the problem that calculation results in derived information (which may be the final result of model prediction (predicted value) or intermediate result of the model (such as the value output by the nodes that make up the model, such as the value in the feature map output by the corresponding neuron of the neural network model) cannot be exported, the inventors believe that the main reason is that the data owner has difficulty in determining whether exporting the value will allow the user to obtain the original data. To address this, the present invention provides a data analysis environment, or data user execution environment, that authorizes users to use the data owner's data for data analysis in a regulated manner. Users can define data analysis process files to execute their required data analysis processes (such as training models and performing prediction tasks based on the trained models) using the data owner's data. However, within the data user execution environment, a corresponding fingerprint set is generated for user-defined variables in the data analysis process file. This fingerprint set indicates the data associated with the corresponding variable. When a user requests to export the value of a custom variable, the system determines whether the variable's value can be exported to the data user execution environment based on preset variable export rules for data security and the corresponding fingerprint set (or fingerprint features). Specifically, the fingerprint set of the variable currently being exported determines whether the value of the requested variable can be exported. The variable currently being exported by the user is the variable corresponding to the value of the variable currently being exported. Therefore, while enabling users to efficiently, conveniently, and reliably utilize the data owner's data for analysis, the data security of the data owner can be protected.

[0052] To facilitate understanding of the principles of this invention, the process of forming this invention will be explained first.

[0053] While researching access control for big data, the inventors discovered that data owners often struggle to provide an environment where users can freely analyze their data. Furthermore, to protect data, it's difficult to share the parameter-based models derived from data analysis with users. While this protects data security, it hinders users from developing in-depth models, impeding the use of existing data to drive technological advancement. The inventors found that this deficiency in existing technologies stems from the lack of a data ownership handling mechanism in traditional program development. Through research into data analysis environments and variable usage processes, the inventors discovered that this deficiency can be addressed by establishing a fingerprint set of original data for user-defined variables. Based on variable derivation rules, the fingerprint set is used to determine if there are conflicts between variables and the derivation rules, preventing conflicting variable values ​​from being simultaneously exported to the controlled environment. An illustrative process is as follows:

[0054] Multiple regression models, as one of the most commonly used data modeling methods, are integrated into various data analysis environments. Let the dependent variable be Y, and the k independent variables be X1, X2, ..., X... k A multiple regression model describes how the dependent variable Y depends on the independent variables X1, X2, ..., X... k The equation for the error term ε can be expressed in general form as follows:

[0055] Y = B0 + B1X1 + B2X2 + ... + B k X k +ε

[0056] Among them, B0, B1, B2, ..., B k These are the parameters of the model; ε is the error term.

[0057] When ε = 0, this expression can also be understood as a user-defined variable being represented by a combination of multiple raw data with coefficients through "accumulation". For example, Z1 = a0 + a1X1 + a2X2 + a3X3 + a4X4 + a5X5 + a6X6.

[0058] From the perspective of ownership, the parameters of the model (a0, a1, a2, a3, a4, a5, a6, etc.) constitute the model obtained by the data user and should belong to the data user. Meanwhile, Z1 is a user-defined variable. Allowing user-defined variables to be exported to the data analysis environment without restriction will result in a loss for the data owner. For example, when keeping the model parameters a0, a1, a2, a3, a4, a5, a6 unchanged, let Z2 = a0 + a1X1 + a2X2 + a3X3 + a4X4 + a5X5. If variables Z1, Z2, a1, a2, a3, a4, a5, a6, etc., are allowed to be exported to the data analysis environment, the value of the original data X6 can be obtained using the formula X6 = (Z1 - Z2) / a6. The values ​​of the original X1, X2, X3, X4, X5 can be obtained in the same way. Furthermore, when the calculation is expressed as Z3 = a1X1, if Z3 is allowed to derive the data analysis environment, the original X1 can be obtained based on Z3 and a1. Therefore, Z3 cannot derive the data analysis environment.

[0059] In addition to the expressions mentioned above, if we set Z1 = a0 + a1X1 + a2X2 + a3X3 + a4X4 + a5X5 + a6X6, Z4 = (a0 / 2) + a1X1 + a2X2 + a3X3, and Z5 = (a0 / 2) + a4X4 + a5X5, and if the data analysis environment allows variables such as Z1, Z4, Z5, a1, a2, a3, a4, a5, and a6 to be exported, the value of the original data X6 can be obtained using the formula X6 = (Z1 - Z4 - Z5) / a6. Similarly, the values ​​of the original data X1, X2, X3, X4, and X5 can be obtained.

[0060] Therefore, the datasets participating in the calculation of Z1, Z2, Z4, and Z5 can be represented using sets: U1 = {X1, X2, X3, X4, X5, X6}, U2 = {X1, X2, X3, X4, X5}, U4 = {X1, X2, X3}, and U5 = {X4, X5}. U1, U2, U4, and U5 are the variable fingerprint sets of Z1, Z2, Z4, and Z5, respectively, and are referred to as fingerprint sets.

[0061] U2, U4, and U5 are subsets of U1; the union of U4 and U5 is U2. This invention defines the subset that differs from its parent set by one element as the largest subset. Referring to the relationship between subsets and parent sets, this invention defines the parent set that differs from its subsets by one element as the smallest parent set. For example: U1 is the smallest parent set of U2; U1 is the smallest parent set of the union of U4 and U5; U2 is the largest subset of U1; the union of U4 and U5 is the largest subset of U1.

[0062] When the fingerprints of two variables are each other's largest subset and smallest parent set, such as Z1 and Z2, if both variables are allowed to be exported to the data analysis environment simultaneously, the original data can be calculated by subtracting Z1 and Z2 as described above. Furthermore, if Z3 is allowed to be exported to the data analysis environment, the original data will also be calculated using the same method. To prevent the leakage of original data, restrictions need to be placed on the custom variables that data users can export. When the fingerprint set corresponding to a variable is an empty set, the value of that variable can be exported to the data analysis environment. Conversely, a set with only one element is the smallest parent set of an empty set; therefore, the value of a variable corresponding to a fingerprint set containing only one element cannot be exported to the data analysis environment. For example, when Z3 = a1X1, Z3 cannot be exported to the data analysis environment.

[0063] Based on the principles of the above process, the method and system of this invention can be established.

[0064] According to one embodiment of the present invention, see Figure 1 This invention provides a method for managing data access in a data management system, or a method for limiting the export behavior of user-defined variable values. The data management system includes a data user execution environment, which is a data analysis environment that authorizes users to use the data owner's data for data analysis in a regulated manner. The method includes steps S1, S2, and S3. To better understand this invention, each step is described in detail below with reference to specific embodiments.

[0065] Step S1: Obtain the data user execution environment preset for the user. This is a data analysis environment in which the user is authorized to use the data owner's data for data analysis in a regulated manner, provided that the user cannot export the data owner's data.

[0066] To use data, it is necessary to ensure the security of the data owner's data assets. According to one embodiment of the present invention, the situation where the user cannot export the data owner's data (or the conditions under which the user cannot export the data owner's data) can be achieved through technical means. For example, the data user's execution environment is a terminal device that does not support exporting the data owner's data, or a functional software that does not support exporting the data owner's data. Illustratively, for a terminal device, it may be a device that does not provide external data storage (e.g., does not configure interfaces and corresponding functional drivers for external storage such as floppy disks or USB flash drives), identifies the data owner's data, and prohibits users from forwarding the data owner's data over the network. For functional software, it may be software that can be installed on any host but does not provide a copy option on the interface displaying data in the software system, stores data in an isolated secure area, and does not support user copying.

[0067] According to one embodiment of the present invention, the data user performs environment parsing of the data owner's data and displays the data to the user in a paginated manner. Alternatively, the data user performs environment parsing of the data owner's data and displays the data to the user in a paginated manner with partial data masking. The technical solution of this embodiment can achieve at least the following beneficial technical effects: In big data scenarios and neural network model training scenarios, the amount of data from the data owner is often large. In order to enable users to use the data owner's data efficiently, this embodiment provides a scenario where users can see the data owner's data. Thus, users can decide which specific local data they need to use to train the corresponding model based on the data provided by the data owner. However, due to the large amount of data, although users can see the data, the data can only be displayed in paginated form, and there is still considerable resistance to obtaining all the data and the original dataset.

[0068] To enable users to deeply develop the model, it is necessary to grant export permissions for the model. According to one embodiment of the present invention, the data user execution environment allows the export of parameter values ​​of the model trained on the data. Preferably, the method further includes: determining the parameter values ​​of the model in a data analysis workflow file within the data user execution environment; and, after executing the user-defined data analysis workflow file, authorizing the user to export the parameter values ​​of the trained model to the data user execution environment. Optionally, user-defined variables can store the model's parameter values.

[0069] Step S2: In the data user execution environment, determine the user-defined variables in the user-defined data analysis process file, and generate a corresponding fingerprint set for the variables, wherein the fingerprint set indicates the data of the data owner associated with the corresponding variable.

[0070] According to one embodiment of the present invention, in the data user execution environment, before (or during) executing a user-defined data analysis process file, user-defined variables in the data analysis process file are determined, and a corresponding fingerprint set is generated for each variable. This fingerprint set indicates the data associated with the corresponding variable. If the fingerprint set of any variable is not empty, it indicates that the value of the variable is calculated based on the data in the fingerprint set corresponding to that variable. In other words, for a user-defined variable, its fingerprint set indicates the data required to calculate the value of that variable in the data analysis process file, i.e., the data associated with the variable.

[0071] According to one embodiment of the present invention, a user-defined data analysis process file includes an analysis model. The user-defined data analysis process file is used to train and / or predict the model in the data user's execution environment. The data analysis process file stores model parameter values, intermediate values ​​obtained based on some parameters of the model, and the model's output value as variables. For example, the model can be based on a regression analysis model. For example, taking regression analysis model A as an example, any user-defined variable can store a parameter value of regression analysis model A, an intermediate value obtained based on some parameters of regression analysis model A, or the output value of regression analysis model A. If the fingerprint set of a parameter value of regression analysis model A is empty, it indicates that the parameter value is not associated with data from the data owner. If it is based on a neural network, the aforementioned intermediate value may be obtained by calculating the model's local parameters using the data in the fingerprint set corresponding to the variable (e.g., an intermediate result output by a neuron), and the aforementioned output value may be the model's output result (e.g., a predicted value).

[0072] To meet the needs of different implementation scenarios, according to one embodiment of the present invention, the value of a user-defined variable can be a single value (single calculation) or multiple values ​​in vector form (batch calculation), depending on the user's implementation needs.

[0073] Since users can perform multiple data analyses using multiple data analysis workflow files and reuse the same variable name to represent different variables in different data analysis workflow files, in order to more accurately track each variable and better ensure data security, according to an embodiment of the present invention, when determining user-defined variables in a user-defined data analysis workflow file, a unique variable internal code is assigned to each user-defined variable to identify the variable's identity, and a fingerprint set is maintained for each variable internal code. Among different fingerprint sets, a description method is provided to identify the identity of the same data (also known as the original data).

[0074] Step S3: In response to a user's request to export the value of a custom variable, based on a preset variable export rule table for data security and the fingerprint set corresponding to the variable, determine whether the value of the requested variable can be exported to the data user's execution environment. The variable export rule table records historical data of one or more variables defined by the user, the fingerprint set corresponding to each variable, and the exportability flags of each variable. This controls the export behavior of variable values.

[0075] According to one embodiment of the present invention, it is determined whether the value of the variable currently requested for export can be exported based on the fingerprint set corresponding to the variable currently requested for export and the fingerprint set corresponding to variables already exported recorded in historical data. It should be understood that the variable currently requested for export is the variable corresponding to the value of the variable currently requested for export; for example, assuming the user currently requests to export the value of variable Z3, then the fingerprint set corresponding to the variable currently requested for export is the fingerprint set of variable Z3. Variables determined to be exportable, whether individually or in combination, cannot deduce the original data indicated by the relevant fingerprint set. Conversely, if the data indicated by the relevant fingerprint set can be deduced based on the value of the currently requested variable or based on the combination of the value of the currently requested variable and the values ​​of already exported variables, it is determined that the value of the currently requested variable cannot be exported. Preferably, based on the fingerprint set corresponding to the variable currently requested for export by the user and the fingerprint set corresponding to the exported variables recorded in historical data, it is determined whether the value of the variable currently requested for export and the values ​​of the exported variables can be used to deduce data. Specifically, if the data can be deduced, it is determined that the value of the variable currently requested for export cannot be used to deduce the data user's execution environment; if the data cannot be deduced, it is determined that the value of the variable currently requested for export can be used to deduce the data user's execution environment. It should be noted that, for simplicity, "exported variable" may be used to refer to "value of exported variable" in some subsequent instances. The technical solution of this embodiment can achieve at least the following beneficial technical effects: the solution of this embodiment allows users to obtain data that is difficult to deduce from variables, thereby preventing some users from deducing data in batches by assigning variable values, thus protecting the data security of the data owner.

[0076] Data associated with variables may be deduced directly from the variable's own value, or it may be deduced by combining the values ​​of multiple variables. According to one embodiment of the present invention, the process of determining whether data can be deduced from the exported variable includes: determining whether data can be deduced from the exported variable based solely on the fingerprint set corresponding to the exported variable; and determining whether data can be deduced from both the exported variable and previously exported variables based on the fingerprint set corresponding to the exported variable and the fingerprint set corresponding to previously exported variables recorded in historical data. Preferably, the value of a variable cannot be exported if any of the following conditions exist: the fingerprint set corresponding to the variable being exported contains only one element; the fingerprint set corresponding to the variable being exported is the smallest parent set or the largest subset of the fingerprint sets corresponding to a single variable that has been exported in historical data; the fingerprint set corresponding to the variable being exported and the fingerprint sets corresponding to two or more variables that have been exported in historical data can be combined through a union operation to form a smallest parent set or a largest subset, wherein the parent set that differs from the subset by one element is called the smallest parent set of the subset, and the subset that differs from the parent set by one element is called the largest subset of the parent set.

[0077] It should be understood that implementers may define other ways to restrict the export behavior of variables based on the fingerprint set, and this invention does not impose any restrictions on this. For example, a variable that is currently applied for export cannot be exported if any of the following conditions exist: the number of elements in the fingerprint set corresponding to the variable currently applied for export is less than or equal to a preset number threshold (such as 2, 3, or 4); the fingerprint set corresponding to the variable currently applied for export is the parent set or child set of the fingerprint set corresponding to a single variable that has been exported in historical data, and the difference in the number of elements between the two is less than or equal to a preset number threshold; the fingerprint set corresponding to the variable currently applied for export is the parent set or child set of the union of the fingerprint sets corresponding to at least two variables that have been exported in historical data, and the difference in the number of elements between the two is less than or equal to a preset number threshold.

[0078] To prevent users from tampering with the content of the variable export rule table and to prevent users from launching multiple data user execution environments, according to an embodiment of the present invention, the method further includes step S4: synchronizing a digest of the variable export rule table determined based on a preset hash operation to the data owner or a business manager authorized by the data owner, so as to prevent abnormal users from using data by verifying the continuity of the digest. Illustratively, the data user execution environment includes: an export rule storage area for storing the variable export rule table; wherein, when the data user execution environment is closed, the information of the export rule storage area is stored in a variable export rule file, and a digest of the variable export rule table is calculated based on the variable export rule file and synchronized to the data owner or a business manager authorized by the data owner; when the data user execution environment is started, a digest of the variable export rule table is calculated based on the local variable export rule file and sent to the data owner or a business manager authorized by the data owner to verify the continuity of the digest. Wherein, if the digest sent when the data user execution environment is currently started is inconsistent with the digest synchronized when it was previously closed, it is determined that the continuity of the digest is impaired and abnormal users corresponding to the data user execution environment are prevented from using data. For example, if a user tampers with the exportable flags of certain variables, the digest continuity will be different the next time the data user execution environment is started to verify the continuity, and it will be judged as a digest continuity anomaly. Similarly, if a user successfully starts a data user execution environment using the corresponding digest, and then attempts to start another data user execution environment using the same digest before exiting, this will also be judged as a digest continuity anomaly. Regarding the digest calculation algorithm, existing hash algorithms (such as MD5 (Message-Digest Algorithm 5), RIPEMD (RACE Original Integrity Verification Message Digest), etc.) can be used, or other hash operation methods defined by the implementer; this invention does not impose any restrictions on this. The technical solution of this embodiment can at least achieve the following beneficial technical effects: By synchronizing the digest to the data owner or the business manager authorized by the data owner, this embodiment allows the data owner or the business manager authorized by the data owner to determine whether a user has tampered with the variable export rule table or started multiple data user execution environments by verifying the continuity of the digest, thus preventing abnormal users from using data and better protecting data security.

[0079] There are multiple ways to trigger the synchronization summary. According to one embodiment of the present invention, the synchronization summary can be triggered periodically, when a user exits, or each time a user requests to export variables, causing a change in the exportable flag of any variable. Of course, other triggering methods defined by the implementer can also be used. Taking user exit as an example, step S4 includes: in response to a user's request to exit the data user execution environment, storing the user's variable export rule table in a variable export rule file and determining the summary of the variable export rule file based on a preset hash operation, and synchronizing it with the data owner or a business manager authorized by the data owner. This is used to verify the continuity of the variable export rule file's summary before the user restarts the data user execution environment and to prevent users with abnormal summary continuity from using the data. Taking user exporting a corresponding variable as an example, step S4 includes: each time a user requests to export variables, causing a change in the exportable flag of any variable, storing the user's variable export rule table in a variable export rule file and determining the summary of the variable export rule file based on a preset hash operation, and synchronizing it with the data owner or a business manager authorized by the data owner. This is used to verify the continuity of the summary before the user updates the export rule file and to prevent users with abnormal summary continuity from using the data. It should be understood that the export rule file is the file that carries the variable export rule table, and the summary of the variable export rule file can be regarded as the summary of the variable export rule table.

[0080] The methods described in the foregoing embodiments can be implemented in corresponding systems to achieve the goals of facilitating user access to data and protecting data security. Illustratively, according to one embodiment of the present invention, see... Figure 2 a. A data management system is provided, including: a data user data processing device 10000, a data owner data management device 20000, and a business manager management service device 30000. The data user data processing device 10000, the data owner data management device 20000, and the business manager management service device 30000 are interconnected via a network 40000 to exchange information.

[0081] The business manager management service device 30000 is used to run the business manager management service; the data owner data management device 20000 is used to run the data owner data authorization service, to grant users access to the data owner's data based on authorization. Since each of the two devices runs a single service, no special distinction is made between the devices and the services running on them. According to one embodiment of the invention, the management service is used to manage user and data owner registration and login, and to provide a platform for information interaction between users and data owners. The data user data processing device 10000 runs a general operating environment 12000 for operating the device. The data user execution environment 13000 is a data analysis environment that uses the general operating environment 12000 as a host and supports data users in modeling and analyzing raw data. The data user execution environment 13000 is provided by the data owner. Simultaneously, the data user execution environment 13000 includes the raw data provided by the data user. The data user execution environment 13000 uses the host general operating environment 12000 to access various sub-devices in the data user data processing device 10000, such as network cards, input consoles, display devices, etc. File exchange between the data user execution environment 13000 and the host general operating environment 12000 is controlled by sub-processes such as reading external files and variable exporting sub-processes.

[0082] To prevent the data user execution environment 13000 from being copied multiple times by the user and to ensure that the data user uses the original data within the scope defined by the data owner, this invention utilizes a state synchronization mechanism to periodically synchronize the summary of the variable export rule file. The data owner data authorization service (hereinafter referred to as: data authorization service 21000) running on the data owner data management device 20000 confirms whether the data user is using a unique copy of the data user execution environment 13000 by checking the continuity of the summary.

[0083] Figure 2 b illustrates the methods by which various entities in the system provide and obtain services. To reduce the coupling between data owners and data users, this embodiment uses SOA to organize these three entities and provide services. SOA (Service-Oriented Architecture) is a coarse-grained, loosely coupled service architecture. Service entities communicate through simple, precisely defined interfaces, without involving underlying programming interfaces and communication models, and are independent of the hardware platform, operating system, and programming language used to implement the services. SOA architecture mainly consists of three entities: service provider, service requester, and service register.

[0084] Business managers use the management business service (hereinafter referred to as "management business service 31000") running on the business manager management device 30000, which is the service registry center, to provide user registration for data users and to provide data users with the network service entry address of the data owner; it also provides service name registration and network service entry address registration for the data owner. The data user execution environment 13000 is the service user, and the data authorization service 21000 is the service provider.

[0085] After the data user execution environment 13000 obtains the network service entry address of the data owner through the data user data processing device 10000, the data user execution environment 13000 initiates a service request to the data owner data authorization service 21000 through the data user data processing device 10000, and the two parties exchange information.

[0086] Based on the fundamental requirements of this system, the following defines the basic storage content and response messages between entities. To provide services to data owners and data users, the management service 31000 needs to store information for both. Data owner information includes, but is not limited to: data owner identifier (name), password (or login password), and data owner network service entry address. Data user information includes, but is not limited to: data user identifier (name) and password. The management service 31000 can recognize, but is not limited to: data owner registration requests, data owner login requests, data owner network service entry address synchronization, data user registration requests, data user login requests, and data user service entry requests; and can generate and return response messages based on conditions such as whether the entity has already registered, logged in, or entered a network service entry address. For requests that cannot be processed, the management service 31000 generally adopts a no-response approach.

[0087] Based on subsequent prompts, developers can develop solutions tailored to specific needs. For example, Management Service 31000 can provide services to both types of entities at different Management Service entry addresses; furthermore, to provide better service, Management Service 31000 can provide separate entity configuration management subsystems for each type of entity. These expansions to the stored information and message responses will provide the system with more new functionalities.

[0088] Figure 3This is a sequence diagram of service provision and acquisition methods. Under the guidance of the management business service 31000, the data user execution environment 13000 exchanges information with the data authorization service 21000 through the data user data processing device 10000. This embodiment provides and obtains services, specifically including: data authorization service 21000 registering with management business service 31000 (corresponding to F1), data authorization service 21000 logging into management business service 31000 (corresponding to F2), data authorization service 21000 synchronizing the entry address of data authorization service 21000 with management business service 31000 (corresponding to F3), data user execution environment 13000 registering with management business service 31000 (corresponding to F4), data user execution environment 13000 logging into management business service 31000 (corresponding to F5), data user execution environment 13000 obtaining the entry address of data authorization service 21000 through management business service 31000 (corresponding to F6), and data user execution environment 13000 requesting services from data authorization service 21000 (corresponding to F7).

[0089] Data authorization service 21000 needs to store information about management business service 31000 and data users respectively. The stored management business service 31000 information includes, but is not limited to: management business service 31000 entry address, data owner identifier, password, and data authorization service 21000 network service entry address (hereinafter referred to as: entry address).

[0090] The stored data user information includes: data user identifier, digest, authorization status, and issued credential status. Since the data user execution environment 13000 is provided by the data owner, the data owner provider has pre-defined the data user identifier for the data user. The digest is divided into two categories: initialization digest and dynamic digest. The initialization digest is a string generated by the data owner and distributed to the data user along with the data user execution environment 13000. When the data user execution environment 13000 first connects to the data authorization service 21000, the initialization digest is used to verify the legitimacy of the data user execution environment 13000.

[0091] The dynamic digest is generated by the data user execution environment 13000 during execution, performing a digest calculation on the variable export rule file, for example, using an MD5 digest. This digest is periodically synchronized with the digest stored in the data authorization service 21000. Using this synchronization mechanism, the data authorization service 21000 checks the continuity of the digest information to confirm whether the data user is using a unique copy of the data user execution environment 13000. Depending on the digest execution method, the authorization status is: waiting for initialization, authorized, or revoked. The three authorization statuses can be distinguished using 0, 1, and 2. Other methods can also be used to distinguish the authorization status.

[0092] When a data user gains access to data, the credential issuance status is used to identify whether the user has already received an issued authorization credential. Data can be used based on this authorization credential. 0 indicates that a credential has been issued, and 1 indicates that a credential has not been issued. During initialization, the credential issuance status is set to "no credential issued."

[0093] Data authorization service 21000 can send and recognize messages that interact with business service 31000. Data authorization service 21000 can send and recognize messages with data user execution environment 13000. Data authorization service 21000 can recognize, but is not limited to: data usage authorization requests, summary updates; and can generate and return response messages to the above messages based on service status, such as whether the data user has authorized the data usage.

[0094] According to one embodiment of the present invention, such as Figure 3 As shown, the schematic exchange process between the various entities is as follows:

[0095] F1: Data owners register with the management service;

[0096] F1-1: Data authorization service 21000 sends a registration message to management business service 31000. The registration message includes at least the following information: message type is registration request, data owner identifier, password, user category is data owner, etc.

[0097] F1-2: Management Business Service 31000 checks if the identifier exists in the data owner database based on the user category. If not, it returns a registration success message, which includes at least the message type "registration response," indicating successful registration.

[0098] F2: Data owner login to manage business services;

[0099] F2-1: Data Authorization Service 21000 sends a message to Management Business Service 31000 to log in to Management Business Service 31000. The login message must include at least the following: message type is login request.

[0100] Information such as data owner identifier, password, and user category (data owner);

[0101] F2-2: Management Business Service 31000 returns a login success message after successfully comparing the user category with the information in the data owner's database. The message must include at least the message type "login response", indicating successful login.

[0102] F3: The entry point for data owner and management business service synchronization;

[0103] F3-1: Data Authorization Service 21000 sends a message to Management Business Service 31000 to synchronize the entry address of Data Authorization Service 21000. The message must include at least: message type "Service Entry Synchronization", data owner identifier, and data authorization service address.

[0104] Information such as the 21000 entry address;

[0105] F3-2: After confirming that the message sender has logged in, the management service 31000 updates the entry address of the data authorization service 21000 based on the data owner identifier and returns a synchronization success message.

[0106] The message must include at least the following: a synchronous response message indicating successful login. F4: Data users register with the management service.

[0107] F4-1: Data user execution environment 13000 sends a registration message to management business service 31000. The registration message includes at least: message type is registration request, user identifier, and...

[0108] Information such as order type and user category is used to identify data users;

[0109] F4-2: Management Business Service 31000 checks if the user identifier exists in the data user database based on the user category. If not, it returns a registration success message.

[0110] The message type should include at least: a registration response, indicating successful registration for F5: Data User Login Management Service;

[0111] F5-1: Data user execution environment 13000 sends a message to management business service 31000 to log in to management business service 31000. The login message includes at least the following information: message type is login request, user ID, password, user category is data user, etc.

[0112] F5-2: Management Business Service 31000 returns a login success message after successfully comparing the user category with the information in the data user database. The message must include at least the message type "login response", indicating successful login.

[0113] F6: Data users obtain the service entry address by managing business services;

[0114] F6-1: Data user execution environment 13000 requests the entry address of data authorization service 21000 from management business service 31000. The request message must include at least the following: message type is service entry request, data owner identifier, and data user identifier.

[0115] F6-2: After confirming that the message sender has logged in, the management service 31000 returns a response message based on the data authorization service 21000 entry address information registered by the data owner in the data owner database. The message includes at least: the message type is service entry response and the data authorization service 21000 entry address.

[0116] F7: Data users request services from data owners;

[0117] F7-1: Based on the entry address of Data Authorization Service 21000, Data User Execution Environment 13000 requests services from Data Authorization Service 21000; The message includes at least: message type, data user identifier, etc., and other information in the message is determined according to the message type.

[0118] F7-2: Data Authorization Service 21000 responds to the request message, which includes at least:

[0119] Message type; other information in the message is determined based on the response to the request message.

[0120] Based on the interaction process between the aforementioned entities, developers can implement the process of data users and data owners creating and registering basic identity information of data users and data owners with the management business service 31000, as well as the mechanism for storing related information.

[0121] Figure 4 This is a schematic diagram of the logical structure of a data user execution environment according to an embodiment of the present invention. This logical structure can be implemented in hardware, software, or a hybrid approach combining both. For example, raw data can be stored in a hardware device and provided to the data user. The specific implementation does not affect the basic functions of each module. The execution environment includes a logic execution unit 13100, an export rule storage area 13200, a raw data area 13300, a currently defined variable area 13400, a system information storage area 13500, a system file storage area 13600, and a user file storage area 13700.

[0122] The logic execution unit 13100 is used to execute the system's and user-defined processing flows. The export rule storage area 13200 is used to store the export rules for variables. The raw data area 13300 is used to store data provided by the data owner. The currently defined variable area 13400 is used to store user-defined variables. The system information storage area 13500 is used to store basic system information: user identifier, password, data owner identifier, entry address of the management business service 31000, initialization summary provided by the data user, etc.; in addition, this area is also used to store information returned by the management business service 31000, such as, but not limited to, the entry address of the data authorization service 21000.

[0123] The system file storage area 13600 stores system files, such as workflows that guide data users to start the environment 13000 and variable export rule files. Files are distinguished by their file extensions. The user file storage area 13700 stores user-written workflow files for data analysis and modeling. These workflow files contain user-defined variables. The system file storage area 13600 and the user file storage area 13700 are independent of each other. User files cannot access files in the system file storage area 13600.

[0124] The variable export rule file is used to store the mapping information of the export rule storage area 13200. When the data user execution environment 13000 starts, the system loads the variable export rule file into the export rule storage area 13200. When it closes, the system stores the information of the export rule storage area 13200 into the variable export rule file.

[0125] The variable export rule file consists of multiple record description blocks, which are distinguished from each other by predefined system identifiers. These identifiers can be semicolons or XML format. The choice of identifier does not affect the distinction between record description blocks. Each record in the variable export rule table is stored in an independent record description block. Each record description block in the variable export rule table must include at least one user-defined variable, the fingerprint set corresponding to the variable, and the variable's exportability flag. Implementers can also add other information, such as: the variable's internal code (corresponding to the user-defined variable), the associated original data table index, the export flag (corresponding to the exportability flag), the number of elements in the fingerprint set (abbreviated as "number of elements in the set"), and the element names in the fingerprint set (abbreviated as "element names in the set," corresponding to the fingerprint set). To distinguish the information recorded in the description blocks, predefined identifiers can be used as distinguishing symbols, such as double colons, to distinguish: variable internal code, associated original data table index, export flag, number of elements in the set, and elements in the set. Since a fingerprint set can include multiple elements, these elements can be enclosed in curly braces, and their contents can be separated by double colons. Information recorded in the description block can also be differentiated using other methods, such as XML; this invention does not impose any restrictions on this. Differentiation symbols and methods can be selected according to actual needs. Differentiation symbols and methods do not affect the differentiation of content. Because element name strings vary in length, each fingerprint set is stored in the export rule storage area 13200 using a combination of indexing and variable-length storage space. The number of elements in the set is used to determine the number of index units for each element in the fingerprint set.

[0126] Figure 5 This is a schematic diagram of the raw data storage area structure according to an embodiment of the present invention. The raw data storage area is used to store raw data provided by the data owner. The raw data can be provided in the form of tables and items. Unless otherwise specified, the raw data provided by the owner is stored in the raw data storage area in an unencrypted form, and this data can be observed by the data user. Schematic, the raw data storage area can store multiple raw data tables 13320, a table can contain multiple raw data items 13330, and a raw data item 13330 can contain multiple data items. To facilitate access to the tables, the storage structure includes a raw data table index table 13310. The table index table includes: table name, maximum number, table address, etc. Figure 5In the original data projects, in addition to the numbers, some illustrative project names are also provided, such as: temperature, height, width, air pressure, number of people, etc. It should be understood that the data owner can set these as needed based on the actual situation of the collected data, and this invention does not impose any restrictions on this. For example, data users can collect settlement monitoring data of engineering structures. The illustrative project names of this data include number, monitoring time, time interval, load, CJ1 elevation, etc. Correspondingly, users can use regression analysis methods to establish a mathematical relationship between the dependent variable and the independent variable (i.e., deformation and influencing factors), i.e., a regression equation. Based on the established regression equation, the deformation of the building (settlement, tilt, horizontal displacement, etc.) can be analyzed, and the displacement at a certain time in the future can be predicted using the regression equation, which is essential for the safe operation of engineering structures. For example, data users can collect data on China's energy consumption over the years. The data includes illustrative items such as number, year, energy consumption, total industrial output, total population, and GDP. Users can build multiple linear regression models, use prediction software to perform multiple linear regression analysis, prediction, model testing, and correction on the historical data, and then conduct causal analysis based on the research results, and propose corresponding countermeasures and suggestions, aiming to provide ideas for relevant departments to formulate energy development strategies.

[0127] According to one embodiment of the present invention, the data owner's data is recorded in tabular form, and a corresponding description method is defined. For example, a complete description method of the original data is defined using the format of "single underscore prefix dot bracket". The specific format for data users to access the original data is: _table name.item name[element number]. The complete description method allows access to a specific piece of data. For ease of representation, all data stored in tables, items within tables, etc., are modified based on the complete description form. Preferably, in the system, the access path for each piece of data can be defined by the table name, the item name within the table, and the element number within the table, or a combination thereof, and users are authorized to access the required data through this definition. Wherein, when only the table name is present, for example, _table name, it indicates access to the table. When the table name and item name appear simultaneously, for example, _table name.item name, it indicates access to all elements under that item in the table; correspondingly, user-defined variables can access all data under the corresponding table name and item name in batches through the access path defined by the table name and item name, and the access path defined by the table name and item name indicates the data of the data owner associated with the variable in the fingerprint set. When both table name and element number appear, for example, `_table_name.[element_number]`, it accesses all elements in the table with the same element number. The element number cannot exceed the maximum number. The standard description of raw data mentioned later generally refers to the dataset described in the format "_table_name.item_name", not the numerical values ​​of the actual data. Removing the square brackets from the complete description of raw data converts it to the standard description. Entering only "_table_name" accesses all data in the table. Except for descriptions with element numbers, all other descriptions are variable descriptions of the raw data.

[0128] From an expression format perspective, raw data variables and user-defined variables differ significantly in their descriptive forms. The purpose of using the "single underscore prefix dotted brackets" descriptive format is primarily to distinguish raw data from user-defined variables. Based on the aforementioned encoding principles for raw data, technicians can easily develop other descriptive formats to describe raw data.

[0129] Figure 6This is a schematic diagram of the storage structure of the currently defined variable area according to an embodiment of the present invention. The currently defined variable area is used to store the names of variables defined in the current execution flow of the data user. This area is mainly used to access variables and store the variable internal code corresponding to the current variable. Different data analysis flows can use the same variable name to name different variables; variables in a single flow can also store different data. In order to track variables, the variable internal code is a number randomly assigned by the system to the current variable. When identifying variables subsequently, the variable internal code can maintain the uniqueness of the tracked variable and eliminate ambiguity between variable names. The variable internal code can also be called the variable internal code.

[0130] The storage structure includes a user-defined variable table 13410 and variable content (variable values) 13420. The user-defined variable table 13410 stores variable control information, including variable name, variable internal code, variable type, and variable address. The variable content 13420 stores the content corresponding to the variable name, and the identifier for accessing this content is stored in the variable address of the user-defined variable table 13410. This identifier can be either the variable name or the address that allows access to the variable content. It should be understood that the variable type is determined by the specific programming language, and this invention does not impose any limitations on it. Since this invention is aimed at computational users, variable types include, but are not limited to, int, long, and float. The storage space required to store the variable content 13420 is allocated by the processing system of the programming language.

[0131] Figure 7 This is a schematic diagram of the storage structure of the export rule storage area according to an embodiment of the present invention. The export rule storage area is used to store the control rules for variable export. The export rule storage area includes: a variable export rule table 13210 and a raw data set 13320. The variable export rule table 13210 includes: variable internal code, associated raw data set index (corresponding to fingerprint set), number of elements in the set, export flag (corresponding to exportable flag), and storage address of the raw data item name set. The raw data set stores set elements using a combination of indexes and variable-length variables. Element 13320 in the raw data set is the storage unit for the raw data standard description; element index 13321 in the raw data set is the address for accessing element content. The number of elements in the set represents the number of elements in the raw data set, i.e., the number of element indices.

[0132] Within the same data analysis workflow file, the same variable name can be assigned different content. Therefore, an associated original data set index is used to identify the original data set involved in the assignment statement. The associated original data set index is generated by random numbers to accommodate variations in the assignment statement. Records in Variable Derivation Rule Table 13210 are used to maintain the mapping relationship between variable internal codes and associated original data set indices. The mapping relationship between variable internal codes and associated original data set indices is sequentially added to Variable Derivation Rule Table 13210 according to the order in which assignment statements appear in the data analysis workflow file. The system workflow can find all original data sets related to a variable name in Variable Derivation Rule Table 13210 based on the relationship between the variable and its internal code. Among all related sets, the last set found in the order it is added to Variable Derivation Rule Table 13210 is defined as the original data set corresponding to the variable name. Unless otherwise specified, the original data set corresponding to the variable name is the last set mentioned above.

[0133] The export flag is used to determine whether the content of a variable can be exported to the data user execution environment 13000. When a request is made to export a user-defined variable to the data user execution environment 13000, the system process first finds the original data set corresponding to the variable based on the associated original data set index, and then determines whether the information contained in the set conforms to the export rules. The export flag status is divided into: pending, exportable, and non-exportable. Different values ​​are assigned to the export flag to represent different states. When the export flag is 1, it indicates that it conforms to the export rules and can be exported; when the export flag is 0, it indicates pending; when the export flag is -1, it indicates that it does not conform to the export rules and cannot be exported. It should be noted that when a user-defined variable has not yet been requested to be exported or has been requested to be exported but its exportability has not yet been determined, the exportability flag of the variable is set to pending.

[0134] Figure 8This is a main control flowchart according to an embodiment of the present invention. After the data user execution environment 13000 automatically loads the process from the system file storage area 13600, it first establishes a connection with the data authorization service 21000 through an initialization sub-process and obtains authorization to use the data; secondly, it waits for user commands to execute related operations. Before running the user-defined data analysis process file, the system preprocesses the variables in the file, filling the information related to the variables into the current defined variable area 13400 and the export rule storage area 13200. During the preprocessing stage, if an error is found in the user-defined data analysis process file, an error code is returned, and the preprocessing stage is exited. When there are no errors in the preprocessing stage, the user-defined data analysis process file is executed. The execution is handled by the language interpretation system that wrote the process file. The system can read the file from the host environment and store it in the user file storage area 13700. When a command to terminate system operation is received, the stored variable control rule sub-process is called to store the information in the export rule storage area 13200 into the variable export rule file, generate a summary, synchronize the new summary with the data authorization service 21000, and terminate system operation. Specifically, such as Figure 8 As shown, the schematic process is as follows:

[0135] S10000: Start;

[0136] S10100: Start invoking the system initialization sub-process;

[0137] S10105: Wait for the user to input a message from the console. If a message is received, proceed to S10110; otherwise, proceed to S10105.

[0138] S10110: Extract commands from messages;

[0139] S10200: Is this command an execution command for a process file? If yes, proceed to S10210; otherwise, proceed to S10500.

[0140] S10210: Extract the filename from the message and read the file from the user file storage area 13700 into the logic execution unit 13100;

[0141] S10220: Call the file preprocessing function and fill in the relevant information in the variable export rule storage area 13200;

[0142] S10221: Determine whether to return an error number. If yes, proceed to S10223; otherwise, proceed to S10230.

[0143] S10223: Display error number, proceed to S10105;

[0144] S10230: Execute file statement commands in sequence, then proceed to S10105;

[0145] S10500: Is this command an export variable command? If yes, go to S10510; otherwise, go to S10800.

[0146] S10510: Extract the variable name from the message, use the variable name as a parameter, call the variable export sub-process, and then proceed to S10105.

[0147] S10800: Is this command a command to read a file from the external host environment (i.e., the host general operating environment 12000) to the data user execution environment 13000? If yes, go to S10810; otherwise, go to S10900.

[0148] S10810: Extract the filename from the message, use the filename as a parameter, call the sub-process of reading external files, and then proceed to S10105;

[0149] S10900: Is this command intended to terminate the execution of the data user's environment 13000? If yes, proceed to S10910; otherwise, proceed to S10120.

[0150] S10120: Input error message, redirecting to S10105;

[0151] S10910: Call the sub-process for exporting stored variables;

[0152] S10130: End.

[0153] Figure 9 An initialization sub-process according to an embodiment of the present invention is used to initialize information in the local variable export rule storage area and request data usage rights from the data owner. An illustrative process is as follows:

[0154] S11000: Start;

[0155] S11100: Call the variable export rule to initialize the sub-process;

[0156] S11200: Invoke the sub-process for requesting data usage;

[0157] S11300: Return.

[0158] Figure 10 A variable export rule initialization sub-process according to an embodiment of the present invention is used to read the mapping information stored in the variable export rule file into the export rule storage area 13200. The illustrative process is as follows:

[0159] S12000: Start;

[0160] S12100: Open the variable export rule file;

[0161] S12200: Has the file ended? If not, proceed to S12300; if it has ended, proceed to S12800.

[0162] S12300: Read control information from the current position of the variable export rule file, including the variable internal code in the description block, the index of the associated original data table, the export flag, and the number of elements in the set;

[0163] S12400: Fill the control information into variable export rule table 13210;

[0164] S12500: Request storage space from the export rule storage area according to the number of elements in the set to store the element index, and fill the address of the storage space into the storage address of the original data item name set in the variable export rule table 13210;

[0165] S12600: Read all element contents of the set from the current record description block, and request storage space for each original data standard description from the exported rule storage area, and fill the address that can access the above contents into the element index;

[0166] S12700: Move to the next record description block in the file, then proceed to S12200;

[0167] S12800: Return.

[0168] Figure 11 This describes a data usage application sub-process according to an embodiment of the present invention. Specifically, it describes the interactive process by which a data user applies to a data owner for data usage rights. The specific interactive process is as follows:

[0169] F5-10: Based on the data owner's data authorization service 21000 entry address, the data user execution environment 13000 requests data usage rights from the data authorization service 21000. The message type is data usage authorization request message: username, summary; and a timer is set to wait for the response from the data authorization service 21000.

[0170] F5-11: After the Data Authorization Service 21000 identifies the message type as a Data Use Authorization Request message, it extracts the username and summary from the message; it then verifies the information against the user information stored in the local database; if correct, it returns a Data Use Authorization Response message.

[0171] F5-20: Data authorization service 21000 sends a response message to the data user execution environment 13000. The message type is data usage authorization response message, indicating that the authorization is approved.

[0172] F5-21: The data user execution environment 13000 identifies the message type. If it is a data user authorization response message, further processing is performed.

[0173] Figure 12 This is a data ownership authorization management process according to an embodiment of the present invention. The data authorization service 21000 confirms whether a data user is using a unique data user execution environment 13000 by checking the continuity of the digest information. This process includes processing authorization requests and digest updates. Illustrative steps of this process are as follows:

[0174] SS1000: Start;

[0175] SS1010: Waiting for network message;

[0176] SS1020: Extract message type and message sender network address;

[0177] SS1040: Is this a data usage authorization request message? If yes, forward to SS1100; otherwise, forward to SS1050.

[0178] SS1100: Extract username and digest from messages;

[0179] SS1110: Search for relevant records in the local user database based on the extracted username. If found, proceed to SS1113; otherwise, proceed to SS1010.

[0180] SS1113: Determine whether the status of the voucher issuance status in the relevant record is "voucher not issued". If it is, proceed to SS1115; otherwise, proceed to SS1130.

[0181] SS1115: Set the status of the voucher issuance in the relevant record to 1, indicating that the voucher has been issued;

[0182] SS1120: Determine if the digest stored in the local database matches the digest in the message. If they match, proceed to SS1122; otherwise, proceed to SS1130.

[0183] SS1122: Is the status in the record authorized? If yes, proceed to SS1150; otherwise, proceed to SS1124.

[0184] SS1124: Is the status in the record revoked? If yes, proceed to SS1010; otherwise, proceed to SS1140.

[0185] SS1130: Set the user's authorization status in the above record to "Cancel Authorization" and proceed to SS1010;

[0186] SS1140: Set the user's authorization status in the above record to authorized;

[0187] SS1150: The system returns a data usage authorization response message to the message sender based on the message sender's network address, indicating that the authorization is approved, and then forwards to SS1010;

[0188] SS1050: Is this a summary update message? If yes, forward to SS1200; otherwise, forward to SS1060.

[0189] SS1200: Extract username and digest from messages;

[0190] SS1210: Search for relevant records in the local data user database based on the extracted username. If found, proceed to SS1220; otherwise, proceed to SS1010.

[0191] SS1220: Determine if the authorization status in the record is authorized. If authorized, proceed to SS1225; otherwise, proceed to SS1010.

[0192] SS1225: Sets the status of voucher issuance in the record to 0, indicating that no voucher has been issued;

[0193] SS1230: Update the summary in the record based on the summary in the message;

[0194] SS1240: The system returns a summary update response message to the message sender based on the message sender's network address, indicating that the update is complete, and then proceeds to SS1010;

[0195] SS1060: Execute other processing procedures, then transfer to SS1010;

[0196] Figure 13 This invention provides a data user authorization request process according to an embodiment of the present invention. This process obtains authorization to use the data from the data owner. If authorization is not obtained, the operation of the data user execution environment 13000 is terminated directly. An illustrative process is as follows:

[0197] S13000: Start

[0198] S13010: Request login to management service 31000 based on the entry address of management service 31000 stored in the system;

[0199] S13012: Start timing, waiting for server response;

[0200] S13014: Has the timeout occurred? If it has, proceed to S13095; otherwise, proceed to S13016.

[0201] S13016: Has a message been received? If a message has been received, proceed to S13018; otherwise, proceed to S13014.

[0202] S13018: Cancel timer;

[0203] S13020: Determine whether the login was successful based on the returned message. If successful, proceed to S13030; otherwise, proceed to S13095.

[0204] S13030: Based on the entry address of management service 31000 and the data owner identifier stored in the system, request the entry address of data authorization service 21000 from management service 31000.

[0205] S13032: Start timing, waiting for server response;

[0206] S13034: Has the timeout occurred? If it has, proceed to S13095; otherwise, proceed to S13036.

[0207] S13036: Has a message been received? If a message has been received, proceed to S13038; otherwise, proceed to S13034.

[0208] S13038: Cancel timer;

[0209] S13040: Determine whether to return the data authorization service 21000 entry address based on the returned message. If yes, proceed to S13050; otherwise, proceed to S13095.

[0210] S13050: Determine if the variable export rule table is empty. If it is empty, go to S13060; otherwise, go to S13052.

[0211] S13052: Summary of the contents of the rule file for deriving calculated variables;

[0212] S13056: Based on the data authorization service 21000 entry address, send a data usage authorization request message. The message includes information such as: username, summary of variable export rule file, etc.

[0213] S13060: Based on the data authorization service 21000 entry address, send a data usage authorization request message, which includes: username, data user initialization token in system information storage area 13500, and then forward to S13070;

[0214] S13070: Start timing, waiting for server response;

[0215] S13071: Has the timeout occurred? If it has, proceed to S13095; otherwise, proceed to S13072.

[0216] S13072: Has a message been received? If a message has been received, proceed to S13073; otherwise, proceed to S13071.

[0217] S13073: Cancel timer;

[0218] S13080: Is this a data usage authorization response message? If yes, proceed to S13090; otherwise, proceed to S13095.

[0219] S13090: Return;

[0220] S13095: Error message, exit system.

[0221] It should be understood that in step S13050, when the environment is started for the first time, the variable export rule table may be empty because the user has not defined any variables.

[0222] Figure 14 This is a file preprocessing sub-flow according to an embodiment of the present invention. Since the variables in the sub-function are temporary, they are not output as model parameters, and therefore the variables in the sub-function are not recognized.

[0223] The main purpose of this process is to illustrate how a static scanning method can be used to generate a set of variable fingerprints based on the variables in the data analysis workflow document. For clarity, Figure 14 The illustrated static scanning method does not recognize variables in sub-functions (sub-procedures). However, based on the teachings of this embodiment, developers can develop methods to recognize variables in sub-procedures.

[0224] Interpreted languages ​​such as Python and JavaScript no longer use explicit main function identifiers, such as `main()`. This means that the interpreter no longer distinguishes the scope of variables in the main function from global variables. This process uses the aforementioned interpreted languages ​​as a reference, does not specifically identify the main function, and treats all identified variables as global variables. When there is an explicit main function identifier, variables defined in the main function can be obtained by adding a module that identifies the reserved string used to identify the main function as defined by the language. While there are some differences between programming languages, most directly or indirectly inherit the C syntax standard. To facilitate variable identification and operations, this preprocessing subprocess uses ANSI C 87 (i.e., the ISO C standard accepted by the International Organization for Standardization in 1990) as the standard for identifying variables and operations on variables. To facilitate the interpretation of the variable processing flow, firstly, variables need to be defined before use; secondly, mathematical operators such as "+", "-", and "* / " only have mathematical operational meaning. An illustrative flow is as follows:

[0225] S20000: Start;

[0226] S20020: Sub-function processing status is set to No;

[0227] S20030: Set the current position of the file to the beginning position of the file;

[0228] S20040: Is the end of the file reached? If yes, proceed to S20090; otherwise, proceed to S20050.

[0229] S20050: Obtain a line of text from the current line of the file and set the next line as the current line;

[0230] S20060: Is the sub-function processing status yes? If yes, proceed to S20100; otherwise, proceed to S20030.

[0231] S20100: Based on the syntax, identify whether this statement marks the end of the sub-function. If it does, go to S20110; otherwise, go to S20040.

[0232] S20110: Set the sub-function processing status to no, then proceed to S20040;

[0233] S20300: Based on the syntax, identify whether the statement is a function definition statement. If it is, proceed to S20310; otherwise, proceed to S20400.

[0234] S20310: Set the sub-function processing status to yes, then proceed to S20040;

[0235] S20400: Based on the syntax, identify whether the statement is a variable definition statement. If it is, proceed to S20410; otherwise, proceed to S20500.

[0236] S20410: Pass this statement as a parameter to call the variable definition preprocessing sub-process, then proceed to S20040;

[0237] S20500: Based on the syntax, identify whether the statement is an assignment statement to a variable. If it is, go to S20520; otherwise, go to S20040.

[0238] S20520: Use this statement as a parameter to call the assignment operation preprocessing subprocess;

[0239] S20522: Determine whether an error number is returned. If yes, proceed to S20095; otherwise, proceed to S20040.

[0240] S20090: Return;

[0241] S20095: Returns the error number.

[0242] Figure 15 This is a variable definition preprocessing sub-process according to an embodiment of the present invention. This sub-process scans the current process and fills the user-defined variables into the user-defined variable table. An illustrative process is as follows:

[0243] S24000: Start;

[0244] S24010: Add a record to the user-defined variable table and write the variable name;

[0245] S24020: Generate a random number and use the random number as the internal code of the variable;

[0246] S24030: Fill in the variable address with the identifier that the variable content can be accessed, so that the variable can be accessed;

[0247] S24040: End.

[0248] Figure 16 This is a preprocessing sub-flow for assignment operations according to an embodiment of the present invention. Assignment operations are the most basic operations. In multiple regression, a model is described by a set of factors. Each factor is obtained by complex calculations on the original data. We divide assignment operations into assignment between variables and assignment of equations. For example, Y = t1 is an assignment between variables, where Y is the variable being assigned and t1 is the assignment variable. For example, Y = t0 + t1*x1 + t2*x2 + t3*x3, Y = t0 + t1*x1 + foo_1(t2,x2)) + foo_2(t3,x3,x4), etc., are assignments of equations, where t0, t1*x1, foo_2(t3,x3,x4), etc., are terms in the equation, with t0 being the variable term, t1*x1 being the multiplication term, and foo_2(t3,x3,x4) being the function term. The specific process for extracting variables from the assignment expressions is as follows:

[0249] S25000: Start;

[0250] S25010: Create a list header, i.e., the first list, to store the names of variables in the statement;

[0251] S25020: Create a header list, i.e., a second list, to store the names of functions in the statement;

[0252] S25030: Create a list header to store the names of the original data, i.e., the fourth list;

[0253] S25040: Determine whether the statement is an assignment between variables based on the syntax. If it is, go to S25050; otherwise, go to S25300.

[0254] S25050: Extract the assigned variable according to the assignment syntax rules between variables;

[0255] S25060: The process of generating the associated original data set is called by taking the variable name and the fourth list as parameters. After the process is executed normally and returns, the fourth list stores the standard description of all the original data related to the current variable.

[0256] S25061: Determine if an error number is returned. If yes, proceed to S25070; otherwise, proceed to S2508.

[0257] S25070: Delete all lists, proceed to S25101;

[0258] S25300: Identify each item in the statement according to the syntax. If it is a variable item, fill the variable name into the first list. If it is a function item or a multiplication item, fill it into the second list.

[0259] S25305: Determine if the second list is empty. If it is, proceed to S25350; otherwise, proceed to S25310.

[0260] S25310: Take one item from the second list and delete that item from the second list;

[0261] S25320: Based on the syntax, identify whether the expression of this item is a function call. If it is, go to S25330; otherwise, go to S25325.

[0262] S25325: Identify all variable names in the multiplication term expression according to the syntax, add all variable names to the first list, and go to S25340;

[0263] S25330: Identify all variable names in the function call expression based on the syntax and add all variable names to the first list;

[0264] S25340: Does the second list contain any items? If yes, proceed to S25310; otherwise, proceed to S25350.

[0265] S25350: Take a variable name from the first list and delete the variable name from the first list;

[0266] S25360: The process of generating the associated original data set is called by taking the variable name and the fourth list as parameters. After the process is executed normally and returns, the fourth list stores all the original data names related to the current variable.

[0267] S25361: Determine if an error number is returned. If yes, proceed to S25365; otherwise, proceed to S25370.

[0268] S25365: Delete all lists, proceed to S25102;

[0269] S25370: Does the first list contain variable names? If so, go to S25350; otherwise, go to S25080.

[0270] S25080: Check if the fourth list is empty. If it is empty, go to S25095; otherwise, go to S25085.

[0271] S25085: Extract the variable to be assigned according to the assignment syntax rules;

[0272] S25090: The variable to be assigned and the fourth list are used as parameters to call the sub-process of establishing variable export rules;

[0273] S25095: Delete all lists;

[0274] S25100: Return;

[0275] S25101: Returns the error number;

[0276] S25102: Returns the error number.

[0277] Figure 17 This is a sub-process for generating an associated raw data set according to an embodiment of the present invention. Specifically, it describes the process of generating the associated raw data set. Based on the descriptive form of the variable name, it determines whether the variable name exists in the raw data area or in the user-defined variable table 13410. The standard description of the raw data corresponding to the variable name is found and added to the list storing the set of standard descriptions of raw data. In the variable export rule table 13210, when a variable's internal code corresponds to multiple records, the record added last in chronological order to the variable export rule table 13210 is selected (the order of records in the variable export rule table is based on the chronological order in which the variable names corresponding to the records appear in the user process file. Therefore, when searching sequentially, the order of the search results has a temporal characteristic). An illustrative process is as follows:

[0278] S25600: Start;

[0279] S25605: Extract the variable names and the list in the parameters, i.e., the fourth list;

[0280] S25610: Determine whether the variable name description format is the original data description method. If it is, convert it to S25625; otherwise, convert it to S25620.

[0281] S25625: Check if the variable name exists in the original data area 13300. If it does, go to S25635; otherwise, go to S25685.

[0282] S25635: Store the original data standard description corresponding to the variable name into the fourth list, then transfer to S25680;

[0283] S25620: Check if the variable name exists in user-defined variable table 13410. If it does, proceed to S25630; otherwise, proceed to S25685.

[0284] S25630: Extract the variable internal code from the record;

[0285] S25640: Check if the variable's internal code exists in variable export rule table 13210. If it does, proceed to S25650; otherwise, proceed to S25680.

[0286] S25650: Locate all records containing the internal code of this variable in sequence;

[0287] S25660: Based on the storage address of the set of original data item names of the last record, retrieve all original data standard descriptions from the set;

[0288] S25670: Add all raw data standard description methods to the fourth list;

[0289] S25680: Return;

[0290] S25685: Returns the error number.

[0291] Figure 18 To establish a variable derivation rule sub-process according to an embodiment of the present invention, the currently assigned variable and all associated original data standard descriptions are added to the records of the variable derivation rule table. The list passed as a parameter contains all original data standard descriptions related to the assigned variable. The set formed by these descriptions is associated with the assigned variable and constitutes the fingerprint set of the assigned variable. For ease of description, the independent original data standard descriptions in the set are referred to as elements of the set, or simply elements. An illustrative process is as follows:

[0292] S25900: Start;

[0293] S25905: Extract the variable names from the parameters, and extract the list from the parameters, i.e., the fourth list;

[0294] S25910: Add a record to variable export rule table 13210 and automatically assign an associated original data table index to the record. This index can be generated by random numbers.

[0295] S25915: Based on the variable name, find the variable code of the variable name in the user-defined variable table 13410, and fill the variable code into the newly generated record above.

[0296] S25920: Set the export flag to pending;

[0297] S25925: Get the number of elements in the list;

[0298] S25930: Based on the number of list elements, request storage space from the export rule storage area to store element indexes (element index list), and fill the address of the storage space into the storage address of the original data item name set of the newly generated record;

[0299] S25940: Determine if there is an element in the list; if there is, proceed to S25945; otherwise, proceed to S25960.

[0300] S25945: Retrieve an element from the list;

[0301] S25950: Request storage space from the exported rule storage area for the original data standard description contained in the element, fill the address that can access the above content into the element index list, and establish the association between the variable and the original data;

[0302] S25955: Delete the element from the list, go to S25940;

[0303] S25960: Determine if the number of list elements is 1. If yes, go to S25965; otherwise, go to S25970.

[0304] S25965: According to the variable export rules, set the export flag of the newly generated record to non-exportable;

[0305] S25970: Return.

[0306] Figure 19 This is a variable derivation sub-process according to an embodiment of the present invention. The process determines whether the value of a variable can be exported to the computing environment based on export rules. When the variable fingerprints of any two variables in the export rule storage area 13200 are each other's largest subset and smallest parent set, or when multiple sets can form a largest subset and smallest parent set through a union operation, only one variable's value among the multiple variable values ​​can be exported to the data user execution environment 13000. An illustrative process is as follows:

[0307] S50000: Start;

[0308] S50010: Find the record containing the variable name from the user-defined variable table 13410 based on the variable name;

[0309] S50020: Read the variable internal code in this record;

[0310] S50030: Based on the variable's internal code, locate all records containing that variable's internal code in variable export rule table 13210, ordered by addition time. Among the found records, the record added latest to the variable export rule table is designated as the record to be processed.

[0311] S50040: Read the set element corresponding to the variable based on the storage address of the set of original data item names in the processed record, and denot it as the processed set;

[0312] S51000: Is the export flag in the processed record set to pending status? If yes, proceed to S51100; otherwise, proceed to S52000.

[0313] S51100: Record the first record of variable export rule table 13210 as the reference record;

[0314] S51200: Determine whether the export flag of the reference record is exportable. If it is, proceed to S51210; otherwise, proceed to S51240.

[0315] S51210: Extract the set from the reference record and set it as the reference set;

[0316] S51220: Apply parameters to the reference set and the set being processed, and invoke the set difference process;

[0317] S51230: Returns whether the number of elements in the set is equal to the number of elements in the processed record. If yes, proceed to S51240; otherwise, proceed to S51260.

[0318] S51260: Returns whether the set is empty. If it is, go to S51262; otherwise, go to S51270.

[0319] S51262: Change the "export flag" of the records corresponding to the set to be processed to exportable.

[0320] S51264: Based on the address information in the record containing the variable name in the user-defined variable table 13410, perform the operation of exporting the variable content (variable value) to the user execution environment 13000. The specific operation process is determined by the computing environment and its host environment. Proceed to S50095.

[0321] S50095: Return;

[0322] S51270: Returns whether the number of elements in the set is equal to 1. If yes, go to S51280; otherwise, go to S51290.

[0323] S51290: Determine if the returned set is a subset of the processed set. If yes, proceed to S53000; otherwise, proceed to S51240.

[0324] S53000: Get the number of elements in the returned set, designate the current returned set as the first returned set, and use the first returned set as a parameter to call the union processing sub-process;

[0325] S53010: Check if the number of elements in the return set is equal to the number of elements in the first return set. If yes, go to S51240; otherwise, go to S51260.

[0326] S51280: Set the records corresponding to the set to be processed to be non-exportable, then proceed to S50095;

[0327] S50095: Return;

[0328] S51240: Has variable export rule table 13210 been read? If yes, go to S51245; otherwise, go to S51250.

[0329] S51245: Set the next record in variable export rule table 13210 as the reference record, and go to S51200;

[0330] S51250: Change the "export flag" of the record corresponding to the set to be processed to exportable, then proceed to S52010;

[0331] S52000: Is the export flag in the processed record set to exportable? If yes, go to S52010; otherwise, go to S50090.

[0332] S52010: Based on the address information in the record containing the variable name in the user-defined variable table 13410, perform the operation of exporting the variable content (the value of the variable) to the user execution environment 13000. The specific operation process is determined by the computing environment and its host environment.

[0333] S50090: Return.

[0334] Figure 20 This is a sub-process for confirming the existence of a child or parent union according to an embodiment of the present invention. Since multiple derivable sets can be combined into a single set through union calculation, the resulting set may be the parent set or a child set of the currently processed set. The currently processed set cannot be derived when the parent set is the smallest parent set, or when the child set is the largest child set. This process searches for the existence of a combined parent or child set in the variable derivation rule table 13210 by calculating the difference between sets, and identifies whether the set is the largest parent set or the smallest child set. An illustrative process is as follows:

[0335] S53001: Start;

[0336] S53010: Create an empty set, assign the set extracted from the parameters to the newly created set, and record this set as the set to be processed;

[0337] S53020: Create an empty set, assign the set to be processed to the newly created set, and record this set as the original set;

[0338] S53030: Create an empty set and denote it as the current smallest set;

[0339] S53040: Record the first record in variable export rule table 13210 as a reference record;

[0340] S53100: Determine whether the export flag of the reference record is exportable. If it is, proceed to S53110; otherwise, proceed to S53210.

[0341] S53110: Extract the set from the reference record and set it as the reference set;

[0342] S53120: Call the set difference process with the reference set and the set to be processed as parameters;

[0343] S53130: Determine whether the number of elements in the returned set is equal to the number of elements in the processed record. If yes, proceed to S53210; otherwise, proceed to S53140.

[0344] S53140: Determine whether the returned set is a subset of the processed set and whether the number of elements in the returned set is greater than 1. If yes, proceed to S53150; otherwise, proceed to S53180.

[0345] S53150: Determine if the current minimum set is empty. If it is, proceed to S53170; otherwise, proceed to S53160.

[0346] S53160: Compare whether the number of elements in the returned set is less than the number of elements in the current minimum set. If yes, go to S53170; otherwise, go to S53210.

[0347] S53170: Assign the returned set to the current smallest set, then go to S53210;

[0348] S53200: Set the next record in variable export rule table 13210 as the reference record, and proceed to S53100;

[0349] S53210: Determine whether all records in the discriminant variable derivation rule table 13210 have been read. If yes, proceed to S53220; otherwise, proceed to S53200.

[0350] S53220: Determine if the current minimum set is empty. If it is, proceed to S53230; otherwise, proceed to S53240.

[0351] S53230: Convert the original set into a return parameter, then go to S53090;

[0352] S53240: Assign a value to the set being processed using the current smallest set;

[0353] S53250: Delete all elements in the current smallest set, making it an empty set, then go to S53040;

[0354] S53180: Use the returned set as the return parameter;

[0355] S53090: Return.

[0356] Figure 21 This is a sub-process for calculating the difference between two sets according to an embodiment of the present invention. The method for calculating the difference is as follows: if the two sets are the same, return empty; if the two sets are different, return the currently processed set; if the processed set is a subset of the reference set, return all elements in the reference set that are not in the processed set as a single set; if the reference set is a subset of the processed set, return all elements in the processed set that are not in the reference set as a single set. An illustrative process is as follows:

[0357] S54000: Start;

[0358] S54010: Extract the reference set and the processed set from the parameters;

[0359] S54100: Determine whether the reference set and the processed set are the same. If they are, S54110; otherwise, S54200.

[0360] S54110: Return an empty set as a parameter, then go to S54090;

[0361] S54200: Determine whether the set being processed is a subset of the reference set. If yes, S54210; otherwise, S54300.

[0362] S54210: Obtain the union of two sets;

[0363] S54220: Return the set after removing the union of the reference set as the return parameter, then go to S54090;

[0364] S54300: Determine whether the reference set is a subset of the set being processed. If yes, S54310; otherwise, S54330.

[0365] S54310: Obtain the union of two sets;

[0366] S54320: Return the set after removing the union of the processed set as the return parameter, then proceed to S54090;

[0367] S54330: The set to be processed is returned as a parameter;

[0368] S54090: Return.

[0369] Figure 22 This is a sub-process for reading an external file according to an embodiment of the present invention. The file is read from the host environment where the data user execution environment 13000 is located, based on parameters (such as the external file name and its access path on the host machine), and stored in the user file storage area 13700. When the external file name is the same as the file name stored in the user file storage area 13700, the user decides whether to store the external file. An illustrative process is as follows:

[0370] S80000: Start;

[0371] S80010: Does it have parameters? If it does, go to S80020; if it doesn't, go to S80095.

[0372] S80020: Get the file name from the parameters;

[0373] S80040: Determine if the file name is the same as the file name stored in user file storage area 13700. If yes, proceed to S80050; otherwise, proceed to S80070.

[0374] S80050: The user is prompted that the file already exists and asked whether to continue the operation. The system is waiting for user commands.

[0375] S80060: Determine whether the user's response is to continue the operation. If the user continues, proceed to S80070; otherwise, proceed to S80090.

[0376] S80070: Reads a file from the external environment and stores the file in the user file storage area 13700, then transfers to S80090;

[0377] S80095: Error message, redirecting to S80090;

[0378] S80090: Return.

[0379] Figure 23 This describes an interactive process for updating a digest according to an embodiment of the present invention. The process describes the synchronization of digests between a data user execution environment 13000 and a data owner data authorization service 21000. An illustrative flow of the digest update interactive process is as follows:

[0380] F5-100: Based on the data owner's data authorization service 21000 entry address, the data user execution environment 13000 sends a summary update message to the data authorization service 21000. The message type is summary update message: username, summary; and it counts down, waiting for the data authorization service 21000's response.

[0381] F5-101: After the Data Authorization Service 21000 identifies the message type as a summary update message, it extracts the username and summary from the message; under authorized status, it updates the summary of the user in the local database; and returns a summary update message response message.

[0382] F5-200: Data Authorization Service 21000 sends a response message to Data User Execution Environment 13000. The message type is Digest Update Response Message, indicating that the digest update is complete.

[0383] Figure 24 This is a sub-process for storing variable control rules according to an embodiment of the present invention. Information in the export rule storage area 13200 is stored in a variable export rule file in the system file storage area 13600, a summary is generated based on this file, and the summary is synchronized with the data authorization service 21000. An illustrative process is as follows:

[0384] S90000: Start;

[0385] S90010: Delete the variable export rule file in the system file storage area 13600, and create a variable export rule file with no content in the system file storage area 13600 to store the information in the export rule storage area 13200;

[0386] S90020: Set the read position to the first record of variable export rule table 13210;

[0387] S90030: Read the index of the associated original data set, variable internal code, number of elements in the set, export flag, and elements in the set in the current record;

[0388] S90040: Write the above data into the file according to the file format of the variable export rule file;

[0389] S90050: Move down one record from the current read position of rule table 13210 derived from variables;

[0390] S90060: Are there any unread records in variable export rule table 13210? If yes, proceed to S90030; otherwise, proceed to S90070.

[0391] S90070: Summary of the contents of the rule file for deriving calculation variables;

[0392] S90075: Based on the data authorization service 21000 entry address, send a summary update message, which includes: username and a summary of the variable export rule file content;

[0393] S90080: Start timing, waiting for server response;

[0394] S90081: Has the timeout occurred? If it has, proceed to S90085; if it has not, proceed to S90082.

[0395] S90082: Has a message been received? If a message has been received, forward to S90083; otherwise, forward to S90081.

[0396] S90083: Cancel timer;

[0397] S90084: Is this a summary update response message? If yes, forward to S90090; otherwise, forward to S90085.

[0398] S90085: Error message displayed, system exit;

[0399] S90090: Return.

[0400] It should be understood that although the above embodiments provide illustrative detailed system solutions with abundant accompanying drawings and technical details, those skilled in the art, inspired by the above embodiments of the present invention, may still obtain other feasible implementation methods to achieve the present invention, and the present invention does not impose any limitations on these methods. For example, in one system embodiment, instead of using a business manager management service device, a data owner data management device manages user registration, login, and the granting of data owner data to users based on authorization.

[0401] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0402] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0403] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, including but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0404] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for managing data access, the method comprising: In a data user execution environment where users are authorized to use data owner's data for data analysis in a regulated manner, user-defined variables in a user-defined data analysis process file are identified, and a corresponding fingerprint set is generated for the variables, wherein the fingerprint set indicates the data of the data owner associated with the corresponding variable; In response to a user's request to export the value of a custom variable, the system determines whether the value of the requested variable can be exported to the data user's execution environment based on a preset variable export rule table for data security and the fingerprint set corresponding to the variable. The determination of whether the value of the requested variable can be exported to the data user's execution environment is made as follows: Based on the fingerprint set corresponding to the variable currently requested for export and the fingerprint set corresponding to variables already exported recorded in historical data, the system determines whether the variable currently requested for export can be exported. If the data indicated by the relevant fingerprint set can be deduced based on the value of the variable currently requested for export or based on the combination of the value of the variable currently requested for export and the values ​​of variables already exported, the system determines that the value of the variable currently requested for export cannot be exported. The variable export rule table records historical data of one or more variables defined by the user in the past, the fingerprint sets corresponding to each variable, and the exportability flags of each variable. The variable currently requested for export is the variable corresponding to the value of the variable currently requested for export.

2. The method according to claim 1, wherein, A variable cannot be exported if any of the following conditions exist in the current application: The fingerprint set corresponding to the variable currently being exported contains only one element; The fingerprint set corresponding to the variable currently being exported is the smallest parent set or the largest child set of the fingerprint sets corresponding to the single variable that has been exported in the historical data. The fingerprint set corresponding to the variable currently being exported can be combined with the fingerprint sets corresponding to two or more variables that have been exported in the historical data records to form a minimum parent set or a maximum child set through a union operation.

3. The method according to claim 1, wherein, When identifying user-defined variables in a user-defined data analysis workflow file, a unique variable internal code is assigned to each user-defined variable to identify its identity, and a corresponding fingerprint set is maintained for each variable internal code. In different fingerprint sets, a description method is set to identify the identity of the same data.

4. The method according to claim 1, wherein, The method further includes: The summary of the variable export rule table determined based on the preset hash operation is synchronized to the data owner or the business manager authorized by the data owner, so as to prevent abnormal users from using the data by checking the continuity of the summary.

5. The method according to claim 1, wherein, The exportable flags include: pending, exportable, and non-exportable.

6. The method according to any one of claims 1-5, wherein, If the fingerprint set of any variable is not empty, it means that the value of the variable is calculated based on the data in the fingerprint set corresponding to that variable.

7. The method according to claim 6, characterized in that, The user-defined data analysis process file is used to train and / or predict the model in the data user's execution environment. The data analysis process file stores the model's parameter values, intermediate values ​​obtained based on some of the model's parameters, and the model's output values ​​in the form of variables.

8. A data management system, characterized in that, include: A data user data processing device for performing the method according to any one of claims 1-7; A data owner data management device, which is used to run a data owner data authorization service to allow users to use the data owner's data based on authorization.

9. The data management system according to claim 8, characterized in that, Also includes: A data user execution environment, which is a data analysis environment that authorizes users to use the data owner's data for data analysis in a regulated manner; The business manager manages the equipment, which is used to run management business services. These management business services are used to manage the registration and login of users and data owners who use data, and to provide a platform for information interaction between the users and data owners.

10. The data management system according to claim 9, characterized in that, in, The management service is used to provide user registration for data users and to provide data users with the network service entry address of the data owner; And to provide service name registration and network service entry address registration for data owners; After the data user execution environment obtains the network service entry address of the data owner through the data user data processing device, the data user execution environment initiates a service request to the data owner's data authorization service through the data user data processing device. The two parties exchange information to determine whether to authorize the user to obtain and use the data owner's data based on the data user execution environment.

11. The data management system according to claim 9, characterized in that, The data user execution environment includes: The export rule storage area is used to store the variable export rule table; Specifically, when the data user's execution environment is shut down, the information of the export rule storage area is stored in the variable export rule file, and a summary of the variable export rule table is calculated based on the variable export rule file and synchronized to the data owner or the business manager authorized by the data owner. When the data user execution environment starts, it will calculate a summary of the variable export rule table based on the local variable export rule file and send it to the data owner or the business manager authorized by the data owner to verify the continuity of the summary.

12. The data management system according to claim 11, characterized in that, When the digest sent when the data user execution environment is currently started is inconsistent with the digest synchronized when it was previously shut down, the continuity of the digest is determined to be impaired and the abnormal user corresponding to the data user execution environment is prevented from using the data.

13. The data management system according to claim 10, characterized in that, The data owner's data is recorded in tabular form, and the access path for each piece of data is defined by the table name, the item name in the table, and the element number in the table, or a combination thereof, and users are authorized to access the required data through this definition.

14. The data management system according to claim 13, characterized in that, User-defined variables can access all data under the corresponding table name and project name in batches through the access path defined by the table name and project name, and indicate the data owner associated with the variable in the fingerprint set using the access path defined by the table name and project name.

15. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1 to 7.

16. An electronic device, characterized in that, include: One or more processors; as well as Memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method according to any one of claims 1 to 7 by executing the executable instructions.

Images