Data analysis method, device, equipment, medium and program product
By adding access control conditions to the initial analysis statements generated by the large language model, the problem of user data access rights not being taken into account is solved, and secure and accurate data analysis is achieved.
Patent Information
- Application Number
- CN202510848327.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-24
- Publication Date
- 2025-09-26
AI Technical Summary
When using large language models for data analysis, the user's data access rights are not taken into consideration, resulting in the analysis results containing information outside the user's data access rights, which has low security.
By using a large language model to generate an initial analysis statement and identifying sensitive fields, access control conditions are added to the initial analysis statement based on the user's access permission information, and a target analysis statement is generated to control the scope of data analysis.
It improves the security and accuracy of data analysis, ensures that the analysis results are only performed within the scope of data accessible to users, and reduces the risk of data leakage.
Smart Images
Figure CN120706565A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and more specifically to a data analysis method, apparatus, device, medium, and program product. Background Art
[0002] With the continuous development of artificial intelligence technology, large language models have been widely used. For example, in order to improve efficiency, large language models are used for data analysis.
[0003] However, when using large language models for data analysis, the user's data access rights are usually not taken into consideration, resulting in the analysis results containing information outside the user's data access rights, which has low security. Summary of the Invention
[0004] In view of the above problems, the present application provides a data analysis method, apparatus, device, medium and program product.
[0005] According to a first aspect of the present application, a data analysis method is provided, comprising: in response to receiving a data analysis request for a target data set from a user, processing the data analysis request using a large language model to generate an initial analysis statement corresponding to the data analysis request, wherein the large language model is trained using the target data set; using sensitive field information corresponding to the target data set, performing sensitive field identification on the initial analysis statement to obtain an identification result; when the identification result indicates that the initial analysis statement includes sensitive fields, adding, based on the access permission information of the user, an access control condition matching the access permission information to the initial analysis statement to obtain a target analysis statement; executing the target analysis statement to analyze the target data set to obtain an analysis result.
[0006] According to an embodiment of the present application, in the case where the above-mentioned recognition result indicates that the above-mentioned initial analysis statement includes sensitive fields, according to the access permission information of the above-mentioned user, an access control condition matching the above-mentioned access permission information is added to the above-mentioned initial analysis statement to obtain a target analysis statement, including: in the case where the above-mentioned access permission information indicates that the above-mentioned user has access permission to the above-mentioned sensitive fields, according to the above-mentioned initial analysis statement, determining the target data table targeted by the above-mentioned initial analysis statement; according to the above-mentioned access permission information, determining the row-level access permission of the above-mentioned user in the above-mentioned target data table, the above-mentioned row-level access permission indicating that the above-mentioned user has permission to access data rows whose preset fields are preset field values; generating the above-mentioned access control condition according to the above-mentioned preset fields and the above-mentioned preset field values; adding the above-mentioned access control condition to the above-mentioned initial analysis statement to obtain the above-mentioned target analysis statement.
[0007] According to an embodiment of the present application, the above-mentioned access control conditions are added to the above-mentioned initial analysis statement to obtain the above-mentioned target analysis statement, including: when the above-mentioned initial analysis statement includes a query condition, the above-mentioned query condition and the above-mentioned access control condition are spliced to obtain the target condition; the query condition in the above-mentioned initial analysis statement is replaced with the above-mentioned target condition to obtain the above-mentioned target analysis statement.
[0008] According to an embodiment of the present application, the above-mentioned data analysis request includes data analysis requirements in natural language form. In response to receiving the user's data analysis request for the target data set, the above-mentioned data analysis request is processed by using a large language model to generate an initial analysis statement corresponding to the above-mentioned data analysis request, including: performing logical compliance identification on the above-mentioned data analysis requirement to obtain a compliance identification result; when the above-mentioned compliance identification result represents the logical compliance of the above-mentioned data analysis requirement, the above-mentioned data analysis requirement is processed by using the above-mentioned large language model to obtain the above-mentioned initial analysis statement.
[0009] According to an embodiment of the present application, when the compliance identification result represents the logical compliance of the data analysis requirement, the data analysis requirement is processed using the large language model to obtain the initial analysis statement, including: inputting the prompt information and the data analysis requirement into the large language model to obtain the target analysis intention corresponding to the data analysis requirement, the prompt information is used to guide the large language model to determine the target analysis intention corresponding to the data analysis requirement from multiple preset analysis intentions; obtaining and displaying target question information matching the target analysis intention, the target question information is used to guide the user to input parameters related to the analysis intention; in response to receiving the answer information for the target question information input by the user, the data analysis requirement and the answer information are input into the large language model to obtain the initial analysis statement.
[0010] According to an embodiment of the present application, the above method also includes: determining a target chart type that matches the above analysis result based on the target data type of the above analysis result and utilizing a mapping relationship between the data type and the chart type; inputting the above analysis result into a target chart generation component corresponding to the above target chart type to generate a target chart for the above analysis result.
[0011] According to an embodiment of the present application, the above method also includes: displaying the above initial analysis statement; in response to the above user's modification operation on the above initial analysis statement, obtaining a revised analysis statement modified by the above user; and using the above data analysis request, the above initial analysis statement and the above revised analysis statement to fine-tune the above large language model.
[0012] A second aspect of the present application provides a data analysis device, comprising: an initial generation module, configured to, in response to receiving a data analysis request from a user for a target data set, process the data analysis request using a large language model to generate an initial analysis statement corresponding to the data analysis request, wherein the large language model is trained using the target data set; a field identification module, configured to identify sensitive fields on the initial analysis statement using sensitive field information corresponding to the target data set to obtain an identification result; a target generation module, configured to, when the identification result indicates that the initial analysis statement includes sensitive fields, add access control conditions matching the access permission information to the initial analysis statement based on the user's access permission information to obtain a target analysis statement; and a statement execution module, configured to execute the target analysis statement, analyze the target data set, and obtain an analysis result.
[0013] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.
[0014] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.
[0015] The fifth aspect of the present application further provides a computer program product, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor.
[0016] According to the embodiments of the present application, by utilizing a large language recognition model, a user's data analysis requirements can be automatically converted into initial analysis statements, reducing the difficulty of data analysis and improving its efficiency. Furthermore, by identifying sensitive fields in the initial analysis statements, and dynamically injecting access control conditions into the initial analysis statements based on the user's access rights information when sensitive fields are included, more precise permission control can be implemented, improving the security of data analysis using the large language model. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The above contents and other objects, features and advantages of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:
[0018] Figure 1 Schematically illustrates an application scenario diagram of the data analysis method, apparatus, device, medium, and program product according to an embodiment of the present application;
[0019] Figure 2 The following schematically shows a flow chart of a data analysis method according to an embodiment of the present application;
[0020] Figure 3 Schematically shows a flow chart of generating a target analysis statement according to an embodiment of the present application;
[0021] Figure 4 A schematic diagram of generating an initial analysis statement according to an embodiment of the present application is schematically shown;
[0022] Figure 5 A schematic diagram of a data analysis device according to an embodiment of the present application is shown; and
[0023] Figure 6 A block diagram of an electronic device suitable for implementing a data analysis method according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION
[0024] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.
[0025] The terms used herein are only for describing specific embodiments and are not intended to limit this application. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.
[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0027] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0028] In related technologies, data analysts often need to manually write data analysis statements to perform data analysis, which makes data analysis difficult and inefficient. Therefore, to improve the efficiency and reduce the difficulty of data analysis, data platforms often embed large language models so that data analysts can use them to perform data analysis.
[0029] However, in a data platform, different users have different data access permissions. When directly using a large language model for data analysis, the user's data access permissions are usually not taken into account. As a result, the large language model cannot generate analysis results normally, or the generated analysis results include information outside the user's permissions, reducing the security of the data platform.
[0030] An embodiment of the present application provides a data analysis method, characterized in that the method includes: in response to receiving a data analysis request from a user for a target data set, processing the data analysis request using a large language model to generate an initial analysis statement corresponding to the data analysis request, wherein the large language model is trained using the target data set; using sensitive field information corresponding to the target data set, performing sensitive field identification on the initial analysis statement to obtain an identification result; when the identification result indicates that the initial analysis statement includes sensitive fields, adding access control conditions matching the access permission information to the initial analysis statement based on the user's access permission information to obtain a target analysis statement; executing the target analysis statement, analyzing the target data set, and obtaining an analysis result.
[0031] The embodiments of the present application add access control conditions to the initial analysis statement containing sensitive information, so that the obtained target analysis statement can perform data analysis within the data range accessible to the user, avoiding the analysis of information outside the user's access rights and improving security.
[0032] Figure 1 The application scenario diagram of the data analysis method, apparatus, device, medium and program product according to the embodiments of the present application is schematically shown.
[0033] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or optical fiber cables.
[0034] A user may use a first terminal device 101, a second terminal device 102, or a third terminal device 103 to interact with a server 105 via a network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, or the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).
[0035] The first terminal device 101 , the second terminal device 102 , and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0036] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process received data such as user requests, and feed back processing results (e.g., web pages, information, or data obtained or generated based on user requests) to the terminal devices.
[0037] It should be noted that the data analysis method provided in the embodiment of the present application can generally be executed by the server 105. Accordingly, the data analysis device provided in the embodiment of the present application can generally be set in the server 105. The data analysis method provided in the embodiment of the present application can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the data analysis device provided in the embodiment of the present application can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.
[0038] For example, a user can use the first terminal device 101, the second terminal device 102 or the third terminal device 103 to send a data analysis request to the server 105 through the network 104. In response to receiving the user's data analysis request for the target data set, the server 105 uses a large language model to process the data analysis request and generates an initial analysis statement corresponding to the data analysis request. The large language model is trained using the target data set; the sensitive field information corresponding to the target data set is used to identify the sensitive fields of the initial analysis statement to obtain an identification result; when the identification result indicates that the initial analysis statement includes sensitive fields, an access control condition matching the access permission information is added to the initial analysis statement according to the user's access permission information to obtain a target analysis statement; the target analysis statement is executed, the target data set is analyzed to obtain an analysis result, and the analysis result is returned to the first terminal device 101, the second terminal device 102 or the third terminal device 103 through the network 104 so that the user can view the analysis result.
[0039] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.
[0040] The following will be based on Figure 1 The scene described by Figures 2 to 4 The data analysis method according to the embodiment of the present application is described in detail.
[0041] Figure 2 The flowchart of the data analysis method according to the embodiment of the present application is schematically shown.
[0042] like Figure 2 As shown, the data analysis method of this embodiment includes operations S210 to S240.
[0043] In operation S210 , in response to receiving a data analysis request from a user for a target data set, the data analysis request is processed using a large language model to generate an initial analysis statement corresponding to the data analysis request.
[0044] In operation S220 , sensitive field information corresponding to the target data set is used to identify sensitive fields in the initial analysis statement to obtain an identification result.
[0045] In operation S230 , when the recognition result indicates that the initial analysis statement includes a sensitive field, an access control condition matching the access permission information is added to the initial analysis statement according to the user's access permission information to obtain a target analysis statement.
[0046] In operation S240 , the target analysis statement is executed to analyze the target data set to obtain an analysis result.
[0047] According to an embodiment of the present application, a user can access a client of a data platform in a terminal and input data analysis requirements in the client. The client can generate a data analysis request based on the data analysis requirements input by the user and send the data analysis request to a server of the data platform.
[0048] According to embodiments of the present application, after a user enters data analysis requirements, they can specify a target dataset to be analyzed in the client. In some embodiments, the target dataset can be obtained from multiple data sources. After obtaining the datasets from multiple data sources, preprocessing operations can be performed on the datasets to ensure data quality and obtain the target dataset. Preprocessing operations can include, for example, cleaning, data conversion, and mapping.
[0049] According to embodiments of the present application, a large language model is trained using a target dataset. For example, after obtaining the target dataset, the server can train the large language model using the target dataset and preset data analysis requirements that match the target dataset, so that the large language model can understand the relationship between natural language related to data analysis and corresponding analysis statements, and learn the patterns and characteristics of the data in the target dataset.
[0050] According to an embodiment of the present application, the initial analysis statement may be in the form of Structured Query Language (SQL). Since the large language model has been trained, the large language model can generate a corresponding initial analysis statement based on the user's data analysis requirements.
[0051] According to embodiments of the present application, sensitive fields can be fields that are restricted from access by certain users, and the sensitive field information can include the field identifiers of the sensitive fields in the target dataset. Therefore, sensitive field identification can be performed on the initial analysis statement based on the field identifiers of the sensitive fields to determine whether the initial analysis statement includes sensitive fields.
[0052] According to an embodiment of the present application, when performing sensitive field identification on an initial analysis statement, a matching operation can be performed on the initial analysis statement using the sensitive field identifier to obtain a sensitive field identification result.
[0053] According to an embodiment of the present application, when the initial analysis statement does not include sensitive fields, executing the initial analysis statement will not obtain information outside the user's access rights, so the initial analysis statement can be directly executed to perform data analysis and obtain analysis results.
[0054] According to an embodiment of the present application, when the initial analysis statement includes sensitive fields, directly executing the initial analysis statement may obtain information outside the user's access rights. Therefore, it is necessary to add access control conditions to the initial analysis statement to control the data analysis process within the user's access rights.
[0055] According to an embodiment of the present application, the access control condition may be generated based on the user's access permission information. Specifically, the user's access scope for sensitive fields may be obtained from the access permission information, and the access control condition may be generated based on the access scope.
[0056] According to the embodiments of the present application, by utilizing a large language recognition model, a user's data analysis requirements can be automatically converted into initial analysis statements, reducing the difficulty of data analysis and improving its efficiency. Furthermore, by identifying sensitive fields in the initial analysis statements, and dynamically injecting access control conditions into the initial analysis statements based on the user's access rights information when sensitive fields are included, more precise permission control can be implemented, improving the security of data analysis using the large language model.
[0057] According to an embodiment of the present application, when the identification result indicates that the initial analysis statement includes sensitive fields, access control conditions that match the access permission information are added to the initial analysis statement based on the user's access permission information to obtain a target analysis statement, including: when the access permission information indicates that the user has access permissions to sensitive fields, determining the target data table targeted by the initial analysis statement based on the initial analysis statement; determining the user's row-level access permissions in the target data table based on the access permission information; generating access control conditions based on preset fields and preset field values; and adding the access control conditions to the initial analysis statement to obtain a target analysis statement.
[0058] According to an embodiment of the present application, the target data table includes sensitive fields. Therefore, access control conditions for users can be determined based on the user's row-level access rights and column-level access rights in the target data table, so as to control the user's access to sensitive fields using the access control conditions. Specifically, row-level access rights indicate that the user has the right to access data rows where a preset field has a preset field value, and column-level access rights indicate that the user has the right to access the field.
[0059] In an embodiment of the present application, before determining the user's row-level access rights, the user's column-level access rights can be obtained from the access rights information. If the column-level access rights indicate that the user does not have access rights to a sensitive field, the user's terminal can be prompted that the user does not have access rights to the sensitive field.
[0060] According to an embodiment of the present application, when column-level access permissions indicate that a user has permission to access sensitive fields, the user's row-level access permissions are determined to further determine the scope of the user's access to sensitive fields. For example, although a user has access to the "Name" field, they can only access user information within their department. Therefore, the user's row-level access permissions are for data rows where the "Department" field indicates the user's department.
[0061] According to embodiments of the present application, access conditions can be similar to query conditions in SQL. Access control conditions can be generated based on the preset fields and preset field values in row-level access permissions. For example, if row-level access permissions indicate that a user can access data for Department A, and the department field is "department," the access control condition could be "WHERE department = 'Department A'" to restrict the user to data within that department.
[0062] According to an embodiment of the present application, after the access control condition is generated, the access control condition can be added to the initial analysis statement and used as a part of the analysis statement to control access rights, thereby obtaining a target analysis statement.
[0063] According to an embodiment of the present application, fine-grained row-level access rights are utilized to generate access control conditions and improve the security of target analysis statements.
[0064] Figure 3 The flowchart of generating a target analysis statement according to an embodiment of the present application is schematically shown.
[0065] like Figure 3 As shown, generating a target analysis statement includes operations S301 to S307.
[0066] In operation S301 , sensitive fields are identified for an initial analysis statement.
[0067] In operation S302, it is determined whether the initial analysis statement includes sensitive fields. If the initial analysis statement includes sensitive fields, operation S303 is performed; otherwise, operation S308 is performed.
[0068] In operation S303, it is determined whether the user has access rights to the sensitive field. If the user has access rights to the sensitive field, operation S304 is performed; otherwise, operation S307 is performed.
[0069] In operation S304 , the target data table targeted by the initial analysis statement is determined.
[0070] In operation S305 , access control conditions are generated according to the row-level access rights of the user in the target data table.
[0071] In operation S306 , the access control condition is added to the initial analysis statement to obtain a target analysis statement.
[0072] In operation S307 , the initial analysis statement is determined to be a target analysis statement.
[0073] In operation S308 , the user is prompted that he or she does not have access rights to the sensitive field.
[0074] According to an embodiment of the present application, access control conditions are added to the initial analysis statement to obtain a target analysis statement, including: when the initial analysis statement includes a query condition, the query condition and the access control condition are spliced to obtain the target condition; and the query condition in the initial analysis statement is replaced with the target condition to obtain the target analysis statement.
[0075] According to an embodiment of the present application, when adding access control conditions to the initial analysis statement, since the access control conditions are similar in form to the query conditions and are both conditional clauses, it is possible to first determine whether the initial analysis statement already includes the conditional clause, and then determine whether the initial analysis statement already includes the query conditions.
[0076] According to an embodiment of the present application, if the initial analysis statement includes a query condition, the query condition can be concatenated with the access control condition to obtain a new conditional clause, which serves as the target condition. For example, if the initial analysis statement already includes the query condition "WHERE age>25", the access control condition "WHERE department = 'Department A' " can be concatenated with the query condition "WHERE " to obtain the target condition "WHERE department = 'Department A' AND age>25".
[0077] According to an embodiment of the present application, the target condition already includes the access control condition and the query condition. In this case, the query condition in the initial analysis statement can be directly replaced with the target condition to obtain the target analysis statement.
[0078] According to an embodiment of the present application, by splicing the query condition with the access control condition when the query condition is already included in the initial analysis statement, the security of generating the target analysis statement is improved.
[0079] According to an embodiment of the present application, in response to receiving a data analysis request from a user for a target data set, the data analysis request is processed using a large language model to generate an initial analysis statement corresponding to the data analysis request, including: performing logical compliance identification on the data analysis requirement to obtain a compliance identification result; when the compliance identification result represents the logical compliance of the data analysis requirement, the data analysis requirement is processed using a large language model to obtain an initial analysis statement.
[0080] According to embodiments of the present application, logical compliance identification can be used to determine whether a user-entered data analysis requirement is logically consistent. Since user data analysis requirements are natural language descriptions entered by the user, to ensure the accuracy of the analysis statement, logical compliance identification can be performed on the data analysis requirement to ensure the logical compliance of the data analysis requirement, thereby ensuring that the initial analysis statement generated by the large language model based on the data analysis requirement is also logically consistent.
[0081] According to an embodiment of the present application, if the data analysis requirements are not logically compliant, a prompt indicating the logical non-compliance of the data analysis requirements can be displayed on the terminal. For example, if the data analysis requirements include both "sales > 1000" and "sales < 500", the data analysis requirements are not logically compliant, and the user can be prompted to "delete the conflicting condition" or "change it to sales > 500?"
[0082] According to an embodiment of the present application, when the logic of the data analysis requirements is compliant, a large language model can be used to generate initial analysis statements corresponding to the data analysis requirements.
[0083] According to an embodiment of the present application, by performing logical compliance identification on the data analysis requirements input by the user to ensure that analysis statements are generated using logically correct data analysis requirements, the accuracy of the analysis statements can be improved.
[0084] According to an embodiment of the present application, when the compliance identification result represents the logical compliance of the data analysis requirement, the data analysis requirement is processed using a large language model to obtain an initial analysis statement, including: inputting prompt information and data analysis requirements into the large language model to obtain a target analysis intention corresponding to the data analysis requirement; obtaining and displaying target question information that matches the target analysis intention; in response to receiving answer information for the target question information input by the user, inputting the data analysis requirement and the answer information into the large language model to obtain an initial analysis statement.
[0085] According to an embodiment of the present application, the prompt information is used to guide the large language model to determine the target analysis intent corresponding to the data analysis requirement from multiple preset analysis intents. When guiding the large language model to identify the user's analysis intent, the prompt information and the user's data analysis requirement can be input into the large language model at the same time, guiding the large language model to determine the user's target analysis intent from multiple preset analysis intents.
[0086] According to an embodiment of the present application, since the user may not consider all the required parameters when inputting data analysis requirements, target question information can be used to guide the user to input parameters related to the analysis intention, and guide the user to input answer information including relevant parameters.
[0087] According to an embodiment of the present application, analysis intents and corresponding question information may be pre-stored. After determining the user's target analysis intent, target question information corresponding to the target analysis intent is determined from multiple question information and displayed to the user. For example, if the target analysis intent is determined to be analyzing sales trends, the prompt information may be "Please select: 1. Time range 2. Comparison dimension 3. Indicator calculation method" to guide the user to complete the parameters required to analyze sales trends.
[0088] According to an embodiment of the present application, when the user's input answer information is ambiguous, candidate fields that match the parameters can also be further listed. For example, when the user inputs the comparison dimension of performance, the candidate fields can include "sales" and "profit", and the prompt message can be "Do you mean [sales] or [profit]?"
[0089] According to an embodiment of the present application, a completion guidance mechanism is used to guide users to supplement their data analysis needs, thereby improving the accuracy of initial analysis statements.
[0090] Figure 4 The diagram schematically shows a method for generating an initial analysis statement according to an embodiment of the present application.
[0091] like Figure 4 As shown, prompt information 401 and logically compliant data analysis requirements 402 are input into the large language model. The large language model is used to identify the intent of the data analysis requirements, resulting in a target analysis intent 403. From multiple question information, target question information 404 that matches the target analysis intent is determined and displayed. Based on the answer information 405 entered by the user for the target question information, the data analysis requirement 402 and the answer information 405 are input into the large language model to generate an initial analysis statement 406.
[0092] According to an embodiment of the present application, the data analysis method also includes: determining a target chart type that matches the analysis result based on the target data type of the analysis result and utilizing a mapping relationship between the data type and the chart type; inputting the analysis result into a target chart generation component corresponding to the target chart type to generate a target chart of the analysis result.
[0093] According to an embodiment of the present application, the data types of the analysis results may include time series data and classification data, etc. The chart type that has a mapping relationship with the time series data may be a line chart, and the chart type that has a mapping relationship with the classification data may be a pie chart.
[0094] According to an embodiment of the present application, the data platform server may be pre-integrated with various types of chart generation components, such as a line chart generation component and a pie chart generation component. When a user needs to generate a target chart for the analysis results, the target chart generation component corresponding to the target chart type may be called to generate the target chart for the analysis results.
[0095] According to an embodiment of the present application, a low-code component may also be integrated into the server of the data platform. When the user needs to display the analysis results using a visual analysis result page, the analysis results and the target chart of the analysis results can be integrated into the analysis result page, and the analysis result page can be designed using the low-code component.
[0096] According to the embodiments of the present application, a target chart that is compatible with the data type of the analysis result is automatically generated, which has the beneficial effect of increasing the richness of the analysis result and improving the user experience.
[0097] According to an embodiment of the present application, the data analysis method also includes: displaying an initial analysis statement; obtaining a revised analysis statement modified by the user in response to a user's modification operation on the initial analysis statement; and fine-tuning the large language model using the data analysis request, the initial analysis statement, and the revised analysis statement.
[0098] According to an embodiment of the present application, when displaying the analysis results, the initial analysis sentence generated by the large language model can also be displayed. If the initial analysis sentence does not meet the user's needs, the user can directly modify the initial analysis sentence to obtain a revised analysis sentence.
[0099] According to an embodiment of the present application, after obtaining a revised analysis statement, the revised analysis statement can be used to replace the initial analysis statement to execute the data analysis method, which will not be described in detail here.
[0100] According to an embodiment of the present application, since the revised analysis sentence is a sentence input by the user that matches the data analysis requirement, the revised analysis sentence, the data analysis request, and the initial analysis sentence can be used to fine-tune the large language model.
[0101] According to an embodiment of the present application, when fine-tuning a large language model, the data analysis requirements in the data analysis request can be used as the input of the large language model, the initial analysis statement can be used as the error label, and the corrected analysis statement can be used as the correct label to fine-tune the large language model.
[0102] In some specific embodiments, data analysis requests, initial analysis statements, and revised analysis statements can be stored in a hallucination sample pool. When the amount of data in the hallucination sample pool reaches a certain threshold, the data in the hallucination sample pool is used to fine-tune the parameters of the large language model, the hallucination sample pool is cleared, and the data platform is incrementally updated.
[0103] According to an embodiment of the present application, when the initial analysis statement is inaccurate, the large model is fine-tuned using the revised analysis statement modified by the user, the initial analysis statement generated by the large model, and the data analysis request. The beneficial effect is to improve the accuracy of the analysis statement generated by the large model.
[0104] Based on the above data analysis method, this application also provides a data analysis device. Figure 5 The device is described in detail.
[0105] Figure 5 The structural block diagram of the data analysis device according to an embodiment of the present application is schematically shown.
[0106] like Figure 5 As shown, the data analysis device 500 of this embodiment includes an initial generation module 510 , a field identification module 520 , a target generation module 530 and a statement execution module 540 .
[0107] Initial generation module 510 is configured to, in response to receiving a user's data analysis request for a target dataset, process the data analysis request using a large language model trained using the target dataset and generate an initial analysis statement corresponding to the data analysis request. In one embodiment, initial generation module 510 may be configured to perform operation S210 described above and will not be further described here.
[0108] The field identification module 520 is used to identify sensitive fields in the initial analysis statement using the sensitive field information corresponding to the target data set to obtain an identification result. In one embodiment, the field identification module 520 can be used to perform the operation S220 described above, which will not be repeated here.
[0109] Target generation module 530 is configured to, when the recognition result indicates that the initial analysis statement includes sensitive fields, add access control conditions that match the user's access rights information to the initial analysis statement to obtain a target analysis statement. In one embodiment, target generation module 530 may be configured to perform operation S230 described above and will not be further described here.
[0110] The statement execution module 540 is used to execute the target analysis statement, analyze the target data set, and obtain the analysis result. In one embodiment, the statement execution module 540 can be used to perform the operation S240 described above, which will not be repeated here.
[0111] According to an embodiment of the present application, the target generation module 530 includes a target table determination submodule, a permission determination submodule, a condition generation submodule, and a condition addition submodule.
[0112] The target table determination submodule is used to determine the target data table targeted by the initial analysis statement according to the initial analysis statement when the access permission information indicates that the user has access permission to the sensitive field.
[0113] The permission determination submodule is used to determine the user's row-level access rights in the target data table based on the access permission information. The row-level access rights indicate that the user has the permission to access the data row with the preset field being the preset field value.
[0114] The condition generation submodule is used to generate access control conditions based on preset fields and preset field values;
[0115] The condition adding submodule is used to add access control conditions to the initial analysis statement to obtain the target analysis statement.
[0116] According to an embodiment of the present application, the conditional adding submodule includes a conditional splicing unit and a conditional replacing unit.
[0117] The condition splicing unit is used to splice the query condition and the access control condition to obtain the target condition when the initial analysis statement includes the query condition.
[0118] The condition replacement unit is used to replace the query condition in the initial analysis statement with the target condition to obtain the target analysis statement.
[0119] According to an embodiment of the present application, the data analysis request includes data analysis requirements in natural language. The initial generation module 510 includes a logic identification submodule and an initial generation submodule.
[0120] The logic identification submodule is used to perform logical compliance identification on data analysis requirements and obtain compliance identification results.
[0121] The initial generation submodule is used to use the large language model to process the data analysis requirements and obtain the initial analysis statement when the compliance recognition result represents the logical compliance of the data analysis requirements.
[0122] According to an embodiment of the present application, the initial generation submodule includes an intention determination unit, a question presentation unit and an initial generation unit.
[0123] The intention determination unit is used to input prompt information and data analysis requirements into the large language model to obtain the target analysis intention corresponding to the data analysis requirements. The prompt information is used to guide the large language model to determine the target analysis intention corresponding to the data analysis requirements from multiple preset analysis intentions.
[0124] The question display unit is used to obtain and display target question information that matches the target analysis intention. The target question information is used to guide the user to input parameters related to the analysis intention.
[0125] The initial generation unit is used to input the data analysis requirements and the answer information into the large language model in response to receiving the answer information for the target question information input by the user, so as to obtain the initial analysis sentence.
[0126] According to an embodiment of the present application, the data analysis device 500 further includes a type determination module and a chart generation module.
[0127] The type determination module is used to determine the target chart type that matches the analysis result based on the target data type of the analysis result and using the mapping relationship between the data type and the chart type.
[0128] The chart generation module is used to input the analysis results into the target chart generation component corresponding to the target chart type to generate a target chart of the analysis results.
[0129] According to an embodiment of the present application, the data analysis device 500 further includes a statement display module, a correction acquisition module and a model fine-tuning module.
[0130] The statement display module is used to display the initial analysis statement.
[0131] The correction acquisition module is used to obtain a corrected analysis statement modified by the user in response to a user's modification operation on the initial analysis statement.
[0132] The model fine-tuning module is used to fine-tune the large language model using data analysis requests, initial analysis sentences, and revised analysis sentences.
[0133] According to an embodiment of the present application, any multiple modules among the initial generation module 510, the field identification module 520, the target generation module 530, and the statement execution module 540 can be combined into a single module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in a single module. According to an embodiment of the present application, at least one of the initial generation module 510, the field identification module 520, the target generation module 530, and the statement execution module 540 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented in hardware or firmware by any other reasonable means of integrating or packaging circuits, or can be implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the initial generation module 510 , the field identification module 520 , the target generation module 530 and the statement execution module 540 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.
[0134] Figure 6 A block diagram of an electronic device suitable for implementing a data analysis method according to an embodiment of the present application is schematically shown.
[0135] like Figure 6 As shown, an electronic device 600 according to an embodiment of the present application includes a processor 601, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present application.
[0136] Various programs and data required for the operation of the electronic device 600 are stored in the RAM 603. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flow according to the embodiment of the present application by executing the programs in the ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 may also perform various operations of the method flow according to the embodiment of the present application by executing the programs stored in the one or more memories.
[0137] According to an embodiment of the present application, electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to bus 604. Electronic device 600 may also include one or more of the following components connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or modem. Communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. Removable media 611, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 610 as needed, so that computer programs read from the removable media can be installed into storage section 608 as needed.
[0138] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of this application is implemented.
[0139] According to an embodiment of the present application, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, a computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.
[0140] The embodiments of the present application also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data analysis method provided in the embodiments of the present application.
[0141] The computer program executes the above functions defined in the system / device of the embodiment of the present application when the computer program is executed by the processor 601. According to the embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0142] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.
[0143] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from a removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present application are performed. According to the embodiment of the present application, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.
[0144] According to an embodiment of the present application, the program code for executing the computer program provided by the embodiment of the present application can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, languages such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).
[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0146] Those skilled in the art will appreciate that the features described in the various embodiments of this application may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in this application. In particular, the features described in the various embodiments of this application may be combined and / or coupled in various ways without departing from the spirit and teachings of this application. All such combinations and / or couplings fall within the scope of this application.
Claims
1. A data analysis method, characterized in that: The method comprises: In response to receiving a data analysis request from a user for a target data set, processing the data analysis request using a large language model to generate an initial analysis statement corresponding to the data analysis request, wherein the large language model is trained using the target data set; Using the sensitive field information corresponding to the target data set, performing sensitive field identification on the initial analysis statement to obtain an identification result; If the recognition result indicates that the initial analysis statement includes a sensitive field, adding an access control condition matching the access permission information to the initial analysis statement based on the user's access permission information to obtain a target analysis statement; The target analysis statement is executed to analyze the target data set to obtain an analysis result.
2. The method according to claim 1, characterized in that When the recognition result indicates that the initial analysis statement includes a sensitive field, adding an access control condition matching the access permission information to the initial analysis statement based on the user's access permission information to obtain a target analysis statement includes: In a case where the access permission information indicates that the user has access permission to the sensitive field, determining, according to the initial analysis statement, a target data table targeted by the initial analysis statement; Determining, based on the access permission information, row-level access permission of the user in the target data table, wherein the row-level access permission indicates that the user has permission to access a data row in which a preset field has a preset field value; generating the access control condition according to the preset field and the preset field value; The access control condition is added to the initial analysis statement to obtain the target analysis statement.
3. The method according to claim 2, characterized in that Adding the access control condition to the initial analysis statement to obtain the target analysis statement includes: In the case where the initial analysis statement includes a query condition, the query condition and the access control condition are concatenated to obtain a target condition; The query condition in the initial analysis statement is replaced with the target condition to obtain the target analysis statement.
4. The method according to claim 1, wherein The data analysis request includes a data analysis requirement in natural language form. In response to receiving the user's data analysis request for the target data set, the large language model is used to process the data analysis request to generate an initial analysis statement corresponding to the data analysis request, including: Performing logical compliance identification on the data analysis requirements to obtain compliance identification results; In a case where the compliance identification result indicates the logical compliance of the data analysis requirement, the data analysis requirement is processed using the large language model to obtain the initial analysis statement.
5. The method according to claim 4, characterized in that When the compliance identification result indicates that the data analysis requirement is logically compliant, processing the data analysis requirement using the large language model to obtain the initial analysis statement includes: Inputting prompt information and the data analysis requirement into the large language model to obtain a target analysis intent corresponding to the data analysis requirement, wherein the prompt information is used to guide the large language model to determine the target analysis intent corresponding to the data analysis requirement from multiple preset analysis intents; Acquire and display target question information that matches the target analysis intent, wherein the target question information is used to guide the user to input parameters related to the target analysis intent; In response to receiving the answer information input by the user to the target question information, the data analysis requirement and the answer information are input into the large language model to obtain the initial analysis sentence.
6. The method according to claim 1, characterized in that The method further comprises: According to the target data type of the analysis result, a target chart type matching the analysis result is determined by using a mapping relationship between data types and chart types; The analysis result is input into a target chart generating component corresponding to the target chart type to generate a target chart of the analysis result.
7. The method according to claim 1, characterized in that The method further comprises: displaying the initial analysis statement; In response to a modification operation performed by the user on the initial analysis statement, obtaining a revised analysis statement modified by the user; The large language model is fine-tuned using the data analysis request, the initial analysis sentence, and the revised analysis sentence.
8. A data analysis device, characterized in that: The device comprises: an initial generation module, configured to, in response to receiving a data analysis request from a user for a target data set, process the data analysis request using a large language model to generate an initial analysis statement corresponding to the data analysis request, wherein the large language model is trained using the target data set; A field identification module is used to identify sensitive fields of the initial analysis statement using the sensitive field information corresponding to the target data set to obtain an identification result; a target generation module configured to, when the recognition result indicates that the initial analysis statement includes a sensitive field, add, based on the user's access permission information, an access control condition matching the access permission information to the initial analysis statement to obtain a target analysis statement; The statement execution module is used to execute the target analysis statement, analyze the target data set, and obtain analysis results.
9. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
11. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.