BERT-based data index intelligent service recommendation method and system

Abstract

The application discloses a kind of data index intelligent service recommendation method and system based on BERT, the method includes: constructing data index model, the correlation analysis training of data index model constructs data index atlas;According to user information and user input information, identify business domain, supplement scene context;With scene context, filter the data index of data index atlas by BERT algorithm, and the quantity of data index screened out is determined;If the quantity of data index screened out is greater than 0, then return the result to user according to user access authority;If the quantity of data index screened out is 0, then call prompt word service, guide user to make correct selection, return the result to user according to user access authority, and revise correlation analysis training.The application can help user to find index quickly and accurately according to user information and user use scene, provide basis for user to use index well.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of big data processing, and in particular to a data index intelligent service recommendation method and system based on BERT. BACKGROUND

[0002] After calculation, the traditional data index usually exists in the form of a physical table or is supplemented by data asset cataloging. When used by business users, it needs to be connected to a physical table (which is generally oriented to IT users). Business users have a threshold for finding indexes, and cannot find or cannot easily find them. Furthermore, physical tables are generally described in IT language, which leads to a threshold for business users to find and use data indexes. Even if they find them, they may not understand them. How to help business users quickly find and use data indexes according to user information and user usage scenarios is also a requirement of digital transformation.

[0003] CN115114424A discloses a query request response method and device. The response method includes: obtaining a query request information input by a user; determining a plurality of candidate query statements associated with the query request information from a plurality of query statements in a question and answer library; for each candidate query statement in the plurality of candidate query statements, determining at least one relevant feature for indicating the relevance of the candidate query statement to the query request information; determining the matching degrees of the plurality of candidate query statements to the query request information according to the at least one relevant feature; determining at least one target query statement from the plurality of candidate query statements according to the matching degrees of the plurality of candidate query statements to the query request information; and obtaining answer information corresponding to the at least one target query statement in the question and answer library as a query result for the query request information. It is needless to say that the technical solution disclosed in the above patent document is a beneficial attempt in the technical field, but the response method is for intelligent question and answer, and is not applicable to data index intelligent service recommendation. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a data index intelligent service recommendation method and system based on BERT, which can help users quickly and accurately find indexes according to user information and user usage scenarios, provide a basis for users to use indexes, improve user satisfaction and the value of data services, and promote the development of data management and application.

[0005] The data index intelligent service recommendation method based on BERT in the present application comprises:

[0006] A data index model is constructed, and a data index graph is constructed by performing correlation analysis and training on the data index model;

[0007] According to user information and user input information, a business field is identified, and a scene context is supplemented;

[0008] In the context of the scene, the data indicators with a correlation greater than a predetermined threshold are screened in the data indicator atlas by the BERT algorithm, and the number of the screened data indicators is determined;

[0009] If the number of the screened data indicators is greater than 0, the result is returned to the user according to the user access permission;

[0010] If the number of the screened data indicators is 0, a prompt word service is called to guide the user to make a correct selection, and the process of the user's selection and the finally selected data indicator are recorded. Based on the finally selected data indicator, the result is returned to the user according to the user access permission, and the correlation analysis training is corrected.

[0011] Further,

[0012] A data indicator model is constructed, and the data indicator model is subjected to correlation analysis training to construct a data indicator atlas, including:

[0013] Metadata information of the data indicator service is obtained from the data indicator service, and the metadata information includes data sources, dimensions, measures, indicator definition descriptions, and calculation formulas;

[0014] The metadata information is used as a definition element, and the business domain, business process information, and data management information of the data source are obtained from the data asset system to construct the data indicator model;

[0015] The BERT algorithm is used to calculate the correlation of the data indicator model, the data indicator correlation is calculated by weighting these correlations, and the data indicator atlas is constructed according to these data indicator correlations;

[0016] Data indicator reports are collected and analyzed from data indicator application scenarios, and the data indicator atlas is fine-tuned according to the use of the data indicators in the data indicator reports.

[0017] Further,

[0018] According to the user information and the user input information, the business domain is identified, and the scene context is supplemented, including:

[0019] The user information and the user input information are obtained;

[0020] According to the definition elements of the data indicator model, relevant information is extracted from the user input information by the BERT algorithm, and the relevant information includes dimension information and measure information used to describe the data indicators;

[0021] According to the dimension information and the measure information, the business domain of the data indicator is identified, the dimension specific information is obtained according to the dimension information and the user information, and the scene context is supplemented.

[0022] Further,

[0023] The dimension refers to the category data of the data index;

[0024] The metric refers to the numerical data of the data index;

[0025] The index definition description refers to the textual description for introducing the definition of the data index;

[0026] The calculation formula refers to the processing logic for indicating the calculation of the data index.

[0027] Further, the result is returned to the user according to the user access permission, including:

[0028] The authentication interface in the data permission system is called, and whether the user has the index access permission is judged according to the user information;

[0029] If it is judged that no, the data index returned to the user is attached with the description of "no permission, please apply for permission to data field XXX", wherein the "data field XXX" is derived from the data management information of the data source obtained from the data asset system;

[0030] If it is judged that yes, the filtered data index or the data index finally selected by the user is returned to the user.

[0031] Further, the predetermined threshold is 85%.

[0032] A data index intelligent service recommendation system based on BERT in the application, comprising:

[0033] A modeling module, configured to construct a data index model, and to perform correlation analysis training on the data index model to construct a data index graph;

[0034] A supplementary context module, configured to identify a business field and supplement scene context according to user information and user input information;

[0035] An analysis recommendation module, configured to filter data indexes with a correlation greater than a predetermined threshold in the data index graph through a BERT algorithm based on the scene context, and to determine the number of the filtered data indexes;

[0036] A prompt word convergence module, configured to call a prompt word service, guide the user to make correct choices, record the process of the user's choices and the data index finally selected by the user, return a result to the user according to the user access permission based on the data index finally selected by the user, and correct the correlation analysis training;

[0037] An index access permission control module, configured to call an authentication interface in a data permission system, and to judge whether the user has the index access permission according to user information.

[0038] The beneficial effects of the present application are: the present application can identify the scene context according to the user information and the user input information by constructing the data index model and the data index atlas, if the user has the permission, the user can quickly and accurately find the index according to the user information and the user use scene, which provides the basis for the user to use the index, is beneficial to improve the user satisfaction and the value of data service, and promotes the development of data management and application. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to make the purpose, technical scheme and beneficial effects of the present application more clear, the present application provides the following drawings for illustration:

[0040] Figure 1 The flowchart of the data index intelligent service recommendation method based on BERT of the present application;

[0041] Figure 2 The detailed flowchart of the data index intelligent service recommendation method based on BERT of the present application;

[0042] Figure 3 The framework diagram of the data index intelligent service recommendation system based on BERT of the present application. DETAILED DESCRIPTION

[0043] The technical scheme of the present application will be described in detail below in combination with the drawings and examples.

[0044] As shown in the figure, the data index intelligent service recommendation method based on BERT in the present embodiment includes: Figures 1-2 Constructing a data index model, performing correlation analysis training on the data index model, and constructing a data index atlas;

[0045] According to the user information and the user input information, identifying the business field, and supplementing the scene context;

[0046] In the scene context, filtering the data index with the correlation greater than the predetermined threshold in the data index atlas through the BERT algorithm, and determining the number of the filtered data index;

[0047] If the number of the filtered data index is greater than 0, the result is returned to the user according to the user access permission;

[0048] If the number of the filtered data index is 0, the prompt word service is called to guide the user to make the correct selection, and the process of the user selection and the finally selected data index are recorded, the result is returned to the user according to the user access permission based on the finally selected data index, and the correlation analysis training is corrected.

[0049]

[0050] ​The BERT algorithm belongs to the prior art. BERT stands for Bidirectional Encoder Representations from Transformers. It is a deep learning model for processing NLP proposed by Google in 2018. It is a bidirectional encoder representation based on Transformer, a pre-trained language representation model that can learn very useful language information and solve various natural language processing tasks such as semantic understanding and information extraction. It is trained on large amounts of unlabeled data to obtain text containing rich semantic information, and then the semantic representation of the text is fine-tuned in a specific NLP task, with strong language representation and feature extraction capabilities. When solving various natural language processing problems, BERT does not need to change the architecture, but only needs to modify a part of the parameters during training.

[0051] In this embodiment, a data index model is constructed, and the data index model is subjected to correlation analysis training to construct a data index map, comprising:

[0052] Metadata information of the data index service is obtained from the data index service, the metadata information including data source, dimension, measure, index definition description, and calculation formula; the dimension refers to category data of the data index, for example, the dimension can be time, organization, region, etc., used to explain the data; the measure refers to numerical data of the data index, for example, the measure can be single vehicle sales amount, sales volume, etc.; the index definition description refers to a textual explanation for introducing the definition of the data index, used to describe and explain the data index when recommending the data index, facilitating user understanding of the meaning of the data index; the calculation formula refers to processing logic for representing calculation of the data index; the data index service is an API form of the data index, and is part of metadata management and service in the data asset system;

[0053] The metadata information is taken as a definition element, and the business domain to which the data source belongs, business process information, and data management information are obtained from the data asset system to construct the data index model; the business process information refers to a process from which the data index is derived, used to evaluate specific corresponding process actions, and the data management information is used for subsequent determination of index access permissions; the data asset system provides global data governance and management capabilities, including data source, metadata, master data, data standard, data quality, data blood relationship, data asset directory, data security, etc., throughout the whole life cycle of managing data;

[0054] The data index model is calculated by the BERT algorithm, the correlations are calculated, the data index correlations are calculated by weighting, and the data index graph is constructed according to the data index correlations; for example, order quantity and inventory quantity have correlations, and the two constitute a correlation in the process; for example, sales and sales same / compared have correlations, and the two constitute a logical correlation;

[0055] Data index reports are collected and analyzed from data index application scenarios (data index application scenarios can be data application products such as reports, screens, mobile terminals, and the like, which information is generally maintained in the blood analysis of the data asset system, and the blood analysis of the index also records this information); according to the use of the data index in the data index report, the data index graph is fine-tuned. For example, brand customer distribution and brand customer trend analysis scenarios often appear together, so the brand customer number and the derived index brand regional customer number correlation are adjusted.

[0056] In this embodiment, according to the user information and the user input information, the business field is identified, and the scene context is supplemented, including:

[0057] User information and user input information are obtained; for example, the user says “I want to see today's deep blue car sales”;

[0058] According to the definition elements of the data index model (i.e., the data source, dimension, measure, index definition description, and calculation formula mentioned above), relevant information is extracted from the user input information by the BERT algorithm, and the relevant information includes dimension information and measure information used to describe the data index; for example, the user says “I want to see today's deep blue car sales”, and the information is extracted by the BERT algorithm as “[{user: I}; {time: today} {brand: deep blue} {measure: sales}]”, wherein the “user” dimension is used to determine whether access permission is available, the “time” and “brand” dimensions are used to conditionally limit the query, and the “sales” is used to locate where to obtain the data;

[0059] According to the dimension information and the measure information, the business field of the data index is identified, the dimension specific information is obtained according to the dimension information and the user information, and the scene context is supplemented. First, the user information is identified, and the user “I” is translated into specific user information, which is translated into “[{user: XXX-specific person}; {field: marketing}; {time: today} {brand: deep blue} {measure: sales}]”, and then the rest of the relevant information is translated, such as “today” into a specific date, i.e., a specific time XXXX year-XX month-XX day. After translation, it is “[{user: XXX-specific person}; {field: marketing}; {time: XXXX year-XX month-XX day} {brand: deep blue} {measure: sales}]”, thereby realizing the supplement of the scene context.

[0060] In this embodiment, the results returned to the user based on the user's access rights include:

[0061] Call the authentication interface in the data permission system to determine whether the user has the indicator access rights based on the user information; the data permission system is a permission management system based on the RBAC model, in which the authorization and authentication of data access are realized;

[0062] If the judgment is no, then the description "No permission, please apply for permission in data field XXX" will be added to the data indicator returned to the user, where "data field XXX" comes from the data management information of the data source obtained from the data asset system;

[0063] If the judgment is yes, the filtered data indicators or the data indicators finally selected by the user are returned to the user.

[0064] in,

[0065] If the judgment is yes, and the number of the screened data indicators with correlation greater than the predetermined threshold is 0, the data indicator finally selected by the user is returned to the user;

[0066] If the judgment is yes, and the number of the filtered data indicators whose correlation is greater than the predetermined threshold is 1, the filtered data indicators are returned to the user;

[0067] If the judgment is yes, and the number of filtered data indicators with correlation greater than the predetermined threshold is greater than 1, the filtered data indicators will be sorted from large to small according to the correlation value, and then the result will be returned to the user for selection.

[0068] In this embodiment, the predetermined threshold is 85%.

[0069] By building data indicator models and data indicator maps, we can identify scenario contexts based on user information and user input information. If the user has permission, we can help the user find indicators quickly and accurately based on user information and user usage scenarios, providing a basis for users to make good use of indicators, which is conducive to improving user satisfaction and the value of data services, and promoting the development of data management and applications.

[0070] like Figure 3 As shown, a data indicator intelligent service recommendation system based on BERT in this embodiment includes:

[0071] The modeling module is used to build a data indicator model, perform correlation analysis and training on the data indicator model, and build a data indicator map;

[0072] The supplementary context module is used to identify the business domain and supplement the scenario context based on user information and user input information;

[0073] The analysis recommendation module is configured to filter data indexes with a correlation greater than a predetermined threshold in a data index map by a BERT algorithm in a scene context, and determine a number of the filtered data indexes;

[0074] The prompt word convergence module is configured to call a prompt word service, guide a user to make a correct selection, record a process and a final selected data index of the user selection, return a result to the user according to a user access authority based on the final selected data index, and correct a correlation analysis training;

[0075] The index access authority control module is configured to call an authentication interface in a data authority system, and determine whether the user has an index access authority according to user information.

[0076] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application rather than limiting the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the purpose and scope of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. A BERT-based data indicator intelligent service recommendation method, characterized in that, Comprise: Build a data index model, and train the data index model to analyze the correlation and build a data index graph; According to the user information and the user input information, identify the business field and supplement the scene context; According to the scene context, filter the data indexes with a correlation greater than a predetermined threshold in the data index graph through the BERT algorithm, and determine the number of filtered data indexes; If the number of filtered data indexes is greater than 0, return the result to the user according to the user access permission; If the number of filtered data indexes is 0, call the prompt word service to guide the user to make the correct selection, record the process and the finally selected data index of the user, return the result to the user according to the user access permission based on the finally selected data index, and correct the correlation analysis training; Wherein, building a data index model, and training the data index model to analyze the correlation and build a data index graph, comprises: Obtain metadata information of a data index service from the data index service, wherein the metadata information includes data source, dimension, measure, index definition description, and calculation formula; Take the metadata information as a definition element, and obtain the business field, business process information, and data management information of the data source from the data asset system to build a data index model; Calculate the correlation of the data index model through the BERT algorithm, weight the correlation to calculate the data index correlation, and build a data index graph according to the data index correlation; Collect and analyze the data index report from the data index application scene, and fine-tune the data index graph according to the use of the data index in the data index report.

2. The BERT-based data index intelligent service recommendation method of claim 1, wherein: According to the user information and the user input information, identify the business field and supplement the scene context, comprises: Obtain user information and user input information; According to the definition elements of the data index model, extract relevant information from the user input information through the BERT algorithm, wherein the relevant information includes dimension information and measure information used to describe the data index; According to the dimension information and the measure information, identify the business field of the data index, obtain the dimension specific information according to the dimension information and the user information, and supplement the scene context.

3. The BERT-based data index intelligent service recommendation method of claim 1, wherein: The dimension refers to the category data of the data index; The measure refers to the numerical data of the data index; The index definition description refers to the textual description used to introduce the definition of the data index; The calculation formula refers to the processing logic used to represent the calculation of the data index.

4. The BERT-based data index intelligent service recommendation method of claim 2, wherein: According to the user access permission, return the result to the user, comprises: Call the authentication interface in the data permission system to determine whether the user has the index access permission according to the user information; If the judgment is no, the data indicator returned to the user is appended with the description "no permission, please apply for permission in data field XXX", where "data field XXX" is derived from the data management information of the data source obtained from the data asset system; If the judgment is yes, the user is returned the filtered data indicator or the data indicator finally selected by the user.

5. The BERT-based data indicator intelligent service recommendation method according to claim 2, characterized in that: The predetermined threshold is 85%.

6. A BERT-based data index intelligent service recommendation system, characterized in that, Comprise: A modeling module for constructing a data indicator model, performing correlation analysis training on the data indicator model, and constructing a data indicator graph; A supplementary context module for identifying a business field and supplementing scene context according to user information and user input information; An analysis recommendation module for filtering data indicators with a correlation greater than a predetermined threshold in the data indicator graph through a BERT algorithm based on the scene context, and determining the number of filtered data indicators; A prompt word convergence module for calling a prompt word service, guiding the user to make a correct selection, recording the process of user selection and the finally selected data indicator, and correcting the correlation analysis training; An indicator access permission control module for calling an authentication interface in a data permission system to determine whether the user has indicator access permission according to user information; The modeling module is specifically configured to: Obtain metadata information of a data indicator service from the data indicator service, wherein the metadata information includes data sources, dimensions, measures, indicator definition descriptions, and calculation formulas; Use the metadata information as a definition element, and obtain the business field, business process information, and data management information of the data source from the data asset system to construct a data indicator model; Calculate the correlation of the data indicator model through a BERT algorithm, weight these correlations to calculate the data indicator correlation, and construct a data indicator graph based on these data indicator correlations; Collect and analyze data indicator reports from data indicator application scenarios, and fine-tune the data indicator graph based on the use of data indicators in the data indicator reports.

Images