Enterprise AI maturity evaluation method based on large language model and knowledge graph

Abstract

The invention discloses an enterprise AI maturity evaluation method based on a large language model and a knowledge graph, and belongs to the technical field of enterprise intelligent management, and the method comprises the steps: obtaining a plurality of evaluation sentences related to artificial intelligence in various public documents in an enterprise in a target time period, performing reasoning prediction on the evaluated sentence by using a pre-trained large language model to generate a plurality of pieces of structured data; wherein in each piece of structured data, the entity is an evaluation sentence or an AI technology category, the relationship is an AI technology category label to which the evaluation sentence belongs or a cooperative relationship between any two evaluation sentences, and the attribute is an enterprise department to which the evaluation sentence belongs or an AI maturity basic score; according to the structured data, based on a knowledge graph construction method, generating a target knowledge graph; and establishing a multi-dimensional evaluation system, calculating according to the target knowledge graph to obtain an AI maturity score of the current enterprise, and generating an enterprise AI maturity evaluation report.

Description

Technical Field

[0001] The present invention belongs to the field of enterprise intelligent management technology, and specifically relates to an enterprise AI maturity assessment method based on a large language model and a knowledge graph. Background Art

[0002] With the widespread adoption of artificial intelligence (AI) technology in enterprises, assessing their AI maturity has become crucial. Accurate assessments not only help companies understand the current level of AI application but also provide a basis for subsequent AI strategic planning and improvement. Traditional assessment methods often rely on manual questionnaires and expert interviews. These methods are inherently subjective, and the results are easily influenced by the evaluator's personal experience and preferences, resulting in inaccurate and subjective results. Furthermore, manual questionnaires and expert interviews are inefficient and require significant time and effort.

[0003] To address the shortcomings of traditional evaluation methods, existing technologies have proposed an automatic evaluation method that evaluates the maturity of an enterprise's AI applications in its business by analyzing its business data. However, these methods mainly focus on the statistical characteristics of the data, such as data volume and data quality, and lack an understanding of the deep semantic information contained in the data, making it difficult to comprehensively evaluate the actual situation of an enterprise's AI applications. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an enterprise AI maturity assessment method based on a large language model and knowledge graph to solve the above technical problems.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for assessing enterprise AI maturity based on a large language model and knowledge graph, including:

[0007] Obtain multiple AI-related evaluation sentences from public documents within the enterprise during the target time period, and use a pre-trained large language model to infer and predict the evaluation sentences to generate multiple structured data. In each structured data, the entity is the evaluation sentence or AI technology category, the relationship is the AI technology category label to which the evaluation sentence belongs or the collaborative relationship between any two evaluation sentences, and the attribute is the enterprise department to which the evaluation sentence belongs or the basic AI maturity score.

[0008] Generate target knowledge graph based on structured data and knowledge graph construction method;

[0009] Establish a multi-dimensional evaluation system, calculate the current enterprise's AI maturity score based on the target knowledge graph, and generate an enterprise AI maturity evaluation report.

[0010] Furthermore, multiple evaluation sentences related to artificial intelligence are obtained from various public documents within the enterprise during the target time period, including: obtaining the full-text content of various public documents within the enterprise during the target time period for data preprocessing to obtain several independent target sentences; screening several independent target sentences according to preset AI keywords to obtain multiple evaluation sentences related to artificial intelligence.

[0011] Furthermore, we use the pre-trained large language model to perform inference and prediction on the evaluation sentences, generating multiple structured data, including:

[0012] Obtain a pre-trained large language model and multiple evaluation sentences; the large language model is the BERT model;

[0013] Using a large language model, deep learning and supervised fine-tuning are performed on each evaluation sentence. Through sentence-by-sentence reasoning and prediction, the AI technology category label for each evaluation sentence is obtained, and the corporate department to which the corresponding public document for each evaluation sentence belongs is obtained to generate first structured data. In each first structured data, the entity is the evaluation sentence, the relationship is the AI technology category label to which the evaluation sentence belongs, and the attribute is the corporate department to which the evaluation sentence belongs.

[0014] Obtaining an AI technology category label, generating multiple AI technology categories, and obtaining a user-defined AI maturity base score for each AI technology category to generate second structured data; wherein, in each second structured data, the entity is the AI technology category, the relationship is the AI technology category label to which the AI technology category belongs, and the attribute is the AI maturity base score;

[0015] Acquire the collaborative relationships between different enterprise departments, calculate the data correlation between each two evaluation sentences in a collaborative relationship, determine the collaborative relationship between each evaluation sentence and other evaluation sentences, and generate third structured data; wherein, in each third structured data, the entity is the evaluation sentence, the relationship is the collaborative relationship between any two evaluation sentences, and the attribute is the enterprise department to which the evaluation sentence belongs;

[0016] The first structured data, the second structured data, and the third structured data are fused to obtain a plurality of structured data.

[0017] Furthermore, based on the collaborative relationships between different enterprise departments, the data correlation between each two evaluation sentences with collaborative relationships is calculated to determine the collaborative relationship between each evaluation sentence and other evaluation sentences, including:

[0018] Obtain the collaborative relationship between different corporate departments within the enterprise through the internal process processing logic;

[0019] Get any two evaluation sentences and determine whether there is a collaborative relationship between the enterprise departments they belong to;

[0020] If so, extract the collaboration keywords from the two evaluation sentences and calculate the text similarity between the two evaluation sentences after removing the collaboration keywords and conjunctions; collaboration keywords include department name, business document number, and business type.

[0021] Calculate the data correlation between the two current evaluation sentences based on text similarity and collaborative keywords:

[0022]

[0023] Among them, K i,j represents the data association between the i-th evaluation sentence and the j-th evaluation sentence, G represents the text similarity between the i-th evaluation sentence and the j-th evaluation sentence after removing the collaborative keywords and conjunctions, b represents the number of collaborative keywords, and f c Indicates whether there is a correlation between the i-th evaluation sentence and the c-th collaborative keyword extracted from the j-th evaluation sentence, f c is 0 or 1, Indicates the preset increase corresponding to the cth collaborative keyword, When f c =0,

[0024] A preset correlation threshold is obtained, and if the data correlation between the two current evaluation sentences is greater than the preset correlation threshold, it is determined that a collaborative relationship exists between the two current evaluation sentences.

[0025] Furthermore, a method for assessing enterprise AI maturity based on a large language model and knowledge graph also includes:

[0026] When the cth collaborative keyword is the department name, f c The values include:

[0027] If the department name corresponding to the cth collaborative keyword of the i-th evaluation sentence is the same as the enterprise department of the j-th evaluation sentence, or the department name corresponding to the cth collaborative keyword of the j-th evaluation sentence is the same as the enterprise department of the i-th evaluation sentence, f c =1;

[0028] If the department name corresponding to the cth collaborative keyword of the i-th evaluation sentence is different from the enterprise department of the j-th evaluation sentence, or the department name corresponding to the cth collaborative keyword of the j-th evaluation sentence is different from the enterprise department of the i-th evaluation sentence, f c =0;

[0029] When the cth collaborative keyword is not a department name, then fc The value of is:

[0030]

[0031] Among them, t i,c is the cth collaborative keyword in the i-th evaluation sentence, t j,c The cth collaborative keyword in the jth evaluation sentence.

[0032] Furthermore, a multi-dimensional evaluation system is established to calculate the current enterprise's AI maturity score based on the target knowledge graph, including:

[0033] Obtain AI technology categories and determine multiple evaluation dimensions;

[0034] A multi-dimensional evaluation system is established based on multiple evaluation dimensions; the evaluation method of the multi-dimensional evaluation system is as follows:

[0035] Obtain a chain structure of any AI technology category connected by an undirected edge of the target knowledge graph; wherein the entity at the initial point of the chain structure is the AI technology category, the entity of each intermediate node is an evaluation sentence, and the last node is connected to only one node via an undirected edge; or the relationship of the structured data corresponding to the last node is a collaborative relationship, and the relationship of the structured data corresponding to the next node connected to the last node via an undirected edge is the AI technology category label to which the AI technology category belongs;

[0036] Obtain the actual number of evaluation sentences contained in all chain structures corresponding to the target AI technology category;

[0037] and obtaining the standard number of evaluation sentences that should be included in the corresponding AI technology category within the target time period;

[0038] Calculate the ratio of the actual number of target AI technology categories to the standard number, and obtain the degree of enterprise use of the corresponding AI technology category during the target time period as the AI participation weight;

[0039] The AI participation weight and the corresponding AI maturity base score are multiplied to obtain the independent score of the target AI technology category.

[0040] Obtain the independent scores of all target AI technology categories and sum them up to obtain the current enterprise's AI maturity score.

[0041] Furthermore, an enterprise AI maturity assessment report is generated, including: obtaining the current enterprise's AI maturity score, obtaining an independent score for each target AI technology category, and calling a chain structure associated with each AI technology category through an undirected edge of the target knowledge graph, and generating an enterprise AI maturity assessment report through a preset report template.

[0042] The beneficial effects of the present invention are:

[0043] This paper proposes a method for assessing enterprise AI maturity based on a large language model and knowledge graph. Through the powerful natural language understanding and generation capabilities of the large language model, it can deeply analyze the semantic information in text data such as enterprise documents and communication records, and mine key knowledge related to AI applications. It also organizes this knowledge in a structured manner through knowledge graph technology to construct an enterprise AI knowledge graph, which clearly displays all aspects of enterprise AI applications and their interrelationships. Finally, the enterprise AI knowledge graph is used to comprehensively and objectively evaluate the enterprise AI maturity, overcoming the subjectivity and limitations of traditional methods.

[0044] Other advantages, objectives, and features of the present invention will be described in the following description and will be apparent to those skilled in the art to some extent, or may be taught by those skilled in the art from the practice of the present invention. The purposes and other advantages of the present invention may be realized and obtained through the structures particularly pointed out in the written description and the accompanying drawings.

[0045] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0047] Figure 1 This is a flow chart of a method for assessing enterprise AI maturity based on a large language model and knowledge graph in an embodiment of the present invention;

[0048] Figure 2 This is a flow chart of an evaluation method for a multidimensional evaluation system in an enterprise AI maturity evaluation method based on a large language model and knowledge graph in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0050] like Figure 1 As shown, the present invention proposes an enterprise AI maturity assessment method based on a large language model and a knowledge graph, comprising:

[0051] S101. Obtain multiple AI-related evaluation sentences from various public documents within the enterprise during the target time period, and use a pre-trained large language model to infer and predict the evaluation sentences to generate multiple structured data. In each structured data, the entity is an evaluation sentence or an AI technology category, the relationship is the AI technology category label to which the evaluation sentence belongs or the collaborative relationship between any two evaluation sentences, and the attribute is the enterprise department to which the evaluation sentence belongs or the basic AI maturity score.

[0052] S102. Generate a target knowledge graph based on the structured data and the knowledge graph construction method;

[0053] S103. Establish a multi-dimensional evaluation system, calculate the current enterprise's AI maturity score based on the target knowledge graph, and generate an enterprise AI maturity assessment report;

[0054] The working principle of the above technical solution is: through machine tagging and other technical means, all evaluation sentences closely related to artificial intelligence in various public documents within the enterprise during the target time period are obtained, and a pre-trained large language model is used to infer and predict each evaluation sentence obtained to obtain multiple structured data, wherein each structured data is composed of (entity, relationship, attribute), and the entity of the obtained structured data is any one of the evaluation sentences or AI technology categories, the relationship is the AI technology category label to which the evaluation sentence belongs or any one of the collaborative relationships between any two evaluation sentences, and the attribute is any one of the enterprise departments to which the evaluation sentence belongs or the basic score of AI maturity; then, according to the structured data, based on the knowledge graph construction method, a target knowledge graph is generated. Since the acquisition method of several related structured data is described above, Large language models are used for auxiliary generation, so the knowledge graph construction method described in this application usually only needs to go through two stages: knowledge fusion and knowledge calculation, and it is iteratively updated according to the continuously acquired structured data. The more specific target knowledge graph construction method belongs to the common knowledge of those skilled in the art and will not be described here. It is worth noting that when forming structured data, it is necessary to form the structured data into a knowledge expression that conforms to the ontology of the pre-built target knowledge graph according to the expression format of the pre-built target knowledge graph; finally, a multi-dimensional evaluation system is established based on the AI technology category, and the AI maturity score of the current enterprise is calculated based on the generated target knowledge graph. Furthermore, a corresponding enterprise AI maturity assessment report is generated and fed back to the user to assist it in understanding the AI application maturity of the current enterprise at various stages and business processes in the target time period;

[0055] The beneficial effects of the above technical solution are: through the above technical solution, the powerful natural language understanding and generation capabilities of the large language model are utilized to deeply analyze the semantic information in text data such as corporate documents and communication records, and to mine key knowledge related to AI applications. This knowledge is then organized in a structured manner through knowledge graph technology to construct an enterprise AI knowledge graph, which clearly displays all aspects of enterprise AI applications and their interrelationships. Finally, the enterprise AI knowledge graph is used to comprehensively and objectively evaluate the maturity of enterprise AI, overcoming the subjectivity and limitations of traditional methods.

[0056] In one embodiment, obtaining multiple AI-related evaluation sentences from various public documents within an enterprise within a target time period includes: obtaining the full text of various public documents within the enterprise within the target time period, performing data preprocessing, and obtaining multiple independent target sentences; screening the multiple independent target sentences according to preset AI keywords to obtain multiple AI-related evaluation sentences;

[0057] The working principle of the above technical solution is as follows: the full text content of various public documents within the enterprise during the target time period is obtained and preprocessed to obtain several independent target sentences. The data preprocessing includes text data cleaning and natural language extraction operations. Then, these independent target sentences are screened according to preset AI keywords to obtain multiple AI-related evaluation sentences. Specifically, machine annotation technology is used to construct an AI vocabulary library using AI keywords for subsequent screening.

[0058] The beneficial effects of the above technical solution are: through the above technical solution, multiple evaluation sentences related to artificial intelligence are collected for evaluating the maturity of AI using the evaluation sentences. At the same time, the above method of using machine annotation technology to build an AI vocabulary to screen the evaluation sentences is simpler and more efficient in operation than the traditional direct recognition method, and can more accurately realize the screening and extraction of evaluation sentences.

[0059] In one embodiment, a pre-trained large language model is used to perform inference and prediction on the evaluation sentence to generate multiple structured data, including:

[0060] Obtain a pre-trained large language model and multiple evaluation sentences; the large language model is the BERT model;

[0061] Using a large language model, deep learning and supervised fine-tuning are performed on each evaluation sentence. Through sentence-by-sentence reasoning and prediction, the AI technology category label for each evaluation sentence is obtained, and the corporate department to which the corresponding public document for each evaluation sentence belongs is obtained to generate first structured data. In each first structured data, the entity is the evaluation sentence, the relationship is the AI technology category label to which the evaluation sentence belongs, and the attribute is the corporate department to which the evaluation sentence belongs.

[0062] Obtaining an AI technology category label, generating multiple AI technology categories, and obtaining a user-defined AI maturity base score for each AI technology category to generate second structured data; wherein, in each second structured data, the entity is the AI technology category, the relationship is the AI technology category label to which the AI technology category belongs, and the attribute is the AI maturity base score;

[0063] Acquire the collaborative relationships between different enterprise departments, calculate the data correlation between each two evaluation sentences in a collaborative relationship, determine the collaborative relationship between each evaluation sentence and other evaluation sentences, and generate third structured data; wherein, in each third structured data, the entity is the evaluation sentence, the relationship is the collaborative relationship between any two evaluation sentences, and the attribute is the enterprise department to which the evaluation sentence belongs;

[0064] Performing data fusion on the first structured data, the second structured data, and the third structured data to obtain a plurality of structured data;

[0065] The working principle and beneficial effects of the above technical solution are: obtaining a pre-trained large language model and multiple evaluation sentences, where the multiple evaluation sentences refer to all the evaluation sentences closely related to AI obtained above. The use of multiple evaluation sentences is to be consistent with the expression of the above technical features. At the same time, the technical features expressed above also refer to all the evaluation sentences closely related to AI obtained; in addition, since the BERT model is a classic model in the field of natural language processing, it can have bidirectional encoding and understanding capabilities through unsupervised learning on a large-scale text corpus. This feature enables the BERT model to quickly adapt to specific task requirements with only simple fine-tuning. Therefore, the large language model used in this application is preferably the BERT model;

[0066] After obtaining the corresponding data, the large language model is used to perform deep learning and supervised fine-tuning on each evaluation sentence. Through sentence-by-sentence reasoning and prediction, the AI technology category label of each evaluation sentence is obtained, and the corporate department to which the corresponding public document of each evaluation sentence belongs is obtained to generate first structured data. In each first structured data, the entity is the evaluation sentence, the relationship is the AI technology category label to which the evaluation sentence belongs, and the attribute is the corporate department to which the evaluation sentence belongs.

[0067] In addition, based on the AI technology category labels involved in the first structured data, multiple AI technology categories are generated, and the AI maturity basic scores for the relevant AI technology categories pre-defined by the user are obtained to generate second structured data; wherein, in each second structured data, the entity is the AI technology category, the relationship is the AI technology category label to which the AI technology category belongs, and the attribute is the AI maturity basic score;

[0068] Since the large language model is limited by the training accuracy of the large language model when inferring the AI technology category label attribution for all evaluation sentences, it may result in the inability to accurately infer the AI technology category label attribution for a small number of evaluation sentences. Therefore, for this part of the data, this application calculates the data correlation between each two evaluation sentences with a collaborative relationship based on the collaborative relationship between different enterprise departments, determines the collaborative relationship between each evaluation sentence and other evaluation sentences, and generates third structured data; wherein, in each third structured data, the entity is the evaluation sentence, the relationship is the collaborative relationship between any two evaluation sentences, and the attribute is the enterprise department to which the evaluation sentence belongs; it is worth noting that in the third structured data, the same entity includes multiple collaborative relationships;

[0069] The method includes calculating the data correlation between each two evaluation sentences with a collaborative relationship based on the collaborative relationship between different enterprise departments, and determining the collaborative relationship between each evaluation sentence and other evaluation sentences, including:

[0070] Obtain the collaborative relationship between different corporate departments within the enterprise through the internal process processing logic;

[0071] Get any two evaluation sentences and determine whether there is a collaborative relationship between the enterprise departments they belong to;

[0072] If so, extract the collaboration keywords from the two evaluation sentences and calculate the text similarity between the two evaluation sentences after removing the collaboration keywords and conjunctions; collaboration keywords include department name, business document number, and business type.

[0073] Calculate the data correlation between the two current evaluation sentences based on text similarity and collaborative keywords:

[0074]

[0075] Among them, K i,j represents the data association between the i-th evaluation sentence and the j-th evaluation sentence, G represents the text similarity between the i-th evaluation sentence and the j-th evaluation sentence after removing the collaborative keywords and conjunctions, b represents the number of collaborative keywords, and f c Indicates whether there is a correlation between the i-th evaluation sentence and the c-th collaborative keyword extracted from the j-th evaluation sentence, f c is 0 or 1, Indicates the preset increase corresponding to the cth collaborative keyword, When f c =0, Replaced by the original default increase

[0076] Indicates that when f c ≠0, When f c =0,

[0077] For f c If the cth collaboration keyword is a department name, there are two cases:

[0078] 1. The department name corresponding to the cth collaborative keyword of the i-th evaluation sentence is the same as the enterprise department of the j-th evaluation sentence, or the department name corresponding to the cth collaborative keyword of the j-th evaluation sentence is the same as the enterprise department of the i-th evaluation sentence, f c =1;

[0079] Second, the department name corresponding to the cth collaborative keyword of the i-th evaluation sentence is different from the enterprise department of the j-th evaluation sentence, or the department name corresponding to the cth collaborative keyword of the j-th evaluation sentence is different from the enterprise department of the i-th evaluation sentence, f c =0;

[0080] If the cth collaboration keyword is not a department name, then f c The value of is:

[0081]

[0082] Among them, t i,c is the cth collaborative keyword in the i-th evaluation sentence, t j,c The cth collaborative keyword in the jth evaluation sentence;

[0083] Obtaining a preset correlation threshold, and if the data correlation between the two current evaluation sentences is greater than the preset correlation threshold, determining that there is a collaborative relationship between the two current evaluation sentences;

[0084] Finally, the first structured data, the second structured data and the third structured data are fused to obtain multiple structured data. It is worth noting that if in a structured data, the entity is an evaluation sentence, and the relationship includes the AI technology category label to which the evaluation sentence belongs or the collaborative relationship between any two evaluation sentences, then the AI technology category label to which the evaluation sentence belongs is mainly used to optimize the structured data so that each structured data contains only one entity, one relationship and one attribute, which is convenient for the subsequent AI maturity assessment based on the target knowledge graph constructed according to the structured data.

[0085] like Figure 2 As shown, in one embodiment, a multi-dimensional evaluation system is established to calculate the AI maturity score of the current enterprise based on the target knowledge graph, including:

[0086] Obtain AI technology categories and determine multiple evaluation dimensions;

[0087] A multi-dimensional evaluation system is established based on multiple evaluation dimensions; the evaluation method of the multi-dimensional evaluation system is as follows:

[0088] S201. Obtain a chain structure of any AI technology category connected by an undirected edge of the target knowledge graph; wherein the entity at the initial point of the chain structure is the AI technology category, the entity of each intermediate node is an evaluation sentence, and the last node is connected to only one node via an undirected edge; or the relationship of the structured data corresponding to the last node is a collaborative relationship, and the relationship of the structured data corresponding to the next node connected to the last node via an undirected edge is the AI technology category label to which the AI technology category belongs;

[0089] S202. Obtain the actual number of evaluation sentences contained in all chain structures corresponding to the target AI technology category;

[0090] S203, and obtaining the standard number of evaluation sentences that should be included in the corresponding AI technology category within the target time period;

[0091] S204. Calculate the ratio of the actual number of target AI technology categories to the standard number, and obtain the degree of enterprise use of the corresponding AI technology category within the target time period as the AI participation weight;

[0092] S205. Obtain an independent score for the target AI technology category by multiplying the AI participation weight and the corresponding AI maturity base score.

[0093] S206. Obtain and sum the independent scores of all target AI technology categories to obtain the current enterprise's AI maturity score;

[0094] The working principle and beneficial effects of the above technical solution are as follows: obtaining AI technology categories and determining multiple evaluation dimensions. For example, if the AI technology categories obtained by the current enterprise through inference and prediction are four categories, then four evaluation dimensions are determined. The name of each evaluation dimension is the name of the corresponding AI technology category, such as data regression technology category, cluster analysis technology category, rule reasoning technology category, etc., and a multidimensional evaluation system is established based on the evaluation dimensions; among which, the evaluation method of the multidimensional evaluation system is:

[0095] Obtain a chain structure of any AI technology category connected by an undirected edge of the target knowledge graph; wherein the entity at the initial point of the chain structure is the AI technology category, the entity of each intermediate node is an evaluation sentence, and the last node is connected to only one node via an undirected edge; or the relationship of the structured data corresponding to the last node is a collaborative relationship, and the relationship of the structured data corresponding to the next node connected to the last node via an undirected edge is the AI technology category label to which the AI technology category belongs;

[0096] It is worth noting that the connecting lines in the chain structure associated with each AI technology category through the undirected edges of the target knowledge graph are undirected edges in the target knowledge graph. There are three cases in total:

[0097] 1. Take any AI technology category as the initial point of the chain structure, obtain the node associated with it through an undirected edge as the second node (wherein, the relationship between the second node and the structured data must be the AI technology category label), and obtain the node associated with the second node through an undirected edge as the third node (wherein, the relationship between the third node and the structured data must be a collaborative relationship), and then obtain the node associated with the third node through an undirected edge as the fourth node, and judge the relationship between the fourth node and the structured data. If the relationship between the structured data of the fourth node is the AI technology category label, then use the third node as the last node to obtain the chain structure of the current AI technology category, where the obtained evaluation sentence entity is obtained from the second node and the third node; if the relationship between the structured data of the fourth node is a collaborative relationship, then continue to obtain the node associated with the fourth node through an undirected edge as the fifth node, and repeat the above relationship judgment until the last node is determined, and the chain structure of the current AI technology category is obtained;

[0098] Continuing with the judgment in Case 1, if the relationship of the structured data of the fifth node is still a collaborative relationship, and the fifth node has no undirected edge connections with other nodes except the fourth node, the fifth node is used as the last node to obtain the chain structure of the current AI technology category;

[0099] 3. Continuation Case 1: If the second node has no undirected edge connections with any other nodes except the initial node, the chain structure of the current AI technology category is obtained by taking the second node as the last node.

[0100] In addition, in each chain structure, if multiple evaluation sentences have the same AI technology category label, there can be multiple evaluation sentences connected to the initial point, thus forming a multi-chain structure;

[0101] The chain structure generated by the above technical solution takes into account the degree of participation of the same entity (evaluation sentence) when it has a collaborative relationship with other entities and participates in the business processing of two or more AI technology categories. Compared with the existing single-link judgment method for participation degree, the multi-dimensional evaluation system proposed in this application is more accurate in AI maturity assessment;

[0102] Obtain the actual number of evaluation sentences contained in all chain structures corresponding to the target AI technology category; that is, the actual number of all evaluation sentences contained in the multi-chain structure;

[0103] And obtain the standard number of evaluation sentences that should be included in the corresponding AI technology category within the target time period; the standard number is a calibrated number determined in advance based on market research data or a user-defined target number;

[0104] Calculate the ratio of the actual number of target AI technology categories to the standard number, and obtain the degree of enterprise use of the corresponding AI technology category during the target time period as the AI participation weight;

[0105] Then, based on the AI participation weight and the corresponding AI maturity base score, the independent score of the target AI technology category is obtained through multiplication.

[0106] Finally, obtain the independent scores of all target AI technology categories and sum them to obtain the current enterprise's AI maturity score:

[0107]

[0108] Among them, P is the current AI maturity score of the enterprise, D n represents the independent score of the nth target AI technology category, m represents the number of dimensions in the multidimensional evaluation system, that is, the number of AI technology categories, represents the AI participation weight in the nth AI technology category, x n represents the actual number of all evaluation sentences contained in the chain structure corresponding to the nth AI technology category, y n represents the pre-set standard number of evaluation sentences that the nth AI technology category should contain within the target time period, It represents the basic AI maturity score corresponding to the nth AI technology category. Through the above technical solution, the relationship between entities in the target knowledge graph is utilized to comprehensively and objectively evaluate the AI maturity of the enterprise, overcoming the subjectivity and limitations of traditional methods.

[0109] In one embodiment, generating an enterprise AI maturity assessment report includes: obtaining the current enterprise's AI maturity score, obtaining an independent score for each target AI technology category, and invoking a chain structure associated with each AI technology category through an undirected edge of a target knowledge graph, and generating the enterprise AI maturity assessment report using a preset report template;

[0110] The working principle and beneficial effects of the above technical solution are: obtaining the AI maturity score of the current enterprise, and obtaining an independent score for each target AI technology category, which is used to intuitively reflect the current enterprise's overall AI maturity and the application performance of each AI technology category in the current enterprise. At the same time, by calling the chain structure associated with each AI technology category through the undirected edge of the target knowledge graph, it reflects the business data and enterprise departments and other important information associated with each AI technology category in the current enterprise, and provides implementation direction for subsequent help to enterprises in promoting AI development. Then, through the preset report template, the above content is filled in with data to generate an enterprise AI maturity assessment report, which helps managers to have a deeper understanding of the current enterprise's external business processing and internal work negotiations in the target time period. When there is a multi-chain structure, the data filling of the multi-chain structure in the preset report template is preferably visualized using the knowledge graph display method.

[0111] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A method for assessing enterprise AI maturity based on a large language model and knowledge graph, characterized by: include: Obtain multiple AI-related evaluation sentences from public documents within the enterprise during the target time period, and use a pre-trained large language model to infer and predict the evaluation sentences to generate multiple structured data. In each structured data, the entity is the evaluation sentence or AI technology category, the relationship is the AI technology category label to which the evaluation sentence belongs or the collaborative relationship between any two evaluation sentences, and the attribute is the enterprise department to which the evaluation sentence belongs or the basic AI maturity score. Generate target knowledge graph based on structured data and knowledge graph construction method; Establish a multi-dimensional evaluation system, calculate the current enterprise's AI maturity score based on the target knowledge graph, and generate an enterprise AI maturity evaluation report.

2. The enterprise AI maturity assessment method based on a large language model and knowledge graph according to claim 1 is characterized in that: Obtain multiple evaluation sentences related to artificial intelligence from various public documents within the enterprise within the target time period, including: obtaining the full text content of various public documents within the enterprise within the target time period for data preprocessing to obtain several independent target sentences; screening several independent target sentences according to preset AI keywords to obtain multiple evaluation sentences related to artificial intelligence.

3. The enterprise AI maturity assessment method based on a large language model and knowledge graph according to claim 1 is characterized in that: Use the pre-trained large language model to perform inference and prediction on the evaluation sentences to generate multiple structured data, including: Obtain a pre-trained large language model and multiple evaluation sentences; the large language model is the BERT model; Using a large language model, deep learning and supervised fine-tuning are performed on each evaluation sentence. Through sentence-by-sentence reasoning and prediction, the AI technology category label for each evaluation sentence is obtained, and the corporate department to which the corresponding public document for each evaluation sentence belongs is obtained to generate first structured data. In each first structured data, the entity is the evaluation sentence, the relationship is the AI technology category label to which the evaluation sentence belongs, and the attribute is the corporate department to which the evaluation sentence belongs. Obtaining an AI technology category label, generating multiple AI technology categories, and obtaining a user-defined AI maturity base score for each AI technology category to generate second structured data; wherein, in each second structured data, the entity is the AI technology category, the relationship is the AI technology category label to which the AI technology category belongs, and the attribute is the AI maturity base score; Acquire the collaborative relationships between different enterprise departments, calculate the data correlation between each two evaluation sentences in a collaborative relationship, determine the collaborative relationship between each evaluation sentence and other evaluation sentences, and generate third structured data; wherein, in each third structured data, the entity is the evaluation sentence, the relationship is the collaborative relationship between any two evaluation sentences, and the attribute is the enterprise department to which the evaluation sentence belongs; The first structured data, the second structured data, and the third structured data are fused to obtain a plurality of structured data.

4. The enterprise AI maturity assessment method based on a large language model and knowledge graph according to claim 3 is characterized in that: Based on the collaborative relationships between different enterprise departments, the data correlation between each two evaluation sentences with collaborative relationships is calculated, and the collaborative relationship between each evaluation sentence and other evaluation sentences is determined, including: Obtain the collaborative relationship between different corporate departments within the enterprise through the internal process processing logic; Get any two evaluation sentences and determine whether there is a collaborative relationship between the enterprise departments they belong to; If so, extract the collaboration keywords from the two evaluation sentences and calculate the text similarity between the two evaluation sentences after removing the collaboration keywords and conjunctions; collaboration keywords include department name, business document number, and business type. Calculate the data correlation between the two current evaluation sentences based on text similarity and collaborative keywords: Among them, K i,j represents the data association between the i-th evaluation sentence and the j-th evaluation sentence, G represents the text similarity between the i-th evaluation sentence and the j-th evaluation sentence after removing the collaborative keywords and conjunctions, b represents the number of collaborative keywords, and f c Indicates whether there is a correlation between the i-th evaluation sentence and the c-th collaborative keyword extracted from the j-th evaluation sentence, f c is 0 or 1, Indicates the preset increase corresponding to the cth collaborative keyword, When f c =0, A preset correlation threshold is obtained, and if the data correlation between the two current evaluation sentences is greater than the preset correlation threshold, it is determined that a collaborative relationship exists between the two current evaluation sentences.

5. The enterprise AI maturity assessment method based on a large language model and knowledge graph according to claim 4 is characterized in that: When the cth collaborative keyword is the department name, f c The values include: If the department name corresponding to the cth collaborative keyword of the i-th evaluation sentence is the same as the enterprise department of the j-th evaluation sentence, or the department name corresponding to the cth collaborative keyword of the j-th evaluation sentence is the same as the enterprise department of the i-th evaluation sentence, f c =1; If the department name corresponding to the cth collaborative keyword of the i-th evaluation sentence is different from the enterprise department of the j-th evaluation sentence, or the department name corresponding to the cth collaborative keyword of the j-th evaluation sentence is different from the enterprise department of the i-th evaluation sentence, f c =0; When the cth collaborative keyword is not a department name, then f c The value of is: Among them, t i,c is the cth collaborative keyword in the i-th evaluation sentence, t j,c The cth collaborative keyword in the jth evaluation sentence.

6. The enterprise AI maturity assessment method based on a large language model and knowledge graph according to claim 1 is characterized in that: Establish a multi-dimensional evaluation system to calculate the current enterprise's AI maturity score based on the target knowledge graph, including: Obtain AI technology categories and determine multiple evaluation dimensions; A multi-dimensional evaluation system is established based on multiple evaluation dimensions; the evaluation method of the multi-dimensional evaluation system is as follows: Obtain a chain structure of any AI technology category connected by an undirected edge of the target knowledge graph; wherein the entity at the initial point of the chain structure is the AI technology category, the entity of each intermediate node is an evaluation sentence, and the last node is connected to only one node via an undirected edge; or the relationship of the structured data corresponding to the last node is a collaborative relationship, and the relationship of the structured data corresponding to the next node connected to the last node via an undirected edge is the AI technology category label to which the AI technology category belongs; Obtain the actual number of evaluation sentences contained in all chain structures corresponding to the target AI technology category; and obtaining the standard number of evaluation sentences that should be included in the corresponding AI technology category within the target time period; Calculate the ratio of the actual number of target AI technology categories to the standard number, and obtain the degree of enterprise use of the corresponding AI technology category during the target time period as the AI participation weight; The AI participation weight and the corresponding AI maturity base score are multiplied to obtain the independent score of the target AI technology category. Obtain the independent scores of all target AI technology categories and sum them up to obtain the current enterprise's AI maturity score.

7. The enterprise AI maturity assessment method based on a large language model and knowledge graph according to claim 1 is characterized in that: Generate an enterprise AI maturity assessment report, including: obtaining the current enterprise's AI maturity score, obtaining an independent score for each target AI technology category, and calling a chain structure associated with each AI technology category through undirected edges of the target knowledge graph, and generating an enterprise AI maturity assessment report through a preset report template.

Images