Enterprise digital transformation degree measurement method, device, equipment, medium and product

Abstract

The invention discloses an enterprise digital transformation degree measurement method and device, equipment, a medium and a product, and relates to the field of enterprise information technology and data analysis, and the method comprises the steps: segmenting an enterprise annual report of each enterprise into a plurality of chapters; a part of chapters and sections are extracted, and the grade value of the to-be-evaluated enterprise digital transformation index corresponding to each chapter and section is manually labeled to obtain a labeled data set; determining an optimal task execution strategy and an optimal knowledge excitation mode according to the annotation data set; generating an instruction type cue word according to the optimal task execution strategy and the optimal knowledge excitation mode; and inputting the instruction type cue word and each remaining chapter of the to-be-measured enterprise into a large language model to obtain a grade value of each digital transformation index of the to-be-evaluated enterprise corresponding to each chapter, and completing the measurement of the digital transformation degree of the to-be-measured enterprise. According to the method and the device, the problems of shallow semantic understanding, single measurement dimension and limited flexibility and expansibility can be solved.

Description

Technical Field

[0001] This application relates to the field of enterprise information technology and data analysis, and in particular to a method, apparatus, equipment, medium and product for measuring the degree of enterprise digital transformation. Background Technology

[0002] As digital technology becomes a core driver of enterprise development, accurately and effectively measuring the progress of enterprise digital transformation has become an important issue in business practice. The results of enterprise digital transformation measurement play an indispensable role in areas such as strategic decision-making, investor decision-making, and industry competitive analysis.

[0003] Corporate strategic decision-making basis: When formulating a digital transformation strategy, corporate managers need to clearly understand their current transformation stage, strengths, and weaknesses. Precise quantitative evaluation can provide companies with detailed diagnostic reports, helping them clarify their transformation direction, rationally allocate resources, formulate targeted transformation strategies, avoid blind investment and decision-making errors, and improve the success rate of transformation.

[0004] Investment Decision Reference: For investors, the degree of a company's digital transformation is a key indicator for assessing its future growth potential and investment value. Accurate quantitative evaluation results can help investors screen out companies with high growth potential and competitiveness, enabling them to make more informed investment decisions.

[0005] Industry competition analysis: Quantitative evaluation helps companies make horizontal comparisons within the industry, understanding their own position and competitiveness. By comparing their digital transformation indicators with those of leading companies in the same industry, companies can identify gaps, learn from best practices, formulate differentiated competitive strategies, and enhance their competitive advantage within the industry.

[0006] Currently, existing technical solutions for measuring the degree of enterprise digital transformation mainly fall into the following categories, and all of them have certain limitations: (1) Measurement methods based on single indicators or comprehensive evaluation indicators. For example, the digitalization level can be indirectly reflected by calculating the proportion of digital assets to total assets, or by constructing a multi-dimensional indicator system and manually setting weights for comprehensive scoring. Data is readily available for this method, but the selection of indicators and the setting of weights are highly subjective.

[0007] (2) Frequency-based text analysis. This method measures the degree of transformation by constructing a digital technology keyword dictionary and statistically analyzing the word frequency in publicly available documents such as corporate annual reports. This is the mainstream method in current empirical research. However, this method has obvious technical bottlenecks: First, it relies heavily on a pre-set keyword dictionary and cannot identify semantically related expressions that are not in the dictionary, leading to semantic recognition bias and missed judgments; Second, it can only make a "presence" judgment of keywords and cannot understand the context, making it prone to misjudgment due to fragmented text (such as "we have not invested heavily in cloud computing"); Third, it is essentially a binary judgment of "existence or non-existence" and cannot quantify the depth and effectiveness of technology application.

[0008] (3) Sentence classification based on supervised machine learning models. To overcome the limitations of traditional text analysis, recent studies have attempted to use pre-trained models for supervised sentence classification. This method has improved the accuracy of semantic understanding, but its model training and recognition process still does not completely break free from the path dependence of the "dictionary-style label system". More importantly, this method is essentially still framed as a binary classification task, and its output is limited to determining whether a certain technology is "mentioned", rather than making a breakthrough in measuring the "degree of application" and "effect" of the technology in enterprises.

[0009] In summary, existing technical solutions face the following key technical bottlenecks in addressing the core issue of measuring the degree of enterprise digital transformation: 1. Superficial semantic understanding: Traditional methods rely on surface keyword matching and lack the ability to reason about the deep semantics and contextual logic of text, resulting in insufficient accuracy and robustness of recognition results.

[0010] 2. Limited Dimensions of Measurement: Existing methods generally only achieve binary judgments of "yes" and "no", without conducting continuous and hierarchical quantitative assessments of "application depth", which is crucial in enterprise digital transformation. This limits the understanding of the complex connotations and multi-dimensional levels of enterprise digital transformation.

[0011] 3. Limited flexibility and scalability: Methods based on fixed dictionaries or labeling systems are difficult to dynamically adapt to the rapid evolution and changes in the connotation of digital technology concepts, and the cost of model maintenance and updates is high. Summary of the Invention

[0012] The purpose of this application is to provide a method, device, equipment, medium, and product for measuring the degree of enterprise digital transformation, which can solve the problems of shallow semantic understanding, single measurement dimension, and limited flexibility and scalability.

[0013] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a method for measuring the degree of enterprise digital transformation, including: obtaining the annual reports of multiple enterprises; the multiple enterprises include the enterprise to be measured.

[0014] Each company's annual report is divided into multiple chapters, resulting in a set of chapters for each company.

[0015] A portion of chapters is extracted from the chapter sets of each enterprise, and the level values ​​of the digital transformation indicators of the enterprises to be evaluated corresponding to the extracted chapters are manually labeled to obtain a labeled dataset; the labeled dataset includes the extracted chapters and the level values ​​of the digital transformation indicators of the enterprises to be evaluated corresponding to the manually labeled extracted chapters.

[0016] The optimal task execution strategy and the optimal knowledge activation method are determined based on the labeled dataset; the task execution strategy is the output method of the large language model; the knowledge activation method is the generation method of instruction prompt words.

[0017] Instructional prompts are generated based on the optimal knowledge activation method; the instructional prompts include: digital transformation indicators of each enterprise to be evaluated and the optimal task execution strategy.

[0018] The imperative prompts and the remaining chapters after the enterprise to be measured are input into the large language model to obtain the level value of the digital transformation indicators of each enterprise to be evaluated corresponding to the remaining chapters, thus completing the measurement of the degree of digital transformation of the enterprise to be measured.

[0019] Secondly, this application provides a device for measuring the degree of enterprise digital transformation, including: a data preprocessing module for obtaining the annual reports of multiple enterprises; the multiple enterprises include the enterprise to be measured, and the annual reports of each enterprise are divided into multiple chapters to obtain the chapter set of each enterprise.

[0020] The large-scale model analysis module is used to extract a portion of chapters from the chapter sets of each enterprise, manually annotate the level values ​​of the digital transformation indicators of the enterprises to be evaluated corresponding to each extracted chapter, and obtain an annotated dataset. Based on the annotated dataset, the optimal task execution strategy and the optimal knowledge activation method are determined. The annotated dataset includes each extracted chapter and the level values ​​of the digital transformation indicators of each enterprise to be evaluated corresponding to each manually annotated chapter. The knowledge activation method is a command-based prompt word generation method. The task execution strategy is the output method of the large language model. Command-based prompt words are generated based on the optimal knowledge activation method. The command-based prompt words include: the digital transformation indicators of each enterprise to be evaluated and the optimal task execution strategy. The command-based prompt words and the remaining chapters of the enterprises to be measured are input into the large language model to obtain the level values ​​of the digital transformation indicators of each enterprise to be evaluated corresponding to the remaining chapters, thus completing the measurement of the degree of digital transformation of the enterprises to be measured.

[0021] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for measuring the degree of enterprise digital transformation.

[0022] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for measuring the degree of enterprise digital transformation.

[0023] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned method for measuring the degree of enterprise digital transformation.

[0024] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method, device, equipment, medium and product for measuring the degree of enterprise digital transformation. This application uses instructional prompts to guide a large language model to perform deep semantic analysis on the enterprise annual report text, enabling the large language model to accurately understand the meaning of the text based on the context, rather than simple keyword matching, thereby effectively avoiding misjudgment caused by semantic understanding bias, improving the accuracy and robustness of the recognition results, and solving the problem of shallow semantic understanding.

[0025] This application's instructional prompts include digital transformation indicators for each enterprise to be evaluated, enabling the large language model to understand relevant information about enterprise digital transformation from multiple indicators. This drives the large language model to perform deep semantic analysis and reasoning on enterprise text data. The large language model outputs a rating value, allowing it to break through the limitations of traditional binary classification and solve the problem of a single measurement dimension.

[0026] Large language models possess powerful generalization and learning capabilities for new knowledge, without relying on fixed dictionaries or labeling systems. This application divides each company's annual report into multiple chapters, including the companies to be evaluated. A portion of the chapters is extracted, and the level values ​​of the digital transformation indicators corresponding to each chapter are manually labeled to obtain a labeled dataset. The optimal task execution strategy and optimal knowledge activation method are determined based on the labeled dataset. Instructional prompts, including the optimal task execution strategy, are generated based on the optimal knowledge activation method. When digital technology concepts change, only the optimal task execution strategy and optimal knowledge activation method need to be adjusted, along with the prompts, to guide the large language model to adapt to new semantic and task requirements. This reduces model maintenance and update costs and solves the problems of limited flexibility and scalability. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a method for measuring the degree of enterprise digital transformation, provided as an embodiment of this application.

[0029] Figure 2 This is a schematic diagram of the functional modules of a device for measuring the degree of enterprise digital transformation, provided in an embodiment of this application.

[0030] Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0032] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] The evaluation paradigm for enterprise digital transformation urgently needs a fundamental shift from "whether it is applied" to "the degree of application and its effectiveness." The groundbreaking capabilities of large language models in deep semantic understanding, contextual reasoning, and instruction following offer a novel technological path to solving this challenge. Based on this, the research concept of this application is to abandon traditional dictionary matching and binary classification frameworks, and instead directly utilize the deep semantic understanding and reasoning capabilities of large language models to construct a new evaluation system capable of multi-dimensional, multi-level quantitative scoring of enterprise digital transformation. This application aims to address the core shortcomings of the aforementioned existing technologies. Specifically, the technical problem to be solved is: how to overcome the limitations of binary judgments and achieve accurate, fine-grained, and interpretable quantitative identification and evaluation of the "degree of application" of various digital technologies by enterprises, thereby providing a more powerful and in-depth methodological tool for empirical research and business analysis of enterprise digital transformation.

[0034] In one exemplary embodiment, such as Figure 1 As shown, a method for measuring the degree of enterprise digital transformation is provided, including: Step 201: Obtain the annual reports of multiple enterprises; the multiple enterprises include the enterprise to be measured.

[0035] Step 202: Divide each company's annual report into multiple chapters to obtain a set of chapters for each company.

[0036] Step 203: Extract a portion of chapters from the chapter sets of each enterprise, and manually annotate the level values ​​of the digital transformation indicators of the enterprises to be evaluated corresponding to the extracted chapters to obtain an annotated dataset; the annotated dataset includes the extracted chapters and the level values ​​of the digital transformation indicators of the enterprises to be evaluated corresponding to the manually annotated extracted chapters.

[0037] Step 204: Determine the optimal task execution strategy and the optimal knowledge activation method based on the labeled dataset; the knowledge activation method is the generation method of instruction prompt words; the task execution strategy is the output method of the large language model.

[0038] Step 205: Generate instructional prompts based on the optimal knowledge activation method; the instructional prompts include: digital transformation indicators of each enterprise to be evaluated and the optimal task execution strategy.

[0039] Step 206: Input the instructional prompts and the remaining chapters after the enterprise to be measured into the large language model to obtain the level values ​​of the digital transformation indicators of each enterprise to be evaluated corresponding to the remaining chapters, and complete the measurement of the degree of digital transformation of the enterprise to be measured.

[0040] In another exemplary embodiment of this application, the optimal task execution strategy and the optimal knowledge activation method are determined based on the labeled dataset, specifically including: dividing the labeled dataset into a training set, a validation set, and a test set.

[0041] The task execution strategy of outputting each single indicator one by one yields prompts for the digital transformation indicators of each enterprise to be evaluated.

[0042] By inputting the prompts for the digital transformation indicators of each enterprise to be evaluated, as well as the chapters of the test set, into the large language model, the first predicted level value of the digital transformation indicators of each enterprise to be evaluated corresponding to each chapter of the test set is obtained.

[0043] The evaluation results corresponding to the task execution strategies for each individual indicator are determined based on the first predicted level value of each enterprise's digital transformation indicator in each chapter of the test set and the level value of each enterprise's digital transformation indicator in each chapter of the test set that is manually annotated.

[0044] The task execution strategy of executing multiple indicators synchronously outputs the overall prompt words, which include all digital transformation indicators of the enterprise to be evaluated.

[0045] By inputting the overall prompts and each chapter of the test set into the large language model, the second predicted level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter of the test set is obtained.

[0046] The evaluation results corresponding to the task execution strategy of simultaneous output of multiple indicators are determined based on the second predicted level value of each enterprise's digital transformation indicator corresponding to each chapter in the test set and the level value of each enterprise's digital transformation indicator corresponding to each chapter in the test set that is manually annotated.

[0047] The optimal task execution strategy is determined based on the evaluation results of the task execution strategies output one by one for each single indicator and the evaluation results of the task execution strategies output simultaneously for multiple indicators.

[0048] Based on the optimal task execution strategy, a knowledge-inducing method for executing a direct prediction strategy is used to obtain direct prediction prompts.

[0049] By inputting the direct prediction prompts into the large language model, the third prediction level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter in the test set is obtained.

[0050] The evaluation results corresponding to the knowledge activation method of the direct prediction strategy are determined based on the third prediction level value of each enterprise's digital transformation indicator corresponding to each chapter in the test set and the level value of each enterprise's digital transformation indicator corresponding to each chapter in the test set that is manually annotated.

[0051] Based on the optimal task execution strategy, the knowledge activation method of the thought chain activation strategy is implemented to obtain thought chain prompts.

[0052] By inputting the thought chain prompts into the large language model, the fourth prediction level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter in the test set is obtained.

[0053] The evaluation results corresponding to the knowledge stimulation method of the mind chain stimulation strategy are determined based on the fourth predicted level value of each chapter of the test set corresponding to the digital transformation indicators of each enterprise to be evaluated, and the level value of each chapter of the test set corresponding to the digital transformation indicators of each enterprise to be evaluated, which are manually annotated.

[0054] The optimal knowledge activation method is determined based on the evaluation results of the knowledge activation methods corresponding to the direct prediction strategy and the thinking chain activation strategy.

[0055] In another exemplary embodiment of this application, before inputting the imperative prompts and the remaining chapters after the enterprise to be measured into the large language model, the method further includes: using low-rank adaptive techniques based on the optimal task execution strategy and the optimal knowledge activation method to fine-tune the large language model on the training set, evaluate the fine-tuned large language model on the validation set, and test the evaluated large language model on the test set.

[0056] In another exemplary embodiment of this application, the method further includes preprocessing each chapter before extracting a portion of the chapters.

[0057] In another exemplary embodiment of this application, the remaining chapters of the large language model output a JSON-formatted file containing the level values ​​of each enterprise's digital transformation indicators to be evaluated. The method for measuring the degree of enterprise digital transformation further includes: converting the JSON-formatted file output by the large language model.

[0058] In another exemplary embodiment of this application, the company annual report is in PDF format. Each company's annual report is divided into multiple chapters to obtain a chapter set for each company. Specifically, for any company, the logical starting page number of each chapter is obtained based on the table of contents of the company's annual report.

[0059] Get the actual starting page number of the first chapter.

[0060] The page number offset is obtained based on the actual starting page number and the logical starting page number of the first chapter.

[0061] The actual starting page number of other chapters is obtained based on the page number offset and the logical starting page number of other chapters; other chapters are all chapters except the first chapter.

[0062] The actual end page number of each chapter is obtained from the actual starting page number of each chapter.

[0063] The company's annual report is divided into multiple chapters based on the actual starting and ending page numbers of all chapters, resulting in a set of chapters for the company.

[0064] In practical applications, in addition to corporate annual report text data, external data such as high-frequency industry reports and news related to corporate digital transformation can be introduced into the large language model. The relevant information on corporate digital transformation from this external data can then be integrated with the corporate annual report data. For example, data mining techniques can be used to extract overall industry digital transformation trends and key indicators from industry reports, and news articles can be used to obtain the latest developments and events of companies in the digital field. This information is then combined with the corporate annual report data and input into a large-scale vertical model for corporate digital transformation to conduct a quantitative evaluation of corporate digital transformation.

[0065] External high-frequency data offers unique advantages, providing more comprehensive and timely background information and industry benchmarks for assessing enterprise digital transformation. Industry reports reflect the overall digital development trajectory of the industry, while news and information provide real-time updates on companies' latest actions in the digital field. This information effectively compensates for the shortcomings of corporate annual reports, which, while rich in content, have a low update frequency, enabling assessment models to more accurately grasp a company's position and actual progress in the digital transformation wave. This data fusion approach facilitates more precise quantitative assessments, aligning perfectly with the purpose of this application: to accurately evaluate the degree of enterprise digital transformation.

[0066] This application also provides a more specific embodiment, which elaborates on the above-mentioned method for measuring the degree of enterprise digital transformation: Step 1: Convert all enterprises' annual reports in PDF format to TXT format and divide them by chapter to provide a suitable data format for subsequent text analysis.

[0067] Step 1.1: For any company's annual report, parse the table of contents, obtain the title of each chapter and its corresponding logical starting page number, and record the range of each chapter in the table of contents, laying the foundation for subsequent chapter-by-chapter document processing.

[0068] First, the physical structure of the PDF document is read, and the text layout is scanned page by page to identify text lines that conform to the format "chapter title...page number". Regular expression matching is used to extract the title of each chapter and its logical starting page number in the table of contents. Simultaneously, the system records the start and end page numbers of the table of contents to distinguish the table of contents area from the main text area. This process does not rely on fixed page number identifiers but dynamically identifies text format features, supporting annual report documents with different layout formats.

[0069] Step 1.2: Locate the actual starting page number of the first chapter. Accurately determine the actual starting page number of the first chapter so that the page number range of each chapter can be accurately calculated in the future, ensuring that the extracted chapter text content is without deviation.

[0070] Because there may be discrepancies between logical and actual page numbers in a company's annual report, after obtaining the logical starting page number of the first chapter in the table of contents, the system skips all table of contents pages and scans each page sequentially starting from the first page of the main text. For the extracted text on each page, the system extracts the core content of the title (e.g., removing prefixes such as "Chapter X") and performs similarity matching (based on a fuzzy text matching algorithm) with the chapter titles recorded in the table of contents. Once the first chapter title is matched on a page of the main text, the physical page number of that page is determined as the actual starting page number of the first chapter.

[0071] Step 1.3: Calculate the page number range of each chapter to accurately define the page number range of each chapter in the document, ensuring the integrity and accuracy of the content when extracting text by chapter.

[0072] The page number offset is calculated based on the difference between the actual starting page number of the first chapter and its logical starting page number in the table of contents. Using this offset, the logical starting page numbers of all chapters in the table of contents are adjusted uniformly to obtain the actual starting page number of each chapter in the document. Then, the actual ending page number of the preceding chapter is determined based on the actual starting page numbers of adjacent chapters (i.e., the actual starting page number of the next chapter minus one), and the actual ending page number of the last chapter is set to the total number of pages in the document minus one. This constructs a structured index that fully maps chapter titles to actual page number ranges.

[0073] Step 1.4: Extract and convert the text format by chapter. Convert the PDF format of the company's annual report into TXT format, which is easier for subsequent text analysis. Then, divide it into chapters so that subsequent analysis can be carried out on specific chapter content, thereby enhancing the relevance and accuracy of the analysis.

[0074] Based on the established chapter and page number range index, all text content within the corresponding page range of the PDF document is extracted chapter by chapter and concatenated into a continuous text stream in natural reading order. The text of each chapter is saved independently as a TXT file, with the filename named after the chapter title (after standardization processing such as removing illegal characters). Ultimately, each company annual report PDF is converted into a folder containing several chapter TXT files, preserving the original document's structured information and facilitating subsequent fine-grained analysis of specific chapters.

[0075] Step 2: Analyze the TXT chapter content converted in Step 1 using a large language model to identify and quantify information related to the degree of enterprise digital transformation.

[0076] Step 2.1: Evaluate performance based on task execution strategy.

[0077] Based on the TXT text obtained through segmentation in Step 1, the annual report text in TXT format acquired in Step 1 is preprocessed, including removing redundant spaces and extracting valid text segments. Based on this preprocessing, a portion of the text is extracted to construct a labeled dataset, which is then divided into training, validation, and test sets in an 8:1:1 ratio. The labeled dataset consists of a text portion and a label portion. The text portion originates from the text obtained in Step 1; the label portion contains the TXT text name and the corresponding level values ​​of various enterprise digital transformation indicators. The level values ​​of these enterprise digital transformation indicators are determined through manual annotation.

[0078] Accuracy, precision, recall, and F1 score were calculated using manually labeled enterprise digital transformation metrics from a test set and the output values ​​of the large language model under two task execution strategies. This comprehensive evaluation of the model's output accuracy, stability, and inference efficiency under both strategies was conducted. Based on the evaluation results, the task execution strategy most favorable to the large language model in measuring enterprise digital transformation was determined.

[0079] The first task execution strategy is to output each indicator one by one. This strategy constructs independent prompts for each enterprise's digital transformation indicator (such as artificial intelligence technology, big data technology, etc.), and inputs them sequentially into the large language model. The model performs semantic parsing and level determination only for a single indicator each time, outputting the level evaluation result of that indicator, which is the level value (0-3) of the enterprise's digital transformation indicator. This strategy reduces the complexity of multi-objective reasoning in the model through task decomposition and is suitable for situations where the semantic independence between indicators is relatively strong.

[0080] The second task execution strategy is multi-indicator simultaneous output. This strategy integrates all evaluation indicators into the same prompt word template, guiding the large language model to simultaneously understand the semantics of each indicator and perform joint inference in a single call. Based on context awareness and the potential correlation between indicators (such as "cloud computing" and "big data" often appearing together), the model outputs a structured JSON result containing the ratings of all indicators (the level values ​​of all enterprise digital transformation indicators). This strategy leverages the model's multi-task collaborative capabilities to improve overall inference efficiency.

[0081] Step 2.2: Evaluate performance based on the knowledge activation method dimension.

[0082] Within the zero-shot learning framework, this study investigates the impact of different reasoning guidance strategies on the knowledge activation and logical reasoning capabilities of a large language model, and utilizes the existing DSPy framework to conduct comparative evaluations of these strategies. Based on test set text data and indicator labels (human-annotated ratings of enterprise digital transformation indicators), following a general analysis process, and under the optimal task execution strategy determined in step 2.1, a thorough comparison of two knowledge activation methods is conducted. Classification accuracy (covering key indicators such as accuracy and F1 score) is considered. Through a meticulous evaluation process, under zero-shot conditions, it accurately determines which knowledge activation method is more conducive to improving the performance of the large language model in the complex classification task of quantitative evaluation of enterprise digital transformation. The optimal knowledge activation method is then applied to subsequent steps.

[0083] The first knowledge activation method is a direct prediction strategy: This strategy leverages the existing DSPy Predict module to directly send predefined structured task signatures—which include task descriptions, input text, and output format requirements (i.e., output methods)—to the large language model. Based on its pre-trained knowledge, the model performs a single-step semantic understanding and mapping, directly outputting the rating results for each indicator (the rating values ​​of enterprise digital transformation indicators). This strategy tests the model's real-time understanding and execution efficiency of task instructions.

[0084] The second knowledge stimulation method is the thought chain stimulation strategy: the system utilizes the existing ChainOfThought module of DSPy to embed instructions that trigger step-by-step reasoning in the task signature (prompt words). The model is guided to first parse relevant evidence in the text (such as identifying technical keywords and judging the descriptive context), then infer the degree of application based on the logical chain, and finally map it to the level evaluation of the task objective (the level value of the enterprise's digital transformation indicator). This strategy explicitly stimulates the model's internal reasoning process, aiming to improve the accuracy of complex judgments and the interpretability of the results.

[0085] Step 2.3: Evaluate performance based on domain adaptability.

[0086] Based on the optimal task execution strategy determined in step 2.1 and the optimal knowledge activation method determined in step 2.2, an existing supervised fine-tuning method based on low-rank adaptive technology is used to finely tune the large language model using the training and validation sets, obtaining the optimal large language model for the task of measuring the degree of enterprise digital transformation. The tuned large language model is then used to reprocess the enterprise annual report text data in the test set according to the optimal task execution strategy determined in step 2.1 and the optimal knowledge activation method determined in step 2.3, resulting in the final evaluation results data in JSON format. Compared to the previous analysis results, this data is expected to show significant improvements in accuracy and stability, providing a more reliable and accurate basis for the quantitative evaluation of the degree of enterprise digital transformation.

[0087] During fine-tuning, iterative training allows the model to gradually learn the feature representations and classification patterns specific to the enterprise digital transformation degree measurement task. Simultaneously, the model performance at different training stages is systematically evaluated on the validation set, and the optimal checkpoints are selected to effectively suppress overfitting. This allows the large language model to better adapt to the enterprise digital indicator application degree classification and judgment task, significantly improving the model's performance in this specific domain. Finally, the model is tested on the test set to compare the effects before and after fine-tuning, resulting in a well-tuned large language model.

[0088] Step 2.4: Measuring the degree of enterprise digital transformation.

[0089] To measure Company A, for each remaining section after extracting Company A's data, based on the evaluation indicator names required by the client, the system automatically generates instructional prompts for the optimal large-scale language model using the best task execution strategy and the best knowledge activation method. These prompts clearly define the task objectives (enterprise digital transformation indicators), evaluation levels, and output format requirements. This clearly communicates the tasks to be completed to the vertical large-scale language model adapted to the enterprise's digital transformation, explicitly specifies the evaluation dimensions and output format, and guides the large-scale language model to analyze the text according to predetermined rules.

[0090] Step 3: Convert the model prediction results output in Step 2 into XLSX format for easy viewing and analysis by users.

[0091] Step 3.1: Read the analysis results and obtain the data from Step 2 to prepare for converting it to Excel format. Write a Python function to read the TXT file containing the analysis results in JSON format output from Step 2.

[0092] Step 3.2: Parse and extract key information. Extract key information that is of great value to users from complex JSON data, so as to organize it into a more intuitive format for users to view and analyze.

[0093] The system parses the read JSON data, extracting the company name, chapter name, character count, and rating information for each evaluation dimension (enterprise digital transformation indicators). If the output field itself is a JSON string, it requires secondary parsing. Regular expressions are used to match complete JSON objects in the output, and the JSON string is converted into a Python dictionary to extract the enterprise digital transformation indicator rating values. If the JSON format is abnormal or the match fails, the system will log an error and skip that record to ensure the robustness of data processing.

[0094] Step 3.3: Convert to Excel format. After organizing the information extracted in Step 3.2 into a DataFrame format, save it as an XLSX file. Converting the extracted data to an XLSX file makes it easier for users to view, analyze, and further process the data using common office software, improving data readability and usability.

[0095] This application constructs structured cue words containing multi-dimensional indicators of enterprise digital transformation, driving a large language model to perform deep semantic analysis and reasoning on unstructured enterprise text data.

[0096] This application enables large language models to break through the traditional binary classification judgment and output quantitative scoring results with specific levels (such as 0-3 points) for various enterprise digital transformation indicators.

[0097] This application uses a large language model as the core processing engine, transforming it from a general algorithm into a specialized tool for solving the specific problem of "precise measurement of enterprise digital transformation". This produces technical effects that surpass conventional information processing methods, achieving a more accurate and in-depth assessment from qualitative to quantitative methods.

[0098] This application overcomes the limitations of binary judgments, achieving precise and fine-grained quantitative assessment. Existing technologies generally only allow for "yes" or "no" binary judgments, failing to provide multi-level quantitative assessments of the "application depth" of enterprise digital transformation. The first application constructs a specific technical process to generate structured prompts containing multi-dimensional indicators of enterprise digital transformation, driving a large language model to perform deep semantic analysis and reasoning on unstructured enterprise text data. This multi-dimensional prompt design enables the large language model to understand enterprise digital transformation-related information from multiple perspectives and output multi-dimensional, multi-level quantitative assessments of enterprise digital transformation. The second application implements a discrete quantitative rating mechanism, allowing the large language model to break through traditional binary classification judgments and output quantitative scores (e.g., 0-3 points) for the application level of various digital technologies. This mechanism clarifies the quantitative standards and levels, thus achieving fine-grained quantitative assessment. In summary, this application can achieve precise, fine-grained, and interpretable quantitative identification and assessment of the "application degree" of enterprises in various digital technologies.

[0099] This application improves the accuracy of semantic understanding. Existing technologies suffer from shallow semantic understanding, relying on surface keyword matching and lacking the ability to reason about the deep semantics and contextual logic of the text, resulting in insufficient accuracy and robustness of the recognition results. This application uses a large language model as its core processing engine and fully leverages its capabilities through a specific technical process. In step 2, templated prompts guide the large language model to perform deep semantic analysis of the corporate annual report text. For example, the generated prompts clearly define the task, specify evaluation dimensions, and output format, enabling the large language model to accurately understand the meaning of the text based on context, rather than simply matching keywords. This effectively avoids misjudgments caused by semantic understanding biases, thus improving the accuracy and robustness of the recognition results. This application significantly improves the accuracy of semantic understanding by utilizing the powerful deep semantic understanding and reasoning capabilities of the large language model.

[0100] This application enhances flexibility and scalability. Existing methods based on fixed dictionaries or tag systems struggle to dynamically adapt to the rapid evolution and changing connotations of digital technology concepts, resulting in high model maintenance and update costs. This application abandons the traditional dictionary matching and binary classification framework, directly utilizing the capabilities of large language models. Large language models possess powerful generalization and learning abilities for new knowledge, eliminating the need for fixed dictionaries or tag systems. When digital technology concepts change, only the construction of prompt words needs adjustment to guide the large language model to adapt to new semantics and task requirements, reducing model maintenance and update costs and enhancing the method's flexibility and scalability. The method presented in this application offers greater flexibility and scalability, better addressing the evolving landscape of digital technologies.

[0101] Based on the same inventive concept, this application also provides an enterprise digital transformation degree measurement device for implementing the above-mentioned enterprise digital transformation degree measurement method. The solution provided by this device is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more enterprise digital transformation degree measurement device embodiments provided below can be found in the limitations of the enterprise digital transformation degree measurement method above, and will not be repeated here.

[0102] In one exemplary embodiment, such as Figure 2 As shown, a device for measuring the degree of enterprise digital transformation is provided, including: a data preprocessing module for obtaining the annual reports of multiple enterprises; the multiple enterprises include the enterprise to be measured, and the annual reports of each enterprise are divided into multiple chapters to obtain the chapter set of each enterprise.

[0103] The large-scale model analysis module is used to extract a portion of chapters from the chapter sets of each enterprise, manually annotate the level values ​​of the digital transformation indicators of the enterprises to be evaluated corresponding to each extracted chapter, and obtain an annotated dataset. Based on the annotated dataset, the optimal task execution strategy and the optimal knowledge activation method are determined. The annotated dataset includes each extracted chapter and the level values ​​of the digital transformation indicators of each enterprise to be evaluated corresponding to each manually annotated chapter. The knowledge activation method is a command-based prompt word generation method. The task execution strategy is the output method of the large language model. Command-based prompt words are generated based on the optimal knowledge activation method. The command-based prompt words include: the digital transformation indicators of each enterprise to be evaluated and the optimal task execution strategy. The command-based prompt words and the remaining chapters of the enterprises to be measured are input into the large language model to obtain the level values ​​of the digital transformation indicators of each enterprise to be evaluated corresponding to the remaining chapters, thus completing the measurement of the degree of digital transformation of the enterprises to be measured.

[0104] In practical applications, the level values ​​of each chapter of the digital transformation indicators of the enterprise to be evaluated, output by the large language model, are in JSON format. The enterprise digital transformation degree measurement device also includes a result processing module, which is used to convert the JSON format file output by the large language model.

[0105] This application improves evaluation efficiency. The various modules in this application have clearly defined functions and work collaboratively. The data preprocessing module automatically converts PDF annual reports to TXT format and segments them by chapter; the large-scale model analysis module automatically analyzes the text and outputs results using a large language model interface; the results processing module automatically converts the analysis results to Excel format. Through the system's modular design and workflow construction, this application achieves an automated process from processing enterprise annual report data to outputting analysis results. The entire process requires minimal manual intervention; computer equipment automatically executes each step according to preset programs and algorithms. Compared to manual operation or existing complex and inefficient methods, this significantly improves evaluation efficiency.

[0106] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data measuring the degree of enterprise digital transformation. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for measuring the degree of enterprise digital transformation.

[0107] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0108] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method embodiments.

[0109] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method embodiments.

[0110] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described method embodiments.

[0111] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0112] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0113] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0114] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for measuring the degree of enterprise digital transformation, characterized in that, The methods for measuring the degree of enterprise digital transformation include: Obtain annual reports from multiple companies, including the company to be measured. Each company's annual report is divided into multiple chapters, resulting in a set of chapters for each company. A portion of chapters is extracted from the chapter sets of each enterprise, and the level values ​​of the digital transformation indicators of the enterprises to be evaluated corresponding to the extracted chapters are manually labeled to obtain a labeled dataset; the labeled dataset includes the extracted chapters and the level values ​​of the digital transformation indicators of the enterprises to be evaluated corresponding to the manually labeled extracted chapters. The optimal task execution strategy and the optimal knowledge activation method are determined based on the labeled dataset; the task execution strategy is the output method of the large language model; the knowledge activation method is the generation method of instruction-based prompt words. Instructional prompts are generated based on the optimal knowledge activation method; these prompts include: digital transformation indicators for each enterprise to be evaluated and the optimal task execution strategy. The imperative prompts and the remaining chapters after the enterprise to be measured are input into the large language model to obtain the level value of the digital transformation indicators of each enterprise to be evaluated corresponding to the remaining chapters, thus completing the measurement of the degree of digital transformation of the enterprise to be measured.

2. The method for measuring the degree of enterprise digital transformation according to claim 1, characterized in that, Based on the labeled dataset, the optimal task execution strategy and the optimal knowledge activation method are determined, specifically including: The labeled dataset is divided into a training set, a validation set, and a test set; The task execution strategy of outputting each single indicator one by one yields prompt words for the digital transformation indicators of each enterprise to be evaluated; Input the prompt words for the digital transformation indicators of each enterprise to be evaluated and the chapters of the test set into the large language model to obtain the first predicted level value of the digital transformation indicators of each enterprise to be evaluated corresponding to each chapter of the test set. The evaluation results corresponding to the task execution strategies for each single indicator are determined based on the first predicted level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter in the test set and the level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter in the test set that is manually annotated. The task execution strategy of executing multiple indicators synchronously outputs the overall prompt words, which include all digital transformation indicators of the enterprise to be evaluated. The overall prompt words and each chapter of the test set are input into the large language model to obtain the second predicted level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter of the test set. The evaluation results corresponding to the task execution strategy of simultaneous output of multiple indicators are determined based on the second predicted level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter in the test set and the level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter in the test set that is manually annotated. The optimal task execution strategy is determined based on the evaluation results of the task execution strategy output one by one for each single indicator and the evaluation results of the task execution strategy output simultaneously for multiple indicators. Based on the optimal task execution strategy, a knowledge-inducing method for executing a direct prediction strategy is used to obtain direct prediction prompts. By inputting the direct prediction prompts into the large language model, the third prediction level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter in the test set is obtained. The evaluation results corresponding to the knowledge activation method of the direct prediction strategy are determined based on the third prediction level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter in the test set and the level value of each digital transformation indicator of the enterprise to be evaluated corresponding to each chapter in the test set that is manually annotated. Based on the optimal task execution strategy, the knowledge activation method of the thought chain activation strategy is used to obtain thought chain prompt words; Inputting the thought chain prompts into the large language model yields the fourth prediction level value for each digital transformation indicator of the enterprise to be evaluated, corresponding to each chapter in the test set. The evaluation results corresponding to the knowledge stimulation method of the thinking chain stimulation strategy are determined based on the fourth predicted level value of each chapter of the test set corresponding to the digital transformation indicators of each enterprise to be evaluated and the level value of each chapter of the test set corresponding to the digital transformation indicators of each enterprise to be evaluated that are manually annotated. The optimal knowledge activation method is determined based on the evaluation results of the knowledge activation methods corresponding to the direct prediction strategy and the thinking chain activation strategy.

3. The method for measuring the degree of enterprise digital transformation according to claim 2, characterized in that, Before inputting the instructional prompts and the remaining chapters after the enterprise to be measured into the large language model, the process further includes: using low-rank adaptive techniques based on the optimal task execution strategy and the optimal knowledge activation method to fine-tune the large language model on the training set, evaluate the fine-tuned large language model on the validation set, and test the evaluated large language model on the test set.

4. The method for measuring the degree of enterprise digital transformation according to claim 1, characterized in that, Before extracting a portion of the chapters, the process also includes preprocessing each chapter.

5. The method for measuring the degree of enterprise digital transformation according to claim 1, characterized in that, The remaining chapters of the large language model output a JSON file containing the level values ​​of each enterprise's digital transformation indicators to be evaluated. The method for measuring the degree of enterprise digital transformation also includes: converting the JSON file output by the large language model.

6. The method for measuring the degree of enterprise digital transformation according to claim 1, characterized in that, The company annual reports are in PDF format. Each company's annual report is divided into multiple chapters, resulting in a set of chapters for each company, specifically including: For any given company, the logical starting page number of each chapter is obtained from the table of contents of the company's annual report; Get the actual starting page number of the first chapter; The page number offset is obtained based on the actual starting page number and the logical starting page number of the first chapter; The actual starting page number of other chapters is obtained based on the page number offset and the logical starting page number of other chapters; other chapters are all chapters except the first chapter; Obtain the actual end page number of each chapter based on the actual start page number of each chapter; The company's annual report is divided into multiple chapters based on the actual starting and ending page numbers of all chapters, resulting in a set of chapters for the company.

7. A device for measuring the degree of enterprise digital transformation, characterized in that, The device for measuring the degree of enterprise digital transformation includes: The data preprocessing module is used to obtain the annual reports of multiple companies, including the company to be measured. The annual reports of each company are divided into multiple chapters to obtain the chapter set of each company. The large-scale model analysis module is used to extract a portion of chapters from the chapter sets of each enterprise, manually annotate the level values ​​of the digital transformation indicators of the enterprises to be evaluated corresponding to each extracted chapter, and obtain an annotated dataset. Based on the annotated dataset, the optimal task execution strategy and the optimal knowledge activation method are determined. The annotated dataset includes each extracted chapter and the level values ​​of the digital transformation indicators of each enterprise to be evaluated corresponding to each manually annotated chapter. The knowledge activation method is a command-based prompt word generation method. The task execution strategy is the output method of the large language model. Command-based prompt words are generated based on the optimal knowledge activation method. The command-based prompt words include: the digital transformation indicators of each enterprise to be evaluated and the optimal task execution strategy. The command-based prompt words and the remaining chapters of the enterprises to be measured are input into the large language model to obtain the level values ​​of the digital transformation indicators of each enterprise to be evaluated corresponding to the remaining chapters, thus completing the measurement of the degree of digital transformation of the enterprises to be measured.

8. A computer device, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that the processor executes the computer program to implement the enterprise digital transformation measurement method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method for measuring the degree of enterprise digital transformation as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the method for measuring the degree of enterprise digital transformation as described in any one of claims 1-6.

Images