A data analysis method and system based on a large language model

Abstract

The application discloses a data analysis method and system based on a large language model, relates to the technical field of data processing, and comprises the following steps: collecting human resource data and task detail data of each department in an enterprise; converting the collected data into high-dimensional vectors recognizable by the large language model; creating a multitask analysis body based on the generated high-dimensional vectors, while meeting three data analysis requirements of task scheduling, task progress prediction and performance evaluation; integrating the created multitask analysis body into the large language model, and fine-tuning the model based on historical data; deploying the fine-tuned large language model to a production environment, and continuously optimizing the large language model according to user feedback data. The application optimizes the application of the large language model in the data analysis process, can provide specific analysis methods according to specific requirements, so as to adapt to complex and changeable data environments; while guaranteeing analysis efficiency, the accuracy and pertinence of analysis are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a data analysis method and system based on a large language model. BACKGROUND

[0002] In today's rapid development of digital economy, data analysis has become an indispensable important link in various industries. With the advent of the big data era, various enterprises have accumulated a large amount of data resources, and how to efficiently analyze and utilize these data resources to mine valuable information has become the key to improving the competitiveness and optimizing the decision-making of enterprises. Traditional data analysis methods often rely on manually preset rules and models, which are low in processing efficiency and difficult to adapt to complex and variable data environments. The advantage of a large language model lies in its ability to be applicable to multiple tasks after a large-scale training, and its high efficiency in capturing long-distance dependency relationships through a self-attention mechanism. Therefore, how to further improve the convenience and pertinence of data analysis by utilizing the language understanding and generation capabilities of a large language model has become an important issue currently faced by researchers in the field. SUMMARY

[0003] The present application provides a data analysis method based on a large language model, comprising:

[0004] Step 1, collecting human resource data and task detail data of each department in the enterprise;

[0005] Step 2, converting the collected data into a high-dimensional vector recognizable by the large language model;

[0006] Step 3, creating a multi-task analysis body based on the generated high-dimensional vector, while meeting the three data analysis requirements of task scheduling, task progress prediction and performance evaluation;

[0007] Step 4, integrating the created multi-task analysis body into the large language model, and fine-tuning the model based on historical data;

[0008] Step 5, deploying the fine-tuned large language model to the production environment, and continuously optimizing the large language model according to user feedback data.

[0009] The data analysis method based on a large language model as described above, wherein the collected data is converted into a high-dimensional vector recognizable by the large language model, and the specific steps are as follows:

[0010] Encoding the collected raw data;

[0011] Splicing the encoded data into character sequences according to the original fields;

[0012] The concatenated character sequence is converted into a high-dimensional vector sequence using a high-dimensional vector generation function.

[0013] The data analysis method based on a large language model as described above, wherein a multi-task analysis body is created based on the generated high-dimensional vectors, and the method comprises the following sub-steps:

[0014] A multi-layer analysis structure is created, and a high-dimensional vector extractor is set at the entrance of each analysis layer.

[0015] The sub-models in each analysis layer are individually adjusted according to the analysis requirements to be met.

[0016] The output rules of the multi-layer analysis structure are set to obtain the final multi-task analysis body.

[0017] The data analysis method based on a large language model as described above, wherein the created multi-task analysis body is integrated into the large language model, and the model is fine-tuned based on historical data, and the method comprises the following sub-steps:

[0018] A fully connected layer is added to the large language model, and the created multi-task analysis body is deployed to the fully connected layer to share the self-attention mechanism of the large language model.

[0019] The historical human data and historical task detail data of each department in the enterprise are processed into a training data set.

[0020] The integrated large language model is supervised trained using the training data set, and the parameters in the large language model are optimized based on the minimum loss principle.

[0021] The adaptability of the trained large language model to the application scenario is tested, and the large language model is further optimized according to the test results.

[0022] The data analysis method based on a large language model as described above, wherein the large language model is continuously optimized according to user feedback data, and the method comprises the following sub-steps:

[0023] An error feedback mechanism is established to collect errors or inconsistencies in the model output.

[0024] The training data is adjusted according to the error feedback information, and the large language model is reinforced trained based on the adjusted training data.

[0025] The application also provides a data analysis system based on a large language model, comprising a data collection module, a data conversion module, a multi-task analysis body creation module, a multi-task analysis body integration module, and a large language model continuous optimization module.

[0026] A data collection module is configured to collect human resource data and task specification data of various departments in an enterprise;

[0027] A data conversion module is configured to convert the collected data into high-dimensional vectors recognizable by a large language model;

[0028] A multi-task analysis body creation module is configured to create a multi-task analysis body based on the generated high-dimensional vectors, while meeting the three data analysis requirements of task scheduling, task progress prediction and performance evaluation;

[0029] A multi-task analysis body integration module is configured to integrate the created multi-task analysis body into the large language model and fine-tune the model based on historical data;

[0030] A large language model continuous optimization module is configured to deploy the fine-tuned large language model to a production environment and continuously optimize the large language model according to user feedback data.

[0031] The beneficial effects realized by the present application are as follows: specific analysis methods can be provided according to specific requirements to adapt to complex and variable data environments; the analysis accuracy and pertinence are improved while ensuring analysis efficiency; the application of the large language model in the data analysis process is optimized, making it more convenient and efficient. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0033] Figure 1 is a large language model-based data analysis method flowchart provided by the first embodiment of the present application;

[0034] Figure 2 is a large language model-based data analysis system schematic diagram provided by the second embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0036] Embodiment one

[0037] As Figure 1As shown, the embodiment one of the present application provides a data analysis method based on a large language model, comprising:

[0038] Step S10: Collect human resource data and task detail data of each department in the enterprise;

[0039] The human resource data includes fields such as employee ID, department, position, skill label, available working hours, current task load, historical performance score, leave record, overtime length, project participation history, etc. The task detail data includes fields such as task ID, department, task description, required skills, priority, deadline, task status (not started / in progress / completed), task history completion time, etc. These data are the data basis for task scheduling, task progress prediction and performance evaluation in a department, which can be obtained by accessing the API of the enterprise internal system.

[0040] Step S20: Convert the collected data into a high-dimensional vector recognizable by the large language model, which is specifically divided into the following sub-steps:

[0041] Step S21: Perform encoding processing on the collected raw data;

[0042] Encoding the raw data can ensure data consistency, and also enable the large language model to better understand the raw data. For numerical data (such as working hours, task priority), Z-score or Min-Max can be used for standardization; for classification data (such as skill label, task status), One-Hot encoding or embedding layer can be used for encoding processing; for text data (such as task description, employee skills), Word2Vec or BERT can be used for encoding processing.

[0043] Step S22: Concatenate the encoded data according to the original fields to form a character sequence;

[0044] For example, there are original fields: employee ID, skill, working hours, task ID, priority and deadline, and the field value encoding is a1-a6. Then, according to the original field concatenation, the following can be obtained: [employee ID, a1] [skill, a2] [working hours, a3] [task ID, a4] [priority, a5] [deadline, a6]. Each square bracket represents a key-value pair, where the first element represents the key and the second element represents the value.

[0045] Step S23: Use a high-dimensional vector generation function to convert the concatenated character sequence into a high-dimensional vector sequence;

[0046] The input parameter of the high-dimensional vector generation function is an encoded data, and the return value is the high-dimensional vector converted from the encoded data. The value (value) in the character sequence is sequentially input into the high-dimensional vector generation function, and the return value is used to replace the original value (value) to obtain a new high-dimensional vector sequence. The mathematical expression of the high-dimensional vector generation function is:

[0047] F(v) = (v·W + b)·Softmax(W Q ·ReLU((v·W + b)·W K )), where v is the input parameter, W is the linear transformation matrix, b is the bias vector, W Q and W K are the query matrix and key matrix in the self-attention mechanism of the large language model respectively, softmax represents the softmax function, and ReLU represents the ReLU function.

[0048] Step S30: based on the generated high-dimensional vector, create a multi-task analysis body, which meets the three data analysis requirements of task scheduling, task progress prediction and performance evaluation;

[0049] It should be noted that the three data analysis requirements listed here can be further analyzed based on the collected data, and if the collected data changes, the corresponding analysis requirements should also change; the multi-task analysis body provides three analysis methods for problems, statistical problems, classification problems and prediction problems, which are implemented by a default sub-model and two sub-models obtained by adjusting the default sub-model.

[0050] The specific creation process of the multi-task analysis body includes the following sub-steps:

[0051] Step S31: create a multi-layer analysis structure, and set a high-dimensional vector extractor at the entrance of each analysis layer;

[0052] A multi-layer analysis structure is created to isolate different analysis requirements. This embodiment proposes three analysis requirements, so three analysis layers are created to meet these three analysis requirements; the high-dimensional vector extractor is used to extract the high-dimensional vector required by the current layer from the input data. For example, when scheduling tasks, the skills label, available working hours, current task load, project participation history, and task description, required skills, priority, and deadline fields need to be considered comprehensively, and finally each task is given to the most suitable person to do. The high-dimensional vector extractor in the task scheduling analysis layer needs to extract the high-dimensional vector converted from these field values as the analysis data of the current layer.

[0053] Step S32: individualize adjust the sub-models in each analysis layer according to the analysis requirements to be met;

[0054] Each newly created analysis layer contains a default submodel, denoted as:

[0055] where Y is the output of the submodel, x i is the i-th analysis data (i-th high-dimensional vector extracted to the layer), is the weight vector of the i-th analysis data, b i is the bias vector of the i-th analysis data, L is a normalization constant, i takes values from 1 to n, and n is the total number of analysis data in the layer.

[0056] This submodel is applicable to general statistical problems, such as the performance evaluation proposed in this embodiment, which is a statistical problem, so the submodel in the analysis layer responsible for performance evaluation does not need to be modified; but when facing classification problems and prediction problems, the submodel in the analysis layer needs to be personalized adjusted, specifically:

[0057] For classification problems, such as task scheduling proposed in this embodiment, the analysis layer responsible for this requirement needs to output the employee who best fits the current task, and the personalized adjusted analysis layer submodel is denoted as:

[0058] where Y is the output of the submodel, x j is the historical matching rate of the j-th employee and the current task (the number of times the task appears in the project participation history divided by the total number of tasks), is the matching coefficient of the u-th employee feature (high-dimensional vector describing the employee extracted to the layer) and the e-th task feature (high-dimensional vector describing the task extracted to the layer), x u represents the u-th employee feature, b u is the bias vector of the u-th employee feature, u takes values from 1 to U, U is the total number of employee features, e takes values from 1 to E, E is the total number of task features, L is a normalization constant, j takes values from 1 to m, and m is the total number of employees in a certain department;

[0059] For prediction problems, such as task progress prediction proposed in this embodiment, the analysis layer responsible for this requirement needs to output the progress prediction result of a certain task in the future period, and the personalized modified analysis layer submodel is denoted as:

[0060] where Y is the output of the submodel, ω T is the output weight matrix of the submodel, x i is the i-th analysis data, is the weight vector of the i-th analysis data, b i is the bias vector of the i-th analysis data, λ i is the trend parameter of the i-th analysis data, and σT is a normalization constant, i takes values from 1 to n, and n is the total number of analysis data in the current layer.

[0061] Step S33: Set the output rules of the multi-layer analysis structure to obtain the final multi-task analysis body;

[0062] The analysis results returned by different analysis layers are integrated in a character splicing manner, and the integrated data is the final output result of the multi-layer analysis body. The results of different layers are separated by special symbols during splicing.

[0063] Step S40: Integrate the created multi-task analysis body into the large language model, and fine-tune the model based on historical data;

[0064] Add a fully connected layer to the large language model, deploy the created multi-task analysis body to the fully connected layer, and make it share the self-attention mechanism of the large language model. The fine-tuning process of the integrated large language model is specifically divided into the following sub-steps:

[0065] Step S41: Process the historical human data and historical task detail data of each department in the enterprise into a training data set;

[0066] Use the historical human data and historical task detail data of each department as input, and splice the corresponding information in the historical performance evaluation results, historical task scheduling, and task progress report as output. After sorting, the training data set is obtained.

[0067] Step S42: Supervised training of the integrated large language model using the training data set, and fine-tuning of the parameters in the large language model based on the minimum loss principle;

[0068] The integrated large language model has multi-task analysis capability, and its loss function is expressed as: Loss = a * Loss_Statistics + b * Loss_Classification + g * Loss_Prediction, where a, b, and g are the loss evaluation weights of different tasks, which need to be dynamically adjusted during training to meet the performance requirements of the model in different analysis tasks. The three loss functions Loss_Statistics, Loss_Classification, and Loss_Prediction can be selected as needed.

[0069] Step S43: Test the adaptability of the trained large language model to the application scenario, and further optimize the large language model according to the test results;

[0070] The tester tests the trained large language model based on the enterprise's recent human resource data, task specification data, and recent performance evaluation results, task scheduling, and task progress reports, verifies the adaptability of the large language model in the current application scenario, and further optimizes the parameters in the large language model according to the test results until the output results of the large language model for all recent data are within the expected range.

[0071] Step S50: deploying the fine-tuned large language model to a production environment and continuously optimizing the large language model according to user feedback data;

[0072] Although the large language model can capture long-distance dependencies efficiently after one training, its performance and applicability will gradually mismatch the current production environment in actual use, so continuous optimization is needed to ensure its stability. Specifically:

[0073] Step S51: Establish an error feedback mechanism to collect errors or inconsistencies in the model output;

[0074] The error feedback mechanism can collect output errors or inconsistencies found by users during use. The collected feedback information provides data support for continuous optimization of the large language model.

[0075] Step S52: Adjust the training data according to the error feedback information, and perform reinforcement training on the large language model based on the adjusted training data;

[0076] The error data in the error feedback information is used as a negative sample, and the corrected data is used as a positive sample. The association between the contrast learning reinforcement model and the positive sample is introduced to gradually make the model output correct. After the model output reaches the expected result, it is redeployed to the production environment for use.

[0077] Embodiment Two

[0078] As shown in Figure 2 Embodiment Two of the present application provides a data analysis system based on a large language model, which includes a data collection module 21, a data transformation module 22, a multi-task analysis body creation module 23, a multi-task analysis body integration module 24, and a large language model continuous optimization module 25.

[0079] The data collection module 21 is used to collect human resource data and task specification data of various departments in the enterprise.

[0080] The human resource data includes fields such as employee ID, department, position, skill label, available working hours, current task load, historical performance score, leave record, overtime length, project participation history, and the task detail data includes fields such as task ID, department, task description, required skill, priority, deadline, task status (not started / in progress / completed), and historical task completion time; these data are the basis for task scheduling, task progress prediction and performance evaluation in a department, and can be obtained by accessing the API of the enterprise internal system.

[0081] The data conversion module 22 is configured to convert the collected data into a high-dimensional vector recognizable by the large language model, and specifically includes the following sub-modules:

[0082] 1. A data encoding sub-module configured to encode the collected raw data;

[0083] Encoding the raw data can ensure data consistency and enable the large language model to better understand the raw data. For numerical data (such as working hours and task priority), Z-score or Min-Max normalization can be used; for classification data (such as skill labels and task status), One-Hot encoding or embedding layer can be used; and for text data (such as task description and employee skills), Word2Vec or BERT can be used.

[0084] 2. A data splicing sub-module configured to splice the encoded data into character sequences according to the original fields;

[0085] For example, there are original fields such as employee ID, skill, working hours, task ID, priority, and deadline, and the field value encoding is a1-a6. Splicing according to the original fields can obtain: [employee ID, a1] [skill, a2] [working hours, a3] [task ID, a4] [priority, a5] [deadline, a6]. Each square bracket represents a key-value pair, where the first element represents the key and the second element represents the value.

[0086] 3. A high-dimensional vector generation sub-module configured to convert the spliced character sequence into a high-dimensional vector sequence using a high-dimensional vector generation function;

[0087] The high-dimensional vector generation function takes an encoded data as an input parameter, and the return value is the high-dimensional vector converted from the encoded data. The values (value) in the character sequence are sequentially used as input parameters of the high-dimensional vector generation function, and the return value is used to replace the original value (value) to obtain a new high-dimensional vector sequence. The mathematical expression of the high-dimensional vector generation function is:

[0088] F(v) = (v W + b) softmax(W Q ReLU((v W + b) W K ), where v is an input argument, W is a linear transformation matrix, b is a bias vector, W Q and W K are a query matrix and a key matrix in a large language model self-attention mechanism, respectively, softmax represents a softmax function, and ReLU represents a ReLU function.

[0089] The multi-task analysis body creation module 23 is configured to create a multi-task analysis body based on the generated high-dimensional vectors, while meeting the three data analysis requirements of task scheduling, task progress prediction, and performance evaluation. The multi-task analysis body creation module 23 includes the following sub-modules:

[0090] 1. A multi-layer analysis structure creation sub-module, configured to create a multi-layer analysis structure and set a high-dimensional vector extractor at the entrance of each analysis layer.

[0091] The multi-layer analysis structure is created to isolate different analysis requirements. In this embodiment, three analysis requirements are proposed, so three analysis layers are created to meet the three analysis requirements. The high-dimensional vector extractor is used to extract the high-dimensional vectors required by the current layer from the input data. For example, when scheduling tasks, the skill tags, available working hours, current task load, project participation history of employees, and the description, required skills, priority, and deadline of the task are considered comprehensively, and each task is assigned to the most suitable person. Therefore, the high-dimensional vector extractor of the task scheduling analysis layer needs to extract the high-dimensional vectors converted from these field values as the analysis data of the current layer.

[0092] 2. An analysis layer adjustment sub-module, configured to adjust the sub-models in each analysis layer according to the analysis requirements to be met.

[0093] Each newly created analysis layer includes a default sub-model, which is represented as:

[0094] where Y is the output of the sub-model, x i is the i-th analysis data (the i-th high-dimensional vector extracted to the current layer), is the weight vector of the i-th analysis data, b i is the bias vector of the i-th analysis data, L is a normalization constant, i takes a value from 1 to n, and n is the total number of analysis data in the current layer.

[0095] This sub-model is applicable to general statistical problems. For example, the performance evaluation proposed in this embodiment is a statistical problem, so the sub-model within the analysis layer responsible for performance evaluation does not need any modification. However, when facing classification and prediction problems, the sub-model within the analysis layer needs to be customized. Specifically:

[0096] For classification problems, such as the task scheduling proposed in this embodiment, the analysis layer responsible for this requirement needs to output the employee that best matches the current task. The personalized adjusted analysis layer sub-model is represented as follows:

[0097] Where Y is the output of the sub-model, X j Let be the historical match rate between the j-th employee and the current task (obtained by dividing the number of times the task appears in the project participation history by the total number of tasks). Let x be the matching coefficient between the u-th employee feature (the high-dimensional vector describing the employee extracted to this layer) and the e-th task feature (the high-dimensional vector describing the task extracted to this layer). u Let b represent the characteristic of the u-th employee. u Let be the bias vector of the u-th employee feature, where u takes values ​​from 1 to U, U is the total number of employee features, e takes values ​​from 1 to E, E is the total number of task features, L is a normalization constant, and j takes values ​​from 1 to m, where m is the total number of employees in a department.

[0098] For prediction-type problems, such as the task progress prediction proposed in this embodiment, the analysis layer responsible for this requirement needs to output the progress prediction result of a certain task within a future period. The customized analysis layer sub-model is represented as follows:

[0099] Where Y is the output of the sub-model, ω T Let x be the output weight matrix of the sub-model. i For the i-th analysis data, Let b be the weight vector for the i-th analysis data. i Let λ be the bias vector for the i-th analysis data. i Let σ be the trend parameter of the i-th analysis data. T L is the correction matrix for the sub-model, and i is a normalization constant. The value of i ranges from 1 to n, and n is the total number of data points analyzed in this layer.

[0100] 3. Output rule setting submodule, used to set the output rules of the multi-layer analysis structure to obtain the final multi-task analysis body;

[0101] The analysis results returned by different analysis layers are integrated by concatenating characters. The integrated data is used as the final output of the multi-layer analysis body. During concatenation, the results of different layers are separated by special symbols.

[0102] A multi-task analysis body integration module 24 is configured to integrate the created multi-task analysis body into a large language model and fine-tune the model based on historical data. The module includes the following sub-steps:

[0103] 1. A training data set creation sub-module is configured to process historical human resource data and historical task detail data of each department in the enterprise into a training data set.

[0104] The historical human resource data and historical task detail data of each department are taken as input, and the corresponding information in the historical performance evaluation results, historical task scheduling, and task progress reports is taken as output. After processing, the training data set is obtained.

[0105] 2. A large language model training sub-module is configured to perform supervised training on the integrated large language model using the training data set and fine-tune the parameters in the large language model based on the minimum loss principle.

[0106] The integrated large language model has multi-task analysis capability, and its loss function is represented as: Loss = a * Loss_Statistics + b * Loss_Classification + g * Loss_Prediction, where a, b, and g are loss evaluation weights for different tasks, which need to be dynamically adjusted during training to meet the performance requirements of the model in different analysis tasks. The three loss functions Loss_Statistics, Loss_Classification, and Loss_Prediction can be selected as needed.

[0107] 3. A large language model testing sub-module is configured to test the adaptability of the trained large language model to the application scenario and further optimize the large language model based on the test results.

[0108] The tester tests the trained large language model based on the enterprise's recent human resource data, task detail data, and recent performance evaluation results, task scheduling, and task progress reports to verify its adaptability in the current application scenario. Further optimization of the parameters in the large language model is performed based on the test results until the output results of the large language model for all recent data are within the expected range.

[0109] A large language model continuous optimization module 25 is configured to deploy the fine-tuned large language model to a production environment and continuously optimize the large language model based on user feedback data. The module includes the following sub-modules:

[0110] 1. A feedback information collection sub-module is configured to set up an error feedback mechanism to collect errors or inconsistencies in the model output.

[0111] The error feedback mechanism can collect output errors or inconsistencies found by the user during use, and the collected feedback information provides data support for continuous optimization of the large language model.

[0112] 2. The reinforcement training submodule is configured to adjust the training data according to the error feedback information, and to perform reinforcement training on the large language model based on the adjusted training data.

[0113] The error data proposed in the error feedback information is used as a negative sample, and the corrected data is used as a positive sample. The association between the contrast learning reinforcement model and the positive sample is introduced to gradually make the model output correct, and the model output result is redeployed to the production environment for use after reaching the expected result.

[0114] Corresponding to the above-mentioned embodiments, the embodiments of the present application provide a computer storage medium, comprising: at least one memory and at least one processor.

[0115] The memory is configured to store one or more program instructions.

[0116] The processor is configured to execute the one or more program instructions to perform a data analysis method based on a large language model.

[0117] Corresponding to the above-mentioned embodiments, the embodiments of the present application provide a computer readable storage medium, the computer storage medium contains one or more program instructions, the one or more program instructions are used to be executed by the processor to perform a data analysis method based on a large language model.

[0118] The embodiments disclosed by the present application provide a computer readable storage medium, the computer readable storage medium stores computer program instructions, when the computer program instructions run on the computer, make the computer execute the above-mentioned data analysis method based on a large language model.

[0119] In the embodiments of the present application, the processor can be an integrated circuit chip with signal processing capability. The processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP for short), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC for short), a field programmable gate array (Field Programmable Gate Array, FPGA for short) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0120] The disclosed methods, steps, and logic block diagrams in the embodiments of the present application can be implemented or performed with a general- purpose processor, a special purpose processor, or any other processor. The steps of the methods disclosed in the embodiments of the present application can be directly embodied to a hardware code, a processor code, or a software code for execution by a processor. The software code can reside in the storage media such as the random access memory (RAM), the flash memory, the read only memory (ROM), the programmable read only memory (PROM), the electrically programmable read only memory (EPROM), the electrically erasable programmable read only memory (EEPROM), the compact disk (CD), the digital versatile disk (DVD), the Blu-ray disk, the hard disk drive (HDD), or any other storage medium. The processor reads information in the storage medium, and performs the steps of the methods in combination with hardware of the processor.

[0121] The storage medium can be the memory, for example, the volatile memory or the non-volatile memory, or can include both the volatile and non-volatile memory.

[0122] The non-volatile memory can be the read only memory (ROM), the programmable read only memory (PROM), the erasable programmable read only memory (EPROM), the electrically EPROM (EEPROM), or the flash memory.

[0123] The volatile memory can be the random access memory (RAM) used as the external cache. By way of example, and not limitation, many forms of RAM are available, for example, the static random access memory (SRAM), the dynamic random access memory (DRAM), the synchronous dynamic random access memory (SDRAM), the double data rate SDRAM (DDR SDRAM), the enhanced SDRAM (ESDRAM), the Synchlink DRAM (SLDRAM), and the direct Rambus RAM (DRRAM).

[0124] The storage medium described in the embodiments of the present application is intended to include, but not limited to, these and any other suitable types of memory.

[0125] Those skilled in the art should be aware that, in one or more examples described above, functions described by the present application can be implemented in combination of hardware and software. When the software is applied, the corresponding functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium. The computer readable medium includes computer storage medium and communication medium, wherein the communication medium includes any medium that facilitates transfer of a computer program from one place to another. The storage medium can be any available medium accessible by a general or special purpose computer.

[0126] The above detailed description further describes the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application should be included in the protection scope of the present application.

Claims

1. A data analysis method based on a large language model, characterized in that, include: Step 1: Collect human resource data and task details from various departments within the company; Step 2: Convert the collected data into high-dimensional vectors that can be recognized by the large language model; Step 3: Create a multi-task analysis body based on the generated high-dimensional vector, which simultaneously meets the three data analysis requirements of task scheduling, task progress prediction, and performance evaluation. The specific creation process of a multi-task analysis body includes the following sub-steps: Create a multi-layered analysis structure and set a high-dimensional vector extractor at the entrance of each analysis layer; Create a multi-layered analysis structure to isolate the needs of different analyses, and use a high-dimensional vector extractor to extract the high-dimensional vectors required for this layer from the input data; The sub-models within each analysis layer are customized according to the analytical needs to be met; Each newly created analysis layer contains a default sub-model, represented as: ,in For the output of the sub-model, For the i-th analysis data, Let i be the weight vector of the i-th data point being analyzed. Let L be the bias vector for the i-th analysis data, where L is a normalization constant, i takes values ​​from 1 to n, and n is the total number of analysis data in this layer. The default sub-model is suitable for general statistical problems, so the sub-models within the performance evaluation analysis layer do not require any modification. However, when dealing with classification and prediction problems, the sub-models within the analysis layer need to be customized. Specifically: For classification problems, the personalized adjusted analysis layer sub-model is represented as follows: ,in For the output of the sub-model, Let be the historical match rate between the j-th employee and the current task. Let be the matching coefficient between the u-th employee feature and the current e-th task feature. This represents the characteristic of the u-th employee. Let be the bias vector of the u-th employee feature, where u takes values ​​from 1 to U, U is the total number of employee features, e takes values ​​from 1 to E, E is the total number of task features, L is a normalization constant, and j takes values ​​from 1 to m, where m is the total number of employees in a department. For prediction problems, the customized analysis layer sub-model is represented as follows: ,in For the output of the sub-model, This is the output weight matrix of the sub-model. For the i-th analysis data, Let i be the weight vector of the i-th data point being analyzed. Let be the bias vector for the i-th analysis data. For the i-th analysis data, L is the correction matrix for the sub-model, and i is a normalization constant. The value of i ranges from 1 to n, and n is the total number of data points analyzed in this layer. Set the output rules for the multi-layered analysis structure to obtain the final multi-task analysis body; Step 4: Integrate the created multi-task analysis body into the large language model, and fine-tune the model based on historical data; Step 5: Deploy the fine-tuned large language model to the production environment and continuously optimize the large language model based on user feedback data.

2. The data analysis method based on a large language model according to claim 1, characterized in that, The collected data is transformed into high-dimensional vectors that can be recognized by large language models, which is specifically divided into the following sub-steps: The collected raw data is encoded. Concatenate the encoded data into a character sequence according to the original fields; Use a high-dimensional vector generation function to convert the concatenated character sequence into a high-dimensional vector sequence.

3. The data analysis method based on a large language model according to claim 1, characterized in that, A multi-task analysis volume is created based on the generated high-dimensional vector, which consists of the following sub-steps: Create a multi-layered analysis structure and set a high-dimensional vector extractor at the entrance of each analysis layer; The sub-models within each analysis layer are customized according to the analytical needs to be met; By setting the output rules of the multi-layered analysis structure, the final multi-task analysis body is obtained.

4. The data analysis method based on a large language model according to claim 1, characterized in that, Integrate the created multi-task analysis body into the large language model, and fine-tune the model based on historical data. This process is divided into the following sub-steps: Add a fully connected layer to the large language model and deploy the created multi-task analysis body to the fully connected layer so that it can share the self-attention mechanism of the large language model; The historical human resources data and historical task details data of each department within the enterprise are processed into a training dataset. The integrated large language model is trained in a supervised manner using the training dataset, and the parameters within the large language model are tuned based on the principle of minimum loss. The large language model trained is tested for its adaptability to application scenarios, and further optimization is performed on the large language model based on the test results.

5. The data analysis method based on a large language model according to claim 1, characterized in that, The large language model is continuously optimized based on user feedback data, specifically through the following sub-steps: Establish an error feedback mechanism to collect errors or inconsistencies in the model output; The training data is adjusted in response to error feedback, and the large language model is reinforced based on the adjusted training data.

6. A data analysis system based on a large language model, characterized in that, To perform a data analysis method based on a large language model as described in any one of claims 1-5, comprising: The data collection module is used to collect human resource data and task details from various departments within the enterprise. The data transformation module is used to transform the collected data into high-dimensional vectors that can be recognized by large language models; The multi-task analysis body creation module is used to create a multi-task analysis body based on the generated high-dimensional vector, which simultaneously meets the three data analysis needs of task scheduling, task progress prediction and performance evaluation. The multi-task analysis body integration module is used to integrate the created multi-task analysis body into the large language model and fine-tune the model based on historical data; The large language model continuous optimization module is used to deploy the fine-tuned large language model to the production environment and continuously optimize the large language model based on user feedback data.

7. A data analysis system based on a large language model according to claim 6, characterized in that, The data transformation module specifically includes the following sub-modules: The data encoding submodule is used to encode the collected raw data. The data concatenation submodule is used to concatenate encoded data into a character sequence based on the original fields. The high-dimensional vector generation submodule is used to convert a concatenated character sequence into a high-dimensional vector sequence using high-dimensional vector generation functions.

8. A data analysis system based on a large language model according to claim 6, characterized in that, The multi-task analysis body creation module specifically includes the following sub-modules: The multi-level analysis structure creation submodule is used to create a multi-level analysis structure and set a high-dimensional vector extractor at the entrance of each analysis layer; The analysis layer adjustment submodule is used to personalize the adjustment of sub-models within each analysis layer according to the analysis requirements to be met; The output rule setting submodule is used to set the output rules for the multi-layer analysis structure to obtain the final multi-task analysis body.

9. A data analysis system based on a large language model according to claim 6, characterized in that, The multi-task analysis module integration module specifically includes the following sub-modules: The training dataset creation submodule is used to process historical human resources data and historical task details data from various departments within the enterprise into training datasets. The large language model training submodule is used to perform supervised training on the integrated large language model using the training dataset and to fine-tune the parameters within the large language model based on the principle of minimum loss. The large language model testing submodule is used to test the adaptability of the trained large language model to the application scenario, and to further optimize the large language model based on the test results.

10. A computer storage medium, characterized in that, include: At least one memory and at least one processor; Memory, used to store one or more program instructions; A processor for running one or more program instructions to perform a data analysis method based on a large language model as described in any one of claims 1-5.

Images