Chart replacement text quality automatic evaluation method and system based on large language model

Abstract

The application discloses a kind of based on big language model's chart alternative text quality automation evaluation method and system, it is related to big language model field and chart alternative text field, including: by demand investigation, obtain chart alternative text quality evaluation scheme;Artificially construct chart alternative text evaluation standard data set carries out data preprocessing, based on GLM4-9B model carries out Finetune training and performance tuning.The chart alternative text quality automation evaluation method proposed in the application is based on chart accessibility reading criteria to design evaluation scheme, and the prediction score of the transfer model obtained by training is highly correlated with the artificial evaluation score, which can provide high-reliability automated chart alternative text quality evaluation results and can be applied to text quality evaluation in the field of chart alternative text.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of natural language processing, and particularly relates to a chart replacement text quality automatic evaluation method and system based on a large language model. BACKGROUND

[0002] Chart replacement text is a text used to describe the content of a chart, and is usually used to help visually impaired persons understand the content of a chart, and plays an important role in promoting the development of information accessibility. In recent years, the development of natural language processing technology has promoted the progress of automatic chart replacement text generation technology, and current multi-modal large language models can receive chart pictures input by users and provide descriptions about the content of the charts. However, due to the lack of reliable automatic evaluation methods for chart replacement text quality, the quality of chart replacement text generated by these large language models cannot be effectively verified at present.

[0003] Existing chart replacement text evaluation methods mainly include manual evaluation methods and automatic evaluation methods. The manual evaluation method is not only inefficient, but also is easily affected by subjective factors; and mainstream automatic text quality evaluation methods (such as BLEU, ROUGE, etc.) mainly evaluate the quality of replacement text based on the content similarity between reference text and generated text, and have low relevance to manual evaluation, so such scores do not have reference significance. In recent years, large language models based on the Transformer architecture have made major breakthroughs in natural language understanding, but in the field of automatic evaluation of chart replacement text quality, due to the lack of professional background knowledge in the field of barrier-free chart understanding, it is impossible to reliably evaluate the quality of chart replacement text. Therefore, it is urgent to invent a chart replacement text quality automatic evaluation method based on a large language model to solve the problem that current chart replacement text quality evaluation cannot achieve automatic evaluation. SUMMARY

[0004] The present application aims at the deficiencies of the prior art, and provides a chart replacement text quality automatic evaluation method and system based on a large language model.

[0005] The purpose of the present application is achieved by the following technical solutions: a chart replacement text quality automatic evaluation method based on a large language model, comprising the following steps:

[0006] S1, determining the evaluation dimensions and weights of chart replacement text according to the barrier-free chart reading guide and the chart reading needs of visually impaired users;

[0007] S2, performing picture preprocessing on the chart to be evaluated, constructing the chart replacement text to be evaluated, the reference chart replacement text, and the manual evaluation score as a data set, and performing text data preprocessing on the text data of the data set;

[0008] S3, model post-processing is performed on the GLM4 model, and the output prediction value of the model is defined; an input of the GLM4 model is pre-processed data, and an output is a prediction value defined in post-processing; the pre-processed data set is input into the GLM4 model after post-processing and fine-tuning training is performed, and the trained migration model is tested and model performance is optimized; wherein the output prediction value of the GLM4 model is obtained by referring to the n-gram overlap degree between the chart replacement text and the chart replacement text to be evaluated, and the weight of each evaluation dimension;

[0009] S4, input the chart replacement text into the optimized GLM4 model to obtain the final evaluation score.

[0010] Further, the S1 includes the following sub-steps:

[0011] Investigate the chart reading needs of visually impaired users, conduct user research, set the evaluation dimensions of the chart replacement text as chart title, chart type, chart coordinate axis, chart specific value, color information description and chart trend analysis, and obtain the user importance score of each dimension through the research results, and obtain the weight W of each dimension through the user importance score of each dimension i :

[0012]

[0013] Wherein, A i is the mean of the user importance score of each dimension.

[0014] Further, the construction of the chart replacement text to be evaluated, the reference chart replacement text and the artificial evaluation score specifically includes: the chart replacement text to be evaluated is obtained by inputting the chart picture into a large language model; the reference chart replacement text is manually made according to the evaluation dimensions for the chart picture, and the artificial evaluation score is obtained by manually evaluating the chart replacement text to be evaluated.

[0015] Further, the standard of the artificial evaluation specifically includes:

[0016] In the chart type dimension, give one point for the correct chart type; in the chart title dimension, give one point for the correct description of the chart title; no description or incomplete chart title description, no points; in the chart coordinate axis dimension, give 0.25 points for the correct description of the horizontal coordinate title, label range, vertical coordinate title, label range; in the chart specific value dimension, the score = the number of correctly described chart plotting area data points / the total number of chart plotting area data points; in the color information description dimension, the score = the number of correctly described colors / the total number of colors in the chart; in the chart trend analysis dimension, give one point for the relevant description of the data trend presented by the chart; no description or description unrelated to the chart data, no points.

[0017] Further, the picture preprocessing includes non-horizontal picture correction, picture size standardization, and smoothing processing.

[0018] The text data preprocessing includes text cleaning, word segmentation, special character removal, standardization processing, and feature splicing; the feature splicing is to convert the to-be-evaluated chart replacement text, reference chart replacement text, and artificial evaluation score into a numerical vector representation using a pre-trained word embedding model, and splice them into a new feature vector; the standardization processing specifically includes:

[0019]

[0020] wherein S normalized is the standardized artificial evaluation score, S is the original artificial evaluation score, S min is the minimum value of the artificial evaluation score; S max is the maximum value of the artificial evaluation score.

[0021] Further, the feature splicing includes: combining the standardized artificial evaluation score with the text features; constructing a feature vector X containing the embedding vector T of the to-be-evaluated chart replacement text and the embedding vector Q of the reference chart replacement text, and the artificial evaluation score S normalized ;

[0022] X = [T; Q; S normalized ]

[0023] wherein ";" represents vector splicing operation.

[0024] Further, the model post-processing specifically includes:

[0025] Using the ROUGE evaluation method, the n-gram overlap degree between the reference chart replacement text and the to-be-evaluated chart replacement text is measured, the evaluation dimension weight is combined, and the output prediction value E of the model is customized:

[0026]

[0027] wherein, R1 represents n-gram precision, representing the proportion of n-gram overlap between the to-be-evaluated alternative text and the reference alternative text; P i represents precision, which is the proportion of the number of correctly described word vectors in the to-be-evaluated alternative text to the total number of word vectors in the to-be-evaluated alternative text; W i represents the respective weight of each evaluation dimension, r i represents recall, which is the proportion of the number of correctly described word vectors in each evaluation dimension of the to-be-evaluated alternative text to the total number of word vectors in each evaluation dimension of the reference alternative text; α and β are balance coefficients for adjusting the weight relationship between n-gram precision, precision, and recall of the key evaluation dimension.

[0028] Further, the loss function used in the fine-tuning training is:

[0029]

[0030] wherein, L(θ) is the loss function, N is the number of samples in the data set, Si is the actual manual evaluation score of the i-th sample, Zi is the predicted score of the i-th sample, and θ is the model parameter.

[0031] Further, the testing and model performance tuning includes:

[0032] The performance of the transfer model is evaluated using the validation set in the chart alternative text evaluation data set, using a correlation index to calculate the difference between the predicted score and the actual manual evaluation score, and the correlation index is:

[0033]

[0034] wherein, ρ is the Pearson correlation coefficient; S i is the actual manual evaluation score of the i-th sample, is the average of the actual manual evaluation score; Z i is the predicted score of the i-th sample, is the average of the predicted score; N is the number of samples;

[0035] According to the performance of the model test, the transfer model is optimized, including hyperparameter adjustment and optimization algorithm.

[0036] According to another aspect of the specification, a chart alternative text quality automatic evaluation system based on a large language model is also provided, which includes a data set construction module, a data preprocessing module, a model training module, and a model testing and performance tuning module.

[0037] The data set construction module is used for collecting high-quality reference chart alternative text, chart alternative text to be evaluated, artificial evaluation scores, and constructing a chart alternative text evaluation data set;

[0038] The data preprocessing module is used for preprocessing data set data, including text cleaning, word segmentation, special character removal, standardization processing and feature splicing, and dividing the data set into a training set, a verification set and a test set for model training;

[0039] The model training module selects GLM4-9B as a basic model, defines a model output prediction value E in combination with reading preferences of visually impaired users, defines a loss function L(θ), uses the chart alternative text evaluation data set to perform Finetune training on the GLM4 model, and continuously adjusts model weights to minimize the loss function;

[0040] The model testing and performance tuning module uses the verification set in the chart alternative text evaluation data set to perform performance evaluation on the performance of the migration model, uses a correlation index to calculate the difference between the model prediction score and the actual artificial evaluation score, and tunes the migration model according to the model testing performance.

[0041] The present application has the following advantages:

[0042] 1. The present application is based on the actual needs of visually impaired users, combined with the current information accessibility field for the design criteria of chart alternative text content, and proposes a new 6-dimensional evaluation scheme for chart alternative text quality.

[0043] 2. The present application provides an automatic chart alternative text quality evaluation method for the current chart alternative text research field, which combines the mainstream evaluation index based on text content similarity and the index in the field of chart information accessibility, and can provide efficient and reliable quality evaluation results for chart alternative text.

[0044] 3. The chart alternative text quality automatic evaluation method based on the large language model proposed in the present application can be deployed and run on different computing devices, and can quickly build an efficient, professional and reliable chart alternative text quality automatic evaluation system.

[0045] 4. The present application can solve the problem of lack of reliable automatic evaluation method in the current chart alternative text generation field, and further promote the related research in the field of chart alternative text automatic generation.

[0046] 5. The present application constructs the current scarce chart alternative text evaluation data set which is completely derived from the real world, wherein the construction of the reference chart alternative text and the artificial evaluation score is loyal to the chart accessibility criteria, has high reference significance, and can provide data set support for chart alternative text evaluation work in the field. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a flow chart of a large language model-based chart replacement text quality automatic evaluation method.

[0048] Figure 2 is a schematic diagram of a large language model-based chart replacement text quality automatic evaluation device. DETAILED DESCRIPTION

[0049] The present application will be described in detail below with reference to the accompanying drawings. The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application and its applications or uses. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0050] Referring to Figure 1 , the large language model-based chart replacement text quality automatic evaluation method of the present application comprises the following steps:

[0051] Step 1: Design evaluation scheme

[0052] Conduct literature research in the field, and based on the current accessibility standards in the field of chart reading, preliminarily set 5 evaluation dimensions. Through offline research on the chart reading needs of visually impaired users, understand the types of information that visually impaired users expect to obtain when reading charts, and the research data shows that the proportion of users who have access needs for color information is more than 80%, therefore, on the basis of the original 5 evaluation dimensions, add 1 color information description evaluation dimension. According to the average importance score of users on the 6 content dimensions, obtain the weight percentage W i of the 6 evaluation dimensions, and establish a reliable chart replacement text quality evaluation scheme. The 6 evaluation dimensions are: chart title, chart type, chart coordinate axis, chart specific value, color information description, and chart trend analysis. The calculation formula of the weight W i of each dimension is as follows:

[0053]

[0054] Among them, A i is the average importance score of each dimension by users.

[0055] Step 2: Build dataset: collect reference chart replacement text, chart replacement text to be evaluated, artificial evaluation score, and build chart replacement text evaluation dataset; preprocess the text data in the chart replacement text evaluation dataset, which includes text cleaning, word segmentation, removal of special characters, standardization processing, and feature splicing; divide the chart replacement text evaluation dataset into training set, validation set and test set.

[0056] In the embodiment, the construction data set further comprises:

[0057] Collect different types of chart pictures and perform picture preprocessing, the picture preprocessing including non-horizontal picture correction, picture size standardization, and smoothing processing; input the chart pictures into a large language model to obtain chart replacement text to be evaluated output by the large language model; manually make reference chart replacement text for the chart pictures, and perform manual evaluation on the chart replacement text to be evaluated to obtain a manual evaluation score S:

[0058]

[0059] wherein Wi represents the weight of each of the six key evaluation dimensions, and s i represents the manual evaluation score of each dimension.

[0060] The manual evaluation standards of each dimension are as follows:

[0061]

[0062]

[0063] In the embodiment, the standardization processing is to standardize the manual evaluation scores in the data set, specifically:

[0064]

[0065] wherein S normalized is the standardized manual evaluation score, S is the original manual evaluation score, S min is the minimum value of the manual evaluation score; S max is the maximum value of the manual evaluation score. The feature splicing includes: combining the standardized manual evaluation score with the text features. A feature vector X is constructed by splicing the embedding vector T of the chart replacement text to be evaluated, the embedding vector Q of the reference chart replacement text, and the standardized manual evaluation score S normalized .

[0066] X = [T; Q; S normalized ]

[0067] The chart pictures are all from the real world and have real significance. The Chinese charts are from the China Economic and Social Big Data Research Platform, the Beijing Judicial Bureau, and the China Internet Network Information Center, and the English charts are from the Pew Research Center (PewResearch Center https: / / www.pewresearch.org).

[0068] Step three: model selection and post-processing: GLM4-9B model is selected as the base model; in order to optimize the prediction results of the model and make it better adapt to specific application scenarios, the original output of the model is customized. The ROUGE evaluation method is used to measure the n-gram overlap between the reference chart replacement text and the chart replacement text to be evaluated; at the same time, the visual impairment user chart reading preference is fully considered, and the evaluation dimension weight in step one is combined. The prediction value E of the model output is as follows:

[0069]

[0070] Wherein, R1 represents the n-gram overlap ratio between the replacement text to be evaluated and the reference replacement text; P i represents the proportion of the number of correctly described word vectors in the replacement text to be evaluated to the total number of word vectors in the replacement text to be evaluated; Wi represents the weight of each of the six key evaluation dimensions, r i represents the recall rate, which is the proportion of the number of correctly described word vectors in each evaluation dimension of the replacement text to be evaluated to the total number of word vectors in each evaluation dimension of the reference replacement text. The balance coefficients a and β are used to adjust the weight relationship between n-gram accuracy, precision and recall rate of key evaluation dimensions. In this embodiment, based on the optimal model training effect, the balance coefficient value selected is 0< a+ β <0.4.

[0071] Step four: fine-tune training and model performance tuning: fine-tune the GLM4 model using the chart replacement text evaluation dataset, and test the trained transfer model and optimize the model performance;

[0072] In this embodiment, the step four further comprises a customized loss function:

[0073] The loss function is defined to make the model's predicted score Z as close as possible to the actual human evaluation score S, and the loss function is:

[0074]

[0075] Wherein, L(θ) is the loss function, N is the number of samples in the dataset, Si is the actual human evaluation score of the i-th sample, Zi is the predicted score of the i-th sample, and θ is the model parameter.

[0076] In this embodiment, the step four further comprises model training:

[0077] Based on the training set divided in step two, the processed feature vector X in step two is taken as the model input, and reasonable model training parameters are set, so that the model training can be performed. The prediction score of the original output of the model is calculated based on the model prediction value formula E defined in step three, and the first training parameters in this embodiment are as follows:

[0078] epochs = 50; batch_size = 20; learning_rate = 2e -5 ; weight_decay = 0.01

[0079] In this embodiment, the step four further comprises model test performance:

[0080] The performance of the migration model is evaluated using a chart instead of a text evaluation dataset, and the difference between the predicted score and the actual manual evaluation score is calculated using the Pearson correlation coefficient. In this embodiment, the correlation index is:

[0081]

[0082] Where, p is the Pearson correlation coefficient; S i is the actual manual evaluation score of the i-th sample, is the average of the actual manual evaluation score; Z i is the predicted score of the i-th sample, is the average of the predicted score; N is the number of samples.

[0083] In this embodiment, the step four further comprises model performance tuning:

[0084] Different optimization algorithms (such as Adam, SGD, etc.) are tried, and their effects are compared. The closer the Pearson correlation coefficient p of the model is to 1, the better the training effect is. Different hyperparameter combinations are tried, the learning rate function, batch size, training rounds, and regularization weight are adjusted, the model training results of different hyperparameter combinations are compared, and the model with the best training effect is automatically saved.

[0085] Step five, using the optimized model to evaluate the quality of the chart instead of the text, and obtaining the evaluation score.

[0086] Corresponding to the foregoing embodiment of the method for automatically evaluating the quality of chart instead of text based on a large language model, the present application also provides an embodiment of a system for automatically evaluating the quality of chart instead of text based on a large language model.

[0087] According to another aspect of the embodiment of the present application, the specification of the present application further provides a system for automatically evaluating the quality of chart instead of text based on a large language model, comprising:

[0088] a data set construction module for collecting high-quality reference chart alternative text, chart alternative text to be evaluated, and manual evaluation scores, and constructing a chart alternative text evaluation data set;

[0089] a data preprocessing module for preprocessing data set data, including text cleaning, word segmentation, special character removal, standardization processing, and feature splicing, and dividing the data set into a training set, a validation set, and a test set for model training.

[0090] a model training module for selecting GLM4-9B as a base model, customizing model prediction value E in combination with reading preferences of visually impaired users, customizing a loss function L(θ), and using the chart alternative text evaluation data set to perform Finetune training on the GLM4 model to continuously adjust model weights to minimize the loss function.

[0091] a model testing and performance tuning module for performing performance evaluation on the performance of the transfer model using the validation set in the chart alternative text evaluation data set, calculating the difference between the model prediction score and the actual manual evaluation score using a correlation index, and tuning the transfer model according to the model testing performance, including hyperparameter adjustment and optimization algorithms such as Adam and SGD.

[0092] Corresponding to the foregoing embodiment of the chart alternative text quality automatic evaluation method based on a large language model, the present application also provides an embodiment of a chart alternative text quality automatic evaluation device based on a large language model.

[0093] Referring to Figure 2 The embodiment of the chart alternative text quality automatic evaluation device based on a large language model provided by the present application comprises a memory and one or more processors, the memory stores executable code, and the processor executes the executable code to implement the chart alternative text quality automatic evaluation method based on a large language model in the foregoing embodiments.

[0094] The embodiment of the chart alternative text quality automatic evaluation device based on a large language model provided by the present application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be realized by software, hardware, or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running by the processor of the device with data processing capability. From the hardware level, for example, Figure 2As shown in the figure, it is a hardware structure diagram of any data processing capable device where the large language model based chart replacement text quality automatic evaluation device provided by the application is located, in addition to Figure 2 In addition to the processor, the memory, the network interface, and the non-volatile memory shown in the figure, any data processing capable device where the device in the embodiment is located can also include other hardware according to the actual functions of the data processing capable device, and details are not described here.

[0095] The implementation process of the functions and roles of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, and details are not described here.

[0096] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part is described in the method embodiment. The device embodiment described above is only schematic, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, some or all of the modules can be selected to achieve the purpose of the present application scheme. Those skilled in the art can understand and implement without creative labor.

[0097] The embodiment of the application also provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the large language model based chart replacement text quality automatic evaluation method in the above embodiment.

[0098] The computer readable storage medium can be an internal storage unit of any data processing capable device, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device of any data processing capable device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of any data processing capable device. The computer readable storage medium is used to store the computer program and other programs and data required by the data processing capable device, and can also be used to temporarily store data that has been output or will be output.

[0099] The application also provides a computer program product, which includes a computer program, and the computer program is executed by a processor to realize the large language model based chart replacement text quality automatic evaluation method.

[0100] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0101] It is to be understood that the above general description and the subsequent detailed description are merely exemplary and explanatory and are not restrictive of the application. The application is not restricted to the precise construction and combinations of parts and steps described above and shown in the accompanying drawings, as such variations and modifications are intended to be included within the scope of the application. The scope of the application is limited only by the claims that follow.

Claims

1. A large language model-based chart replacement text quality automatic evaluation method, characterized in that, The method comprises the following steps: S1, determining the evaluation dimensions and weights of chart alternative text according to the barrier-free chart reading guide and the chart reading needs of visually impaired users; S2, obtaining a chart to be evaluated, performing picture preprocessing, constructing chart alternative text to be evaluated, reference chart alternative text, and artificial evaluation score as a data set, and performing text data preprocessing on the data set text data; S3, performing model post-processing on the GLM4 model, and defining the output prediction value of the model; The input of the obtained GLM4 model is the preprocessed data, and the output is the prediction value defined in the post-processing; the preprocessed data set is input into the GLM4 model after post-processing and fine-tuning training, and the trained transfer model is tested and the model performance is optimized; The output prediction value of the GLM4 model is calculated by the n-gram overlap degree between the reference chart alternative text and the chart alternative text to be evaluated, and the weight of each evaluation dimension; S4, inputting the chart alternative text into the optimized GLM4 model to obtain the final evaluation score.

2. The method of claim 1, wherein the method is based on a large language model. The S1 comprises the following sub-steps: Investigate the chart reading needs of visually impaired users, conduct user research, set the evaluation dimensions of chart alternative text as chart title, chart type, chart coordinate axis, chart specific value, color information description and chart trend analysis, and obtain the user importance scores of each dimension through the research results, and obtain the weight of each dimension through the user importance scores of each dimension : ， where A i is the mean of the user importance score for each dimension.

3. The method of claim 1, wherein the method is based on a large language model. The construction of the chart alternative text to be evaluated, the reference chart alternative text, and the artificial evaluation score specifically comprises: the chart alternative text to be evaluated is obtained by inputting the chart picture into a large language model; the reference chart alternative text is manually made according to the evaluation dimensions for the chart picture, and the artificial evaluation score is obtained by manually evaluating the chart alternative text to be evaluated.

4. The method of claim 2, wherein the method is based on a large language model. The standard of the artificial evaluation specifically comprises: In the chart type dimension, one point is obtained for correctly giving the chart type; in the icon title dimension, one point is obtained for correctly describing the chart title; no description or incomplete description of the chart title is not scored; in the chart coordinate axis dimension, 0.25 points are obtained for correctly describing the horizontal coordinate title, label range, vertical coordinate title, and label range; in the chart specific value dimension, the score = the number of correctly described chart plotting area data points / the total number of chart plotting area data points; in the color information description dimension, the score = the number of correctly described colors / the total number of colors in the chart; in the chart trend analysis dimension, one point is obtained for describing the data trend presented by the chart; no description or description unrelated to the chart data is not scored.

5. The method of claim 1, wherein the method is based on a large language model. The picture preprocessing comprises non-horizontal picture correction, picture size standardization, and smoothing processing; The text data preprocessing comprises text cleaning, word segmentation, special character removal, standardization processing, and feature splicing; the feature splicing is to convert the chart alternative text to be evaluated, the reference chart alternative text, and the artificial evaluation score into a numerical vector representation using a pre-trained word embedding model, and splice them into a new feature vector; the standardization processing specifically comprises: ， where S normalized is the normalized human evaluation score, S is the raw human evaluation score, S min is the minimum value of the human evaluation score; S max is the maximum value of the human evaluation score.

6. The method of claim 5, wherein the method is based on a large language model. The feature splicing comprises: combining the normalized artificial evaluation score with the text features; constructing a feature vector X comprising an embedding vector T of the to-be-evaluated chart substitute text and an embedding vector Q of the reference chart substitute text, and the artificial evaluation score S normalized ; ， Where ";" represents vector splicing operation.

7. The method of claim 1, wherein the method is based on a large language model. The model post-processing specifically comprises: Using the ROUGE evaluation method, the degree of n-gram overlap between the reference chart replacement text and the chart replacement text to be evaluated is measured, and the output prediction value of the custom model is combined with the evaluation dimension weight : ， wherein, R1 represents n-gram accuracy, representing the proportion of n-gram overlap between the to-be-evaluated alternative text and the reference alternative text; P i represents precision, which is the proportion of the number of correct word vectors in the to-be-evaluated alternative text to the total number of word vectors in the to-be-evaluated alternative text; W i represents the respective weight of each evaluation dimension, r i represents recall, which is the proportion of the number of correct word vectors in each evaluation dimension of the to-be-evaluated alternative text to the total number of word vectors in each evaluation dimension of the reference alternative text; and α and β are balance coefficients for adjusting the weight relationship between n-gram accuracy, precision, and recall of the key evaluation dimension.

8. The method of claim 1, wherein the method is based on a large language model. The loss function used in the fine-tuning training is: ， where L(θ) is the loss function, N is the number of samples in the dataset, is the actual human-evaluated score of the ith sample, is the predicted score of the ith sample, and θ is the model parameters.

9. The method of claim 1, wherein the method is based on a large language model. The testing and model performance optimization comprise: The performance of the migration model is evaluated using a chart replacement text evaluation dataset, using a correlation index to calculate the difference between the predicted score and the actual human evaluation score, and the correlation index is: ， wherein p is the Pearson correlation coefficient; is the actual human-evaluated score of the i-th sample, is the average of the actual human-evaluated scores; is the predicted score of the i-th sample, is the average of the predicted scores; N is the number of samples; The migration model is optimized according to the model test performance, including hyperparameter adjustment and optimization algorithm.

10. A system for implementing the method of any one of claims 1 to 9, characterized in that, The system comprises a dataset construction module, a data preprocessing module, a model training module, and a model test and performance optimization module. The dataset construction module is used to collect high-quality reference chart replacement text, chart replacement text to be evaluated, and human evaluation scores to construct a chart replacement text evaluation dataset. The data preprocessing module is used to preprocess the dataset data, including text cleaning, word segmentation, special character removal, standardization processing, and feature splicing; and the dataset is divided into a training set, a validation set, and a test set for model training; The model training module selects GLM4-9B as the base model and combines the reading preferences of visually impaired users to customize the model prediction value E; a custom loss function L(θ) is used to fine-tune the GLM4 model using the chart replacement text evaluation dataset, and the model weight is continuously adjusted to minimize the loss function; The model test and performance optimization module uses the validation set in the chart replacement text evaluation dataset to evaluate the performance of the migration model, uses a correlation index to calculate the difference between the model predicted score and the actual human evaluation score; and optimizes the migration model according to the model test performance.

Images