Method and system for realizing digital analysis and processing of image-text image data

Abstract

The invention relates to the technical field of image-text processing, and discloses an image-text image data digital analysis processing method and system, and the method comprises the steps: collecting historical image-text image data, carrying out the analysis of the historical image-text image data, and determining an initial learning rate of a preset Transform model based on an analysis result; inputting the multi-modal historical fusion feature vector matrix into a preset Transform model to obtain a historical prediction result; calculating a historical prediction deviation coefficient of a preset Transform model based on an analysis result; obtaining a correction learning rate; obtaining a real-time image-text image influence coefficient corresponding to the real-time image-text image data; obtaining a compensation learning rate; and training a preset Transform model according to the compensation learning rate. According to the invention, the accuracy and efficiency of digital analysis and processing of image-text image data are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image and text processing, and in particular to a method and system for realizing digital analysis and processing of image and text data. Background Art

[0002] The digitization of graphic image data is the process of converting traditional graphic image data into digital format, which is called the digitization of graphic image data. This process is designed to facilitate the storage, retrieval and analysis of data. Through digitization technology, important data such as historical documents and works of art can be effectively protected and facilitated to share and disseminate on the Internet. The digitization process usually covers scanning, image processing, feature extraction and data storage to ensure the integrity and accessibility of data.

[0003] In the current field of image processing technology, the analysis and processing of image data often relies on manual labeling and classification. This method is not only inefficient, but also easily affected by subjective factors, thus affecting the accuracy of the analysis results.

[0004] Therefore, it is necessary to design a method and system for digital analysis and processing of graphic image data to solve the problems existing in the current technology. Summary of the invention

[0005] In view of this, the present invention proposes a method and system for realizing digital analysis and processing of graphic image data, aiming at improving the efficiency and accuracy of graphic image processing.

[0006] In one aspect, the present invention provides a method for digitally analyzing and processing graphic image data, comprising the following steps:

[0007] S100: collecting historical image data, analyzing the historical image data, and determining an initial learning rate of a preset Transformer model based on the analysis result;

[0008] S200: extracting features from the historical graphic image data to obtain a historical image feature vector and a historical text feature vector, and fusing the historical image feature vector and the historical text feature vector to obtain a multi-modal historical fusion feature vector matrix;

[0009] S300: Inputting the multimodal historical fusion feature vector matrix into the preset Transformer model to obtain a historical prediction result;

[0010] S400: Analyze the historical prediction results, and calculate the historical prediction deviation coefficient of the preset Transformer model based on the analysis results;

[0011] S500: judging whether the initial learning rate needs to be corrected according to the historical prediction deviation coefficient, and if so, setting a correction coefficient corresponding to the initial learning rate, and obtaining a corrected learning rate;

[0012] S600: collecting real-time graphic image data, analyzing the real-time graphic image data, judging whether the modified learning rate needs to be compensated based on the analysis result, and if so, obtaining the real-time graphic image influence coefficient corresponding to the real-time graphic image data;

[0013] S700: Setting a compensation coefficient of the modified learning rate based on the real-time graphic image influence coefficient, and obtaining a compensated learning rate;

[0014] S800: Training the preset Transformer model according to the compensatory learning rate.

[0015] Furthermore, the historical graphic image data is parsed, and the initial learning rate of the preset Transformer model is determined based on the parsing result, including:

[0016] Analyze the historical graphic image data, and obtain image complexity and text complexity based on the analysis results;

[0017] Comparing the image complexity with the image complexity threshold, comparing the text complexity with the text complexity threshold, and determining the initial learning rate of the preset Transformer model according to the comparison results;

[0018] When the image complexity is less than or equal to the image complexity threshold, and the text complexity is less than or equal to the text complexity threshold, determining that the initial learning rate of the preset Transformer model is a first initial learning rate;

[0019] When the image complexity is less than or equal to the image complexity threshold, and the text complexity is greater than the text complexity threshold, determining that the initial learning rate of the preset Transformer model is a second initial learning rate;

[0020] When the image complexity is greater than the image complexity threshold, and the text complexity is less than or equal to the text complexity threshold, determining that the initial learning rate of the preset Transformer model is a third initial learning rate;

[0021] When the image complexity is greater than the image complexity threshold and the text complexity is greater than the text complexity threshold, determining the initial learning rate of the preset Transformer model to be a fourth initial learning rate.

[0022] Further, feature extraction is performed on the historical graphic image data to obtain a historical image feature vector and a historical text feature vector, and the historical image feature vector and the historical text feature vector are fused to obtain a multimodal historical fusion feature vector matrix, including:

[0023] Using a convolutional neural network to extract image features from the historical graphic image data to obtain the historical image feature vector;

[0024] Using natural language processing technology to extract text features from the historical graphic image data to obtain the historical text feature vector;

[0025] The historical image feature vector and the historical text feature vector are fused by weighted fusion to obtain the multimodal historical fusion feature vector matrix.

[0026] Furthermore, the historical prediction results are analyzed, and the historical prediction deviation coefficient of the preset Transformer model is calculated based on the analysis results, including:

[0027] Analyze the historical prediction results to obtain normal prediction behavior and abnormal prediction behavior;

[0028] Determine the prediction deviation factor corresponding to each abnormal prediction behavior and construct a prediction deviation factor sequence;

[0029] Counting the number of the normal prediction behaviors, and recording it as a first number;

[0030] Counting the number of the abnormal prediction behaviors, and recording it as a second number;

[0031] The historical prediction deviation coefficient of the preset Transformer model is calculated according to the prediction deviation factor sequence, the first quantity and the second quantity.

[0032] Further, judging whether the initial learning rate needs to be corrected according to the historical prediction deviation coefficient includes:

[0033] Comparing the historical prediction deviation coefficient with the historical prediction deviation coefficient threshold, and determining whether the initial learning rate needs to be corrected according to the comparison result;

[0034] When the historical prediction deviation coefficient is less than or equal to the historical prediction deviation coefficient threshold, it is determined that there is no need to correct the initial learning rate;

[0035] When the historical prediction deviation coefficient is greater than the historical prediction deviation coefficient threshold, it is determined that the initial learning rate needs to be corrected.

[0036] Furthermore, setting a correction coefficient corresponding to the initial learning rate and obtaining a corrected learning rate includes:

[0037] Setting a correction coefficient interval, wherein the correction coefficient interval includes a first correction coefficient, a second correction coefficient, and a third correction coefficient;

[0038] Calculating a coefficient ratio of the historical prediction deviation coefficient to the historical prediction deviation coefficient threshold;

[0039] When the coefficient ratio is greater than 1 and less than or equal to 1.15, the first correction coefficient is selected as the correction coefficient corresponding to the initial learning rate, and the product of the first correction coefficient and the initial learning rate is used as the corrected learning rate;

[0040] When the coefficient ratio is greater than 1.15 and less than or equal to 1.3, the second correction coefficient is selected as the correction coefficient corresponding to the initial learning rate, and the product of the second correction coefficient and the initial learning rate is used as the corrected learning rate;

[0041] When the coefficient ratio is greater than 1.3, the third correction coefficient is selected as the correction coefficient corresponding to the initial learning rate, and the product value of the third correction coefficient and the initial learning rate is used as the corrected learning rate.

[0042] Furthermore, real-time graphic image data is collected, and the real-time graphic image data is analyzed, and based on the analysis result, it is determined whether the modified learning rate needs to be compensated, including:

[0043] Classifying the real-time graphic image data, and obtaining graphic image feature values ​​corresponding to each category of the real-time graphic image data;

[0044] Determine the image standard value corresponding to each type of real-time image data;

[0045] If all the image characteristic values ​​are less than or equal to the corresponding image standard values, it is determined that there is no need to compensate the modified learning rate;

[0046] If there is a graphic image characteristic value that is greater than the corresponding graphic image standard value, it is determined that the modified learning rate needs to be compensated.

[0047] Further, when obtaining the real-time image and text influence coefficient corresponding to the real-time image and text data, it includes:

[0048] Extract all image feature values ​​greater than the image standard value, and calculate the deviation between each image feature value and the image standard value, which is recorded as the image deviation value;

[0049] Determine all image deviation values, and generate a first image deviation value set according to all image deviation values ​​that are less than or equal to the image deviation value threshold;

[0050] Generate a second image-text image deviation value set according to all image-text image deviation values ​​greater than the image-text image deviation value threshold;

[0051] Calculating a first average image deviation value of the first image deviation value set, and calculating a second average image deviation value of the second image deviation value set;

[0052] The real-time image influence coefficient corresponding to the real-time image data is calculated based on the first average image deviation value and the second average image deviation value.

[0053] Furthermore, when setting the compensation coefficient of the modified learning rate based on the real-time graphic image influence coefficient and obtaining the compensated learning rate, it includes:

[0054] Setting a real-time image-text influence coefficient interval, wherein the real-time image-text influence coefficient interval includes a first real-time image-text influence coefficient and a second real-time image-text influence coefficient; the first real-time image-text influence coefficient is less than the second real-time image-text influence coefficient;

[0055] Setting a compensation coefficient interval, wherein the compensation coefficient interval includes a first compensation coefficient, a second compensation coefficient, and a third compensation coefficient;

[0056] When the first condition is identified, the first compensation coefficient is selected as the compensation coefficient corresponding to the modified learning rate, and the product value of the first compensation coefficient and the modified learning rate is used as the compensation learning rate;

[0057] When the second condition is identified, the second compensation coefficient is selected as the compensation coefficient corresponding to the modified learning rate, and the product value of the second compensation coefficient and the modified learning rate is used as the compensation learning rate;

[0058] When the third condition is identified, the third compensation coefficient is selected as the compensation coefficient corresponding to the modified learning rate, and the product value of the third compensation coefficient and the modified learning rate is used as the compensated learning rate;

[0059] Among them, the first condition includes that the real-time image and text influence coefficient is less than the first real-time image and text influence coefficient, the second condition includes that the real-time image and text influence coefficient is greater than or equal to the first real-time image and text influence coefficient, and is less than the second real-time image and text influence coefficient, and the third condition includes that the real-time image and text influence coefficient is greater than or equal to the second real-time image and text influence coefficient.

[0060] Compared with the prior art, the beneficial effect of the present invention is that the method for realizing digital analysis and processing of graphic image data provided by the present invention can better adapt to the complexity changes of graphic image data by dynamically adjusting the initial learning rate of the preset Transformer model, thereby improving the generalization ability and prediction accuracy of the model. Through the collection and analysis of real-time graphic image data, the system can adjust the learning rate in time to adapt to changes in data distribution and ensure the efficiency and effectiveness of model training. By introducing the historical prediction deviation coefficient and the real-time graphic image influence coefficient, the system can more finely control the learning rate to avoid the occurrence of overfitting or underfitting, thereby maintaining a high prediction performance in a variety of different application scenarios.

[0061] In another aspect, the present invention also proposes a system for realizing digital analysis and processing of graphic image data, comprising:

[0062] A collection module is configured to collect historical image data, analyze the historical image data, and determine an initial learning rate of a preset Transformer model based on the analysis result;

[0063] The historical prediction module is configured to perform feature extraction on the historical graphic image data to obtain a historical image feature vector and a historical text feature vector, fuse the historical image feature vector and the historical text feature vector to obtain a multimodal historical fusion feature vector matrix; input the multimodal historical fusion feature vector matrix into the preset Transformer model to obtain a historical prediction result;

[0064] A correction module is configured to analyze the historical prediction results, calculate the historical prediction deviation coefficient of the preset Transformer model based on the analysis results; determine whether the initial learning rate needs to be corrected according to the historical prediction deviation coefficient, and if so, set the correction coefficient corresponding to the initial learning rate, and obtain the corrected learning rate;

[0065] A compensation module is configured to control the acquisition module to acquire real-time graphic image data, and analyze the real-time graphic image data, and determine whether the modified learning rate needs to be compensated based on the analysis result, and if so, obtain the real-time graphic image influence coefficient corresponding to the real-time graphic image data; set the compensation coefficient of the modified learning rate based on the real-time graphic image influence coefficient, and obtain the compensated learning rate;

[0066] A training module is configured to train the preset Transformer model according to the compensatory learning rate.

[0067] It is understandable that the above-mentioned method and system for realizing digital analysis and processing of graphic image data have the same beneficial effects, which will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0069] Figure 1 A flowchart of a method for digital analysis and processing of graphic image data provided by an embodiment of the present invention;

[0070] Figure 2 A structural block diagram of a system for digital analysis and processing of graphic and text image data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0072] See also Figure 1 As shown, in some embodiments of the present application, this embodiment provides a method for digital analysis and processing of graphic image data, including the following steps:

[0073] S100: collecting historical image data, analyzing the historical image data, and determining an initial learning rate of a preset Transformer model based on the analysis result;

[0074] S200: extracting features from the historical graphic image data to obtain a historical image feature vector and a historical text feature vector, and fusing the historical image feature vector and the historical text feature vector to obtain a multi-modal historical fusion feature vector matrix;

[0075] S300: Inputting the multimodal historical fusion feature vector matrix into the preset Transformer model to obtain a historical prediction result;

[0076] S400: Analyze the historical prediction results, and calculate the historical prediction deviation coefficient of the preset Transformer model based on the analysis results;

[0077] S500: judging whether the initial learning rate needs to be corrected according to the historical prediction deviation coefficient, and if so, setting a correction coefficient corresponding to the initial learning rate, and obtaining a corrected learning rate;

[0078] S600: collecting real-time graphic image data, analyzing the real-time graphic image data, judging whether the modified learning rate needs to be compensated based on the analysis result, and if so, obtaining the real-time graphic image influence coefficient corresponding to the real-time graphic image data;

[0079] S700: Setting a compensation coefficient of the modified learning rate based on the real-time graphic image influence coefficient, and obtaining a compensated learning rate;

[0080] S800: Training the preset Transformer model according to the compensatory learning rate.

[0081] In an embodiment of the present invention, the preset Transformer model may be a deep learning model with a multi-layer structure, such as BERT, GPT or its variants. These models can process text and image information in graphic image data, and capture long-distance dependencies through a self-attention mechanism, thereby improving the accuracy of predictive analysis.

[0082] It can be understood that the method for realizing digital analysis and processing of graphic image data provided by this embodiment can better adapt to the changes in the complexity of graphic image data by dynamically adjusting the initial learning rate of the preset Transformer model, thereby improving the generalization ability and prediction accuracy of the model. Through the collection and analysis of real-time graphic image data, the system can adjust the learning rate in time to adapt to changes in data distribution and ensure the efficiency and effectiveness of model training. By introducing the historical prediction deviation coefficient and the real-time graphic image influence coefficient, the system can more finely control the learning rate to avoid overfitting or underfitting, thereby maintaining a high prediction performance in a variety of different application scenarios.

[0083] Specifically, the historical graphic image data is parsed, and the initial learning rate of the preset Transformer model is determined based on the parsing result, including:

[0084] Analyze the historical graphic image data, and obtain image complexity and text complexity based on the analysis results;

[0085] Comparing the image complexity with the image complexity threshold, comparing the text complexity with the text complexity threshold, and determining the initial learning rate of the preset Transformer model according to the comparison results;

[0086] When the image complexity is less than or equal to the image complexity threshold, and the text complexity is less than or equal to the text complexity threshold, determining that the initial learning rate of the preset Transformer model is a first initial learning rate;

[0087] When the image complexity is less than or equal to the image complexity threshold, and the text complexity is greater than the text complexity threshold, determining that the initial learning rate of the preset Transformer model is a second initial learning rate;

[0088] When the image complexity is greater than the image complexity threshold, and the text complexity is less than or equal to the text complexity threshold, determining that the initial learning rate of the preset Transformer model is a third initial learning rate;

[0089] When the image complexity is greater than the image complexity threshold and the text complexity is greater than the text complexity threshold, determining the initial learning rate of the preset Transformer model to be a fourth initial learning rate.

[0090] In this embodiment, the steps for calculating the image complexity include: preprocessing the image; using an edge detection algorithm to identify edge information in the image and calculate the edge density; applying a grayscale co-occurrence matrix to evaluate the complexity of the image texture; analyzing the color distribution of the image, calculating a color histogram, and measuring the color diversity by the entropy value of the color histogram; evaluating the richness of the details of the image in combination with the size and resolution of the image; and weighted summing the parameters such as edge density, texture complexity, color diversity, and richness of details obtained in the above steps to obtain the image complexity. For example, for an image with high resolution, rich texture, complex edges, and uneven color distribution, the calculation result of the image complexity will be higher, thereby affecting the setting of the initial learning rate of the preset Transformer model. Through such a calculation process, it can be ensured that the model can adjust to a suitable initial learning rate when facing images of different complexities to optimize the training effect of the model.

[0091] In this embodiment, the calculation steps of text complexity include: performing word segmentation on the text and extracting keywords from the text; using natural language processing technology to analyze the grammatical structure and semantic content of the text; calculating the diversity of vocabulary in the text, such as using a vocabulary richness index; evaluating the sentence complexity of the text, such as through sentence length and the number of clauses; analyzing the emotional tendency of the text and judging the emotional color of the text; and weighted summing the parameters such as the number of keywords, grammatical structure complexity, vocabulary richness, sentence complexity and emotional tendency obtained in the above steps to obtain text complexity. For example, for a scientific paper containing professional terms, long sentences and complex sentences, the calculation result of its text complexity will be higher, which will cause the initial learning rate of the preset Transformer model to be adjusted accordingly to adapt to the complexity of the text. Through this calculation method, the model can effectively learn and understand text content of different complexities, thereby improving the accuracy of digital analysis and processing of graphic image data.

[0092] In this embodiment, the image complexity threshold and the text complexity threshold are obtained based on statistical analysis of a large amount of historical graphic and text image data.

[0093] Specifically, feature extraction is performed on the historical graphic image data to obtain a historical image feature vector and a historical text feature vector, and the historical image feature vector and the historical text feature vector are fused to obtain a multimodal historical fusion feature vector matrix, including:

[0094] Using a convolutional neural network to extract image features from the historical graphic image data to obtain the historical image feature vector;

[0095] Using natural language processing technology to extract text features from the historical graphic image data to obtain the historical text feature vector;

[0096] The historical image feature vector and the historical text feature vector are fused by weighted fusion to obtain the multimodal historical fusion feature vector matrix.

[0097] It is understandable that the use of convolutional neural networks can effectively extract spatial hierarchical features from images, while natural language processing technology can capture semantic information in text. By combining these two technologies, the system can obtain multimodal feature vectors containing rich information, which can better represent the essential characteristics of graphic image data. In the process of fusing feature vectors, the weighted fusion strategy can ensure that image and text features are properly valued in the multimodal historical fusion feature vector matrix. For example, for image features, higher weights can be given because images usually contain more intuitive information; while for text features, the weights are adjusted according to their importance in specific application scenarios. In this way, the fused feature vector matrix can more comprehensively reflect the comprehensive information of graphic image data.

[0098] For example, the historical image feature vector is [1, 0.2, 0.3, 0.5], where each element represents the weight of different features in the image, such as edge information, texture complexity, color diversity, and detail richness. The historical text feature vector is [0.4, 0.3, 0.2, 0.1], where each element represents the weight of different aspects in the text, such as the number of keywords, grammatical structure complexity, vocabulary richness, and sentence complexity. Then the historical image feature vector and the historical text feature vector are fused to obtain a multimodal historical fusion feature vector matrix of [1.2, 0.5, 0.5, 0.6], where each element is the weighted sum of the corresponding elements in the corresponding image and text feature vectors. For example, the first element 1.2 is obtained by adding the first element 1 of the image feature vector and the first element 0.4 of the text feature vector and multiplying them by appropriate weights. In this way, the system can comprehensively consider the features of the image and text to generate a multimodal feature vector matrix that can represent the overall features of the graphic image data.

[0099] Specifically, analyzing the historical prediction results and calculating the historical prediction deviation coefficient of the preset Transformer model based on the analysis results includes:

[0100] Analyze the historical prediction results to obtain normal prediction behavior and abnormal prediction behavior;

[0101] Determine the prediction deviation factor corresponding to each abnormal prediction behavior and construct a prediction deviation factor sequence;

[0102] Counting the number of the normal prediction behaviors, and recording it as a first number;

[0103] Counting the number of the abnormal prediction behaviors, and recording it as a second number;

[0104] The historical prediction deviation coefficient of the preset Transformer model is calculated according to the prediction deviation factor sequence, the first quantity and the second quantity.

[0105] In this embodiment, normal prediction behavior means that the difference between the prediction result of the preset Transformer model and the actual data is within an acceptable range, while abnormal prediction behavior means that the difference between the prediction result and the actual data exceeds the acceptable range. By analyzing the historical prediction results, it is possible to identify in which aspects the model has deviations, so as to make targeted adjustments and optimizations to the model. The purpose of calculating the historical prediction deviation coefficient is to quantify the accuracy of the model prediction so as to facilitate the subsequent adjustment of the learning rate. The specific calculation method can be:

[0106] In this embodiment, the calculation formula of the historical forecast deviation coefficient is:

[0107] ;

[0108] Among them, K represents the historical prediction deviation coefficient; N1 represents the first number; N2 represents the second number; pi represents the i-th prediction deviation factor in the prediction deviation factor sequence.

[0109] It is understood that by calculating the historical prediction deviation coefficient, the system can evaluate the performance of the model on historical data and adjust the learning rate accordingly, in order to reduce the deviation and improve the accuracy in future predictions. This dynamic adjustment mechanism is an important feature of the present invention, which enables the model to self-optimize and adapt to changes in data, thereby maintaining good prediction performance in practical applications.

[0110] Specifically, judging whether the initial learning rate needs to be corrected according to the historical prediction deviation coefficient includes:

[0111] Comparing the historical prediction deviation coefficient with the historical prediction deviation coefficient threshold, and determining whether the initial learning rate needs to be corrected according to the comparison result;

[0112] When the historical prediction deviation coefficient is less than or equal to the historical prediction deviation coefficient threshold, it is determined that there is no need to correct the initial learning rate;

[0113] When the historical prediction deviation coefficient is greater than the historical prediction deviation coefficient threshold, it is determined that the initial learning rate needs to be corrected.

[0114] It is understandable that the historical prediction deviation coefficient threshold is set based on the model's performance on historical data and the expected prediction accuracy. If the historical prediction deviation coefficient exceeds this threshold, it means that there is a large deviation between the model's prediction results and the actual values. At this time, the initial learning rate needs to be corrected to improve the model's prediction performance.

[0115] Specifically, setting the correction coefficient corresponding to the initial learning rate and obtaining the corrected learning rate includes:

[0116] Setting a correction coefficient interval, wherein the correction coefficient interval includes a first correction coefficient, a second correction coefficient, and a third correction coefficient;

[0117] Calculating a coefficient ratio of the historical prediction deviation coefficient to the historical prediction deviation coefficient threshold;

[0118] When the coefficient ratio is greater than 1 and less than or equal to 1.15, the first correction coefficient is selected as the correction coefficient corresponding to the initial learning rate, and the product of the first correction coefficient and the initial learning rate is used as the corrected learning rate;

[0119] When the coefficient ratio is greater than 1.15 and less than or equal to 1.3, the second correction coefficient is selected as the correction coefficient corresponding to the initial learning rate, and the product of the second correction coefficient and the initial learning rate is used as the corrected learning rate;

[0120] When the coefficient ratio is greater than 1.3, the third correction coefficient is selected as the correction coefficient corresponding to the initial learning rate, and the product value of the third correction coefficient and the initial learning rate is used as the corrected learning rate.

[0121] It can be understood that the selection of the correction coefficient is determined based on the ratio of the historical prediction deviation coefficient to the historical prediction deviation coefficient threshold, so as to adjust the learning rate more accurately. By setting different correction coefficient intervals, the system can adopt different degrees of adjustment strategies according to the degree of deviation. For example, when the prediction deviation is relatively small, that is, the coefficient ratio is between 1 and 1.15, the first correction coefficient is selected for fine-tuning; when the prediction deviation is medium, that is, the coefficient ratio is between 1.15 and 1.3, the second correction coefficient is selected for medium adjustment; and when the prediction deviation is large, that is, the coefficient ratio exceeds 1.3, the third correction coefficient is selected for a larger adjustment. Such a segmented adjustment mechanism helps the model find a suitable learning rate at different deviation levels, thereby improving the adaptability and prediction accuracy of the model.

[0122] Specifically, collecting real-time graphic image data, analyzing the real-time graphic image data, and judging whether the modified learning rate needs to be compensated based on the analysis result, includes:

[0123] Classifying the real-time graphic image data, and obtaining graphic image feature values ​​corresponding to each category of the real-time graphic image data;

[0124] Determine the image standard value corresponding to each type of real-time image data;

[0125] If all the image characteristic values ​​are less than or equal to the corresponding image standard values, it is determined that there is no need to compensate the modified learning rate;

[0126] If there is a graphic image characteristic value that is greater than the corresponding graphic image standard value, it is determined that the modified learning rate needs to be compensated.

[0127] In the present embodiment, the types of real-time graphic image data include image data and text data. The graphic image feature values ​​corresponding to the image data are preferably the clarity, contrast and brightness of the image, while the graphic image feature values ​​corresponding to the text data include the length, complexity and semantic relevance of the text, etc. Through such classification and acquisition of feature values, the system can perform a detailed analysis of the real-time data, thereby determining whether it is necessary to compensate for the modified learning rate. For example, if the clarity of the real-time image data is lower than the standard value, indicating that the current image quality is poor, it may affect the prediction accuracy of the model, and at this time it is necessary to compensate for the modified learning rate to improve the adaptability of the model to low-quality images.

[0128] In this embodiment, the real-time image standard value is obtained based on the statistical analysis of real-time data, and is used to measure the complexity and change trend of the current image data. When the image feature value exceeds the standard value, it indicates that the complexity or change rate of the current data exceeds expectations. At this time, the modified learning rate needs to be compensated to ensure that the model can adapt to these changes.

[0129] Specifically, when obtaining the real-time image and text influence coefficient corresponding to the real-time image and text data, it includes:

[0130] Extract all image feature values ​​greater than the image standard value, and calculate the deviation between each image feature value and the image standard value, which is recorded as the image deviation value;

[0131] Determine all image deviation values, and generate a first image deviation value set according to all image deviation values ​​that are less than or equal to the image deviation value threshold;

[0132] Generate a second image-text image deviation value set according to all image-text image deviation values ​​greater than the image-text image deviation value threshold;

[0133] Calculating a first average image deviation value of the first image deviation value set, and calculating a second average image deviation value of the second image deviation value set;

[0134] The real-time image influence coefficient corresponding to the real-time image data is calculated based on the first average image deviation value and the second average image deviation value.

[0135] In this embodiment, the calculation formula of the graphic image influence coefficient is:

[0136] ;

[0137] Wherein, C represents the real-time image influence coefficient; α represents the first weight coefficient; D1 represents the first average image deviation value; β represents the second weight coefficient; D1 represents the second average image deviation value;

[0138] In this embodiment, the value of the first weight coefficient α is preferably 0.6, and the value of the second weight coefficient β is preferably 0.4. Through such weight distribution, the system can pay more attention to those eigenvalues ​​with large deviations, thereby making more targeted adjustments to the modified learning rate.

[0139] It can be understood that by calculating the real-time image and text influence coefficient, the system can quantify the degree of influence of real-time data on the model prediction performance. When the deviation between the image and text feature value and the standard value is large, that is, the deviation value exceeds the image and text deviation value threshold, it indicates that the complexity or change rate of the current data exceeds expectations, and the correction learning rate needs to be compensated. The calculation of the real-time image and text influence coefficient takes into account the size and distribution of the deviation value, so as to more accurately reflect the impact of real-time data on the model prediction performance. In this way, the system can dynamically adjust the learning rate to adapt to changes in real-time data, ensuring that the model can maintain a high prediction accuracy when facing image and text data of different complexities and change rates.

[0140] Specifically, the compensation coefficient of the modified learning rate is set based on the real-time graphic image influence coefficient, and the compensated learning rate is obtained, including:

[0141] Setting a real-time image-text influence coefficient interval, wherein the real-time image-text influence coefficient interval includes a first real-time image-text influence coefficient and a second real-time image-text influence coefficient; the first real-time image-text influence coefficient is less than the second real-time image-text influence coefficient;

[0142] Setting a compensation coefficient interval, wherein the compensation coefficient interval includes a first compensation coefficient, a second compensation coefficient, and a third compensation coefficient;

[0143] When the first condition is identified, the first compensation coefficient is selected as the compensation coefficient corresponding to the modified learning rate, and the product value of the first compensation coefficient and the modified learning rate is used as the compensation learning rate;

[0144] When the second condition is identified, the second compensation coefficient is selected as the compensation coefficient corresponding to the modified learning rate, and the product value of the second compensation coefficient and the modified learning rate is used as the compensation learning rate;

[0145] When the third condition is identified, the third compensation coefficient is selected as the compensation coefficient corresponding to the modified learning rate, and the product value of the third compensation coefficient and the modified learning rate is used as the compensated learning rate;

[0146] Among them, the first condition includes that the real-time image and text influence coefficient is less than the first real-time image and text influence coefficient, the second condition includes that the real-time image and text influence coefficient is greater than or equal to the first real-time image and text influence coefficient, and is less than the second real-time image and text influence coefficient, and the third condition includes that the real-time image and text influence coefficient is greater than or equal to the second real-time image and text influence coefficient.

[0147] It is understandable that by setting different real-time graphic image influence coefficient intervals and corresponding compensation coefficients, the system can adopt different learning rate compensation strategies according to the complexity and change rate of real-time data. For example, when the real-time graphic image influence coefficient is low, it indicates that the complexity and change rate of the current data are within an acceptable range. At this time, the first compensation coefficient is selected for minor adjustments; and when the influence coefficient is high, it indicates that the data complexity and change rate are beyond expectations and require a greater degree of adjustment. At this time, the second or third compensation coefficient is selected for compensation. In this way, the accuracy and efficiency of digital analysis and processing of graphic image data can be effectively improved.

[0148] See also Figure 2 As shown, in some embodiments of the present application, this embodiment provides a system for realizing digital analysis and processing of graphic image data, including:

[0149] A collection module is configured to collect historical image data, analyze the historical image data, and determine an initial learning rate of a preset Transformer model based on the analysis result;

[0150] The historical prediction module is configured to perform feature extraction on the historical graphic image data to obtain a historical image feature vector and a historical text feature vector, fuse the historical image feature vector and the historical text feature vector to obtain a multimodal historical fusion feature vector matrix; input the multimodal historical fusion feature vector matrix into the preset Transformer model to obtain a historical prediction result;

[0151] A correction module is configured to analyze the historical prediction results, calculate the historical prediction deviation coefficient of the preset Transformer model based on the analysis results; determine whether the initial learning rate needs to be corrected according to the historical prediction deviation coefficient, and if so, set the correction coefficient corresponding to the initial learning rate, and obtain the corrected learning rate;

[0152] A compensation module is configured to control the acquisition module to acquire real-time graphic image data, and analyze the real-time graphic image data, and determine whether the modified learning rate needs to be compensated based on the analysis result, and if so, obtain the real-time graphic image influence coefficient corresponding to the real-time graphic image data; set the compensation coefficient of the modified learning rate based on the real-time graphic image influence coefficient, and obtain the compensated learning rate;

[0153] A training module is configured to train the preset Transformer model according to the compensatory learning rate.

[0154] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, methods or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0155] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (methods) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for realizing digital analysis and processing of graphic image data, characterized in that: include: Collecting historical graphic image data, analyzing the historical graphic image data, and determining an initial learning rate of a preset Transformer model based on the analysis result; Performing feature extraction on the historical graphic image data to obtain a historical image feature vector and a historical text feature vector, and fusing the historical image feature vector and the historical text feature vector to obtain a multimodal historical fusion feature vector matrix; Inputting the multimodal historical fusion feature vector matrix into the preset Transformer model to obtain historical prediction results; Analyze the historical prediction results, and calculate the historical prediction deviation coefficient of the preset Transformer model based on the analysis results; Determine whether the initial learning rate needs to be corrected according to the historical prediction deviation coefficient, and if so, set the correction coefficient corresponding to the initial learning rate to obtain the corrected learning rate; Collecting real-time graphic image data, analyzing the real-time graphic image data, judging whether the modified learning rate needs to be compensated based on the analysis result, and if so, obtaining the real-time graphic image influence coefficient corresponding to the real-time graphic image data; Setting a compensation coefficient of the modified learning rate based on the real-time graphic image influence coefficient, and obtaining a compensated learning rate; The preset Transformer model is trained according to the compensatory learning rate.

2. The method for realizing digital analysis and processing of graphic image data according to claim 1, characterized in that: The historical graphic image data is analyzed, and an initial learning rate of a preset Transformer model is determined based on the analysis result, including: Analyze the historical graphic image data, and obtain image complexity and text complexity based on the analysis results; Comparing the image complexity with the image complexity threshold, comparing the text complexity with the text complexity threshold, and determining the initial learning rate of the preset Transformer model according to the comparison results; When the image complexity is less than or equal to the image complexity threshold, and the text complexity is less than or equal to the text complexity threshold, determining that the initial learning rate of the preset Transformer model is a first initial learning rate; When the image complexity is less than or equal to the image complexity threshold, and the text complexity is greater than the text complexity threshold, determining that the initial learning rate of the preset Transformer model is a second initial learning rate; When the image complexity is greater than the image complexity threshold, and the text complexity is less than or equal to the text complexity threshold, determining that the initial learning rate of the preset Transformer model is a third initial learning rate; When the image complexity is greater than the image complexity threshold and the text complexity is greater than the text complexity threshold, determining the initial learning rate of the preset Transformer model to be a fourth initial learning rate.

3. The method for realizing digital analysis and processing of graphic image data according to claim 2, characterized in that: Extracting features from the historical graphic image data to obtain historical image feature vectors and historical text feature vectors, fusing the historical image feature vectors and historical text feature vectors to obtain a multimodal historical fusion feature vector matrix, includes: Using a convolutional neural network to extract image features from the historical graphic image data to obtain the historical image feature vector; Using natural language processing technology to extract text features from the historical graphic image data to obtain the historical text feature vector; The historical image feature vector and the historical text feature vector are fused by weighted fusion to obtain the multimodal historical fusion feature vector matrix.

4. The method for realizing digital analysis and processing of graphic image data according to claim 3, characterized in that: Analyzing the historical prediction results and calculating the historical prediction deviation coefficient of the preset Transformer model based on the analysis results includes: Analyze the historical prediction results to obtain normal prediction behavior and abnormal prediction behavior; Determine the prediction deviation factor corresponding to each abnormal prediction behavior and construct a prediction deviation factor sequence; Counting the number of the normal prediction behaviors, and recording it as a first number; Counting the number of the abnormal prediction behaviors, and recording it as a second number; The historical prediction deviation coefficient of the preset Transformer model is calculated according to the prediction deviation factor sequence, the first quantity and the second quantity.

5. The method for realizing digital analysis and processing of graphic image data according to claim 4, characterized in that: When judging whether the initial learning rate needs to be corrected according to the historical prediction deviation coefficient, it includes: Comparing the historical prediction deviation coefficient with the historical prediction deviation coefficient threshold, and determining whether the initial learning rate needs to be corrected according to the comparison result; When the historical prediction deviation coefficient is less than or equal to the historical prediction deviation coefficient threshold, it is determined that there is no need to correct the initial learning rate; When the historical prediction deviation coefficient is greater than the historical prediction deviation coefficient threshold, it is determined that the initial learning rate needs to be corrected.

6. The method for realizing digital analysis and processing of graphic image data according to claim 1, characterized in that: The correction coefficient corresponding to the initial learning rate is set, and the correction learning rate is obtained, including: Setting a correction coefficient interval, wherein the correction coefficient interval includes a first correction coefficient, a second correction coefficient, and a third correction coefficient; Calculating a coefficient ratio of the historical prediction deviation coefficient to the historical prediction deviation coefficient threshold; When the coefficient ratio is greater than 1 and less than or equal to 1.15, the first correction coefficient is selected as the correction coefficient corresponding to the initial learning rate, and the product of the first correction coefficient and the initial learning rate is used as the corrected learning rate; When the coefficient ratio is greater than 1.15 and less than or equal to 1.3, the second correction coefficient is selected as the correction coefficient corresponding to the initial learning rate, and the product of the second correction coefficient and the initial learning rate is used as the corrected learning rate; When the coefficient ratio is greater than 1.3, the third correction coefficient is selected as the correction coefficient corresponding to the initial learning rate, and the product value of the third correction coefficient and the initial learning rate is used as the corrected learning rate.

7. The method for realizing digital analysis and processing of graphic image data according to claim 1, characterized in that: Collecting real-time graphic image data, analyzing the real-time graphic image data, and judging whether it is necessary to compensate the modified learning rate based on the analysis result, including: Classifying the real-time graphic image data, and obtaining graphic image feature values ​​corresponding to each category of the real-time graphic image data; Determine the image standard value corresponding to each type of real-time image data; If all the image characteristic values ​​are less than or equal to the corresponding image standard values, it is determined that there is no need to compensate the modified learning rate; If there is a graphic image characteristic value that is greater than the corresponding graphic image standard value, it is determined that the modified learning rate needs to be compensated.

8. The method for realizing digital analysis and processing of graphic image data according to claim 7, characterized in that: When obtaining the real-time image and text influence coefficient corresponding to the real-time image and text data, it includes: Extract all image feature values ​​greater than the image standard value, and calculate the deviation between each image feature value and the image standard value, which is recorded as the image deviation value; Determine all image deviation values, and generate a first image deviation value set according to all image deviation values ​​that are less than or equal to the image deviation value threshold; Generate a second image-text image deviation value set according to all image-text image deviation values ​​greater than the image-text image deviation value threshold; Calculating a first average image deviation value of the first image deviation value set, and calculating a second average image deviation value of the second image deviation value set; The real-time image influence coefficient corresponding to the real-time image data is calculated based on the first average image deviation value and the second average image deviation value.

9. The method for digital analysis and processing of graphic image data according to claim 8, characterized in that: When the compensation coefficient of the modified learning rate is set based on the real-time graphic image influence coefficient and the compensated learning rate is obtained, it includes: Setting a real-time image and text influence coefficient interval, wherein the real-time image and text influence coefficient interval includes a first real-time image and text influence coefficient and a second real-time image and text influence coefficient; the first real-time image and text influence coefficient is less than the second real-time image and text influence coefficient; Setting a compensation coefficient interval, wherein the compensation coefficient interval includes a first compensation coefficient, a second compensation coefficient, and a third compensation coefficient; When the first condition is identified, the first compensation coefficient is selected as the compensation coefficient corresponding to the modified learning rate, and the product value of the first compensation coefficient and the modified learning rate is used as the compensation learning rate; When the second condition is identified, the second compensation coefficient is selected as the compensation coefficient corresponding to the modified learning rate, and the product value of the second compensation coefficient and the modified learning rate is used as the compensation learning rate; When the third condition is identified, the third compensation coefficient is selected as the compensation coefficient corresponding to the modified learning rate, and the product value of the third compensation coefficient and the modified learning rate is used as the compensated learning rate; Among them, the first condition includes that the real-time image and text influence coefficient is less than the first real-time image and text influence coefficient, the second condition includes that the real-time image and text influence coefficient is greater than or equal to the first real-time image and text influence coefficient, and is less than the second real-time image and text influence coefficient, and the third condition includes that the real-time image and text influence coefficient is greater than or equal to the second real-time image and text influence coefficient.

10. A system for realizing digital analysis and processing of graphic image data, applied to a method for realizing digital analysis and processing of graphic image data as claimed in any one of claims 1 to 9, characterized in that: include: A collection module is configured to collect historical image data, analyze the historical image data, and determine an initial learning rate of a preset Transformer model based on the analysis result; A historical prediction module is configured to perform feature extraction on the historical graphic image data to obtain a historical image feature vector and a historical text feature vector, and fuse the historical image feature vector and the historical text feature vector to obtain a multimodal historical fusion feature vector matrix; Inputting the multimodal historical fusion feature vector matrix into the preset Transformer model to obtain historical prediction results; A correction module is configured to analyze the historical prediction results, calculate the historical prediction deviation coefficient of the preset Transformer model based on the analysis results; determine whether the initial learning rate needs to be corrected according to the historical prediction deviation coefficient, and if so, set the correction coefficient corresponding to the initial learning rate, and obtain the corrected learning rate; A compensation module is configured to control the acquisition module to acquire real-time graphic image data, and analyze the real-time graphic image data, and determine whether the modified learning rate needs to be compensated based on the analysis result, and if so, obtain the real-time graphic image influence coefficient corresponding to the real-time graphic image data; set the compensation coefficient of the modified learning rate based on the real-time graphic image influence coefficient, and obtain the compensated learning rate; A training module is configured to train the preset Transformer model according to the compensatory learning rate.

Images