Multi-dimensional large model data quality evaluation method and system

Abstract

The invention discloses a multi-dimensional large model data quality evaluation method and system, and the method comprises the steps: extracting data features in a data set and a reference data set which meet the lowest inherent quality requirements, and carrying out the evaluation of the quality of data based on the information field effect between data samples in the data set; and feature updating and context semantic association enhancement processing are carried out on each data sample, so that the semantic feature expression ability of each data sample is improved, and more accurate data quality evaluation is realized. Therefore, by introducing the context association information of the data set, the semantic information of each data sample can be more accurately understood, so that the accuracy of overall data quality evaluation is improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent quality assessment, and more specifically, to a multi-dimensional large model data quality assessment method and system. Background Art

[0002] With the rapid development of mobile networks, sensor networks, and crowd-sensing technologies, a large amount of diverse data is being generated. At the same time, data-based information services have also emerged significantly, and data quality plays a vital role in this. High-quality data can provide sufficient and accurate information to complete specific tasks, such as training high-quality machine learning models and helping smart city systems make wise decisions. However, existing work mostly evaluates data quality for a single data unit, lacking a method to evaluate the overall quality of the data set.

[0003] In this regard, the invention patent with publication number CN111612783A discloses a data quality assessment method, which first evaluates the internal features of the data set that are not related to the task to obtain a data set that meets the minimum intrinsic quality requirements; then, feature extraction is performed on each data in the data set that meets the minimum intrinsic quality requirements and the sample data set, and a local sensitive hashing method is used to perform feature comparative analysis on the data set that meets the minimum intrinsic quality requirements and the sample data set to obtain quality assessment results of the data set's task relevance, content diversity, completeness, appropriateness of data volume, and timeliness, thereby meeting the assessment needs for large-scale data quality and effectively improving the comprehensiveness, accuracy, and efficiency of data quality assessment.

[0004] However, in the process of data quality assessment, the above scheme uses a deep learning model to perform independent feature extraction for each data sample in the data set. Given that there are generally contextual associations between the data samples in the data set, this independent feature extraction method may ignore the correlation information between data samples, resulting in deviations in the feature representation of the data, which cannot fully and accurately reflect the true meaning of the data samples, and may have an adverse impact on the accuracy of data quality assessment.

[0005] Therefore, an optimized multi-dimensional large model data quality assessment method is expected. Summary of the invention

[0006] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a multi-dimensional large model data quality assessment method and system, which, after extracting each data feature in the data set and the reference data set that meet the minimum intrinsic quality requirements, performs feature update and context semantic association enhancement processing on each data sample based on the information field effect between each data sample in the data set, thereby improving the semantic feature expression ability of each data sample and achieving more accurate data quality assessment. In this way, by introducing the contextual association information of the data set, the semantic information of each data sample can be understood more accurately, thereby improving the accuracy of the overall data quality assessment.

[0007] According to one aspect of the present application, a multi-dimensional large model data quality assessment method is provided, which includes: evaluating the data internal features that are not related to the task of a data set to obtain a data set that meets the minimum intrinsic quality requirements; extracting features from each data in the data set that meets the minimum intrinsic quality requirements and the reference sample data set to obtain a set of minimum quality requirement data sample feature vectors and a set of reference sample feature vectors; based on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors, performing contextual quality assessment on the data set that meets the minimum intrinsic quality requirements and the sample data set to obtain a quality assessment result; and ranking the quality assessment results by quality, characterized in that based on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors, performing contextual quality assessment on the data set that meets the minimum intrinsic quality requirements and the sample data set to obtain a quality assessment result, including: Performing context-sensitive feature reconstruction enhancement processing on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors respectively to obtain a set of enhanced minimum quality requirement data sample feature vectors and a set of enhanced reference sample feature vectors; Based on the set of enhanced minimum quality requirement data sample feature vectors and the set of enhanced reference sample feature vectors, contextual quality assessment is performed on the data set that meets the minimum intrinsic quality requirement and the sample data set to obtain the quality assessment result.

[0008] According to another aspect of the present application, a multi-dimensional large model data quality assessment system is provided, which includes: The data evaluation module is used to evaluate the internal characteristics of the data set that are not related to the task, and obtain a data set that meets the minimum intrinsic quality requirements; A feature extraction module, used for performing feature extraction on each data in the data set that meets the minimum intrinsic quality requirement and the reference sample data set to obtain a set of minimum quality requirement data sample feature vectors and a set of reference sample feature vectors; A context quality assessment module, configured to perform context quality assessment on the data set meeting the minimum intrinsic quality requirement and the sample data set based on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors to obtain a quality assessment result; The quality ranking module is used to rank the quality evaluation results.

[0009] Compared with the prior art, the present application provides a multi-dimensional large-model data quality assessment method and system, which, after extracting the data set that meets the minimum intrinsic quality requirements and each data feature in the reference data set, performs feature update and context semantic association enhancement processing on each data sample based on the information field effect between each data sample in the data set, thereby improving the semantic feature expression ability of each data sample and achieving more accurate data quality assessment. In this way, by introducing the contextual association information of the data set, the semantic information of each data sample can be understood more accurately, thereby improving the accuracy of the overall data quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 It is a flowchart of a multi-dimensional large model data quality assessment method according to an embodiment of the present application; Figure 2 A data flow diagram of a multi-dimensional large model data quality assessment method according to an embodiment of the present application; Figure 3 It is a flowchart of sub-step S3 of the multi-dimensional large model data quality assessment method according to an embodiment of the present application; Figure 4 It is a block diagram of a multi-dimensional large model data quality assessment system according to an embodiment of the present application. DETAILED DESCRIPTION

[0012] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0013] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0014] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0015] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0016] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0017] Patent CN111612783A proposes a data quality assessment method, which includes: evaluating the internal features of the data set that are independent of the task to obtain a data set that meets the minimum intrinsic quality requirements; performing feature extraction on each data in the data set that meets the minimum intrinsic quality requirements and the reference sample data set to obtain a set of minimum quality requirement data sample feature vectors and a set of reference sample feature vectors; based on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors, performing contextual quality assessment on the data set that meets the minimum intrinsic quality requirements and the sample data set to obtain quality assessment results; and performing quality sorting on the quality assessment results.

[0018] In the process of data quality assessment, the above data quality assessment method uses a deep learning model to extract independent features from each data sample in the data set. However, given that there are generally contextual associations between each data sample in the data set, this independent feature extraction method may ignore the correlation information between data samples, resulting in deviations in the feature representation of the data and failing to fully and accurately reflect the true meaning of the data samples.

[0019] To address this problem, this application proposes an optimized multi-dimensional large-model data quality assessment method, which extracts the data features in the data set and the reference data set that meet the minimum intrinsic quality requirements, and then updates the features of each data sample and strengthens the contextual semantic association based on the information field effect between each data sample in the data set, thereby improving the semantic feature expression ability of each data sample and achieving more accurate data quality assessment. In this way, by introducing the contextual association information of the data set, the semantic information of each data sample can be understood more accurately, thereby improving the accuracy of the overall data quality assessment.

[0020] Specifically, in the technical solution of the present application, a multi-dimensional large model data quality assessment method is proposed. Figure 1 It is a flowchart of a multi-dimensional large model data quality assessment method according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of a multi-dimensional large model data quality assessment method according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the multi-dimensional large model data quality assessment method of the embodiment of the present application, the steps include: S1, evaluating the internal data features of the data set that are independent of the task to obtain a data set that meets the minimum intrinsic quality requirements; S2, performing feature extraction on each data in the data set that meets the minimum intrinsic quality requirements and the reference sample data set to obtain a set of minimum quality requirement data sample feature vectors and a set of reference sample feature vectors; S3, based on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors, performing contextual quality assessment on the data set that meets the minimum intrinsic quality requirements and the sample data set to obtain a quality assessment result; S4, performing quality sorting on the quality assessment results.

[0021] In particular, S1 evaluates the internal characteristics of the data set that are not related to the task, and obtains a data set that meets the minimum intrinsic quality requirements. In other words, the data set is preliminarily screened to remove data that obviously does not meet the standards (such as noise, too many missing values, etc.), ensuring the basic data quality for subsequent processing.

[0022] In particular, S2 performs feature extraction on each data in the data set that meets the minimum intrinsic quality requirements and the reference sample data set to obtain a set of minimum quality requirement data sample feature vectors and a set of reference sample feature vectors. By extracting key features from each data in the data set that meets the minimum intrinsic quality requirements and the reference sample data set, the essential information of the data can be better captured and the data characterization capability can be improved. The obtained set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors provide a quantitative basis for subsequent quality evaluation.

[0023] In particular, the step S3 performs contextual quality assessment on the data set that meets the minimum intrinsic quality requirement and the sample data set based on the set of the minimum quality requirement data sample feature vectors and the set of the reference sample feature vectors to obtain a quality assessment result. Figure 3 As shown, the S3 includes: S31, performing context-sensitive feature reconstruction enhancement processing on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors to obtain a set of enhanced minimum quality requirement data sample feature vectors and a set of enhanced reference sample feature vectors; S32, performing contextual quality assessment on the data set that meets the minimum intrinsic quality requirements and the sample data set based on the set of enhanced minimum quality requirement data sample feature vectors and the set of enhanced reference sample feature vectors to obtain the quality assessment result.

[0024] Specifically, the S31 performs context-sensitive feature reconstruction and enhancement processing on the set of feature vectors of the data samples with the minimum quality requirement and the set of feature vectors of the reference samples, respectively, to obtain a set of enhanced feature vectors of the data samples with the minimum quality requirement and a set of enhanced feature vectors of the reference samples. In the technical solution of the present application, since the data set that meets the minimum intrinsic quality requirements has been preliminarily screened, each data sample therein has a certain correlation with the task, which makes there also a potential context association between each data sample. Therefore, in order to optimize the feature representation of each data sample, the present application proposes a context-sensitive feature reconstruction and enhancement method based on feature field effect to process the set of feature vectors of the data samples with the minimum quality requirement and the set of feature vectors of the reference samples, respectively. It updates the feature representation of each data sample and performs context association enhancement processing by mining the information field effect distribution of each data sample in the data set, so that it can better reflect the actual semantics of each data sample in the context of the data set, thereby enhancing the accuracy of subsequent data quality assessment. In an embodiment of the present application, taking the set of minimum quality requirement data sample feature vectors as an example, first, based on the semantic feature field effect of the set of minimum quality requirement data sample feature vectors, the set of minimum quality requirement data sample feature vectors is updated to obtain an updated set of minimum quality requirement data sample feature vectors.

[0025] First, using a semantic reconstructor to perform feature reconstruction on each minimum quality requirement data sample feature vector in the set of minimum quality requirement data sample feature vectors to obtain a set of reconstructed minimum quality requirement data sample feature vectors. The specific steps include: using a first weight parameter matrix and a second weight parameter matrix to perform continuous linear transformation on the minimum quality requirement data sample feature vectors to obtain the reconstructed minimum quality requirement data sample feature vectors.

[0026] Specifically, the process can be expressed as:

[0027]

[0028] in, represents the set of data sample feature vectors with the minimum quality requirement, , , and respectively represent the first, second, and third in the set of the minimum quality requirement data sample feature vectors and minimum quality requirement data sample feature vector, is the number of feature vectors of data samples with the minimum quality requirement, and denote the first weight parameter matrix and the second weight parameter matrix respectively, For the The minimum quality requirement for reconstructing the data sample feature vector.

[0029] That is, the semantic reconstructor is used to perform domain transformation on each of the minimum quality requirement data sample feature vectors, mapping them to a unified semantic space to unify the feature dimensions, thereby generating a set of reconstructed minimum quality requirement data sample feature vectors. The minimum quality requirement data sample feature vectors after linear transformation can better capture the potential relationship between data samples because the overall information field effect of the data set is considered during the transformation process. This approach can strengthen those features that are critical to understanding the context while weakening irrelevant or redundant features, thereby improving the semantic representation ability of each data sample.

[0030] More specifically, the specific steps of calculating the semantic depth implicit representation value of each reconstructed minimum quality requirement data sample feature vector in the set of reconstructed minimum quality requirement data sample feature vectors to obtain a set of semantic depth implicit representation values ​​of the minimum quality requirement data samples include: calculating the square of the Euclidean norm of the reconstructed minimum quality requirement data sample feature vector to obtain the initial semantic depth implicit representation value of the minimum quality requirement data sample; calculating the ratio of the sum of the initial semantic depth implicit representation value of the minimum quality requirement data sample and the difference between the sum of the initial semantic depth implicit representation value of the minimum quality requirement data sample and the initial semantic depth implicit representation value of the minimum quality requirement data sample to obtain the semantic depth representation modulation coefficient; calculating the logarithmic function value of the semantic depth representation modulation coefficient with base 2 to obtain the semantic depth implicit representation value of the minimum quality requirement data sample.

[0031] Specifically, the process can be expressed as:

[0032] in, For the Reconstruct the minimum quality required data sample feature vector, represents the calculation of the Euclidean norm of a vector, Indicates The minimum quality requirement is the semantic depth implicit representation value of the data sample.

[0033] Here, by calculating the semantic depth implicit representation value of each reconstructed minimum quality requirement data sample feature vector, we can measure the information density or complexity carried by each data sample in the overall information of the data set, and quantify the importance and influence of each data sample in the overall information of the data set.

[0034] Then, based on the sorting of the set of semantic depth implicit representation values ​​of the data samples with minimum quality requirement, the set of feature vectors of the data samples with minimum quality requirement is reordered to obtain the set of updated feature vectors of the data samples with minimum quality requirement; that is, according to the semantic depth implicit representation value of each data sample, the order of features in the set is further adjusted, and the information structure of the data set is reorganized to ensure that key information is given priority in the subsequent contextual semantic association enhancement encoding process.

[0035] Next, the set of updated minimum quality requirement data sample feature vectors is subjected to bidirectional contextual semantic association enhancement to obtain the set of enhanced minimum quality requirement data sample feature vectors. Specifically, first, the set of updated minimum quality requirement data sample feature vectors is input into the first semantic encoder based on the forward LSTM model to obtain the set of forward minimum quality requirement data sample contextual semantic encoding feature vectors; and the set of updated minimum quality requirement data sample feature vectors is input into the second semantic encoder based on the backward LSTM model to obtain the set of backward minimum quality requirement data sample contextual semantic encoding feature vectors; that is, the set of updated minimum quality requirement data sample feature vectors is encoded from the forward and backward directions respectively using a bidirectional long short-term memory (LSTM) model. Among them, the forward LSTM captures the information flow pattern from front to back, while the backward LSTM operates in reverse to identify the dependency from back to front. This dual-perspective encoding mechanism can capture the global dependency of data set information at different levels, and thereby enhance the semantic feature expression of each data sample. Then, the set of the forward minimum quality requirement data sample context semantic encoding feature vectors and the set of the backward minimum quality requirement data sample context semantic encoding feature vectors are fused to obtain the set of enhanced minimum quality requirement data sample feature vectors. Here, the bidirectional context semantic information of each data sample is integrated by fusing the encoding results of the bidirectional LSTM model to form a richer and more comprehensive set of enhanced minimum quality requirement data sample feature vectors, thereby achieving an in-depth understanding of the information of each data sample in the data set that meets the minimum intrinsic quality requirements.

[0036] Among them, the specific steps of fusing the set of forward minimum quality requirement data sample context semantic coding feature vectors and the set of backward minimum quality requirement data sample context semantic coding feature vectors to obtain the set of enhanced minimum quality requirement data sample feature vectors include: using the first weighting parameter and the second weighting parameter to weighted modulate the set of forward minimum quality requirement data sample context semantic coding feature vectors and the set of backward minimum quality requirement data sample context semantic coding feature vectors respectively, and calculating the set of minimum quality requirement data sample context semantic coding feature vectors after weighted modulation, the set of minimum quality requirement data sample context semantic coding feature vectors after weighted modulation, and the position points between the bias items to obtain the set of enhanced minimum quality requirement data sample feature vectors.

[0037] Specifically, the process can be expressed as:

[0038]

[0039]

[0040]

[0041] in, Represents the set of feature vectors of data samples with minimum quality requirements for updating, , , and respectively represent the first, second, and third in the set of updated minimum quality requirement data sample feature vectors and Update the minimum quality requirement data sample feature vector, represents the forward LSTM model, represents the backward LSTM model, represents the set of contextual semantic encoding feature vectors of the forward minimum quality requirement data sample, Represents the set of contextual semantic encoding feature vectors of the backward minimum quality requirement data sample, and denote the first weighting parameter and the second weighting parameter respectively, is the bias term, A set of data sample feature vectors with minimum quality requirements for the enhancement.

[0042] Through the above-mentioned data processing method, the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors are processed separately to obtain a set of enhanced minimum quality requirement data sample feature vectors and a set of enhanced reference sample feature vectors, which can more effectively identify and utilize useful information in the data set and provide a more solid feature foundation for subsequent data quality assessment work.

[0043] Specifically, the S32, based on the set of enhanced minimum quality requirement data sample feature vectors and the set of enhanced reference sample feature vectors, performs contextual quality assessment on the data set that meets the minimum intrinsic quality requirements and the sample data set to obtain the quality assessment result. It should be understood that the contextual quality assessment process is a prior art, which can use the principles disclosed in Chinese patent CN111612783A to process the set of enhanced minimum quality requirement data sample feature vectors and the set of enhanced reference sample feature vectors to obtain the quality assessment result. Of course, it can also use other principles to perform feature comparison analysis between the set of enhanced minimum quality requirement data sample feature vectors and the set of enhanced reference sample feature vectors, which is not limited to this application.

[0044] In particular, S4 ranks the quality assessment results. By ranking, the data samples with the highest quality can be preferentially selected for model training. High-quality data can reduce noise interference, making it easier for the model to learn useful information, thereby improving the generalization ability and prediction accuracy of the model.

[0045] In summary, the multi-dimensional large-model data quality assessment method according to the embodiment of the present application is explained, which, after extracting the data set that meets the minimum intrinsic quality requirements and each data feature in the reference data set, performs feature update and context semantic association enhancement processing on each data sample based on the information field effect between each data sample in the data set, thereby improving the semantic feature expression ability of each data sample and achieving more accurate data quality assessment. In this way, by introducing the contextual association information of the data set, the semantic information of each data sample can be understood more accurately, thereby improving the accuracy of the overall data quality assessment.

[0046] Furthermore, a multi-dimensional large model data quality assessment system is also provided.

[0047] Figure 4 FIG. 1 is a block diagram of a multi-dimensional large model data quality assessment system according to an embodiment of the present application. Figure 4 As shown, according to the multi-dimensional large model data quality assessment system 300 of the embodiment of the present application, it includes: a data evaluation module 310, which is used to evaluate the internal data features of the data set that are independent of the task, and obtain a data set that meets the minimum intrinsic quality requirements; a feature extraction module 320, which is used to extract features from each data in the data set that meets the minimum intrinsic quality requirements and the reference sample data set to obtain a set of minimum quality requirement data sample feature vectors and a set of reference sample feature vectors; a context quality assessment module 330, which is used to perform context quality assessment on the data set that meets the minimum intrinsic quality requirements and the sample data set based on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors to obtain a quality assessment result; a quality sorting module 340, which is used to quality sort the quality assessment results.

[0048] As described above, the multi-dimensional large model data quality assessment system 300 according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a multi-dimensional large model data quality assessment algorithm. In one possible implementation, the multi-dimensional large model data quality assessment system 300 according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the multi-dimensional large model data quality assessment system 300 can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the multi-dimensional large model data quality assessment system 300 can also be one of the many hardware modules of the wireless terminal.

[0049] Alternatively, in another example, the multidimensional large model data quality assessment system 300 and the wireless terminal may also be separate devices, and the multidimensional large model data quality assessment system 300 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interactive information in accordance with an agreed data format.

[0050] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A multi-dimensional large model data quality assessment method, comprising: Evaluate the internal features of the data set that are not related to the task, and obtain a data set that meets the minimum intrinsic quality requirements; Performing feature extraction on each data in the data set that meets the minimum intrinsic quality requirement and the reference sample data set to obtain a set of minimum quality requirement data sample feature vectors and a set of reference sample feature vectors; Based on the set of data sample feature vectors with minimum quality requirements and the set of reference sample feature vectors, contextual quality assessment is performed on the data set that meets the minimum intrinsic quality requirements and the sample data set to obtain a quality assessment result; and quality ranking is performed on the quality assessment results, characterized in that, based on the set of data sample feature vectors with minimum quality requirements and the set of reference sample feature vectors, contextual quality assessment is performed on the data set that meets the minimum intrinsic quality requirements and the sample data set to obtain a quality assessment result, including: Performing context-sensitive feature reconstruction enhancement processing on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors respectively to obtain a set of enhanced minimum quality requirement data sample feature vectors and a set of enhanced reference sample feature vectors; Based on the set of enhanced minimum quality requirement data sample feature vectors and the set of enhanced reference sample feature vectors, contextual quality assessment is performed on the data set that meets the minimum intrinsic quality requirement and the sample data set to obtain the quality assessment result; Among them, the set of minimum quality requirement data sample feature vectors is subjected to context-sensitive feature reconstruction enhancement processing, including: based on the semantic feature field effect of the set of minimum quality requirement data sample feature vectors, the set of minimum quality requirement data sample feature vectors is updated to obtain an updated set of minimum quality requirement data sample feature vectors; and the set of updated minimum quality requirement data sample feature vectors is subjected to bidirectional context semantic association enhancement to obtain the enhanced set of minimum quality requirement data sample feature vectors.

2. The multi-dimensional large model data quality assessment method according to claim 1 is characterized in that: Based on the semantic feature field effect of the set of minimum quality requirement data sample feature vectors, updating the set of minimum quality requirement data sample feature vectors to obtain an updated set of minimum quality requirement data sample feature vectors, including: Using a semantic reconstructor to perform feature reconstruction on each minimum quality requirement data sample feature vector in the set of minimum quality requirement data sample feature vectors to obtain a set of reconstructed minimum quality requirement data sample feature vectors; Calculating the semantic depth implicit representation value of each reconstructed minimum quality requirement data sample feature vector in the set of reconstructed minimum quality requirement data sample feature vectors to obtain a set of minimum quality requirement data sample semantic depth implicit representation values; Based on the sorting of the set of the semantic depth implicit representation values ​​of the data samples with the minimum quality requirement, the set of the feature vectors of the data samples with the minimum quality requirement is re-sorted to obtain the set of the updated feature vectors of the data samples with the minimum quality requirement.

3. The multi-dimensional large model data quality assessment method according to claim 2 is characterized in that: Using a semantic reconstructor to perform feature reconstruction on each minimum quality requirement data sample feature vector in the set of minimum quality requirement data sample feature vectors to obtain a set of reconstructed minimum quality requirement data sample feature vectors, including: The minimum quality requirement data sample feature vector is continuously linearly transformed using the first weight parameter matrix and the second weight parameter matrix to obtain the reconstructed minimum quality requirement data sample feature vector.

4. The multi-dimensional large model data quality assessment method according to claim 3 is characterized in that: Calculating the semantic depth implicit representation value of each reconstructed minimum quality requirement data sample feature vector in the set of reconstructed minimum quality requirement data sample feature vectors to obtain a set of minimum quality requirement data sample semantic depth implicit representation values, including: Calculating the square of the Euclidean norm of the reconstructed minimum quality requirement data sample feature vector to obtain an initial semantic depth implicit representation value of the minimum quality requirement data sample; Calculate a ratio of a sum of an initial semantic depth implicit representation value of the data sample with the minimum quality requirement and a difference with the initial semantic depth implicit representation value of the data sample with the minimum quality requirement to obtain a semantic depth representation modulation coefficient; The logarithmic function value with base 2 of the semantic depth representation modulation coefficient is calculated to obtain the semantic depth implicit representation value of the minimum quality requirement data sample.

5. The multi-dimensional large model data quality assessment method according to claim 4 is characterized in that: Performing bidirectional context semantic association enhancement on the set of updated minimum quality requirement data sample feature vectors to obtain the set of enhanced minimum quality requirement data sample feature vectors, including: Inputting the set of updated minimum quality requirement data sample feature vectors into a first semantic encoder based on a forward LSTM model to obtain a set of forward minimum quality requirement data sample context semantic encoding feature vectors; Inputting the set of updated minimum quality requirement data sample feature vectors into a second semantic encoder based on a backward LSTM model to obtain a set of backward minimum quality requirement data sample context semantic encoding feature vectors; The set of forward minimum quality requirement data sample context semantic coding feature vectors and the set of backward minimum quality requirement data sample context semantic coding feature vectors are fused to obtain the set of enhanced minimum quality requirement data sample feature vectors.

6. The multi-dimensional large model data quality assessment method according to claim 5 is characterized in that: The set of forward minimum quality requirement data sample context semantic coding feature vectors and the set of backward minimum quality requirement data sample context semantic coding feature vectors are merged to obtain the set of enhanced minimum quality requirement data sample feature vectors, including: The set of forward minimum quality requirement data sample context semantic coding feature vectors and the set of backward minimum quality requirement data sample context semantic coding feature vectors are weighted modulated using the first weighting parameter and the second weighting parameter, and the set of weighted modulated minimum quality requirement data sample context semantic coding feature vectors, the set of weighted modulated minimum quality requirement data sample context semantic coding feature vectors, and the set of enhanced minimum quality requirement data sample feature vectors are obtained by adding the position points between the bias items.

7. A multi-dimensional large model data quality assessment system, characterized in that: include: The data evaluation module is used to evaluate the internal characteristics of the data set that are not related to the task, and obtain a data set that meets the minimum intrinsic quality requirements; A feature extraction module, used for performing feature extraction on each data in the data set that meets the minimum intrinsic quality requirement and the reference sample data set to obtain a set of minimum quality requirement data sample feature vectors and a set of reference sample feature vectors; A context quality assessment module, configured to perform context quality assessment on the data set meeting the minimum intrinsic quality requirement and the sample data set based on the set of minimum quality requirement data sample feature vectors and the set of reference sample feature vectors to obtain a quality assessment result; The quality ranking module is used to rank the quality evaluation results.

Images