Preprocessing method and system for multimodal large model data cleaning and feature enhancement

Abstract

The application discloses a multi-modal large model data cleaning and feature enhancement preprocessing method and system, and relates to the technical field of data processing. The method comprises information acquisition: obtaining data required by a multi-modal large model to obtain to-be-processed data. The application performs corresponding cleaning processing on the to-be-processed data according to the classification result through the set data processing method, and performs corresponding enhancement processing on the completed data according to the classification result through the data enhancement method. In the process of data cleaning and feature enhancement, the data is cleaned and enhanced correspondingly in combination with the field information of the data to adapt to data in different scenes, improve the dynamic adaptability of data cleaning and feature enhancement, reduce the probability of enhancement deviation and semantic fragmentation, improve the data processing efficiency and quality, and the range of the classification result can be determined according to user demand, so that the user can determine whether to pursue accuracy or efficiency in the process of data cleaning and feature enhancement to improve the user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a preprocessing method and system for data cleaning and feature enhancement of a multi-modal large model. BACKGROUND

[0002] A multi-modal large model refers to an artificial intelligence model capable of processing and fusing multiple data types simultaneously. Before training the multi-modal large model, data used for training the multi-modal large model needs to be cleaned and enhanced in features to improve the efficiency and quality of training the multi-modal large model.

[0003] A multi-modal large model pre-training data cleaning and balancing method with a patent publication number CN117932219A includes the following steps: S01, feature extraction is performed on images and texts in a graphic-text pair; S02, cosine similarity of the same graphic-text pair is calculated, and graphic-text pairs with cosine similarity less than a similarity threshold are deleted; S03, text segmentation is performed on the retained graphic-text pair to obtain text segmentation; S04, the number of times each item in the metadata appears in the text segmentation is counted as the word frequency, and the position of the text segmentation in the metadata is recorded; S05, the word frequency probability of each item is calculated; S06, a random probability p is generated, and the corresponding word frequency probability is indexed according to the position of each text segmentation in the metadata in each pair of graphic-text pairs to be balanced, and when all corresponding word frequency probabilities are greater than p, the graphic-text pair is retained, otherwise the graphic-text pair is removed, and the final retained data is the pre-training data after cleaning and balancing;

[0004] The above and similar principle methods rely on manual annotation rules or traditional algorithms, and are difficult to dynamically adapt to complex correlation scenarios of multi-modal data. Traditional cleaning methods such as filtering and redundancy filtering are not thorough in processing cross-modal coupling noise, and the rules excessively rely on specific data sets, which are difficult to migrate to new scenarios, thereby causing the above and similar principle methods to have the disadvantages of insufficient dynamic adaptability and limited reliability of generation. In the process of feature enhancement, the existing technology often uses enhancement based on GAN or diffusion model, which may generate samples deviating from the real scene, leading to model overfitting, and cross-modal generation lacks controllable constraints, which is easy to produce semantic fragmentation and needs to rely on post-processing rules for correction, thereby reducing the efficiency of the feature enhancement process and being not conducive to improving the overall data processing efficiency. Therefore, the present application is proposed. SUMMARY

[0005] The present application aims to provide a preprocessing method and system for data cleaning and feature enhancement of a multi-modal large model to solve the problems raised in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solution: a preprocessing method for data cleaning and feature enhancement of a multi-modal large model, comprising:

[0007] Information acquisition: obtaining data used by a multi-modal large model to obtain to-be-processed data, the to-be-processed data including text, images and audio, obtaining user requirements and feedback address information, the feedback address information being address information for sending information to the user;

[0008] Information classification: classifying the to-be-processed data by a classification method to obtain a classification result;

[0009] Classification processing: the classification result includes a plurality of sub-results, an association between the sub-results and the to-be-processed data is established, sub-data corresponding to the sub-results is extracted based on the association to obtain the sub-data, a sub-database is established based on the sub-results as names for storing the sub-data, and a storage database is established for storing the sub-database;

[0010] It is characterized in that it comprises:

[0011] Data processing: data cleaning and feature enhancement are performed on the sub-data, corresponding cleaning processing is performed on the data of different classification results by a data processing method to obtain a cleaning result, the cleaning result includes completed data and incomplete data, a classification result of the completed data is extracted based on the association to obtain a target result, and corresponding enhancement processing is performed on the data of different classification results by a data enhancement method based on the target result and the completed data to obtain an enhancement result, the enhancement result includes enhanced data and unenhanced data;

[0012] Judgment and backtracking: a backtracking threshold is preset, re-classification is performed by a backtracking method based on the incomplete data, the unenhanced data and the classification result, and data cleaning and feature enhancement are performed to obtain a re-classification result, and the number of times of backtracking of the same data in the backtracking method is recorded to obtain a backtracking number;

[0013] Feedback generation: the relationship between the backtracking number and the backtracking threshold is judged, when the backtracking number exceeds the backtracking threshold, data at the backtracking number is extracted to obtain feedback information, and the feedback information is sent to the user based on the feedback address information.

[0014] Further, the data processing method comprises: presetting a processing format, performing normalization processing on the sub-data based on the processing format to obtain preliminary data, splitting the preliminary data to obtain a plurality of data groups, judging the number of single data in the data groups to obtain a target number, extracting the data group with the target number of single data to obtain a first data group, extracting the data group with the target number of multiple data to obtain a second data group, performing monitoring based on the single data group in the first data group by a detection method to obtain a detection result, extracting the single data group with a detection success feedback from the detection result to obtain first completed data, extracting the single data group with a detection failure feedback from the detection result to obtain first incomplete data, performing verification based on the single data group in the second data group by a cross-validation method to obtain a verification result, extracting the single data group with a verification success feedback from the verification result to obtain second completed data, extracting the single data group with a verification failure feedback from the verification result to obtain second incomplete data, integrating the first completed data and the second completed data to obtain completed data, and integrating the first incomplete data and the second incomplete data to obtain incomplete data.

[0015] Further, the detection method comprises: obtaining a first result by acquiring a sub-result of the first data group, obtaining a to-be-detected data group by acquiring a single data group in the first data group, splitting the to-be-detected data group to obtain single to-be-detected information, obtaining reference information by acquiring domain information of the first result, performing abnormality detection and correction on the to-be-detected information based on the reference information to obtain a detection result and processed to-be-detected information, obtaining the first completed data and the first incomplete data based on the detection result, extracting the processed to-be-detected information to obtain the first completed data when the detection result feedback is that the processed to-be-detected information is normal, and extracting the processed to-be-detected information to obtain the first incomplete data when the detection result feedback is that the processed to-be-detected information is abnormal.

[0016] Further, the process of performing abnormality detection and correction on the to-be-detected information to obtain a detection result and a correction result is: performing paragraph processing on the to-be-detected information to obtain a plurality of sub-detection information and processing positions, performing basic format comparison based on the sub-detection information and the reference information and extracting sub-detection information with basic format errors to obtain abnormal values, correcting the abnormal values based on the reference information to obtain corrected sub-detection information, integrating the sub-detection information based on the processing positions to obtain the processed to-be-detected information, extracting specific features from the processing information to obtain specific features, judging the relationship between the specific features and the reference information to obtain a target relationship, obtaining the detection result based on the target relationship, extracting the processed to-be-detected information to the first completed data when the target relationship feedback is that the specific features belong to the reference information, and extracting the processed to-be-detected information to the first incomplete data when the target relationship feedback is that the specific features do not belong to the reference information.

[0017] Further, the cross-validation method comprises: obtaining a second result by obtaining a sub-result of a second data set; obtaining a to-be-verified data set by obtaining a single data set in the second data set; splitting the to-be-verified data set to obtain a plurality of sub-information; obtaining a target feature by obtaining a feature of the plurality of sub-information; obtaining a judgment result by judging whether the plurality of target features are consistent; obtaining a second completed data by extracting the verification data set when the judgment result is that the plurality of target features are consistent and belong to the second result; and obtaining a second incomplete data by extracting the verification data set when the judgment result is that the plurality of target features are not consistent and belong to the second result.

[0018] Further, the data enhancement method comprises: obtaining a plurality of associated features by recording a feature identified in the data cleaning process of the completed data; splitting the completed data to obtain sub-completed data; establishing a corresponding relationship between the associated features and the sub-completed data; deriving a derived feature based on the associated features and the target result; adding the derived feature to the sub-completed data corresponding to the associated features based on the corresponding relationship to obtain enhanced data; and extracting the sub-completed data without adding the derived feature to obtain non-enhanced data.

[0019] Further, the classification method comprises: obtaining a past category by obtaining a past to-be-processed data; obtaining a filled category by further obtaining category information based on the past category; obtaining a category set by integrating the past category and the filled category; determining a range size based on user demand; determining a classification category based on the range size in the category set to obtain a category standard; splitting the to-be-processed data to obtain sub-processing data; matching the sub-processing data based on the category standard to obtain a sub-result; and integrating all sub-results to obtain a classification result.

[0020] Further, the backtracking method comprises: obtaining a re-classification result by re-classifying the non-completed data based on the non-completed data through the classification method; the re-classification result is different from the classification result; obtaining a re-cleaning result based on the re-classification result and the non-completed data through the data processing method until the re-cleaning result does not contain the non-completed data; obtaining a re-enhanced result by the data enhancement method based on the completed data in the re-cleaning result until the re-enhanced result does not contain the non-enhanced data.

[0021] The preprocessing system for multi-modal large model data cleaning and feature enhancement uses the preprocessing method for multi-modal large model data cleaning and feature enhancement.

[0022] Compared with the prior art, the beneficial effects of the present application are:

[0023] The multi-modal large model data cleaning and feature enhancement preprocessing method and system can clean and enhance the data according to the field information of the data, adapt to different scenes of the data, improve the dynamic adaptability of data cleaning and feature enhancement, reduce the probability of enhancement deviation and semantic fragmentation, improve the data processing efficiency and quality, and improve the user experience.

[0024] Meanwhile, the relationship between the backtracking number and the backtracking threshold is judged to determine whether the data is in a processable state. When the backtracking number exceeds the backtracking threshold, the data is determined to be in an unprocessable state, and feedback information is generated and sent to the user for processing. The set backtracking method provides a channel for reclassifying and cleaning and enhancing the data that has not been cleaned and enhanced. The specific cleaning and enhancement process is the same as the original process, which improves the efficiency of data processing and reduces the number of times that the staff intervenes in processing the data.

[0025] Meanwhile, the length of the paragraph is divided to shorten the length of a single piece of data and improve the efficiency of anomaly detection and correction. The length of the specific paragraph division is determined according to the actual use. Whether the specific feature belongs to the reference information is analyzed to determine whether the to-be-detected information is abnormal. The to-be-detected information is classified and processed according to whether it is abnormal. The to-be-detected information that is abnormal is reclassified and cleaned to improve the accuracy of the data cleaning and data enhancement of the data processing method and the data enhancement method. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 The main flow structure diagram of the present application is shown in the figure;

[0027] Figure 2 The feedback information generation structure diagram of the present application is shown in the figure;

[0028] Figure 3 The data processing method structure diagram of the present application is shown in the figure;

[0029] Figure 4 The structure diagram of the third embodiment of the present application is shown in the figure;

[0030] Figure 5 The classification method structure diagram of the present application is shown in the figure. DETAILED DESCRIPTION

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Multimodal large-scale models utilize a variety of training data types, including text, images, audio, and video. The core of multimodal large-scale models lies in jointly learning features from different modalities to achieve deep understanding and generation of cross-modal information. Examples include generating images from text and combining speech and images for question answering. Data cleaning and feature enhancement improve the quality of model training; cleaned, high-quality data reduces training bias, and enhanced features improve the model's adaptability to complex tasks.

[0033] like Figures 1-5 As shown, the present invention provides a technical solution: a preprocessing method for cleaning and feature enhancement of multimodal large model data, comprising:

[0034] Information Acquisition: Acquire the data required for the multimodal large model to obtain the data to be processed, which includes text, images, and audio. Acquire user needs and feedback address information, which is the address information to which the information is sent to the user.

[0035] It is important to note that the data to be processed is the training data used by the multimodal large model during training, the user requirements are the user's processing needs, which can be obtained by the user themselves, and the feedback address information is the address information, which can be the user's email address, phone number, and IP address, etc.

[0036] Information classification: The data to be processed is divided into domain categories using classification methods to obtain classification results;

[0037] It is important to note that before performing data cleaning and feature enhancement on the data to be processed, the data should be classified and processed according to its different categories to improve the accuracy and efficiency of the processing.

[0038] Classification processing: The classification results include several sub-results. Establish the association between the sub-results and the data to be processed. Based on the association, extract the data to be processed corresponding to the sub-results to obtain sub-data. Create a sub-database named after the sub-results to store the sub-data, and create a storage database to store the sub-databases.

[0039] It should be noted that the sub-result in the classification result is the category information of the sub-data, and the corresponding category of the sub-data can be determined through the establishment of the association relationship. The sub-database can be established by taking the sub-result as the name to store the sub-data, so as to facilitate the storage of data of the same category, and to facilitate subsequent data cleaning and feature enhancement of the corresponding category.

[0040] Embodiment 1:

[0041] The core object of the multi-modal large model preprocessing is the training data, including text, image, audio, video and other multi-modal original data. These data usually come from various sources (such as web pages, books, sensors, etc.), have complex formats (such as HTML, PDF, EPUB, audio waveform, etc.), and need to be structured before use. The information transmission interface is established, the user demand information is obtained by sending the demand information to the information transmission interface, the feedback address information is obtained according to the address information of the user sending information, the category interval size of classification is determined, and all category intervals are in the same standard, for example, the sub-result is traffic, medical, etc. The specific sub-result can also be road, medical image, etc. The classification result is obtained by classifying the data to be processed according to the determined classification.

[0042] It is characterized in that it comprises:

[0043] Data processing: data cleaning and feature enhancement of sub-data, corresponding cleaning processing of data of different classification results is obtained by data processing method to obtain cleaning result, cleaning result includes completed data and incomplete data, classification result of completed data is obtained based on association relationship to obtain target result, corresponding enhancement processing of data of different classification results is obtained based on target result and completed data by data enhancement method to obtain enhancement result, enhancement result includes enhanced data and unenhanced data;

[0044] It should be noted that the data to be processed is processed according to the classification result by setting the data processing method, and the completed data is processed according to the classification result by setting the data enhancement method, so that in the process of data cleaning and feature enhancement, the data can be cleaned and enhanced according to the field information of the data to adapt to different scenes of the data, improve the dynamic adaptability of data cleaning and feature enhancement, and reduce the probability of enhancement deviation and semantic fragmentation, so as to improve the efficiency and quality of data processing.

[0045] Judgment backtracking: presetting backtracking threshold, reclassifying and data cleaning and feature enhancement are performed based on incomplete data, unenhanced data and classification result by backtracking method to obtain reclassification result, and the number of times of backtracking of the same data in the backtracking method is recorded to obtain backtracking number;

[0046] It should be noted that the backtracking threshold is specific quantity information, and the backtracking function is provided by the set backtracking method. The data that has not been successfully cleaned and enhanced is reclassified and then cleaned and enhanced to reduce the amount of data processed by humans and improve the efficiency of data processing.

[0047] Feedback generation: determine the relationship between the number of backtracking and the backtracking threshold. When the number of backtracking exceeds the backtracking threshold, extract the data at the number of backtracking to obtain feedback information, and send the feedback information to the user based on the feedback address information.

[0048] It should be noted that the relationship between the number of backtracking and the backtracking threshold is determined to determine whether the data is in a processable state. When the number of backtracking exceeds the backtracking threshold, the data is determined to be in an unprocessable state, and feedback information is generated and sent to the user for processing.

[0049] Embodiment 2:

[0050] The backtracking threshold is set to 3, and when the number of backtracking is 4, the number of backtracking exceeds the backtracking threshold, and the data is extracted to generate feedback information and feedback to the user. When the number of backtracking is 1, the data is reclassified, cleaned, and enhanced. In the data cleaning process, the corresponding data cleaning model in the prior art can be used in combination with the data processing method to clean the data to improve the efficiency of data cleaning. Similarly, in the process of feature enhancement, the corresponding feature enhancement model in the prior art can be used in combination with the data enhancement method to enhance the data to improve the efficiency of data enhancement. Specifically, after data cleaning and feature enhancement are completed, the sampling quantity and sampling interval can be preset. The sampling quantity can be set to 10 data, and the sampling interval can be set to every 30 minutes. According to the sampling quantity, the sampled data is obtained by sampling the processed data, and the judgment standard is obtained by judging whether the sampled data meets the demand state. According to the judgment standard, the corresponding data cleaning model and data enhancement model are adjusted and optimized to improve the accuracy and efficiency of data processing.

[0051] As Figure 1 and Figure 3As shown, the data processing method comprises: presetting a processing format, normalizing the sub-data based on the processing format to obtain preliminary data, splitting the preliminary data to obtain a plurality of data groups, judging the number of single data in the data group to obtain a target number, extracting the data group with a single target number to obtain a first data group, extracting the data group with multiple target numbers to obtain a second data group, monitoring the single data group in the first data group based on the detection method to obtain a detection result, extracting the single data group with a detection success feedback from the detection result to obtain a first completed data, extracting the single data group with a detection failure feedback from the detection result to obtain a first incomplete data, verifying the single data group in the second data group based on the cross-validation method to obtain a verification result, extracting the single data group with a verification success feedback from the verification result to obtain a second completed data, extracting the single data group with a verification failure feedback from the verification result to obtain a second incomplete data, integrating the first completed data and the second completed data to obtain completed data, and integrating the first incomplete data and the second incomplete data to obtain incomplete data.

[0052] It should be noted that the processing format is a standard format for data training of a multi-modal large model, which can be obtained by an administrator. The sub-data is normalized according to the processing format to improve the efficiency of subsequent processing. Multi-modal large model data has different characteristics, which can be manifested as a group of data that may contain two or more image and text data, or a group of data that contains only single image or text data. By analyzing the number of data in the data group, the data cleaning is performed according to the number of data, which can improve the efficiency of data processing. The cross-validation method is used to judge whether the data expression is consistent for the data group with multiple data, and the detection method is used to detect whether the format information of the data conforms to the field and correct the data group with single data. In actual use, the detection method can be added to the cross-validation method to detect outliers and correct the data group with multiple data, so as to improve the running efficiency of the cross-validation method. Figure 3 In a, the processing flow of the first data group is, Figure 3 In b, the processing flow of the second data group is.

[0053] As Figure 3As shown, the detection method comprises: obtaining a first result by obtaining a sub result of a first data set, obtaining a single data set in the first data set to obtain a to-be-detected data set, splitting the to-be-detected data set to obtain single to-be-detected information, obtaining reference information by obtaining domain information of the first result, performing abnormal detection and correction on the to-be-detected information based on the reference information to obtain a detection result and processed to-be-detected information, obtaining first completed data and first incomplete data based on the detection result, when the detection result feedback is that the processed to-be-detected information is normal, extracting the processed to-be-detected information to obtain the first completed data, and when the detection result feedback is that the processed to-be-detected information has an abnormality, extracting the processed to-be-detected information to obtain the first incomplete data.

[0054] It should be noted that the process of obtaining reference information by obtaining domain information of the first result, that is, obtaining the domain information of the classification result, specifically, when the first result is a road class, the reference information is the domain information of the road class, and specifically can be common sense information and specific information of the road class. According to the reference information, the abnormal detection and correction of the to-be-detected information can improve the accuracy of data processing and improve the dynamic adaptability of data processing.

[0055] The process of performing abnormal detection and correction on the to-be-detected information to obtain the detection result and the correction result is: performing paragraph processing on the to-be-detected information to obtain a plurality of sub-detection information and processing positions, performing basic format comparison based on the sub-detection information and the reference information and extracting sub-detection information with errors in basic format to obtain abnormal values, modifying the abnormal values based on the reference information to obtain modified sub-detection information, integrating the sub-detection information based on the processing positions to obtain processed to-be-detected information, extracting features in the processing information to obtain specific features, judging the relationship between the specific features and the reference information to obtain a target relationship, obtaining a detection result based on the target relationship, when the target relationship feedback is that the specific features belong to the reference information, the detection result is that the processed to-be-detected information is normal, and the processed to-be-detected information is extracted to obtain the first completed data, and when the target relationship feedback is that the specific features do not belong to the reference information, the detection result is that the processed to-be-detected information has an abnormality, and the processed to-be-detected information is extracted to obtain the first incomplete data.

[0056] It should be noted that the process of abnormality detection and correction of the to-be-detected information is combined with the reference information of the to-be-detected information category, and the abnormality detection and correction can be performed by establishing an abnormality detection and correction model. In the preliminary abnormality detection and correction of the to-be-detected information, the to-be-detected information is divided into paragraphs to shorten the length of a single data and improve the efficiency of abnormality detection and correction. The length of the specific paragraph division is determined according to the actual use. Whether the specific feature belongs to the reference information is analyzed to determine whether the to-be-detected information is abnormal. The to-be-detected information is classified and processed according to whether it is abnormal. The to-be-detected information with abnormality is reclassified and subjected to data cleaning processing to improve the accuracy of data cleaning and data enhancement by the data processing method and the data enhancement method.

[0057] Embodiment 3

[0058] The abnormality detection and correction of the to-be-detected information is performed by establishing an abnormality detection and correction model under the category of the to-be-detected information. The process of establishing the abnormality detection and correction model can be: obtaining training data, which includes but is not limited to data with format errors in this category and data with semantic defects in this category, etc. The specific format errors and semantic defects are labeled in the training data. The training data also includes data after changing format errors and semantic errors, etc. A deep learning model is selected as a model base. The training data is imported into the model base for learning. After learning, an initial model is obtained. Data that does not exist in the training data is selected as validation data. The validation data is consistent with the training data, and the format errors or semantic defects in the validation data do not exist. The validation data is imported into the initial model to obtain an abnormality detection result. When the abnormality detection result feedback is that the validation data has errors, a correction result is generated. When the abnormality detection result feedback is that the validation data does not have errors, no correction result is generated. The validation result is obtained by manually judging whether the validation data has errors. The initial model is adjusted and optimized according to the difference between the validation result and the abnormality detection result to obtain an abnormality detection and correction model. The specific process is shown in Figure 4 .

[0059] As shown in Figure 3 , the cross-validation method includes: obtaining a second result by obtaining a sub-result of a second data group, obtaining a to-be-verified data group by obtaining a single data group in the second data group, splitting the to-be-verified data group to obtain a plurality of sub-information, obtaining a target feature by obtaining a feature of the plurality of sub-information, determining whether the plurality of target features are consistent to obtain a determination result, extracting a second completed data from the to-be-verified data group when the determination result feedback is that the plurality of target features are consistent and belong to the second result, and extracting a second incomplete data from the to-be-verified data group when the determination result feedback is not that the plurality of target features are consistent and belong to the second result.

[0060] It should be noted that the process of obtaining target features from the characteristics of several sub-information can be performed by training a dedicated feature recognition model for feature recognition, or by manually recognizing by staff. The specific recognition method is determined according to the actual use condition. Through cross-validation method, the data of different expressions with the same meaning in the data set to be verified can be cross-verified and combined with the category information of the data set to improve the reliability of data cleaning. The specific data cleaning can also be combined with the deeper cleaning technology and data processing method in the prior art to improve the data cleaning accuracy.

[0061] As shown in Figure 1 The data enhancement method includes: obtaining several associated features from the features recognized in the data cleaning process of the completed data, splitting the completed data to obtain sub-completed data, establishing a corresponding relationship between the associated features and the sub-completed data, deriving features based on the associated features and target results to obtain derived features, adding the derived features to the sub-completed data corresponding to the associated features based on the corresponding relationship to obtain enhanced data, and extracting the sub-completed data without adding the derived features to obtain non-enhanced data.

[0062] It should be noted that the associated features can be obtained from the system operation log in the actual operation process. The specific associated features include target features and specific features. The associated features are obtained by recording the features of the completed data, which can reduce the amount of data processing and improve work efficiency. The process of deriving features based on the associated features and target results can be obtained by recording the associated features and searching the associated features combined with the target results, or by training a dedicated feature derivation model to obtain the corresponding derived features by inputting the associated features and the target results. At the same time, manual derivation by staff can also be performed. The specific feedback to the staff for further determination can be obtained by adding a verification mechanism. The enhanced data is randomly sampled and judged by the staff to improve the data processing accuracy.

[0063] As shown in Figure 1 and Figure 5 The classification method includes: obtaining the category of the past data to be processed to obtain the past category, further obtaining the category information based on the past category to obtain the filled category, integrating the past category and the filled category to obtain the category set, determining the range size based on the user demand, determining the classification category based on the range size in the category set to obtain the category standard, splitting the data to be processed to obtain the sub-processing data, matching the sub-processing data based on the category standard to obtain the sub-result, and integrating all sub-results to obtain the classification result.

[0064] It should be noted that the process of obtaining the past category of the past to-be-processed data to obtain the past category can be classified by the staff according to the past to-be-processed data, or can be classified by the known category of the to-be-imported multi-modal large model data, and the process of further obtaining the category information to obtain the filled category based on the past category, that is, on the basis of the original past category, further classifying the category to obtain the filled category, the process of determining the range size based on the user demand, that is, according to the user demand, determining the category range needed to be used, which can be specifically understood as whether the fine classification or the coarse classification is needed, for example, the coarse classification can be traffic, and the fine classification can be an extension of the traffic category, specifically road, marking line and signal lamp, by selecting different depth of classification according to the user demand, the user can decide whether the data cleaning and data enhancement process pursues precision or efficiency, so as to improve the user experience. Figure 5 The middle extension path represents the containing relationship between the filled category and the past category.

[0065] As shown in Figure 1 and Figure 2 , the backtracking method includes: based on the unfinished data, the reclassification result is obtained by the classification method, the reclassification result is different from the classification result, the reclassification result and the unfinished data are obtained by the data processing method, the reclassification result is obtained until the reclassification result does not contain the unfinished data, the completed data in the reclassification result is obtained by the data enhancement method until the reclassification result does not contain the enhanced data.

[0066] It should be noted that by setting the backtracking method, a channel for reclassifying and cleaning and enhancing data that has not been cleaned and enhanced is provided, and the process of re-cleaning and enhancing is the same as the original, which is beneficial to improve the efficiency of data processing and reduce the number of times of staff intervention in processing data.

[0067] Although the embodiments of the present application have been shown and described, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended embodiments and their equivalents.

Claims

1. A preprocessing method for multimodal large model data cleaning and feature enhancement, comprising: information acquisition: obtaining data required by a multimodal large model to obtain to-be-processed data, the to-be-processed data including text, images and audio, obtaining user demand and feedback address information, the feedback address information being address information for sending information to a user; information classification: classifying the to-be-processed data by a classification method to obtain a classification result; classification processing: the classification result includes a plurality of sub-results, an association between the sub-results and the to-be-processed data is established, sub-data corresponding to the sub-results is extracted based on the association to obtain the sub-data, a sub-database is established based on the sub-results as names to store the sub-data, and a storage database is established to store the sub-database; characterized in that it comprises: data processing: data cleaning and feature enhancement are performed on the sub-data, different classification results are processed by a data processing method to obtain cleaning results, the cleaning results include completed data and incomplete data, classification results of the completed data are extracted based on the association to obtain target results, and different classification results are processed by a data enhancement method based on the target results and the completed data to obtain enhancement results, the enhancement results include enhanced data and unenhanced data; judgment and backtracking: a backtracking threshold is preset, the incomplete data, the unenhanced data and the classification results are reclassified by a backtracking method and data cleaning and feature enhancement are performed to obtain reclassification results, and the number of times the same data is backtracked in the backtracking method is recorded to obtain a backtracking number; feedback generation: the relationship between the backtracking number and the backtracking threshold is judged, when the backtracking number exceeds the backtracking threshold, data at the backtracking number is extracted to obtain feedback information, and the feedback information is sent to the user based on the feedback address information; the data processing method comprises: a preset processing format, the sub-data is normalized based on the processing format to obtain preliminary data, the preliminary data is split to obtain a plurality of data groups, the number of single data in the data group is judged to obtain a target number, data groups with a target number of single data are extracted to obtain first data groups, data groups with a target number of multiple data are extracted to obtain second data groups, single data groups in the first data groups are monitored by a detection method to obtain detection results, single data groups with detection success feedback in the detection results are extracted to obtain first completed data, single data groups with detection failure feedback in the detection results are extracted to obtain first incomplete data, single data groups in the second data groups are verified by a cross-validation method to obtain verification results, single data groups with verification success feedback in the verification results are extracted to obtain second completed data, single data groups with verification failure feedback in the verification results are extracted to obtain second incomplete data, the first completed data and the second completed data are integrated to obtain completed data, and the first incomplete data and the second incomplete data are integrated to obtain incomplete data.

2. The multimodal large model data cleaning and feature enhancement preprocessing method according to claim 1, characterized in that: The detection method comprises: obtaining a first result by obtaining a sub result of a first data group, obtaining a to-be-detected data group by obtaining a single data group in the first data group, splitting the to-be-detected data group to obtain single to-be-detected information, obtaining reference information by obtaining domain information of the first result, performing abnormality detection and correction on the to-be-detected information based on the reference information to obtain a detection result and processed to-be-detected information, obtaining first completed data and first incomplete data based on the detection result, when the detection result feedback is that the processed to-be-detected information is normal, extracting the processed to-be-detected information to obtain the first completed data, and when the detection result feedback is that the processed to-be-detected information is abnormal, extracting the processed to-be-detected information to obtain the first incomplete data.

3. The multimodal large model data cleaning and feature enhancement preprocessing method of claim 2, wherein: The process of performing abnormality detection and correction on the to-be-detected information to obtain a detection result and a correction result is: performing paragraph processing on the to-be-detected information to obtain a plurality of sub-detection information and processing positions, performing basic format comparison based on the sub-detection information and the reference information and extracting sub-detection information with basic format errors to obtain abnormal values, correcting the abnormal values based on the reference information to obtain corrected sub-detection information, integrating the sub-detection information based on the processing positions to obtain processed to-be-detected information, extracting features in the processing information to obtain specific features, judging the relationship between the specific features and the reference information to obtain a target relationship, obtaining the detection result based on the target relationship, when the target relationship feedback is that the specific features belong to the reference information, the detection result is that the processed to-be-detected information is normal, and the processed to-be-detected information is extracted to the first completed data, and when the target relationship feedback is that the specific features do not belong to the reference information, the detection result is that the processed to-be-detected information is abnormal, and the processed to-be-detected information is extracted to obtain the first incomplete data.

4. The multimodal large model data cleaning and feature enhancement preprocessing method of claim 1, wherein: The cross-validation method comprises: obtaining a second result by obtaining a sub result of a second data group, obtaining a to-be-verified data group by obtaining a single data group in the second data group, splitting the to-be-verified data group to obtain a plurality of sub information, obtaining target features by obtaining features of the plurality of sub information, and judging whether the plurality of target features are consistent to obtain a judgment result, when the judgment result feedback is that the plurality of target features are consistent and belong to the second result, extracting the verification data group to obtain second completed data, and when the judgment result feedback is not that the plurality of target features are consistent and belong to the second result, extracting the verification data group to obtain second incomplete data.

5. The multimodal large model data cleaning and feature enhancement preprocessing method of claim 1, wherein: The data enhancement method comprises: obtaining a plurality of associated features by recording features identified in data cleaning processing of completed data, splitting the completed data to obtain sub-completed data, establishing a corresponding relationship between the associated features and the sub-completed data, deriving features based on the associated features and target results to obtain derived features, adding the derived features to the sub-completed data corresponding to the associated features based on the corresponding relationship to obtain enhanced data, and extracting sub-completed data without adding derived features to obtain non-enhanced data.

6. The multimodal large model data cleaning and feature enhancement preprocessing method of claim 1, wherein: The classification method comprises: obtaining a category of previous to-be-processed data to obtain a previous category, further obtaining category information based on the previous category to obtain a filled category, integrating the previous category and the filled category to obtain a category set, determining a range size based on user demand, determining a divided category in the category set based on the range size to obtain a category standard, splitting the to-be-processed data to obtain sub-processing data, matching the sub-processing data based on the category standard to obtain a sub-result, and integrating all sub-results to obtain a classification result.

7. The multimodal large model data cleaning and feature enhancement preprocessing method of claim 1, wherein: The backtracking method comprises: obtaining a reclassification result by reclassifying the uncompleted data based on the classification method, the reclassification result being different from the classification result, obtaining a re-cleaning result based on the reclassification result and the uncompleted data by a data processing method until the re-cleaning result does not contain the uncompleted data, and obtaining a re-enhanced result based on the completed data in the re-cleaning result by a data enhancement method until the re-enhanced result does not contain the unenhanced data.

Images