Humanoid robot data set quality evaluation method and system based on multi-dimensional evaluation

Abstract

The invention provides a humanoid robot data set quality evaluation method and a humanoid robot data set quality evaluation system based on multi-dimensional evaluation, and provides a systematized and standardized evaluation process aiming at the quality problems of the existing robot multi-modal data set in the aspects of normalization, integrity, accuracy, consistency, timeliness, accessibility, availability and the like. According to the method, an automatic tool is combined with manual auditing, the structure, metadata, annotation content, data formats, multi-modal alignment and the like of a data set are comprehensively detected, a deep learning algorithm is introduced to achieve detection of pornographic pictures and sensitive words of visual content, and content safety and compliance are effectively improved. The system can flexibly adjust the weight of each evaluation dimension according to different application scenes, automatically generates a quality evaluation report, and supports continuous monitoring and optimization and improvement. Practical application shows that the method can significantly improve the overall quality of the robot data set, and provides reliable data basis and technical support for intelligent training and application of the robot.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and robotics technology, specifically relating to a method and system for evaluating the quality of humanoid robot datasets based on multi-dimensional assessment. Background Technology

[0002] With the rapid development of artificial intelligence technology, humanoid robots, as an important carrier of AI technology, have been widely used in service robots, industrial robots, medical robots, and other fields. The level of intelligence of humanoid robots largely depends on the quality of their training datasets; high-quality datasets are the foundation for ensuring that robots accurately perform various tasks.

[0003] Currently, training datasets for humanoid robots typically contain multimodal data, such as visual images, audio signals, sensor data, action sequences, and environmental information. These data are prone to various quality issues during acquisition, annotation, storage, and processing, including but not limited to:

[0004] Incomplete data: Due to equipment failure, network interruption or human error, the data collection is incomplete, with missing values ​​or blank data segments.

[0005] Inaccurate data: Issues such as incorrect labeling, sensor drift, and timestamp asynchrony can cause data to be inconsistent with the actual situation.

[0006] Data inconsistency: Data collected by different sensors in the same scene may conflict, or similar data may have inconsistent formats at different time periods.

[0007] Poor data timeliness: Data collection time is too long, and it cannot reflect the actual situation of the current environment or task.

[0008] Poor data accessibility: The data is poorly organized and lacks effective indexing and retrieval mechanisms.

[0009] Furthermore, existing data quality assessment methods also have shortcomings, making it difficult to ensure the quality of the training dataset. The main shortcomings of existing data quality assessment methods are as follows:

[0010] First, there is a lack of a dedicated evaluation system for the characteristics of multimodal data from humanoid robots. Traditional data quality assessment methods mainly target structured data or single-modal data, making it difficult to effectively handle the complex relationships between multiple modalities such as vision, hearing, touch, and motion in humanoid robot data.

[0011] Secondly, existing methods often employ a single evaluation dimension, lacking comprehensiveness and systematicity. Most methods focus only on a single aspect such as data completeness or accuracy, failing to comprehensively reflect the overall quality of the dataset.

[0012] Secondly, the level of automation is low, relying mainly on manual inspection, which is inefficient and prone to subjective bias. Faced with massive amounts of robot training data, purely manual inspection is not only time-consuming and labor-intensive, but also makes it difficult to guarantee the consistency and objectivity of the inspection.

[0013] Finally, there is a lack of standardized evaluation processes and tools. Existing evaluation methods are often ad hoc and project-specific, lacking reusability and standardization.

[0014] Therefore, there is an urgent need for a quality evaluation method that can be tailored to the characteristics of humanoid robot datasets, employ a multi-dimensional evaluation system, and combine automated detection with manual review, in order to improve the accuracy, efficiency, and standardization of dataset quality evaluation. Summary of the Invention

[0015] This invention addresses the aforementioned problems by providing a method and system for quality assessment of humanoid robot datasets based on multi-dimensional evaluation. It combines automated detection with manual review to comprehensively assess and control the quality of humanoid robot training datasets. Specifically, this invention aims to: establish a seven-dimensional quality assessment system suitable for the characteristics of multimodal humanoid robot data; achieve an efficient assessment process combining automated detection and manual review; provide standardized dataset quality assessment tools and platforms; and ensure the accuracy, objectivity, and repeatability of the assessment results.

[0016] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0017] This invention provides a method for evaluating the quality of humanoid robot datasets based on multi-dimensional assessment. This method is used to evaluate robot datasets containing multimodal data to determine their quality level. It has the following technical features: Step S1, preprocessing and classifying various types of raw data in the robot dataset, including at least data structure parsing, data type identification and classification, and file format and naming convention detection, to obtain a preprocessed dataset; Step S2, constructing multi-dimensional quality feature data based on the preprocessed dataset, including standardization features, completeness features, accuracy features, consistency features, timeliness features, accessibility features, and usability features; Step S3, based on the multi-dimensional feature data, utilizing pre-trained multi-dimensional... A multi-dimensional quality assessment model comprehensively evaluates the preprocessed dataset and, combined with manual review, obtains a quality assessment report for the robot dataset, which includes quality level information. The multi-dimensional quality assessment model includes a feature extraction module, a deep learning detection module, and a comprehensive scoring module. The feature extraction module extracts basic features and statistical information of various data types from the preprocessed dataset and inputs them into the deep learning detection module. Based on the identified data types and their basic features, the deep learning detection module performs corresponding detections on each data type and outputs the corresponding detection results to the comprehensive scoring module. The comprehensive scoring module performs weighted fusion of the various basic features and the detection results and outputs the quality assessment report for the robot dataset.

[0018] The humanoid robot dataset quality evaluation method based on multi-dimensional assessment provided by this invention may also have the following technical features, wherein step S1 includes the following sub-steps: Step S1-1, performing structure and format parsing on all data files in the robot dataset, and selecting data files that meet the requirements as qualified data files; Step S1-2, performing data type identification and file naming standardization detection on the qualified data files, and classifying the qualified data files into corresponding data types based on the detection results; Step S1-3, based on the classification results of step S1-2, performing corresponding file format and naming standardization detection on the qualified data files of each data type; Step S1-4, filtering out invalid information in the classification results, and normalizing the qualified data files of each data type to form the preprocessed dataset.

[0019] The humanoid robot dataset quality evaluation method based on multi-dimensional assessment provided by the present invention may also have the following technical features, wherein steps S1-2 further include: classifying the robot dataset into the corresponding application scenario type according to a preset application scenario classification standard; identifying the specific task type corresponding to the robot dataset according to a preset task classification standard; extracting the technical tag information contained in the robot dataset; and establishing a mapping relationship between skills and data based on the extracted technical tag information.

[0020] The humanoid robot dataset quality evaluation method based on multi-dimensional assessment provided by the present invention may also have the following technical features, wherein, in steps S1-3, the file format and naming convention detection includes file existence check, mandatory field coverage check, data type compliance check, and format consistency check.

[0021] The humanoid robot dataset quality evaluation method based on multi-dimensional assessment provided by this invention may also have the following technical features, wherein, in step S2, the normative features include file format or data format normativity, probability of images containing pornography, and sensitive word labeling; the integrity features include data integrity features and variable integrity features; the accuracy features include annotation accuracy, reasonableness of numerical range, correctness of category labels, and accuracy of timestamps; the consistency features include internal consistency, format consistency, and multimodal consistency; the timeliness features include data freshness, dataset version timeliness, and dataset environmental adaptability; the accessibility features include data organization structure accessibility; and the usability features include application adaptability, model training effect, and model generalization ability.

[0022] The humanoid robot dataset quality evaluation method based on multi-dimensional assessment provided by this invention may also have the following technical features: Among the standardization features, the file format or data format standardization includes data format standardization, naming standardization, structural standardization, annotation standardization, and content standardization. Among the integrity features, the data integrity features include data missing rate, data missing pattern, key data missing situations, and the degree of impact of missing data; the variable integrity features include the integrity of necessary variables, the integrity of variable types, the integrity of variable relationships, and the integrity of variable value ranges. Among the accuracy features, the annotation accuracy includes annotation consistency, annotation accuracy rate, and annotation error type; the numerical range reasonableness includes the reasonableness of numerical statistical distribution, outlier situations, reasonableness of range boundaries, and reasonableness of physical meaning; the category label correctness includes the correctness of label enumeration, the correctness of label semantics, the reasonableness of label distribution, and the correctness of label hierarchical structure; the timestamp accuracy includes the correctness of time format, the reasonableness of time logic, time zone consistency, and time synchronization accuracy. The consistency features include: internal consistency (data relationship consistency, constraint consistency, business rule consistency, statistical feature consistency); format consistency (data type consistency, encoding format consistency, date and time format consistency, data precision consistency); and multimodal consistency (semantic consistency, spatiotemporal consistency). The timeliness features include: data freshness (collection time distribution, timeliness, expired data status, freshness threshold); dataset version timeliness (version update frequency, version lifecycle, version compatibility, version iteration rationality); and dataset environment adaptability (application scenario matching degree, technical environment adaptability, impact of business environment changes). The accessibility features include: data organization structure accessibility (directory structure clarity, file accessibility, classification system rationality, retrieval path optimization degree). The usability features include: application adaptability (demand matching degree, functional coverage, scenario adaptability, customization degree); model training effect (training convergence, model performance indicators, model training efficiency, model training resource consumption); and generalization ability (cross-domain generalization ability, cross-task generalization ability, robustness, and generalization ability quantification indicators).

[0023] The humanoid robot dataset quality evaluation method based on multi-dimensional assessment provided by this invention may also have the following technical features: the deep learning detection module includes multiple format verification submodules, a pornography detection submodule, a sensitive word recognition submodule, and an anomaly detection submodule. The format verification submodule is used to verify the file format or data format specifications of files of corresponding data types. The pornography detection submodule is a pornography detection model based on convolutional neural networks, used to detect pornography-related images in image data. The sensitive word recognition submodule is a sensitive word detection model based on OCR technology, used to perform OCR sensitive word detection on text data and image data. The anomaly detection submodule is used to detect outliers in sensor data.

[0024] The humanoid robot dataset quality assessment method based on multi-dimensional evaluation provided by this invention may also have the following technical features, wherein step S3 includes the following sub-steps: Step S3-1, inputting multi-dimensional quality feature data into a pre-trained multi-dimensional quality assessment model, the model outputs preliminary assessment results, including scores for each dimension and detected high-risk items; Step S3-2, generating corresponding manual review tasks based on the high-risk items, which are then manually reviewed by reviewers; Step S3-3, generating the quality assessment report based on the preliminary assessment results after manual review and the predetermined weights for each dimension, wherein the comprehensive quality score of the robot dataset is obtained by weighted fusion of the scores for each dimension.

[0025] The humanoid robot dataset quality assessment method based on multi-dimensional evaluation provided by this invention may also have the following technical features, wherein the training of the multi-dimensional quality assessment model includes the following steps: Step T1, constructing an initial multi-dimensional quality assessment model structure; Step T2, using a robot dataset containing multiple sets of labeled quality levels as a training set, sequentially inputting it into the initial multi-dimensional quality assessment model, the initial multi-dimensional quality assessment model performs feature extraction and deep learning detection based on the input data, and outputs quality level information; Step T3, calculating the loss error between the output quality level information and the labeled quality level information, and updating the model parameters using the backpropagation algorithm; Step T4, repeating steps T2 to T3 until the predetermined training completion conditions are met, thereby obtaining the trained multi-dimensional quality assessment model.

[0026] This invention provides a quality evaluation system for humanoid robot datasets based on multi-dimensional assessment. This system evaluates robot datasets containing multimodal data to determine their quality level. It features the following technical characteristics: A preprocessing and classification unit performs preprocessing and classification on various types of raw data in the robot dataset, including at least data structure parsing, data type identification and classification, and file format and naming convention detection, thereby obtaining a preprocessed dataset; a feature data construction unit constructs multi-dimensional quality feature data based on the preprocessed dataset, including features such as standardization, completeness, accuracy, consistency, timeliness, accessibility, and usability; and a comprehensive evaluation unit performs a comprehensive evaluation of the preprocessed dataset based on the multi-dimensional feature data, using a pre-trained multi-dimensional quality evaluation model, and combines this with manual review to obtain a quality evaluation report for the robot dataset, which includes quality level information. The multi-dimensional quality assessment model includes a feature extraction module, a deep learning detection module, and a comprehensive scoring module. The feature extraction module extracts basic features and statistical information of various types of data from the preprocessed dataset and inputs them into the deep learning detection module. The deep learning detection module performs corresponding detection on the data of various types based on the identified data types and their basic features, and outputs the corresponding detection results to the comprehensive scoring module. The comprehensive scoring module performs weighted fusion of the basic features and the detection results of various types, and outputs the quality assessment report of the robot dataset.

[0027] The role and effect of invention

[0028] The humanoid robot dataset quality evaluation method and system based on multi-dimensional assessment provided by this invention, by constructing a dataset quality evaluation system covering seven dimensions—standardization, completeness, accuracy, consistency, timeliness, accessibility, and usability—can systematically and comprehensively evaluate multimodal datasets of humanoid robots. The method of this invention not only establishes detailed quality inspection standards for multiple data modalities such as vision, hearing, kinematics, and dynamics, but also combines automated inspection with manual review, greatly improving the efficiency and accuracy of the evaluation. Through automated tools, it is possible to quickly detect basic standardization aspects such as data format, naming conventions, and annotation content, while simultaneously performing comprehensive automated reviews of data completeness, variable coverage, time series continuity, and multimodal alignment, significantly reducing manual workload. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of a humanoid robot dataset quality evaluation method based on multi-dimensional assessment in an embodiment of the present invention;

[0030] Figure 2 This is a flowchart of the humanoid robot dataset quality evaluation method based on multi-dimensional assessment in an embodiment of the present invention;

[0031] Figure 3 This is a schematic diagram of the structure of the humanoid robot dataset quality evaluation system based on multi-dimensional evaluation in an embodiment of the present invention;

[0032] Figure 4 This is a flowchart of step S1 in an embodiment of the present invention;

[0033] Figure 5 This is a flowchart of step S3 in an embodiment of the present invention;

[0034] Figure 6 This is a flowchart of the training process of the multi-dimensional quality assessment model in an embodiment of the present invention. Detailed Implementation

[0035] To make the technical means, creative features, objectives and effects of this invention easy to understand, the following describes in detail the humanoid robot dataset quality evaluation method and system based on multi-dimensional evaluation.

[0036] <Example>

[0037] This embodiment provides a method and system for quality evaluation of humanoid robot datasets based on multi-dimensional assessment, used for quality evaluation of home service robot datasets. The following will use a humanoid robot dataset containing 100,000 data points from home service scenarios as an example to illustrate the implementation process of this invention in detail.

[0038] Figure 1 This is a schematic diagram of the humanoid robot dataset quality evaluation method based on multi-dimensional assessment in this embodiment. Figure 2 This is a flowchart of the humanoid robot dataset quality evaluation method based on multi-dimensional assessment in this embodiment.

[0039] like Figure 1 and Figure 2 As shown, the method includes the following steps:

[0040] Step S1: Perform corresponding preprocessing and classification on various types of raw data in the robot dataset to obtain a preprocessed dataset.

[0041] Step S2: Construct multi-dimensional quality feature data based on the preprocessed dataset.

[0042] Step S3: Use the pre-trained multi-dimensional quality assessment model to comprehensively evaluate the multi-dimensional quality feature data and obtain the corresponding robot dataset quality assessment report.

[0043] Figure 3 This is a schematic diagram of the structure of the humanoid robot dataset quality evaluation system based on multi-dimensional assessment in this embodiment.

[0044] like Figure 3 As shown, this embodiment also provides a humanoid robot dataset quality evaluation system 10 based on multi-dimensional evaluation, corresponding to the above method. It includes a dataset acquisition unit 11, a preprocessing and classification unit 12, a feature data construction unit 13, a comprehensive evaluation unit 14, an evaluation result output unit 15, and a control unit 16 for controlling the operation of each unit. The dataset acquisition unit 11 interfaces with a robot or robot server to obtain the robot dataset to be evaluated. The preprocessing and classification unit 12, the feature data construction unit 13, and the comprehensive evaluation unit 14 respectively perform data preprocessing and classification, feature data construction, and comprehensive evaluation using the methods described in steps S1 to S3. The evaluation result output unit 15 outputs the comprehensive evaluation result for the robot dataset.

[0045] The steps described above will be explained in detail below.

[0046] Step S1: Perform corresponding preprocessing and classification on various types of raw data in the robot dataset to obtain a preprocessed dataset.

[0047] In this step, preprocessing includes at least data structure parsing, data type identification and classification, and file format and naming convention checks. Data type identification and classification includes the identification and classification of data types, application scenarios, and task types.

[0048] Figure 4 This is a flowchart of step S1 in this embodiment. For example... Figure 4 As shown, specifically, step S1 includes the following sub-steps:

[0049] Step S1-1: Analyze the structure and format of all data files in the robot dataset and select the data files that meet the requirements as qualified data files.

[0050] Step S1-2: Perform data type identification and file naming standardization checks on qualified data files. Based on the check results, automatically classify qualified data files into various data types, such as image files, audio files, text files, sensor data files, annotation files, metadata files, etc., and output the classification results.

[0051] This step may also include: classifying the robot dataset into corresponding application scenario types according to preset application scenario classification standards, such as home services, industrial operations, medical assistance, education and entertainment, etc.; identifying the specific task types corresponding to the robot dataset according to preset task classification standards, such as object recognition, motion imitation, voice interaction, path planning, etc.; extracting skill tag information contained in the robot dataset and establishing a mapping relationship between skills and data. The classification results also include the aforementioned application scenario types, task types, and mapping relationships.

[0052] In this embodiment, the humanoid robot dataset for home service scenarios is first subjected to automated structural parsing. Through directory traversal and file type identification, RGB images, depth images, audio, various sensor data, action sequences, and labeled files are distinguished. A Python script combined with regular expressions is used to automatically identify filenames, paths, and extensions, generating a data structure tree and recording the quantity and distribution of each file type.

[0053] The system incorporates a T / SAIAS standard scene and task type encoding table. Based on this table, the system automatically parses the scene and task fields from the humanoid robot dataset. If any fields are missing, it uses a Natural Language Processing (NLP) model (such as BERT) to semantically classify the descriptive text, assisting manual confirmation of the scene or task attribution. For example, if the descriptive text contains keywords such as "kitchen" or "cleaning," it is automatically categorized as a "housekeeping-cleaning" scene.

[0054] Next, for the labeled files and descriptive text, skill tags (such as "grab", "move", "voice interaction") are extracted using a Network Entity Recognition (NER) model, establishing a mapping relationship between skills and data, and outputting them in JSON format for subsequent review.

[0055] Steps S1-3 involve performing file format and naming convention checks on each type of file based on the classification results. In other words, the metadata and description files of the dataset are automatically evaluated.

[0056] In this step, the file format and naming convention checks include file existence checks, required field coverage checks, data type compliance checks, and format consistency checks.

[0057] For file existence checks, the necessary metadata and description files are checked for their existence, including dataset description files, annotation description files, version information files, etc.

[0058] For mandatory field coverage checks, verify the coverage of mandatory fields in the metadata to ensure the integrity of critical information.

[0059] For data type compliance checks, verify whether the data type of each field conforms to the predefined specification requirements.

[0060] For format consistency checks, the file format, encoding method, naming conventions, etc. are checked to see if they conform to the standards.

[0061] In this embodiment, for file existence checks and format consistency checks, the system automatically detects description files (such as README, dataset.json, label_map.txt, etc.) in the dataset root directory and its subdirectories, and attempts to parse and load them. If a file is corrupted or has an incorrect format, it is automatically recorded as an exception.

[0062] For mandatory field coverage checks and data type compliance checks, the system uses JSON Schema or a custom YAML template to validate each metadata field. The system calculates the coverage of mandatory fields, and if any are missing, it automatically marks them and outputs a list of missing fields. For each field, the system checks whether its data type (e.g., string, integer, boolean) is consistent with the standard.

[0063] Steps S1-4 involve filtering out invalid information in the classification results and normalizing parameters such as resolution, sampling rate, and time series of various data files to form the preprocessed dataset mentioned above.

[0064] Step S2: For the preprocessed dataset, construct multi-dimensional quality feature data respectively.

[0065] In this step, the multi-dimensional quality characteristic data includes normative characteristics, completeness characteristics, accuracy characteristics, consistency characteristics, timeliness characteristics, accessibility characteristics, and usability characteristics.

[0066] The multi-dimensional quality feature data is represented by Q = (F, D), where Q represents the multi-dimensional quality feature data, F represents various basic features (such as data volume, missing rate, format distribution, etc.), and D represents the detection results based on deep learning (such as the probability of pornography, sensitive word marking, outlier distribution, etc.).

[0067] In this embodiment, the normative features include file format or data format norms, probability of images containing pornography, and sensitive word marking. Specifically, file format or data format norms include data format norms, naming norms, structural norms, annotation norms, and content norms.

[0068] Completeness features include data completeness features and variable completeness features. Data completeness features include data missing rate, data missing patterns, key data missing information, and the degree of impact of missing data. Variable completeness features include the completeness of necessary variables, the completeness of variable types, the completeness of variable relationships, and the completeness of variable value ranges. Assessing dataset completeness ensures that the dataset is complete in all aspects, providing a complete data foundation for subsequent applications.

[0069] Accuracy characteristics include annotation accuracy, reasonableness of numerical range, correctness of category labels, and timestamp accuracy. Annotation accuracy includes annotation consistency, annotation accuracy rate, and annotation error types. Reasonableness of numerical range includes the reasonableness of numerical statistical distribution, outlier cases, reasonableness of range boundaries, and reasonableness of physical meaning. Correctness of category labels includes the correctness of label enumeration (i.e., whether the labels are within the predetermined enumeration range), the correctness of label semantics, the reasonableness of label distribution, and the correctness of label hierarchical structure. Timestamp accuracy includes the correctness of time format, the reasonableness of time logic, time zone consistency, and time synchronization accuracy. Evaluating the accuracy of a dataset ensures the correctness and reliability of the data content; this is a core indicator of data quality.

[0070] Consistency characteristics include internal consistency, format consistency, and multimodal consistency. Internal consistency includes consistency of data relationships, constraints, business rules, and statistical characteristics. Format consistency includes consistency of data types, encoding formats, date and time formats, and data precision. Multimodal consistency includes semantic consistency and spatiotemporal consistency. Consistency assessment ensures the consistency of datasets both internally and externally, improving data reliability and usability.

[0071] Timeliness characteristics include data freshness, version timeliness, and environmental adaptability. Data freshness includes the distribution of collection time, timeliness, expired data, and freshness thresholds. Version timeliness includes version update frequency, version lifecycle, version compatibility, and the rationality of version iterations. Environmental adaptability includes application scenario matching, adaptability to the technical environment, and the impact of changes in the business environment. Timeliness assessment ensures data freshness and timeliness, guaranteeing the data's validity for current applications.

[0072] Accessibility characteristics primarily include the accessibility of the data organization structure, which encompasses the clarity of the directory structure, file accessibility, the rationality of the classification system, and the optimization of the search path. Accessibility assessment ensures that data can be effectively accessed and used, thereby improving data usability.

[0073] Usability characteristics include application adaptability, model training performance, and generalization ability. Application adaptability includes requirement matching (i.e., the degree to which the dataset matches the application requirements), functional coverage, scenario adaptability, and degree of customization. Model training performance includes training convergence, performance metrics, training efficiency, and training resource consumption. Generalization ability includes cross-domain generalization ability, cross-task generalization ability, robustness, and quantitative metrics of generalization ability. Usability evaluation is the ultimate goal of dataset quality evaluation, ensuring that the data can effectively support the target application.

[0074] The above features will be further explained in step S3, along with the corresponding detection and evaluation methods.

[0075] Step S3: Use the pre-trained multi-dimensional quality assessment model to comprehensively evaluate the multi-dimensional quality feature data to obtain the quality level information of the corresponding robot dataset.

[0076] Figure 5 This is a flowchart of step S3 in this embodiment.

[0077] like Figure 5 As shown, step S3 specifically includes the following sub-steps:

[0078] Step S3-1: Input the multi-dimensional quality feature data into the pre-trained multi-dimensional quality assessment model. The model outputs preliminary assessment results, which include the scores of each dimension and the detected high-risk items.

[0079] In this step, the multi-dimensional quality assessment model includes a feature extraction module, a deep learning detection module, and a comprehensive scoring module.

[0080] The feature extraction module is used to extract basic features and statistical information of various types of data from the preprocessed dataset, and input the extracted information into the deep learning detection module.

[0081] The deep learning detection module includes sub-modules for multiple format verification, pornography detection, sensitive word recognition, and anomaly detection.

[0082] The format verification submodule is used to verify the file format or data format specifications of files of the corresponding data types.

[0083] The pornography detection submodule is a pornography detection model based on convolutional neural networks (CNN), used to detect image data and determine whether the image contains pornography.

[0084] The sensitive word recognition submodule is a sensitive word detection model based on OCR technology, used to perform OCR recognition on text and image data and detect whether they contain sensitive words.

[0085] The anomaly detection submodule is an outlier detection model used to detect outliers in sensor data.

[0086] After each of the above sub-modules performs its corresponding test, the test results are output to the comprehensive scoring module.

[0087] The comprehensive scoring module includes a feature normalization layer, a weighted fusion layer, a quality level classification layer, and a confidence output layer. These layers are used to normalize various basic features, perform weighted fusion of various basic features and their detection results, and obtain the quality level information of the robot dataset, including the confidence level of each quality level, through a classifier or regressor based on the fusion results.

[0088] The multi-dimensional quality assessment model performs the following specific detection and evaluation on the above seven quality characteristic data.

[0089] I. Normative Assessment

[0090] (i) Document or data format standardization

[0091] File format check: Verify whether image files (such as JPEG, PNG), audio files (such as WAV, MP3), video files (such as MP4, AVI), etc. conform to standard format specifications;

[0092] Encoding format verification: Check whether the encoding format (UTF-8, GBK, etc.) of the text data is consistent;

[0093] Metadata format: Check whether the metadata format of various types of data is standardized.

[0094] (II) Naming Standards

[0095] File naming rules: Verify that the file name follows a predefined naming pattern, such as whether it follows the format "timestamp_category_serial number";

[0096] Field naming consistency: Check whether database fields, JSON key names, etc. follow camelCase or underscore naming conventions;

[0097] Directory structure naming: Verify that the naming of folders and directories conforms to hierarchical naming conventions;

[0098] Identifier specifications: Check the naming format of various identifiers (ID, number, etc.).

[0099] (III) Structural Norms

[0100] Dataset hierarchy: Check whether the directory structure of the training set, validation set, and test set is reasonable;

[0101] Modal data classification: Validate the classification and storage structure of different modal data (images, text, audio, etc.).

[0102] (iv) Standardization of labeling

[0103] Annotation format standards: Check whether the annotation files (such as XML, JSON, COCO formats) conform to the corresponding standards;

[0104] Completeness of annotation content: Verify that the annotation information contains all necessary fields;

[0105] Annotation hierarchy: Check the structural regularity of multi-level annotations (such as bounding boxes, categories, and attributes in object detection);

[0106] Annotation quality control: Verify whether the annotation process follows quality control standards.

[0107] (V) Content Standardization

[0108] Content security: Check images or videos for pornography, violence, or other inappropriate content;

[0109] Text security: Check if the text still contains sensitive words.

[0110] In this embodiment, for file or data format compliance checks, the system has built-in multiple format verification modules to automatically detect file formats such as images (JPEG / PNG), audio (WAV / MP3), and video (MP4 / AVI). For text data, the system detects the encoding format (such as UTF-8) and converts it uniformly. For naming compliance, regular expressions and scripts are used to batch check the naming compliance of files, fields, and directories, and automatically output non-compliant items.

[0111] To determine the probability of images involving pornography, the system integrates a pornography detection model based on deep convolutional neural networks (such as ResNet and EfficientNet), which can automatically identify pornographic images. The model takes an image as input and outputs a pornography probability score. The specific process is as follows: All images in the dataset are preprocessed in batches, including scaling and normalization; the preprocessed images are input into the trained pornography detection model to obtain a pornography probability score p for each image; based on a set threshold (e.g., p > 0.7), images with a pornography probability score p higher than the threshold are automatically marked as suspected pornography images; for suspected pornography images, the system automatically generates a manual review task to ensure that false alarms are controllable.

[0112] For sensitive word tagging, the system integrates OCR and sensitive word detection models. OCR models, such as PaddleOCR and EasyOCR, can extract text content from images and video frames. Sensitive word detection models, such as BERT or LSTM-based text classifiers, are used to identify whether the extracted text contains sensitive words related to politics, terrorism, or violence. The specific process is as follows: All images in the dataset are batch-processed with OCR to extract all identifiable text; the identified text is then segmented and denoised before being input into the sensitive word detection model; if the model detects a sensitive word, it automatically marks the file path and content fragments of files containing the sensitive word and outputs a corresponding list for manual review.

[0113] II. Integrity Assessment

[0114] (I) Data Integrity

[0115] Data missing rate statistics: Calculate the missing proportion of each type of data. The formula is: Missing rate = Missing data volume / Total data volume × 100%;

[0116] Data missing pattern analysis: Identify patterns of missing data, such as random missing data, systematic missing data, etc.

[0117] Key data missing status: Focus on checking for missing key fields and core data;

[0118] Missing data impact assessment: Assess the extent to which missing data affects the overall quality of the dataset.

[0119] (ii) Variable completeness

[0120] Required variable check: Verify that all required variables and fields exist completely;

[0121] Variable type integrity: Check the integrity of variables of each type (e.g., numeric, categorical, time, etc.);

[0122] Variable relationship integrity: Verify whether the relationships between variables are complete;

[0123] Variable range integrity: Check whether the range of values ​​of the variable completely covers the expected range.

[0124] In this embodiment, for data integrity, the system calculates the missing rate of each type of data using the formula: Missing rate = Missing data amount / Total data amount × 100%, automatically generates a data missing report, and focuses on checking predetermined key fields (such as action sequences and main sensor data).

[0125] For variable integrity, such as time series integrity, the system uses a sliding window to detect breakpoints and abnormal intervals in timestamp data, and automatically identifies problems such as inconsistent sampling frequencies and data loss. For multimodal alignment, the system automatically verifies the one-to-one correspondence between multimodal data of the same task (e.g., image data and IMU data of the same task) based on timestamps and task IDs. If alignment failure is found, a list of abnormal pairs is output.

[0126] III. Accuracy Assessment

[0127] (I) Labeling accuracy

[0128] Sampling verification method: Stratified sampling method is used to select samples for manual verification;

[0129] Annotation consistency check: Verify the consistency of annotations by comparing annotations from multiple people;

[0130] Accuracy calculation: The accuracy of annotation is calculated using the formula: Accuracy = Number of correct annotations / Total number of annotations × 100%;

[0131] Error type analysis: Analyze the types and distribution patterns of labeled errors.

[0132] (II) Reasonableness of numerical range

[0133] Statistical distribution check: Analyze whether the statistical distribution of the numerical data is reasonable;

[0134] Outlier detection: Statistical methods (such as the 3σ principle, IQR method, etc.) are used to detect outliers;

[0135] Range boundary verification: Checks whether the value is within a predefined reasonable range;

[0136] Physical meaning verification: Verify whether the numerical value conforms to physical or business logic.

[0137] (III) Correctness of Category Labels

[0138] Tag enumeration check: Verifies whether the category tag is within the predefined enumeration range;

[0139] Tag semantic correctness: Check whether the semantic meaning of the tag is correct;

[0140] Tag distribution rationality: Analyze whether the tag distribution meets expectations;

[0141] Tag hierarchy: Verify the correctness of the hierarchical relationship between multi-level tags.

[0142] (iv) Timestamp accuracy

[0143] Time format verification: Check if the timestamp format conforms to the standard (such as ISO 8601);

[0144] Time logic validity: Verify the logical validity of the timestamp (e.g., the creation time should be earlier than the modification time);

[0145] Time zone consistency: Check whether the time zone information of the timestamps is consistent;

[0146] Time synchronization accuracy: Verify the accuracy of time synchronization between multiple devices.

[0147] In this embodiment, to ensure label accuracy, the system employs a stratified sampling algorithm (such as StratifiedSampling) to randomly select samples (e.g., 5%) and output them. The output samples are then manually reviewed. For annotation files (such as COCOJSON and VOC XML), the system can automatically compare the consistency between the annotation content and the image content (e.g., whether the target bounding box falls within the target area).

[0148] For numerical reasonableness and anomaly detection, the system utilizes statistical analysis and deep anomaly detection models (such as AutoEncoder) to automatically detect outliers in sensor data. Outlier detection employs the 3σ principle: outlier threshold = mean ± 3 × standard deviation. The system automatically marks values ​​exceeding the outlier threshold.

[0149] Regarding the correctness of tag categories, the system automatically verifies whether all tags are within the predefined tag set and performs statistical analysis on the tag distribution, automatically issuing warnings when extreme imbalances are detected.

[0150] IV. Consistency Assessment

[0151] (a) Internal consistency

[0152] Data relationship consistency: Check whether the logical relationships between data items are consistent;

[0153] Constraint consistency: Verify whether the data meets the predefined constraints;

[0154] Business rule consistency: Check whether the data conforms to the business logic rules;

[0155] Statistical feature consistency: Verify whether the statistical features of different parts of the dataset are consistent.

[0156] (ii) Format consistency

[0157] Data type consistency: Check whether the data type of the same field is consistent in different records;

[0158] Encoding format consistency: Verify whether the encoding format of the text data is consistent;

[0159] Date and time format: Check if the format of the date and time fields is consistent;

[0160] Numerical precision consistency: Verify whether the precision and format of numerical fields are consistent.

[0161] (III) Multimodal Consistency

[0162] Semantic consistency: Verify whether the semantics of data expressed in different modalities are consistent;

[0163] Spatiotemporal consistency: Check the consistency of multimodal data in time and space dimensions.

[0164] In this embodiment, for internal consistency detection, the system automatically checks the logical relationships between data (e.g., whether the action sequence and image annotation under the same task correspond), and uses a rule engine and NLP to assist in detecting the consistency between the descriptive text and the actual data.

[0165] Regarding format consistency, the system automatically checks whether the type, length, and precision of all data fields are consistent, and compares them with the corresponding industry labels (such as ISO, T / SAIAS), and outputs a compatibility report.

[0166] V. Timeliness Assessment

[0167] (a) Data freshness

[0168] Data collection time analysis: Distribution of statistical data collection time;

[0169] Timeliness calculation: The timeliness index of the data is calculated by formula: Timeliness = 1 / (1+α×time interval), where α is the decay factor, which is used to control and adjust the influence of time interval on freshness.

[0170] Expired data identification: Identify data that has exceeded its expiration date based on the collection time and a predetermined freshness threshold;

[0171] Freshness threshold setting: Set the data freshness threshold according to application requirements.

[0172] (ii) Version Timeliness

[0173] Version update frequency: The frequency of version updates for statistical datasets;

[0174] Version lifecycle: Analyze the lifecycle of dataset versions;

[0175] Version compatibility: Check compatibility between different versions;

[0176] Version iteration rationality: Evaluate the rationality and necessity of version iteration.

[0177] (III) Environmental adaptability

[0178] Application scenario matching degree: Evaluates the degree to which the data matches the current application scenario;

[0179] Technological adaptability: Examine the data's adaptability to the current technological environment;

[0180] Business environment changes: Analyze the impact of changes in the business environment on data validity;

[0181] Adaptive adjustment recommendations: Propose suggestions for improving data adaptability.

[0182] In this embodiment, for data freshness, the system statistically analyzes the data collection time distribution and calculates a timeliness index. Expired data (such as data collected before a set threshold) is automatically marked.

[0183] Regarding version timeliness, the system automatically detects version information in the dataset, analyzes update frequency and lifecycle, and outputs version compatibility suggestions.

[0184] VI. Accessibility Assessment

[0185] Directory structure clarity: Assess the clarity of the data directory structure;

[0186] File accessibility: Detects whether a file can be opened;

[0187] Classification system rationality: Evaluate the rationality of the data classification system;

[0188] Search path optimization level: Analyze the degree of optimization of the data retrieval path.

[0189] In this embodiment, the system automatically analyzes the directory and index structure to assess the clarity of the directory structure and the rationality of the classification system. It then uses a graph database to visualize the data organization relationships and outputs a structural clarity score.

[0190] Regarding file accessibility, the system automatically detects data access permission configurations, tests API interface stability and document integrity, and simulates error handling processes.

[0191] To assess the optimization of the retrieval path, the system uses automated scripts to simulate batch queries, calculate query response time and retrieval accuracy, and output a performance report.

[0192] VII. Usability Assessment

[0193] (I) Application adaptability

[0194] Requirements matching analysis: Analyze the degree of matching between the dataset and application requirements;

[0195] Functional coverage: Calculates the coverage of the dataset to the application's functionalities;

[0196] Scenario adaptability: Evaluate the adaptability of the dataset to different application scenarios;

[0197] Customization level: Analyze the degree of customization and flexibility of the dataset.

[0198] (II) Model Training Results

[0199] Training convergence: Evaluate the convergence of the model trained using this dataset;

[0200] Performance metrics evaluation: Performance metrics of the test model on the validation set;

[0201] Training efficiency: Analyzing the impact of datasets on model training efficiency;

[0202] Resource consumption: Evaluate the resource consumption during the training process.

[0203] (iii) Generalization ability

[0204] Cross-domain generalization ability: Test the model's ability to generalize to data from different domains;

[0205] Cross-task generalization ability: Evaluate the model's generalization performance on relevant tasks;

[0206] Robustness testing: Conduct robustness testing on adversarial examples and noisy data;

[0207] Quantification of generalization ability: Establish quantitative evaluation indicators for generalization ability.

[0208] In this embodiment, for application adaptability, the system automatically compares the functional coverage of the dataset with the application scenario and requirements, and outputs an adaptability score and suggestions.

[0209] In addition, for each of the above dimensions, the system will calculate the score for that dimension based on the predetermined total score and the status of each detection item.

[0210] Step S3-2: Based on the detected high-risk items, generate corresponding manual review tasks, which are then manually reviewed by reviewers.

[0211] In this embodiment, for high-risk items automatically detected and marked, such as pornographic images, sensitive words, serious omissions, and abnormal annotations, the system automatically generates manual review tasks. The platform samples from these tasks and provides them to reviewers for manual review. Simultaneously, the platform automatically adjusts the sampling ratio based on the data volume to ensure both review efficiency and coverage.

[0212] Reviewers can directly view high-risk items (abnormal items), confirm or correct them in batches through the platform interface, and provide feedback on the automated detection results. The system automatically records the consistency indicators of manual review.

[0213] Step S3-3: Based on the preliminary evaluation results after manual review and the predetermined weights of each dimension, a detailed quality evaluation report is generated. The comprehensive quality score of the robot dataset is obtained by weighted fusion of the scores of each dimension.

[0214] In this step, the overall quality score of the dataset is automatically calculated according to the following formula:

[0215]

[0216] Where w i Let Q be the weight of the i-th dimension. i The score is for the i-th dimension.

[0217] For example, for home service robots, the weights of the seven dimensions can be set as follows: standardization 15%, completeness 20%, accuracy 25%, consistency 15%, timeliness 10%, accessibility 10%, and usability 5%.

[0218] The generated detailed quality assessment report includes an overall quality score, scores for each dimension, and key issues.

[0219] Figure 6 This is a flowchart of the training process of the multi-dimensional quality assessment model in this embodiment.

[0220] like Figure 6 As shown, the above multi-dimensional quality assessment model can be trained through the following steps:

[0221] Step T1: Construct the initial multi-dimensional quality assessment model structure.

[0222] Step T2 involves using a robot dataset containing multiple labeled quality levels as a training set, which is then sequentially input into the initial multi-dimensional quality assessment model. The initial multi-dimensional quality assessment model performs feature extraction and deep learning detection based on the input data, and outputs quality level information.

[0223] Step T3: Calculate the loss error between the quality level information output by the model and the labeled (real) quality level information, and update the model parameters using the backpropagation algorithm.

[0224] Step T4: Repeat steps T2 to T3 until the predetermined training completion conditions are met, thereby obtaining the multi-dimensional quality assessment model after training is completed.

[0225] The role and effect of the embodiments

[0226] The humanoid robot dataset quality evaluation method and system based on multi-dimensional assessment provided in this embodiment can systematically and comprehensively evaluate multimodal humanoid robot datasets by constructing a dataset quality evaluation system covering seven dimensions: standardization, completeness, accuracy, consistency, timeliness, accessibility, and usability. The method of this invention not only establishes detailed quality inspection standards for multiple data modalities such as vision, hearing, kinematics, and dynamics, but also combines automated inspection with manual review, greatly improving the efficiency and accuracy of the evaluation. Through automated tools, it is possible to quickly detect basic standardization aspects such as data format, naming conventions, and annotation content, while simultaneously performing comprehensive automated reviews of data completeness, variable coverage, time series continuity, and multimodal alignment, significantly reducing manual workload.

[0227] In practical applications, this method and system can effectively identify and locate various quality issues in datasets. For example, during the evaluation of a home service robot dataset, it can promptly identify problems such as inconsistent image formats, missing sensor data, and non-standard file naming, and output detailed improvement suggestions. By introducing deep learning algorithms, it can automatically detect pornographic images and OCR-sensitive words in visual data, improving the security and compliance of the data content. For multimodal data, it can also automatically verify the temporal, spatial, and semantic alignment between different modalities, ensuring high consistency and high availability of the dataset.

[0228] The quality assessment method and system proposed in this invention have been validated in multiple real-world projects. Experimental results show that datasets evaluated and optimized using the methods and systems of this invention exhibit significant improvements in robot model training and downstream tasks, with substantial enhancements in task accuracy and generalization ability. The automatically generated quality assessment reports provide a scientific basis for continuous improvement and version updates of the datasets, helping data providers and users quickly identify and resolve data quality bottlenecks, thereby enhancing the overall value of data assets.

[0229] Furthermore, the method and system of this invention possess excellent adaptability and scalability, allowing for flexible adjustment of evaluation weights and detection rules across various dimensions according to different application scenarios. This supports quality evaluation of multiple robot datasets, including those for home services, industrial operations, and medical assistance. Through continuous monitoring and trend analysis, the system can promptly detect changes in data quality, promoting dynamic optimization and long-term healthy development of the dataset, thus providing a solid data foundation and technical support for the intelligent application of humanoid robots.

[0230] The above embodiments are merely illustrative of specific implementations of the present invention, and the present invention is not limited to the scope of the description of the above embodiments. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are only for illustrating the principles of the present invention. Various changes and modifications can be made to the present invention without departing from the spirit and scope thereof, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for quality evaluation of humanoid robot dataset based on multi-dimension evaluation, for evaluating a robot dataset containing multi-modal data to determine a quality level of the dataset, characterized in that, The method comprises the following steps: Step S1, respectively pre-process and classify each type of raw data in the robot data set, at least including data structure analysis, data type identification and classification, file format and naming standard detection, so as to obtain a pre-processed data set; Step S2, based on the pre-processed data set, a multi-dimensional quality feature data is constructed, the multi-dimensional quality feature data includes standardization feature, integrity feature, accuracy feature, consistency feature, timeliness feature, accessibility feature and usability feature; Step S3, based on the multi-dimensional feature data, a pre-trained multi-dimensional quality evaluation model is used to comprehensively evaluate the pre-processed data set, and combined with artificial review, a quality evaluation report of the robot data set is obtained, which contains quality level information, Wherein, the multi-dimensional quality evaluation model includes a feature extraction module, a deep learning detection module and a comprehensive scoring module, The feature extraction module is used to extract the basic features and statistical information of various types of data from the pre-processed data set and input them into the deep learning detection module, The deep learning detection module respectively detects various types of data based on the identified various data types and the basic features, and outputs the corresponding detection results to the comprehensive scoring module, The comprehensive scoring module weights and fuses various basic features and detection results, and outputs the quality evaluation report of the robot data set.

2. The method of claim 1, wherein the method is based on a multi-dimensional evaluation of a humanoid robot dataset quality assessment, It is characterized in that: Wherein, step S1 includes the following sub-steps: Step S1-1, structure and format analysis is performed on all data files in the robot data set, and data files meeting the requirements are selected as qualified data files; Step S1-2, data type identification and file naming standard detection are performed on the qualified data files, and based on the detection results, the qualified data files are classified into corresponding data types; Step S1-3, based on the classification results of step S1-2, the qualified data files of each data type are respectively detected for file format and naming standard; Step S1-4, invalid information in the classification results is excluded, and the qualified data files of each data type are normalized to form the pre-processed data set.

3. The method of claim 2, wherein the method is based on a multi-dimensional evaluation of a humanoid robot dataset quality assessment, It is characterized in that: Wherein, step S1-2 further includes: According to the pre-set application scenario classification standard, the robot data set is classified into corresponding application scenario types; According to the pre-set task classification standard, the specific task type corresponding to the robot data set is identified; Extract the technical tag information contained in the robot data set, and establish the mapping relationship between skills and data based on the extracted technical tag information.

4. The robot data set quality evaluation method based on multi-dimensional evaluation according to claim 2, characterized in that: wherein, In step S1-3, the file format and naming standard detection includes file existence check, mandatory field coverage check, data type compliance check and format consistency check.

5. The humanoid robot dataset quality evaluation method based on multi-dimensional evaluation according to claim 1, wherein: wherein In step S2, the normative features include file format or data format normative, image pornography probability, sensitive word marking, The integrity features include data integrity features, variable integrity features, The accuracy features include annotation accuracy, numerical range rationality, category label correctness, timestamp accuracy, The consistency features include internal consistency, format consistency, multi-modal consistency, The timeliness features include data freshness, dataset version timeliness, dataset environment adaptability, The accessibility features include data organization structure accessibility, The usability features include application adaptability, model training effect, model generalization ability.

6. The humanoid robot dataset quality evaluation method based on multi-dimensional evaluation according to claim 5, wherein: wherein, In the normative features, the file format or data format normative includes data format normative, naming normative, structure normative, annotation normative, and content normative, In the integrity features, the data integrity features include data missing rate, data missing mode, key data missing situation, and missing impact degree; the variable integrity features include necessary variable integrity, variable type integrity, variable relationship integrity, and variable value domain integrity, In the accuracy features, the annotation accuracy includes annotation consistency, annotation accuracy rate, and annotation error type; the numerical range rationality includes numerical statistical distribution rationality, abnormal value situation, range boundary rationality, and physical meaning rationality; the category label correctness includes label enumeration correctness, label semantic correctness, label distribution rationality, and label hierarchy structure correctness; the timestamp accuracy includes time format correctness, time logic rationality, time zone consistency, and time synchronization precision, In the consistency features, the internal consistency includes data relationship consistency, constraint condition consistency, business rule consistency, and statistical feature consistency; the format consistency includes data type consistency, coding format consistency, time and date format consistency, and data precision consistency; the multi-modal consistency includes semantic consistency and spatio-temporal consistency, In the timeliness features, the data freshness includes collection time distribution, timeliness, expired data situation, and freshness threshold; the dataset version timeliness includes version update frequency, version life cycle, version compatibility, and version iteration rationality; the dataset environment adaptability includes application scenario matching degree, technical environment adaptability, and business environment change impact situation, In the accessibility features, the data organization structure accessibility includes directory structure clarity, file accessibility, classification system rationality, and retrieval path optimization degree, In the usability features, the application adaptability, model training effect, and model generalization ability. The application adaptability includes demand matching degree, function coverage, scene adaptability, and customization degree; the model training effect includes training convergence, model performance index, model training efficiency, and model training resource consumption; and the generalization capability includes cross-domain generalization capability, cross-task generalization capability, robustness, and generalization capability quantitative index.

7. The humanoid robot data set quality evaluation method based on multi-dimensional evaluation according to claim 5, characterized in that: wherein The deep learning detection module comprises a plurality of format checking sub-modules, a yellow-related detection sub-module, a sensitive word recognition sub-module, and an abnormality detection sub-module. The format checking sub-module is configured to check the file format or data format specification of a file of a corresponding data type. The yellow-related detection sub-module is a yellow-related detection model based on a convolutional neural network and is configured to perform yellow-related image detection on image data. The sensitive word recognition sub-module is a sensitive word detection model based on OCR technology and is configured to perform OCR sensitive word detection on text data and image data. The abnormality detection sub-module is configured to perform abnormal value detection on sensor data.

8. The method of claim 1, wherein the method is based on a multi-dimensional evaluation of a humanoid robot dataset quality assessment, The method comprises the following steps: Step S3 comprises the following sub-steps: Step S3-1: inputting the multi-dimensional quality feature data into a pre-trained multi-dimensional quality evaluation model to obtain a preliminary evaluation result, wherein the preliminary evaluation result comprises scores of each dimension and high-risk items detected; Step S3-2: generating a corresponding manual review task based on the high-risk items, and manually reviewing the high-risk items by an examiner; Step S3-3: generating the quality evaluation report according to the preliminary evaluation result after manual review and predetermined weights of each dimension, wherein a comprehensive quality score of the robot data set is obtained by weighting and fusing scores of each dimension.

9. The method of claim 1, wherein the method is based on a multi-dimensional evaluation of a humanoid robot dataset quality assessment, The training of the multi-dimensional quality evaluation model comprises the following steps: Step T1: constructing an initial multi-dimensional quality evaluation model structure; Step T2: inputting a robot data set comprising a plurality of groups of labeled quality grades as a training set into the initial multi-dimensional quality evaluation model in sequence, performing feature extraction and deep learning detection based on the input data, and outputting quality grade information; Step T3: calculating a loss error between the output quality grade information and the labeled quality grade information, and updating model parameters by using a back propagation algorithm; Step T4: repeating steps T2 to T3 until a predetermined training completion condition is met, thereby obtaining the trained multi-dimensional quality evaluation model. The method comprises the following steps:

10. A multi-dimensional evaluation based humanoid robot dataset quality evaluation system for evaluating a robot dataset containing multi-modal data to determine a dataset quality grade, the system comprising: A preprocessing and classification unit is configured to perform corresponding preprocessing and classification on each type of raw data in the robot data set, at least including data structure analysis, data type identification and classification, and file format and naming specification detection, thereby obtaining a preprocessed data set; A feature data construction unit is configured to construct multi-dimensional quality feature data based on the preprocessed data set, wherein the multi-dimensional quality feature data comprises specification features, integrity features, accuracy features, consistency features, timeliness features, accessibility features, and usability features. ​ The comprehensive evaluation unit is configured to perform comprehensive evaluation on the preprocessed data set based on the multi-dimensional feature data by using a pre-trained multi-dimensional quality evaluation model, and obtain a quality evaluation report of the robot data set by combining manual review, wherein the quality evaluation report includes quality grade information. The multi-dimensional quality evaluation model includes a feature extraction module, a deep learning detection module, and a comprehensive scoring module. The feature extraction module is configured to extract basic features and statistical information of various types of data from the preprocessed data set and input the basic features and statistical information into the deep learning detection module. The deep learning detection module is configured to perform corresponding detection on data of various types of data based on the identified various types of data and the basic features of the various types of data, and output corresponding detection results to the comprehensive scoring module. The comprehensive scoring module is configured to perform weighted fusion on the basic features and the detection results, and output the quality evaluation report of the robot data set.

Images