An industrial high-quality data set construction and application method for the automobile field

Abstract

The application discloses a kind of industrial high-quality data set construction and application method for automobile field.The data set construction method of the application is: the application scene of automobile field is selected using scene driving mode;According to the link involved in application scene, collect data and carry out preliminary cleaning, retain the complete industrial data of effective information and preprocess it, obtain the complete industrial data set of unified format;High-value seed scene is extracted from complete industrial data set, for data synthesis;Synthetic data is converted into complete industrial data of unified format and stored in complete industrial data set;Complete industrial data set is annotated with industrial data scene, to obtain annotated sample set and carry out privacy protection, to obtain the annotated sample set after desensitization;After compliance and integrity verification of the annotated sample set after desensitization, form general knowledge data set and expert knowledge data set;General knowledge data set and expert knowledge data set data set are respectively evaluated, packaged, to obtain industrial high-quality data set.

Description

Technical Field

[0001] This invention belongs to the field of automotive industry data processing, and relates to a method for constructing and applying high-quality industrial datasets for the automotive industry. Background Technology

[0002] Computing power, algorithms, and data are the three core elements of artificial intelligence development. If algorithms are the controller (brain) of large models and computing power is the engine (heart) of large models, then data is the fuel (blood) that supports their operation. Currently, the industry consensus in artificial intelligence is shifting from "model-centric" to "data-centric." With open-source algorithms (algorithm architectures like Transformer and its variants are gradually converging) and affordable computing power (computing power can be built up through capital investment), data has upgraded from the "fuel" of artificial intelligence to a "strategic resource" that determines the outcome of competition. Data is a key factor in determining the upper limit of model performance. From the initial collection of open corpora to refined annotation and cleaning, to formatted dialogue template data, and finally to generative alignment and feedback reinforcement, data has run through the entire lifecycle of model building. The scale, source, structure, and intrinsic quality of data have become the core competitiveness of large models.

[0003] Industrial data serves as the carrier of industrial knowledge accumulation and reuse, and is the source of industrial intelligent innovation and development. In the modern industrial system, industrial data permeates the entire value chain, from R&D and design to manufacturing, operations management, and after-sales service, carrying the most core knowledge assets in the industrial field. Industrial data possesses inherent characteristics that significantly distinguish it from data in other fields: multi-source heterogeneity, strong real-time performance, and high correlation. A typical medium-sized smart factory has as many as 1200 to 1500 connection points for various elements such as people, machines, materials, methods, and environment, generating massive amounts of multi-dimensional data every moment. This data includes time-series data from automation systems such as sensors, PLCs, and SCADA, structured data from management systems such as MES and ERP, and multimodal unstructured data such as product design drawings, process documents, video surveillance, and audio signals, forming the fundamental elements of industrial digital transformation. In the practice of promoting industrial intelligent applications, the industry generally agrees on the basic principle of "70% data, 30% model," therefore, industrial data has become the "key" to industrial digital transformation.

[0004] According to the "Guidelines for the Construction of High-Quality Datasets" released in 2025, a high-quality dataset refers to "a collection of data that, after collection and processing, can be directly used to develop and train artificial intelligence models, effectively improving model performance." Its high quality is mainly reflected in multiple dimensions such as scale, completeness, accuracy, consistency, diversity, security, and timeliness. Compared with traditional datasets, high-quality datasets emphasize the authenticity, completeness, and usability of data, and place greater emphasis on the value evolution capability of data in complex business scenarios. Industrial high-quality datasets are vertical domain-specific datasets formed by deeply integrating the special attributes of the industrial field on the basis of general standards for high-quality datasets. In 2026, the National Data Administration issued the "Implementation Plan for Promoting the Construction of High-Quality Industry Datasets," which clearly proposed to deploy six special actions around key aspects such as the supply, circulation, and application of high-quality industry datasets: strengthening the foundation and expanding capacity, tackling annotation challenges, improving quality and efficiency, empowering applications, providing management services, and releasing value, forming a "data flywheel" of "scenario-driven data, data-driven models, models empowering applications, and applications creating value." The professionalism, scenario-specificity, and complexity of industrial high-quality datasets determine that their data value far exceeds that of other industry datasets.

[0005] With breakthroughs in artificial intelligence technologies, exemplified by generative pre-trained large language models (LLM), data, as a key production factor, has seen its quality become a core bottleneck restricting the performance of AI models. The automotive industry, a pillar of the national economy, is undergoing a critical period of transformation and upgrading, deeply integrating electrification, connectivity, intelligence, and sharing. Automotive industrial data is characterized by its wide range of sources, diverse types, massive volume, and high level of specialization, covering the entire chain from R&D and design, manufacturing, operation and management, customer service, and industrial collaboration. However, current automotive data suffers from problems such as strong heterogeneity from multiple sources, inconsistent collection standards, uneven data quality, high barriers to professional annotation, and fragmented dataset management, severely hindering the deep application of AI technology in the automotive industry. Therefore, there is an urgent need to construct a systematic and engineered method for building high-quality automotive industrial datasets to provide a high-quality data foundation for large-scale model training and intelligent applications in the industry.

[0006] The automotive industry is one of the most data-intensive and complex sectors in the industrial field. From simulation calculations, digital prototypes, and wind tunnel testing in the R&D and design phase, to the four major processes of stamping, welding, painting, and final assembly in the production and manufacturing phase, to supply chain collaboration, production scheduling, and quality traceability in the operation and management phase, and to vehicle networking operation, user behavior analysis, and after-sales maintenance in the customer service phase, the amount of data generated throughout the entire lifecycle of a vehicle is enormous. According to statistics, a single intelligent connected vehicle can generate tens of gigabytes of data every day, covering multi-dimensional information such as vehicle operating status, driving behavior, environmental perception, and human-machine interaction.

[0007] However, the construction of datasets in the current automotive sector faces multiple challenges. At the data acquisition level, some enterprises have low data acquisition equipment access rates and insufficient acquisition accuracy. Older equipment and equipment used in special environments lack sensors, resulting in a large amount of dormant data. Of the 17 billion smart devices globally, 80% of industrial data remains undeveloped and unutilized. At the data quality level, industrial data originates from diverse and heterogeneous systems such as sensors, equipment logs, and control systems, exhibiting problems such as high noise levels, high missing data rates, inconsistent formats, and poor correlation, making it difficult to support efficient governance and modeling. At the data annotation level, industrial data annotation involves non-standardized professional semantic systems and tacit knowledge, requiring the involvement of domain experts, resulting in high annotation costs, low efficiency, and difficulty in ensuring consistency. At the data circulation level, enterprises have low willingness to share data, the problem of data silos is prominent, and there is a lack of unified data standards and circulation mechanisms. At the data application level, dataset construction is disconnected from business scenarios.

[0008] The inventors' field research revealed that, due to the novelty of high-quality industrial datasets, enterprises generally face confusion regarding "what to build and how to build it." Regarding "what to build," they haven't shifted from a "data-driven" to a "scenario-driven" approach; data is merely stored in a different location if it's not integrated into a specific scenario. Regarding "how to build," their planning of the construction path is insufficiently systematic. Many enterprises, despite possessing massive datasets, haven't deeply integrated much of this data with specific business scenarios, exhibiting a "data-first, scenario-lagging" characteristic. Even those that have built preliminary datasets still face the dilemma of being "unusable" and "unsuitable for use," failing to form a closed loop of "data supporting models, models empowering scenarios," making it difficult to translate data value into actual effectiveness. They fail to achieve "out-of-the-box usability," resulting in a disconnect between dataset construction and application scenarios. Furthermore, the "data → model → application → iteration" chain is incomplete, lacking tracking and evaluation of model application effects and scenario implementation, and unable to provide feedback for dataset supplementation, improvement, and optimization—a typical case of "use without depth."

[0009] According to Chinese invention patent applications CN122112625A ("A Method and System for Constructing High-Quality Datasets for Model Training"), CN120671136A ("A Method and Device for Constructing High-Quality Datasets"), CN120277328A ("A Method and System for Constructing High-Quality Datasets for Large Models in the Urban Rail Transit Field"), and CN119647595A ("A Method and System for Constructing High-Quality Datasets for Large Models in Vertical Domains"), the current public domain only offers general processing workflows for general visual datasets and general time-series datasets. There is currently no complete and clear method for constructing and applying high-quality industrial datasets specifically for the automotive industry, covering the entire data processing process. Without a unified technical path within the industry, enterprises lack clear methods for dataset construction, model training, and industrial intelligent agent applications. They can only blindly copy general internet data processing workflows. These general solutions have low adaptability to the complex operating conditions, long-tail distribution of defects, and industrial black-box nature of automotive production lines, making it easy for intelligent agent applications to experience AI "illusions." Summary of the Invention

[0010] In view of the problems existing in the prior art, the purpose of this invention is to provide a method for constructing and applying high-quality industrial datasets for the automotive field.

[0011] To achieve the above objectives, the present invention adopts the following technical solution: A method for constructing high-quality industrial datasets for the automotive industry, such as Figure 1 As shown, the steps include: A scenario-driven approach was adopted to select application scenarios in the automotive field. Collect automotive industry scenario data based on the stages involved in the selected application scenarios; The automotive industry scenario data is initially cleaned to retain complete industrial data with valid information. The complete industrial data is preprocessed to obtain a complete industrial dataset in a unified format; High-value seed scenes that meet the set conditions are extracted from the complete industrial dataset by means of preset screening rules, and the target attributes of the synthetic data are defined based on the high-value seed scenes. The data is synthesized by means of a hybrid generation strategy that integrates physical constraints based on the target attributes, and the synthetic data is converted into complete industrial data in a unified format and stored in the complete industrial dataset. The complete industrial dataset is annotated with industrial data scenarios to obtain an annotated sample set; The labeled sample set is then subjected to secondary verification and cleaning. Privacy protection is applied to the labeled sample set that has undergone secondary verification and cleaning to obtain a de-identified labeled sample set. Perform compliance and integrity checks on the anonymized labeled sample set; Using labeled samples that have been verified for compliance and completeness as input, and based on high-quality dataset classification standards and industry norms, general knowledge datasets and specialized knowledge datasets are formed. The general knowledge dataset and the specialized knowledge dataset were evaluated based on data completeness, annotation accuracy, category balance, working condition coverage, and target model adaptability. The datasets that meet the evaluation requirements are packaged to obtain high-quality industrial datasets corresponding to the selected application scenarios.

[0012] Preferably, the automotive industry scenario data includes equipment data, production line data, factory data, enterprise data, and industrial chain ecosystem data.

[0013] Preferably, the method for preliminary cleaning of the automotive industry scenario data is as follows: S31. Data Asset Inventory and Cataloging: Register the automotive industry scenario data as assets, establish a data catalog list, and record the data source, collection time window, channel identifier, data scale and storage format of each batch of automotive industry scenario data to form a traceable data list. S32. Eliminate completely invalid samples: Perform an integrity pre-check on each record in the data catalog list and eliminate completely invalid samples. The completely invalid samples include, but are not limited to: empty frame data with all load values ​​being empty, fixed value records with all zero or full range generated when the acquisition channel is invalid or the sensor is offline, and records that cannot be parsed normally due to data index disorder or header information damage caused by storage abnormalities. S33. Data deduplication: Deduplicating duplicate samples stored repeatedly under the same sensor channel or the same acquisition timestamp; the duplicate sample refers to two samples with completely identical content, or two samples with adjacent timestamps and Euclidean distance or cosine similarity between feature vectors that is lower than a preset threshold. S34. Coarse-grained filtering of extreme abnormal data: The records are filtered. If the data in a record exceeds the value range corresponding to the data or the industry standard threshold, the record is deleted and candidate valid data is obtained. S35. Initial screening and labeling of missing samples: The missing rate of fields in the candidate valid data is statistically analyzed, and the candidate valid data with a missing rate exceeding the preset threshold is judged as severely missing samples and removed; for the candidate valid data with a missing rate below the preset threshold, the location and missing pattern of the missing fields are marked. S36. File integrity and readability verification: Perform file header verification, verification and comparison, and format specification checks on unstructured or semi-structured data stored in file form among the candidate valid data. Eliminate candidate valid data that fails the verification or check to obtain complete industrial data with retained valid information.

[0014] Preferably, the method for preprocessing the complete industrial data is as follows: S41. Format standardization processing: Perform unified format conversion and field mapping on the complete industrial data according to the preset automotive general data specifications to obtain general storage format data, and establish a mapping relationship table between the original fields and standard fields; S42. Refined processing of missing values: For complete industrial data marked with the location and pattern of missing fields after processing in S41, the missing value processing strategy is adaptively selected to fill in the missing values ​​according to the data type, missing rate and missing pattern of the missing fields. S43. Noise Smoothing and Signal Correction: Based on the signal type and noise characteristics, a differentiated filtering strategy is used to smooth and correct the complete industrial data processed by S42. S44, Multi-source data spatiotemporal alignment: Performs synchronization in the time dimension and coordinate registration in the spatial dimension on the complete industrial data after processing by S43; S45. Numerical Normalization and Standardization: For numerical characteristics of different dimensions and orders of magnitude in the complete industrial data after S4 processing, dimensionless transformation is performed while maintaining their physical meaning and distribution pattern. S46. Data Sharding and Sampling Equalization Preprocessing: Based on the requirements of downstream model training, the data transformed in S45 is organized and resampled to obtain a complete industrial dataset with a uniform format and a preset equalization ratio for the number of samples in each category.

[0015] Preferably, the method for performing industrial data scenario-based annotation on the complete industrial dataset is as follows: S71. Construct a tagging system that integrates expert experience: Based on the automotive industry's design specifications, maintenance manuals, fault mode libraries, and process standards, and combined with expert experience, construct a multi-dimensional hierarchical tagging system for specific business scenarios. The multi-dimensional hierarchical tagging system includes equipment-level tags, process-level tags, quality-level tags, operating condition tags, and behavioral semantic tags. Each level is further refined into four levels of labelable atomic tags according to specific application goals, and the association relationship and mutual exclusion constraint rules between tags are established. At the same time, each tag is accompanied by a tagging description document, which clarifies the tag's business definition, judgment basis, tagging boundary, and typical positive and negative examples. S72. Precise annotation based on human-machine collaboration: For tags with high requirements for complex semantic understanding and professional judgment, a human-machine collaborative annotation mode is adopted to annotate complete industrial data; S73. Large-scale automated labeling based on semi-supervised and weakly supervised learning: For label items with large data volume, relatively clear labeling rules, and formalizable expert knowledge, a combination of semi-supervised learning and weakly supervised learning is used to perform large-scale automated labeling of complete industrial data. S74. Cross-modal joint annotation of multimodal data: For complex scenarios where complete industrial data simultaneously contains multiple modalities, a cross-modal joint annotation strategy is adopted to establish a consistency and complementarity association method between labels of different modalities to annotate the complete industrial data; the cross-modal joint annotation strategy uses business semantic labels as anchors and adapts corresponding annotation tools and annotation methods to different modalities respectively; the annotation results of each modality are associated through a unified sample ID and timestamp to form cross-modal label-aligned annotation samples; S75. Cross-source information association: Vertically associate labeled samples with equipment numbers, process parameter formulas, and quality inspection results in the production and manufacturing stages to establish a full-chain label traceability chain.

[0016] Preferably, the method for performing secondary verification and cleaning on the labeled sample set is as follows: S81. Error Screening and Removal: Systematic error detection is performed on the labeled samples in the labeled sample set, including hard error screening based on a rule engine, automatically detecting and marking labeled samples with logical conflicts, out-of-bounds labels, and violations of business constraints; the labeling results are cross-validated using a verification model, and labeled samples whose model-predicted labels are inconsistent with human labels and whose confidence difference exceeds a preset threshold are marked as suspected errors; an anomaly detection method using isolated forests is used to identify abnormal labeled samples whose labeling patterns deviate significantly from the mainstream distribution; labeled samples marked as suspected errors and abnormal labeled samples are uniformly pushed to the manual review interface for labeling experts to judge one by one; S82. Annotation Consistency Verification: The annotation samples processed in S81 undergo multi-dimensional verification of data annotation consistency and stability. This consistency verification includes: Inter-annotator consistency assessment: Selecting annotation samples of the same type independently completed by different annotators from the same batch, calculating the Kappa coefficient of the annotation results to measure the consistency of different annotators' judgments on the same semantic label. When the consistency coefficient is lower than a preset threshold, the batch of annotation samples is re-annotated; Intra-annotator consistency verification: Selecting a certain proportion of annotation samples, re-annotating them after a period of time, calculating the consistency ratio of the annotation results before and after by the same annotator. If the consistency ratio is lower than a set threshold, the relevant annotation samples are re-annotated; Cross-modal consistency verification: Cross-comparing the labels of the same annotation object in different modal data to ensure semantic consistency across modal labels, and reviewing inconsistent annotation samples. S83. Low-quality labeled sample filtering: The quality of labeled samples is comprehensively scored and graded. The quality score of each labeled sample is calculated based on the labeling confidence, labeler proficiency, labeling time and labeling consistency. Low-quality samples with a quality score below the preset threshold are filtered out or returned for relabeling. At the same time, based on the labeling interaction behavior log recorded by the system during labeling, labeled samples with abnormal behavior during the labeling process are detected and removed. S84. Overall Data Balance Verification and Repair: Perform global-level distribution statistical analysis on the labeled samples processed by S81~S83, and repair the labeled samples based on the analysis results. The sample distribution, label distribution, and cross-distribution of each dimension of the labeled sample set all meet the corresponding preset balance requirements.

[0017] The preferred method for protecting the privacy of the labeled sample set after secondary verification and cleaning is as follows: S91. Sensitive Information Identification and Classification: Sensitivity assessment of each data field in the labeled sample after secondary verification and cleaning. Based on automotive industry data security regulations and the company's own data classification and grading standards, each data field is divided into different sensitivity levels. Corresponding desensitization strategies and processing strengths are formulated for each sensitivity level data field, and sensitivity level marking is completed in the data asset catalog. S92. Numerical Sensitive Data Perturbation Processing Based on Differential Privacy: For continuous numerical fields identified as important data levels, a differential privacy mechanism is used for data perturbation processing. Specifically, this includes: setting a privacy budget parameter ε for each numerical field, determining the noise scale parameter of the required Laplace mechanism based on the field sensitivity, calculating the amount of random noise added to the true value of the continuous numerical field, so that the processed value satisfies (ε,δ)-differential privacy guarantee. S93. Identity information desensitization based on de-identification and anonymization: For the identifier fields and quasi-identifier combinations that can identify vehicles, equipment and individuals in the labeled sample set of secondary verification and cleaning, hierarchical de-identification and anonymization processing is adopted.

[0018] A method for applying high-quality industrial datasets, comprising the following steps: The basic model of the automotive industry is trained using the general knowledge dataset in the high-quality dataset, and then the trained basic model of the automotive industry is trained and fine-tuned using the specialized knowledge dataset to obtain a special model for specific business scenarios. Based on the basic model of the automotive industry and the special model produced for specific business scenarios, an industrial intelligent agent is built by overlaying task planning, memory storage and tool calling modules. After being containerized and encapsulated, it is deployed in the edge or cloud inference environment and realizes real-time perception, inference and decision-making by connecting with business systems through API.

[0019] A method for applying high-quality industrial datasets, comprising the following steps: The basic model of the automotive industry is trained using the general knowledge dataset in the high-quality dataset, and then the trained basic model of the automotive industry is trained and fine-tuned using the specialized knowledge dataset to obtain a special model for specific business scenarios. The trained automotive industry basic model and the specific business scenario-based model are used as a cognitive engine to generate decision reports for the selected application scenarios with confidence scores. A lightweight discriminant model is used to perform real-time cross-validation of the decision report to ensure its authenticity. Based on the basic model of the automotive industry and the special model produced for specific business scenarios, an industrial intelligent agent is built by overlaying task planning, memory storage and tool calling modules. After being containerized and encapsulated, it is deployed in the edge or cloud inference environment and realizes real-time perception, inference and decision-making by connecting with business systems through API.

[0020] A method for applying high-quality industrial datasets, comprising the following steps: The basic model of the automotive industry is trained using the general knowledge dataset in the high-quality dataset, and then the trained basic model of the automotive industry is trained and fine-tuned using the specialized knowledge dataset to obtain a special model for specific business scenarios. The trained automotive industry basic model and the specific business scenario-based model are used as a cognitive engine to generate decision reports for the selected application scenarios with confidence scores. For equipment control and parameter setting in the automotive field, an unbypassable rule firewall is set up to physically verify the feasibility of candidate commands. Based on the basic model of the automotive industry and the special model produced for specific business scenarios, an industrial intelligent agent is built by overlaying task planning, memory storage and tool calling modules. After being containerized and encapsulated, it is deployed in the edge or cloud inference environment and realizes real-time perception, inference and decision-making by connecting with business systems through API.

[0021] The advantages of this invention are as follows: 1. This invention provides automotive companies with a closed-loop construction path that covers the entire chain, from raw data collection and processing to the construction of high-quality industrial datasets, industrial model training, and the deployment and application of industrial intelligent agents. It provides automotive companies with a "ready-to-use" construction and application method, effectively solving the current technical problems of fragmented dataset construction, difficulty in ensuring quality, and difficulty in releasing application value, and ultimately realizing the efficient monetization from "data assets" to "commercial value-added".

[0022] 2. This invention meticulously breaks down and optimizes the entire data process from collection to use, effectively solving the "garbage in, garbage out" problem caused by the rough data processing in existing technologies. It significantly improves the upper limit of dataset quality, realizing a qualitative leap from "usable data" to "high-quality data," ensuring the high purity, high completeness, high generalization, and high compliance of the output dataset, and providing strong support for the performance of industrial models from the data source.

[0023] 3. This invention innovatively employs a dual-optimization architecture, effectively overcoming the static defects of existing technologies that rely on "one-time collection and fixed use." The first optimization embeds a dataset evaluation stage during the dataset construction phase, enabling rigorous quality screening and value prediction of the raw data before model training. This eliminates low-quality and long-tail data at the source, ensuring that the "nutrients" input into the model are pure and efficient. The second optimization, through a feedback data injection stage, feeds back the agent's performance and deviation data in real-time to the front-end dataset, forming a self-evolving closed loop of "what you use is what you get, and what you get is what you've trained." This dual-optimization architecture allows the dataset and model performance to continuously iterate and optimize as the application progresses, completely changing the degradation trend of traditional solutions that "become older and less efficient with use."

[0024] 4. This invention constructs a multi-layered AI illusion defense architecture in the industrial intelligent agent application stage, effectively overcoming the limitations of existing technologies that rely on a single verification method. First, in the model training (step 15) and inference generation (step 16) stages, the probability of illusion generation is reduced from the source through high-quality datasets and industrial-grade training processes. Second, a lightweight neural network for real-time illusion detection is innovatively introduced (step 17) to perform instant quality screening and anomaly capture on the generated content at the inference output end. Then, a deterministic rule engine (step 18) imposes hard boundary constraints on key decisions to ensure that the output conforms to the physical laws and safety specifications of the automotive industry. Finally, relying on multi-agent cross-validation (step 19), agents with different architectures and training strategies mutually verify and collaboratively adjudicate, forming a four-fold mechanism of "generation-detection-constraint-verification". This layered and progressive defense architecture reduces the risk of illusion at multiple stages, providing reliable technical protection for high-safety-level scenarios in the automotive field. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method for constructing high-quality industrial datasets for the automotive industry according to the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0027] Research revealed that a common question for companies when encountering high-quality industrial datasets is: "Why do we need to build high-quality datasets when we have already invested in data governance for many years?" To answer this question, we need to start by looking at the three states that data goes through in an organization.

[0028] From its generation to its use by the model, data exists in three states: raw data, data resources, and high-quality datasets. These are not quality levels or substitutes, but rather different forms of data tailored to different purposes. Raw data consists of original records generated from business activities, possessing authenticity and originality, but it is scattered, heterogeneous, and inconsistent in its definitions, lacking stable reusability. Data resources are organizational-level assets formed from raw data after governance, possessing standardization, credibility, and manageability, supporting querying, statistics, and system calls. High-quality datasets are capability units of data resources reconstructed for artificial intelligence scenarios, organized task-oriented, containing semantic structure, sample relationships, and effect verification information, directly serving model training, retrieval, evaluation, and execution. The difference lies in their service targets: raw data serves business records, data resources serve management and analysis, and high-quality datasets serve models and agents. These three constitute a continuous data value chain: raw data records facts, data resources form supply, and high-quality datasets provide capacity.

[0029] Surveys show that one-third of practitioners cite data quality as their primary challenge. Data errors can translate into biases and misleading predictions in AI, making high-quality industrial datasets a core element of AI industry development. Their fundamental value lies in transforming scattered and complex raw industrial data into "industrial fuel" that can directly empower AI models. Data, as a new type of production factor, possesses unique attributes such as replicability, shareability, scalability, and mineability. However, its raw form is like unmined ore or crude oil—low value density, high noise, and weak practicality. Only through systematic collection, cleaning, enhancement, labeling, and privacy protection—can raw data be transformed into standardized data products with uniform format, controllable quality, and strong scenario adaptability, forming "industrial fuel" driving AI model training and intelligent decision-making. Without a continuous influx of high-quality, large-scale industrial data, general-purpose large models are like "water without a source," and specialized small models are like "trees without roots," unable to truly take root in industrial scenarios. High-quality industrial datasets are not only the foundation for model training and algorithm iteration but also the "industrial blood" that connects the entire manufacturing chain through artificial intelligence.

[0030] High-quality industrial datasets possess both the basic attributes of general datasets and the unique qualities of the industrial sector, exhibiting six core characteristics: First, multimodal fusion: industrial data is highly complex, integrating multiple modalities such as images, videos, time-series sensing, and audio signals; second, strong mechanistic correlation: the data is rooted in physical world production practices and is influenced by multiple factors such as material characteristics, process formulations, and environmental conditions; third, high temporal precision: data acquisition frequencies in high-end manufacturing scenarios often need to reach millisecond or even microsecond levels; fourth, sample imbalance: while normal operation data is abundant, high-value anomaly samples such as faults and defects are extremely scarce; fifth, high security and compliance: industrial data carries core production processes, technical parameters, and other trade secrets; and sixth, value orientation: the construction is not aimed at data scale but rather at serving practical value such as process optimization, cost reduction, and efficiency improvement.

[0031] An embodiment of the present invention provides a method for constructing and applying high-quality industrial datasets for the automotive field, comprising the following steps: Step 1: Selection of application scenarios in the automotive field Adhering to a problem-oriented, market-oriented, and value-oriented approach, and employing the "one map, four lists" scenario deconstruction method, we identified key entry points for solving problems and prioritized the construction and application of datasets in the automotive field.

[0032] Currently, the construction of high-quality industrial datasets faces a fundamental choice: should data construction be "resource-driven" or "demand-driven"? The former points to a data-driven model, while the latter points to a scenario-driven model. The data-driven model follows the logic of "first having data resources, then using data to drive demand upgrades." Its core advantage lies in its ability to quickly generate large-scale datasets, providing foundational materials for tasks requiring massive amounts of data, such as pre-training general-purpose large models. The scenario-driven model follows the logic of "first having needs or scenarios, then building corresponding data support." Its core advantages are: clear objectives, avoiding blind construction and resource waste; and controllable quality, using application results as the evaluation standard.

[0033] This invention adopts a scenario-driven model and establishes a collaborative method of "scenario-data-model" to promote collaboration among scenario users, data providers, and model developers. Scenario applications drive data supply, and data empowers model iteration.

[0034] Step 2: Industrial Data Acquisition Adhering to scenario-driven demand, covering five levels of equipment, production lines, factories, enterprises, and industrial chain ecosystem, and utilizing interface tools, AI and other means, we will standardize and promote the unified collection of multi-source heterogeneous, multi-level distributed and deeply coupled industrial data, open up data links in key industrial scenarios, establish a data resource catalog, and build an enterprise data platform.

[0035] Equipment: Targeting physical equipment or production units, utilizing sensors, instruments, CNC equipment, intelligent machine tools, industrial robots, warehousing equipment, processing units, assembly units, barcode tags, and other sensing terminals to collect data on equipment operation, manufacturing process status, or tooling. Production Line: Targeting production lines or workshops, utilizing programmable logic controllers (PLCs), distributed control systems (DCS), human-machine interfaces (HMIs), intelligent controllers, gateways, remote terminal units (RTUs), supervisory control and data acquisition and monitoring systems (SCADA) to collect data on control systems and network parameters. Factory: Targeting the factory's production and manufacturing processes, utilizing computer-aided manufacturing systems, computer-aided design systems, computer-aided engineering systems, computer-aided process design, manufacturing execution systems, equipment management systems, quality management systems, warehouse management systems, laboratory management systems, metrology systems, health, safety and environmental management systems, product lifecycle management systems, etc., to collect factory-level management data, process data, planning data, safety and environmental data. For enterprises: Focusing on enterprise operations management and maintenance services, this involves using enterprise asset management systems, enterprise resource planning (ERP) systems, supply chain management systems, logistics information systems, financial systems, human resource systems, customer relationship management (CRM) systems, cybersecurity systems, and laboratory management systems to collect enterprise-level data on human resources, technology, costs, finance, and supply chain. For the industrial ecosystem: Focusing on the industrial ecosystem, this involves using customer relationship management, remote operation and maintenance, collaborative design and development, collaborative procurement, collaborative production, resource sharing, supply and demand matching, common applications, and supply chain collaboration to collect data on collaborative strategies and management.

[0036] Step 3: Preliminary cleaning of industrial data The collected automotive industry scenario data undergoes data asset inventory and cataloging, complete invalid samples are removed, data deduplication is performed, coarse-grained filtering of extreme anomalies is performed, missing samples are initially screened and marked, and file integrity and readability are verified. After removing invalid industrial data, complete industrial data with valid information is retained.

[0037] The specific methods for preliminary industrial data cleaning are as follows: S31, Data Asset Inventory and Cataloging: Register the raw data collected in the previous step, establish a data catalog list, record the data source, collection time window, channel identifier, data scale and storage format of each batch of data, forming a traceable data list to provide a global view for subsequent cleaning; S32, Eliminate Completely Invalid Samples: Perform an integrity pre-check on each record in the data catalog list to determine whether it is a completely invalid sample. The completely invalid samples include, but are not limited to: empty frame data with all load values ​​empty, and all zero or all full values ​​generated when the acquisition channel is invalid or the sensor is offline. Records with fixed values, records that cannot be parsed normally due to data index corruption or header information damage caused by storage anomalies, and such samples do not contain any valid information payload and are directly discarded with the reason for removal recorded; S33, Data deduplication: Fingerprint comparison and deduplication are performed on redundant records stored repeatedly under the same sensor channel or the same acquisition timestamp. The deduplication includes completely identical duplicate samples, as well as approximately duplicate samples with adjacent timestamps and Euclidean distance or cosine similarity between feature vectors below a preset threshold. Only a unique valid copy is retained after deduplication; S34, Coarse-grained filtering of extreme abnormal data: Based on the preset physically reasonable value range of each data type. Alternatively, industry standard thresholds are used to perform boundary checks on the data fields in the records, eliminating significant outliers that exceed physical limits. These thresholds, for power domain signals, refer to the vehicle design limit parameters, and for sensing domain signals, refer to the sensor range and three standard deviations of the statistical distribution under typical operating conditions. The filtered data will be marked as candidate valid data and proceed to the next processing stage. S35, Initial screening and marking of missing samples: The missing rate of fields in the candidate valid data is statistically calculated. Candidate valid data with a missing rate exceeding a preset threshold are judged as severely incomplete samples and eliminated. For candidate valid data with a missing rate below the threshold, the location and magnitude of the missing fields are marked. The method retains the main body of candidate valid data for subsequent refined missing data processing modules to interpolate or repair, and generates a missing data mask matrix to accompany the data transmission; S36, File integrity and readability verification: For unstructured or semi-structured data (including but not limited to images, point clouds, video streams and log files) stored in the form of files in the candidate valid data, file header verification, verification and comparison and format specification checks are performed to remove incomplete data fragments that are corrupted due to storage media damage, transmission packet loss or write interruption, cannot be opened normally or fail to be decoded, so as to obtain complete industrial data with retained valid information, ensuring that the downstream links receive only valid files that can be completely parsed.

[0038] Step 4: Industrial Data Preprocessing The preprocessing of complete industrial data after initial cleaning includes format standardization, fine-grained processing of missing values, noise smoothing and signal correction, spatiotemporal alignment of multi-source data, numerical normalization and standardization, and data fragmentation and sampling equalization preprocessing.

[0039] The preprocessing method for industrial data is as follows: S41, Format standardization processing: For the initially cleaned automotive industry data, according to the preset automotive general data specifications, a unified format conversion and field mapping are performed. The format standardization includes: unifying the timestamp accuracy to the microsecond level and converting it to a standard time format, unifying the physical quantity measurement units to the International System of Units (SI), unifying the encoding method and byte order of numerical data, converting semi-structured or unstructured data (including but not limited to JSON logs, CSV time-series records, images and point cloud files) into general storage format data that is accepted by the downstream algorithm module, and establishing a mapping relationship table between the original fields and standard fields to ensure the fusion of multi-source data at the format level. S42. Refined processing of missing values: Based on samples with a missing rate below a preset threshold, an adaptive missing value processing strategy is selected according to the data type, missing rate, and missing pattern of the missing field. The processing strategies include: for continuous numerical fields, linear interpolation, spline interpolation, forward and backward imputation based on time series, and quantile imputation based on historical distribution under the same working conditions are used to complete missing values; for discrete categorical fields, mode imputation and prediction imputation based on decision tree models are used; for scenarios with a moderate missing rate and strong correlation between fields, multiple regression and matrix factorization methods are used for joint estimation imputation. S43. Noise Smoothing and Signal Correction: To address high-frequency random noise, spikes, and periodic interference introduced by the sensor itself or the transmission channel during the acquisition process, differentiated filtering strategies are employed for smoothing and correction based on signal type and noise characteristics. These methods include: for time-series signals with predominantly low-frequency trends (such as temperature, battery voltage, and rotational speed), sliding window mean filtering or median filtering is used to suppress random impulse noise, while a reasonable filter cutoff frequency is set through frequency analysis to preserve effective physical characteristics; for event-type signals containing transient details (such as vibration and pressure pulses), adaptive wavelet denoising or statistical outlier detection and replacement strategies are used to eliminate noise while preserving key event waveform characteristics as much as possible; for visually perceived data, bilateral filtering or non-local mean filtering is used to smooth noise while maintaining edge clarity; the corrected signal must be verified for fidelity using signal-to-noise ratio and waveform similarity indicators to ensure that the preprocessed signal does not introduce significant distortion. S44. Spatiotemporal Alignment of Multi-Source Data: For data streams from different sensors, different acquisition channels, and different sampling frequencies, precise synchronization in the time dimension and coordinate registration in the spatial dimension are performed. The time alignment adopts a timestamp interpolation resampling method based on a common reference clock (such as GPS / PTP synchronization clock) to unify the data of each channel onto the same time axis. The spatial alignment is for the sensing domain data. According to the relative calibration parameters between each sensor (including rotation matrix and translation vector), the multi-sensor data is transformed into a unified coordinate system to ensure the spatiotemporal consistency of the same physical entity in multimodal data.S45. Numerical Normalization and Standardization: For numerical features of different dimensions and orders of magnitude, while maintaining their physical meaning and distribution pattern, a dimensionless transformation is performed to eliminate the impact of dimensional differences on model training. The transformation methods include: for features that conform to or are approximately Gaussian, Z-score standardization is used to convert them into a standard normal distribution with a mean of 0 and a standard deviation of 1; for features with unknown distributions or significant outliers, Min-Max normalization is used to linearly scale them to a preset interval; for features with physical upper and lower boundary constraints, a fixed interval mapping strategy is used to preserve their original boundary meaning; the transformation parameters of each feature must be persistently stored so that the same parameters can be used for inverse transformation or mapping during the inference stage to ensure the reversibility and consistency of data transformation. S46. Data Sharding and Sampling Equalization Preprocessing: After completing the above processing, the data is organized and resampled according to the needs of downstream model training. This includes: using a fixed time window to slide-window sharding for long-term continuously recorded time-series data to generate sequence samples with temporal context, and setting a preset overlap rate for adjacent shards to enhance the temporal continuity between samples; uniformly resampling unstructured data according to preset frame rate and sampling interval; for datasets with extremely imbalanced classification distribution, using undersampling to discard majority class samples, oversampling to copy or synthesize minority class samples, or a hybrid sampling strategy combining both, to ensure that the number of samples in each class reaches a preset balanced ratio; simultaneously, based on the data collection time or working condition label, the processed dataset is divided into training set, validation set, and test set according to a preset ratio, and it is ensured that the data in each set is independent in terms of time dimension and working condition distribution to avoid future information leakage; the preprocessed data is output uniformly in a standardized format, with a complete preprocessing log recording the parameter configuration and processing results of each step, ensuring the traceability and reproducibility of the preprocessing process.

[0040] Step 5: Industrial Data Synthesis Using real-world data as anchors, high-value seed scenarios that meet set conditions (such as representing extreme working conditions, rare faults, and dangerous events) are extracted through preset filtering rules. Based on these high-value seed scenarios, target attributes are defined for the synthesized data, encompassing sensor modalities, physical parameter ranges, and scene semantic tags. A hybrid generation strategy incorporating physical constraints is employed to synthesize the data, ensuring physical consistency while selectively filling in gaps in the distribution of real-world data. The synthesized data is then converted into an industry-standard format, and corresponding metadata is generated simultaneously.

[0041] The method for obtaining high-value seed scenarios is as follows: Develop multi-level screening rules: Combine domain knowledge to develop a rule base, with rules covering three types: physical thresholds, signal mutations, and complex logic.

[0042] Automated coarse screening: Scan the entire dataset using sliding window technology, calculate statistical features within the window and match them with the rule base to mark all abnormal candidate segments.

[0043] Manual verification and confirmation: Candidate segments are visually reviewed to eliminate false alarms, and then entered into the database after confirmation. In practice, this is often combined with equipment self-diagnostic logs (such as fault codes) for accurate localization.

[0044] Structured tag management: Label confirmed seeds with metadata (such as operating condition type and severity level) to form high-value seed scenarios that can be directly used for data synthesis.

[0045] Step 6: Industrial Small Sample Data Augmentation Based on existing data, small sample data augmentation techniques are used to expand the scale and diversity of the data.

[0046] Implement differentiated enhancement strategies based on different modalities of industrial data: For numerical continuous data (such as sensor time series and process parameters): noise injection, amplitude scaling, time axis distortion, and physical rule-based interpolation transformations are employed. For discrete categorical data (such as fault codes and process status labels): use distribution-based prior sampling and feature space interpolation methods such as SMOTE; For spatially perceived data (such as images, point clouds, voxel meshes): geometric transformations (rotation, scaling, cropping), color / intensity adjustments, and spatial perturbation enhancements are employed. For text / log data (such as maintenance records and alarm information): synonym replacement, syntactic trimming, and semantic enhancement based on domain knowledge base are employed. For multimodal combined data: Implement a synchronous transformation strategy to ensure that the spatiotemporal correspondence between each modality remains consistent after enhancement.

[0047] Step 7: Contextualized Labeling of Industrial Data By combining the experience of veteran mechanics in the automotive field, a tagging system integrating expert experience is constructed. Around specific application scenarios, a labeling method based on human-machine collaboration, semi-supervised and weakly supervised, and cross-modal joint labeling is adopted to label the standardized data in the previous step with business tags, covering text logs, images, videos, audio, code, process parameters, numerical indicators, 3D point clouds, etc., and linking cross-source information such as equipment, process, and quality.

[0048] The industrial data scenario-based annotation method is as follows: S71. Construct a tagging system that integrates expert experience: Based on the automotive industry's design specifications, maintenance manuals, fault mode libraries, and process standards, and combined with the practical experience of senior engineers from fields such as vehicle engineering, powertrain systems, chassis control, intelligent driving, manufacturing, and after-sales quality, a multi-dimensional hierarchical tagging system for specific business scenarios is constructed. This tagging system is divided into the following categories according to the business perspective: equipment-level tags, process-level tags, quality-level tags, operating condition tags, and behavioral semantic tags. Each level is further refined into four levels of annotable atomic tags based on specific application goals. Relationships and mutual exclusion rules between tags are established, and each tag is accompanied by an annotation documentation that clarifies the tag's business definition, judgment criteria, annotation boundaries, and typical positive and negative examples, ensuring consistency in the understanding of the same tag among different annotators. S72. Accurate Labeling Based on Human-Machine Collaboration: For labels with complex semantic understanding and high requirements for professional judgment, a human-machine collaborative labeling mode of "machine pre-labeling + manual review and correction" is adopted. Specifically, this includes: using a pre-trained deep learning model to perform preliminary pre-labeling on standardized data and outputting candidate labels with confidence scores; samples with confidence scores higher than a preset high threshold are directly adopted as valid labels, while samples with confidence scores lower than a preset low threshold are pushed to a manual labeling platform for fine-grained interpretation and labeling by labelers with relevant domain qualifications; ambiguous samples with confidence scores between the high and low thresholds are pushed to the review interface for manual confirmation, correction, or rejection of the pre-labeling results. During the labeling process, the system simultaneously displays auxiliary judgment information, including the historical maintenance records of the equipment corresponding to the sample, the distribution of similar case labels, and the label voting results of the same batch of data, to help labelers improve judgment efficiency and accuracy. S73. Large-scale automated labeling based on semi-supervised and weakly supervised learning: For label items with large data volume, relatively clear labeling rules, and formalizable expert knowledge, a combination of semi-supervised and weakly supervised learning is used for large-scale automated labeling. The semi-supervised labeling uses a small number of high-quality manually labeled labels as seeds, and diffuses label information from seed labeled samples to a large number of unlabeled samples through self-training, collaborative training, or graph-based label propagation algorithms. The weakly supervised labeling compiles expert experience into a set of executable labeling rule functions (including keyword matching rules, threshold judgment rules, temporal pattern rules, and logical combination rules). Multiple weakly supervised rules are input into the label modeling module for potentially conflicting labeling results for the same sample. Probabilistic graph inference is performed to estimate the confidence of each rule and the true label probability distribution of each sample, thereby generating large-scale weakly supervised labels with confidence assessment without requiring a large amount of manual labeling. Labeling results with consistent rules and high confidence are directly stored in the database. Samples with conflicting rules or insufficient confidence are diverted to the human-machine collaborative labeling pipeline for manual arbitration according to the conflict mode.S74. Cross-modal joint annotation of multimodal data: For complex scenarios where complete industrial data simultaneously contains multiple modalities (including but not limited to text logs, images, video sequences, audio signals, source code, process parameter curves, numerical index time series, and 3D point clouds), a cross-modal joint annotation strategy is adopted to establish consistency and complementarity associations between labels of different modalities. The cross-modal joint annotation strategy uses business semantic tags as anchors and adapts corresponding annotation tools and annotation methods to different modalities: For image data: polygon segmentation and bounding box annotation are used; for video sequences: keyframe annotation is used. The tracking and annotation method using interpolation propagation generates a consistent label sequence for the same object across consecutive frames. For text logs, it employs annotation methods based on named entity recognition, relation extraction, and event type classification. For audio signals, it uses annotation methods based on temporal segmentation and event category labeling. For process parameters and numerical indicators, it combines rule-based baseline offset detection annotation with expert annotation. For 3D point clouds, it employs 3D bounding box annotation, point-level semantic segmentation, and key point annotation. The annotation results of each modality are associated with a unified sample ID and timestamp to form cross-modal label-aligned annotation samples. S75. Cross-source information association: Vertically associate labeled samples with equipment numbers, process parameters, formulas, and quality inspection results in the production and manufacturing stages to establish a full-chain label traceability chain; Horizontally associate cross-system and cross-stage information such as design parameters, maintenance records, and fault handling logs related to the same vehicle and the same component to provide rich business context for the current labeled sample, assisting labelers in making more accurate labeling judgments when information is insufficient or boundaries are ambiguous; The associated information is attached to the basic label in the form of label metadata, so that the final labeled sample set contains not only atomic business labels, but also equipment traceability chain, operating condition context, and quality closed-loop information.

[0049] Step 8: Secondary verification and cleaning of industrial data A systematic "check-up" and "correction" of the labeled sample set is carried out, mainly including screening and removing labeling errors, verifying labeling consistency, filtering low-quality labeled samples, and verifying and repairing the overall data balance.

[0050] The secondary verification and cleaning method for industrial data is as follows: S81, Error Screening and Removal: Systematic error detection is performed on the labeled samples. The error detection adopts a multi-level composite screening strategy, including hard error screening based on a rule engine, which automatically detects and marks labeled samples with obvious errors such as logical conflicts, out-of-bounds annotations, and violations of business constraints; cross-validation of the annotation results is performed using an independently trained verification model, and labeled samples whose model-predicted labels are inconsistent with human labels and whose confidence difference exceeds a preset threshold are marked as suspected errors; an anomaly detection method using isolated forest is used to identify abnormal labeled samples whose annotation patterns significantly deviate from the mainstream distribution; the suspected error labeled samples and abnormal labeled samples output by the screening are uniformly pushed to the manual review interface, where annotation experts judge them one by one. Samples confirmed as incorrectly labeled are corrected or removed, and the error type and correction operation log are recorded for subsequent targeted training of annotation personnel and rule optimization. S82. Annotation Consistency Verification: Multi-dimensional verification of data annotation consistency and stability is performed. The consistency verification includes: Inter-annotator consistency assessment: Selecting annotation samples of the same type independently completed by different annotators in the same batch, calculating the Kappa coefficient of the annotation results to measure the consistency of different annotators' judgments on the same semantic label. When the consistency coefficient is lower than a preset threshold, the batch is re-annotated; Intra-annotator consistency test: Selecting a certain proportion of annotation samples, re-annotating them after a period of time, calculating the consistency ratio of the annotation results before and after by the same annotator, and assessing the stability and fatigue of the annotator. If the consistency ratio is lower than a set threshold, the relevant annotation samples are re-annotated; Temporal consistency verification: For the same annotation object, the label sequence formed by its labels in continuous time frames is detected. If the label undergoes unreasonable abrupt changes between adjacent time slices, it is marked and pushed to the manual review interface for temporal smoothing correction; Cross-modal consistency verification: Cross-comparing the labels of the same annotation object in different modal data to ensure that the labels of each modality are semantically consistent. Inconsistent annotation samples are marked and returned for cross-modal joint annotation review. S83. Low-quality labeled sample filtering: The quality of labeled samples is comprehensively scored and graded. A quality score for each sample is calculated from multiple dimensions, including labeling confidence, labeler proficiency, labeling time, and labeling consistency. Low-quality samples with quality scores below a preset threshold are filtered out or returned for re-labeling. Simultaneously, based on the labeling interaction behavior logs recorded by the system during labeling, labeled samples exhibiting abnormal behavior are detected and removed. Abnormal behavior includes: labeling time significantly shorter than the average time for similar samples, frequent modifications to labeling results during the labeling process, and labeling samples exhibiting fatigued or arbitrary operations by the labeler. The filtering operation must generate a filtering log, recording the sample ID of the filtered sample, the reason for filtering, and detailed quality scores to ensure the traceability of the filtering decision.S84. Overall Data Balance Verification and Repair: Perform global-level distribution statistical analysis on the labeled samples processed by S81~S83 to verify whether the sample distribution, label distribution, and cross-distribution of each dimension of the data meet the preset balance requirements. The balance verification includes: category balance verification: statistically analyze the sample quantity and proportion of each label category, identify weak categories with severely insufficient sample size, and for significant imbalances, adopt targeted resampling strategies to mark them as gap categories to be supplemented; operating condition coverage balance verification: statistically analyze the sample distribution under different operating conditions (including but not limited to temperature range, speed range, load range, lighting conditions, and weather conditions), identify sparse operating condition coverage areas, and generate gap reports; time distribution balance verification: check whether the distribution of samples on the collection time axis is uniform, and identify sample clustering patterns with concentrated time and missing periods; for various imbalance problems identified during the verification process, automatically trigger compensation measures such as data augmentation, targeted supplementary collection, and sample weight adjustment according to preset repair strategies. For gap categories that cannot be automatically repaired, generate a special supplementary collection suggestion list.

[0051] Step 9: Industrial Data Privacy Protection Sensitive information identification and hierarchical classification are performed on the labeled sample dataset that has undergone secondary verification and cleaning; numerical sensitive data perturbation processing is based on differential privacy; identity information desensitization is based on de-identification and anonymization; and cross-domain collaborative data protection is based on federated learning, so as to achieve "data usable but invisible".

[0052] The specific methods for protecting industrial data privacy are as follows: S91. Sensitive information identification and classification: A comprehensive inventory and sensitivity assessment of each data field in the labeled sample after secondary verification and cleaning is carried out. Based on the data security regulations of the automotive industry and the company's own data classification and grading specifications, each data field is divided into different sensitivity levels. Corresponding de-identification strategies and processing strengths are formulated for each sensitive level of data field, and sensitivity level marking is completed in the data asset catalog. S92. Numerical Sensitive Data Perturbation Processing Based on Differential Privacy: For continuous numerical fields identified as important data levels, a differential privacy mechanism is used for data perturbation processing. Specifically, this includes: setting a privacy budget parameter ε (privacy loss, used to control the overall strength of privacy protection) for each numerical field. A smaller ε means more noise is added, resulting in stronger protection, but reduced data availability; a larger ε means weaker protection and higher data accuracy. Based on the field sensitivity, the required noise scale parameter of the Laplace mechanism is determined. The amount of random noise added to the true value of the continuous numerical field is calculated, ensuring that the processed value satisfies (ε,δ)-differential privacy guarantee. This ensures that attackers cannot reverse-engineer the original value of any labeled sample through differential attacks. Simultaneously, the perturbation amplitude is controlled within an acceptable range through reasonable configuration of the privacy budget, ensuring that the processed labeled sample maintains its original distribution characteristics and related structure at the statistical level. δ is the failure probability, representing the upper limit of the probability that privacy protection is allowed to "fail," which is usually a very small number (e.g., 10). -5 Or 10 -6 The statement indicates that it is almost impossible, but there is a very small probability that the protection will fail to meet the standards. S93. Identity information desensitization based on de-identification and anonymization: For the identifier fields and quasi-identifier combinations that can identify specific vehicles, equipment and individuals in the labeled sample set after secondary verification and cleaning, hierarchical de-identification and anonymization processing is adopted. The processing includes: using irreversible encrypted hashing to de-identify direct identifiers (including VIN code, device serial number, license plate number, vehicle owner's mobile phone number and ID card number, etc.), mapping the original identifier to an anonymous unique code without business meaning, and storing the mapping relationship independently in the controlled access security key management module, so that the source of the data cannot be traced back from the data itself in the data circulation process; using the L-diversity model to process quasi-identifiers (including downgraded processing of collection timestamp accuracy, gridded fuzzy processing of geographical location coordinates, generalized processing of vehicle configuration information, etc.), ensuring that any record in the labeled sample dataset is indistinguishable from at least K-1 other records in terms of quasi-identifier combinations, blocking the possibility of re-identifying individuals through association analysis.

[0053] Step 10: Compliance and Integrity Verification The data processed through the above steps (i.e., the anonymized labeled samples) undergoes a final compliance and integrity check before leaving the factory. The compliance check includes a data anonymization compliance check: verifying whether personal information, geolocation-sensitive information, and corporate trade secrets in the data have been anonymized in accordance with regulatory requirements, ensuring that the output data meets data security and privacy protection compliance requirements; a format compliance check, verifying the completeness of data fields, the correctness of data types, and the legality of value ranges according to automotive industry format specifications; and a completeness check, verifying whether the data fully covers the preset label levels, meets the preset total sample size target, and meets the minimum sample size requirements for each label category.

[0054] Step 11: Classify and build a high-quality dataset Using labeled samples that have passed compliance and integrity checks as input, and benchmarking against high-quality dataset classification standards and industry norms, two major categories are formed: general knowledge datasets and specialized knowledge datasets.

[0055] The methods for constructing high-quality datasets are as follows: S111. Construction of high-quality general knowledge datasets in the automotive field: General knowledge datasets are geared towards common perception, common understanding, and basic decision-making scenarios. Data sources cover public literature, intellectual property, standards and specifications, industry reports, etc. Data selection follows the principles of typicality, representativeness, and diversity to ensure broad coverage of common scenarios. Quality requirements include a labeling accuracy rate not lower than a preset threshold and sample distribution covering the preset set of common working conditions. S112. Construction of high-quality specialized knowledge datasets in the automotive field: Specialized knowledge datasets are geared towards in-depth scenarios in specific business domains (including powertrain failure mode library, battery thermal runaway feature set, extreme working condition driving trajectory, specific vehicle calibration parameters, and long-tailed event samples). Data sources include historical fault work orders, bench test records, extreme environment test data, and expert-calibrated samples. Domain experts are involved in data screening and quality arbitration during the construction process. Quality requirements include high sample purity, a labeling consistency coefficient not lower than a preset threshold, and coverage of typical patterns and marginal cases in the target sub-domain.

[0056] Step 12: Dataset Evaluation (First Optimization: If unsatisfactory, return to Step 3) Using the completed high-quality general knowledge dataset and specialized knowledge dataset as input, the dataset is comprehensively evaluated by multiple indicators from dimensions such as data completeness, annotation accuracy, class balance, working condition coverage, and target model adaptability. If all indicators meet the preset admission standards, the dataset is judged to be qualified and enters the next packaging and delivery stage with the evaluation report. If any indicator fails to meet the standard, the dataset is judged to be unqualified. The system automatically generates a diagnostic report containing the unqualified dimensions, evaluation values, and threshold deviations. The dataset and the diagnostic report are returned to step 3 for targeted repair and reconstruction. After the repair is completed, the dataset re-enters the evaluation process.

[0057] Step 13: Dataset Packaging For datasets that meet the evaluation requirements, a unified encapsulation is performed. The encapsulation includes data entities, label files, metadata description documents, and a data dictionary. Data entities are organized according to a preset directory structure; label files and data entities are indexed and mapped using unique sample IDs; the metadata description documents record the dataset name, version number, construction date, data size, modal composition, label system, and usage instructions. After encapsulation, a digital summary of the dataset is generated, supporting version traceability and integrity verification; all encapsulation products are output according to preset naming conventions and have the ability to be directly imported into downstream training pipelines.

[0058] Step 14: Reliable Supply of High-Quality Datasets Using a high-quality, fully packaged dataset as input, the system connects data supply and demand parties through data ownership registration, contractual authorization, and blockchain-based evidence storage and auditing processes. It employs encrypted transmission channels and secure API interfaces to achieve controlled data delivery. Through digital watermarking and access logs, it enables traceability and leak prevention of data flow. After delivery, settlement is completed based on the billing model and usage measurement agreed upon in the contract, forming a complete data supply chain.

[0059] Step 15: Industrial Model Training Using high-quality datasets from reliable supply chains as training materials, we conduct multi-level model training and application for automotive application scenarios: We use a constructed general knowledge dataset to pre-train the basic automotive industry model, giving the model general perception and understanding capabilities; on this basis, we load scenario-specific knowledge datasets and use efficient parameter fine-tuning to adapt to specific business scenarios such as fault diagnosis, state prediction, and behavioral decision-making to produce dedicated models.

[0060] Step 16: Model inference generates candidate decisions The base model and specialized model trained on high-quality datasets are used as a cognitive engine to generate decision reports for the selected application scenarios, along with confidence scores.

[0061] Step 17: Lightweight Neural Network Real-time Hallucination Detection Deploy a separate, lightweight discriminative model (with few parameters and no GPU acceleration) at the output of the industrial model to perform real-time cross-validation of the decision report from the previous step, keeping the inference latency within 50ms and intercepting factual errors.

[0062] Case Study: The industrial model output is concatenated with retrieved knowledge fragments (equipment manuals, historical normal records), and fed into the discriminative model, which outputs three-category labels: "implied / neutral / contradictory". If the result is determined to be "contradictory" and the confidence level is ≥0.80, an interception is triggered.

[0063] Step 18: Deterministic Rule Engine For high-risk operations involving equipment control and parameter setting in the automotive field, an unbypassable rule firewall is set up to physically verify the feasibility of candidate instructions.

[0064] Case Study: Establish a whitelist of equipment parameters, clearly defining the safety threshold range for each machine (e.g., temperature ≤ 85℃, pressure ≤ 0.8MPa, speed range 1000-3000rpm). If any parameter in a candidate command exceeds the safety threshold, discard the command and return a rejection code, triggering an alarm and requiring manual confirmation, ensuring that the output of the large model never exceeds the safety limit.

[0065] Step 19: Industrial Intelligent Agent Application With basic and specialized models as the core, and by overlaying task planning, memory storage, and tool invocation modules, an industrial intelligent agent is constructed. After being containerized and encapsulated, it is deployed in edge or cloud inference environments and achieves real-time perception, inference, and decision-making by connecting with business systems through APIs.

[0066] Step 20: Feedback data reinjection to achieve closed-loop optimization (second optimization) Feedback data generated during real-world scenarios is anonymized and then reinjected into the dataset construction process, triggering incremental updates to the dataset and a new round of iterative fine-tuning of the model, thus achieving a continuous evolution loop and ensuring that the dataset continues to release business value.

[0067] Case Study: After erroneous samples are labeled as "negative examples," they are incrementally updated to the training set, triggering the next round of model fine-tuning iterations. This allows the model to gradually learn to avoid similar errors, achieving a continuous evolutionary closed loop where it becomes more accurate with use.

[0068] In one embodiment, taking the business data of an automotive company as an example, it can be divided into three categories: structured, semi-structured, and unstructured. The data storage formats, structural rules, and noise characteristics of the three categories are significantly different, making it impossible to process them using a unified script.

[0069] 1. Structured data This type of data follows a strict relational model, with predefined schema constraints. Field dimensions, data types, and value ranges are fixed before being stored in the database. Its physical carriers are primarily relational database management systems such as MySQL and Oracle, as well as flat spreadsheet files such as CSV and Excel. This type of data has extremely high computational affinity, supporting standard SQL queries and multidimensional statistical analysis, but it has significant limitations in semantic expression, unable to support non-standardized long text contexts or fuzzy logical reasoning.

[0070] 2. Semi-structured data This type of data does not rely on a fixed relational schema, but maintains independent data organization logic through self-describing hierarchical tags and key-value pairs. Common formats include JSON, JSONL, XML, and YAML, and it is widely found in API response payloads, server operation logs, user behavior tracking, and lightweight configuration files. Due to the lack of a globally unified data dictionary, this type of data generally suffers from compatibility issues such as excessively deep nesting, inconsistent field naming styles (mixed underscores / camelCase / Chinese characters), and different expressions of fields with the same business meaning, forming the first barrier to data integration.

[0071] 3. Unstructured data This type of data constitutes the largest volume of enterprise data assets, contains the highest density of business knowledge, and is also the most difficult to govern. Its content has no fixed field constraints, its layout is flexible, and semantic units are scattered throughout paragraphs, tables, and charts. In enterprise scenarios, Word documents and PDF files are the core carriers, broadly covering key knowledge assets such as laws and regulations, technical patents, equipment maintenance manuals, project completion reports, and industry standards and specifications. Original documents generally face multiple noise interferences, including paragraph breaks across pages, interference from mixed text and images, redundant information in headers and footers, watermark obscuring, and garbled characters introduced during identification. Effective semantic signals are wrapped in a large amount of non-informative noise, requiring deep cleaning through methods such as layout restoration, text sequence reconstruction, and multimodal content stripping.

[0072] One embodiment of the present invention is a unified output paradigm design for the model ecosystem, in which a standardized output strategy of heterogeneous isomorphism is implemented for three types of data, the core output format is established as JSONL (JSONLines), CSV is used as an auxiliary format, and a complete asset description document is provided.

[0073] 1. Standardized translation of structured data For structured data, a dual-track output of JSONL and CSV is adopted to balance the convenience of machine batch loading and manual review. Each record is laid out flat to eliminate all nesting redundancy, and standardized metadata fields (including globally unique identifiers, data lineage, timestamps, and quality scores) are forcibly injected to ensure its universality for cross-system distribution.

[0074] Case Study: {"sample_id":"S0718","vin":"XX","fault_code":"P0509","mileage_km":26701,"maintain_date":"2026-07-18","dealer_name":"XX","data_source":"XX","quality_score":0.98}.

[0075] 2. Flattening and reshaping semi-structured data To address the issue of excessively deep nesting in the original JSON / XML data, a path flattening algorithm is employed to transform deep nodes into flat fields. The field naming convention is standardized to lowercase + underscore format, and null values ​​and irregular data types are forcibly corrected, resulting in a highly regular collection of key-value pairs.

[0076] Case Study: {"sample_id":"S0718","interface_name":"XX","request_id":"REQ_112233","response_code":105,"battery_soc":92.3,"battery_temp_avg":56.2,"data_source":"XX","quality_score":0.97}.

[0077] 3. Semantic unit reconstruction of unstructured data To adapt to RAG retrieval and text generation tasks, the traditional coarse-grained whole-text extraction method is abandoned, and an intelligent slicing strategy based on semantic coherence is adopted. The output format retains the complete document structure, including the chapter, source file fingerprint, and character offset, ensuring that the generated results have accurate traceability.

[0078] Case Study: {"sample_id":"S0718","content":"XX","doc_title":"XX","chapter_path":"XX","source_file":"BMS_Maintenance_V2.0.pdf","char_length":86,"quality_score":0.97}. Specific implementation examples: (I) Selection of application scenarios in the automotive field: Failure prediction of moving parts of stamping line presses This embodiment follows the "One Map, Four Lists" deconstruction method in the "Reference Guidelines for Promoting Digital Transformation of Key Industries in a Scenario-Based and Map-Based Approach (2025 Edition)". A certain enterprise has a very strong need for "predictive maintenance of equipment," especially for predicting the failure of "moving parts" in stamping line presses. The "One Map" refers to the "Industry Scenario Map," and the "Four Lists" refer to: data elements, knowledge models, software tools, and talent skills.

[0080] The above-mentioned automotive industry scenario map is shown in the table below. Stamping lines are a key process in automobile manufacturing: body panels and structural components are all produced here. If a line stops, the entire assembly line has to wait. In reality, there have been instances where a stamping production line experienced a sudden 8-hour shutdown due to bearing wear, resulting in direct losses exceeding 600,000 yuan, with emergency repair costs three times higher than routine maintenance. However, with the introduction of predictive maintenance, the same equipment's bearing wear is detected in its early stages, allowing for replacement during off-peak hours, reducing downtime losses by over ten million yuan annually.

[0081] Step 1: Taking a 6600kN mechanical press in the stamping workshop of a certain vehicle manufacturer as an example, the "moving parts" of the press include crankshaft, connecting rod and sliding bearing.

[0082] (ii) Industrial Data Acquisition Step 2: Install a PCB 352C33 accelerometer (sensitivity 100mV / g, range ±50g) in both the horizontal and vertical directions on the crankshaft bearing housing of the press. Set the sampling rate to 51.2kHz and continuously acquire time-domain vibration signals. Read the main motor current (every 200ms), lubricating oil temperature (every 1s), and slide block position (every 50ms) from the press PLC via the PROFINET protocol. Synchronously acquire the corresponding stamping cycle time (approximately 12 pieces per minute) and scrap rate data for this press via the MES interface.

[0083] The data collection period is three consecutive months (April 2026 to June 2026), with a total raw data volume of approximately 1.2TB.

[0084] (III) Industrial Data Processing: Step 3: Preliminary cleaning of industrial data The acquired raw vibration data was sliced ​​into 1-second windows, with 51,200 sampling points per window. Preliminary cleaning was performed on each data segment: sensor DC bias was removed (subtracting the mean), eliminating zero drift effects; a 0.5Hz-15kHz bandpass filter was applied to filter out high-frequency circuit noise and low-frequency environmental disturbances, while fully preserving the sensitive frequency band containing the bearing fault characteristic frequencies. The preliminarily cleaned data then entered the preprocessing pipeline.

[0085] Step 4: Industrial Data Preprocessing Based on the initial cleaning, a 5-level decomposition was performed using the db8 wavelet basis. A general threshold soft thresholding process was applied to the detail coefficients of each level, effectively suppressing background white noise while preserving the abrupt change characteristics of the fault impulse signal. After preprocessing, time-domain features (peak value, RMS value, kurtosis, skewness, waveform factor) and frequency-domain features (spectral peak value, sideband amplitude ratio) were extracted and merged with the original waveform to form a structured feature vector.

[0086] Step 5: Industrial Data Synthesis Because the complete degradation process of the bearing from normal to severe wear was only recorded once during the actual operation of the stamping line (approximately 200 failure samples), while there were 1000 normal samples, the number of normal and failure samples was severely unbalanced. Therefore, a data synthesis process was initiated: based on the dynamic mechanism of the press crankshaft-bearing system, a multibody dynamics simulation model including bearing clearance was established in Simulink. Different wear clearance amounts (0.05mm, 0.10mm, 0.15mm, 0.20mm) and speed conditions (30rpm, 45rpm, 60rpm) were set to simulate and generate vibration waveforms at different degradation stages, resulting in a total of 1200 simulated failure samples.

[0087] Step 6: Industrial Small Sample Data Augmentation Based on the mechanism simulation and synthesis, 200 real fault samples were further augmented using multiple strategies: first, time axis stretching (scaling factor 0.9-1.1) to simulate the time-domain scaling effect under different speeds; second, amplitude scaling (0.8-1.2) to simulate amplitude changes caused by load fluctuations; and third, the addition of Gaussian white noise with a signal-to-noise ratio of 30dB to improve the model's robustness to noisy environments. Each original sample was expanded into 5 variants, resulting in a total of 1000 augmented samples.

[0088] Step 7: Contextualized Labeling of Industrial Data Using the fault baseline time recorded in the maintenance work order as "crankshaft bearing cage fracture, one set of bearings replaced," all data within the acquisition period were subjected to scenario-based time-series labeling. The fault evolution was divided into four stages: Stage I, labeled with health_label=0, fault_type=0, and RUL>600 hours (normal operation period); Stage II, confirmed by spectral review with BPFO and its sidebands, labeled with health_label=1, fault_type=1, and RUL=400-600 hours (early wear); Stage III, labeled with RUL=80-400 hours (intermediate wear, kurtosis continuously rising to above 5.0); and Stage IV, labeled with RUL<80 hours (late-stage degradation, significant impact pulses). In addition, 1000 samples were randomly selected from the normal operation data as a normal baseline.

[0089] Step 8: Secondary verification and cleaning of industrial data Secondary verification and cleaning were performed on the labeled multimodal samples. Current, temperature, and location data were strictly aligned to the start time of the vibration data according to the timestamps to ensure the spatiotemporal consistency of the multimodal data. Simultaneously, the labeling results were cross-checked: at least two labeling engineers independently verified the stage division and RUL labels, and samples with inconsistent labeling were re-evaluated and confirmed to ensure labeling accuracy. Approximately 50 samples with alignment failures or questionable labeling were removed and included in the next stage.

[0090] Step 9: Industrial Data Privacy Protection Privacy protection measures are implemented for portions of the dataset that may contain sensitive information about equipment operating parameters. Equipment serial numbers, production line serial numbers, and operator information in maintenance work orders are anonymized, and the original identifiers are replaced using an irreversible hash algorithm. The raw sensor data retains vibration waveforms and operating parameters, and does not involve any personnel or supply chain sensitive information, ensuring that the data cannot be traced back to specific operators in subsequent circulation stages.

[0091] Step 10: Compliance and Integrity Verification All processed data undergoes compliance and integrity verification. Regarding compliance, the effectiveness of data anonymization is checked to ensure no sensitive information remains and that the data meets corporate data security management standards. Regarding integrity, each sample's field integrity (whether there are missing values), annotation coverage (whether all samples have been segmented and annotated with RUL), and multimodal alignment (whether the current, temperature, and position channels are all successfully aligned) are checked item by item.

[0092] Step 11: Classify and build a high-quality dataset The validated multimodal samples are classified according to application level to construct a general knowledge dataset and a specialized knowledge dataset.

[0093] (1) General knowledge dataset The dataset samples are derived from vibration and multimodal monitoring data of bearings in stamping line presses during four stages: normal operation, initial wear, intermediate wear, and late-stage deterioration. It covers normal and fault samples under different operating speeds (30 rpm, 45 rpm, and 60 rpm). This dataset does not limit itself to specific equipment models or fault types, but focuses on learning the general vibration representation of rotating machinery faults: including distinguishing between normal and abnormal states, identifying abnormal patterns in vibration amplitude and spectrum, and extracting general representations of time-frequency domain features. The general dataset comprises approximately 5000 labeled samples (approximately 2500 normal samples and approximately 2500 fault samples).

[0094] The core characteristics of knowledge datasets are broad coverage and strong generalization, providing a model starting point with industrial common sense for subsequent domain adaptation.

[0095] (2) Specialized knowledge dataset The specialized knowledge dataset, based on the general knowledge dataset, has undergone contextualized deep annotation and refined augmentation, specifically including the following three subsets: 1) Mechanism simulation subset: Based on the dynamic model of the crankshaft-bearing system of the press, a multibody dynamic simulation model with bearing clearance was established in Simulink. Different wear clearance amounts (0.05mm, 0.10mm, 0.15mm, 0.20mm) and speed conditions (30rpm, 45rpm, 60rpm) were set to simulate and generate vibration waveforms at different deterioration stages. After feature extraction and annotation, the waveforms were included in the specialized dataset, totaling 1200 simulation samples.

[0096] 2) Data augmentation subset: The time axis was stretched (scaling factor 0.9-1.1), the amplitude was scaled (0.8-1.2), and Gaussian white noise with a signal-to-noise ratio of 30dB was added to 200 real fault samples collected in actual operation. Each sample was expanded into 5 variants, and 1,000 augmented samples were added to improve the robustness of the model to operating condition fluctuations.

[0097] 3) Contextualized Labeling Subset: Using the "crankshaft bearing cage fracture" recorded in the maintenance work order as the baseline time of failure, all samples are finely labeled according to four degradation stages: Stage I (normal operation, RUL>600 hours), Stage II (early wear, BPFO sidebands appear, RUL=400-600 hours), Stage III (intermediate wear, kurtosis>5.0, RUL=80-400 hours), and Stage IV (late degradation, significant impact pulses, RUL<80 hours). Each sample fully includes vibration waveform, time-frequency domain features, current / temperature / position multimodal aligned data, stage label (health_label 0-1), and continuous RUL value label. This part contains approximately 1200 samples (including 200 actually collected samples and 1000 data-augmented samples).

[0098] The core features of the specialized dataset are scene specialization, fine annotation, and a complete labeling system, which directly serve the training, fine-tuning, and online inference deployment of a dedicated model for predicting press bearing failures.

[0099] The general knowledge dataset and the specialized knowledge dataset form a progressive data supply architecture of "pre-training-fine-tuning": the general knowledge dataset provides a broad range of basic capabilities for industrial equipment fault perception, enabling the model to possess common sense for general anomaly recognition; the specialized knowledge dataset, based on the general knowledge dataset, injects prior knowledge of fault evolution for specific equipment (press bearings) and fine annotations, achieving a leap from general perception to specialized prediction capabilities through transfer learning. Together, they constitute a high-quality dataset system for stamping scenarios in the automotive field, totaling approximately 6200 labeled samples, balancing the model's generalization ability and scenario adaptation accuracy.

[0100] Step 12: Dataset Evaluation (First Optimization: If unsatisfactory, return to Step 3) The constructed dataset underwent quality evaluation. Evaluation dimensions included: class balance (normal:faulty ≈ 1:1.07, meeting the standard), label consistency (200 randomly selected samples showed a labeling accuracy of 98.5%), feature separability (the ratio of inter-class distance to intra-class distance between normal and faulty classes in the frequency domain feature space > 2.0), and generated sample similarity (CGAN-generated samples were validated using DTW distance; low-quality samples were removed, and the remaining samples met the similarity standard). The evaluation revealed an issue: some simulated synthetic samples exhibited burrs on the vibration waveform boundaries. These samples were returned to step 3 for re-filtering and preprocessing before being included in the dataset.

[0101] Step 13: Dataset Packaging The qualified datasets are packaged according to industry-standard formats. They are stored uniformly in HDF5 file format, organized into "normal / fault" directories. Each sample contains three parts: the original waveform, feature vector, and label field, along with a data description (including metadata descriptions such as acquisition device parameters, sensor model, sampling frequency, preprocessing parameters, and annotation rules), ensuring that the datasets can be directly imported into mainstream deep learning frameworks for model training.

[0102] Step 14: Reliable Supply of High-Quality Datasets The packaged dataset is versioned and released through an enterprise-level data asset management platform, annotating it with information such as version number, release date, data size, and quality assessment report. This dataset is then distributed to the enterprise's internal R&D teams and authorized ecosystem partners, supporting downstream industrial model training and intelligent agent application deployment.

[0103] (iv) Industrial model training and industrial intelligent agent applications: Step 15: Industrial Model Training Based on the general and specialized datasets for 6600kN press bearing fault prediction constructed in step 11, a fault prediction model was trained using the TCN architecture. The model input consisted of the most recent 60 seconds of multi-sensor time-series data (vibration waveform downsampled to 1024 points, temperature sequence, and current sequence), and the outputs included binary classification of health status, multi-class classification of fault types, and RUL regression values. A "pre-training + fine-tuning" strategy was employed: first, 200 epochs of masked self-supervised pre-training was performed using three months of operating data from the press; then, supervised fine-tuning was performed using the labeled dataset until convergence. Validation set performance: health status classification accuracy 96.2%, fault type classification F1-score 0.89, and RUL prediction MAPE = 12.5%.

[0104] Step 16: Model inference generates candidate decisions The equipment engineer asked "What is the current status of the press?" through the intelligent agent interface. The model took the multi-sensor data of the equipment in the last hour as input and inferred and output candidate decisions: health score 0.73 (threshold 0.5), anomaly probability 0.81, judged as abnormal, the fault type is bearing wear, RUL is about 85 hours.

[0105] Step 17: Lightweight Neural Network Real-time Hallucination Detection The candidate decision input is a lightweight discriminator (MLP architecture, approximately 20,000 parameters). It takes the model confidence distribution, input signal-to-noise ratio, and distance between the working condition and the training set distribution as inputs, and outputs a confidence score of 0.92. If the score is higher than the threshold of 0.70, it passes the detection and proceeds to the next step.

[0106] Step 18: Deterministic Rule Engine Candidate decisions passed by hallucination detection are entered into the rule engine for verification of three hard rules: RUL is not less than 0 hours, a "critical" label is forcibly applied when the health score is <0.4 and the anomaly probability is >0.90, and the fault type is consistent with the RUL stage labeling logic. All results passed the verification.

[0107] Step 19: Industrial Intelligent Agent Application The intelligent agent uses Qwen2.5-72B as its base, is configured with equipment maintenance expert prompts, calls the MCP tool to obtain prediction results, and then queries the maintenance knowledge base to obtain maintenance SOPs.

[0108] Step 20: Feedback data reinjection to achieve closed-loop optimization (second optimization) The intelligent agent integrates the results to generate maintenance recommendations: "The crankshaft bearing of the press has moderate wear (health score 0.73, abnormal probability 81%), RUL approximately 85 hours, it is recommended to stop the machine for retesting within 48 hours, spare part NSK 23248CAE4." After engineer confirmation, a maintenance work order is generated and pushed to the CMMS system. The input data, model output, and maintenance feedback of this inference are fed back as feedback data to the data cleaning stage in step 3. After full-process processing, they are incorporated into the dataset for incremental training of the next version of the model, forming a continuous self-evolving closed loop of "inference-application-feedback-iteration".

[0109] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing high-quality industrial datasets for the automotive industry, comprising the following steps: A scenario-driven approach was adopted to select application scenarios in the automotive field. Collect automotive industry scenario data based on the stages involved in the selected application scenarios; The automotive industry scenario data is initially cleaned to retain complete industrial data with valid information. The complete industrial data is preprocessed to obtain a complete industrial dataset in a unified format; High-value seed scenes that meet the set conditions are extracted from the complete industrial dataset by means of preset screening rules, and the target attributes of the synthetic data are defined based on the high-value seed scenes. The data is synthesized by means of a hybrid generation strategy that integrates physical constraints based on the target attributes, and the synthetic data is converted into complete industrial data in a unified format and stored in the complete industrial dataset. The complete industrial dataset is annotated with industrial data scenarios to obtain an annotated sample set; The labeled sample set is then subjected to secondary verification and cleaning. Privacy protection is applied to the labeled sample set that has undergone secondary verification and cleaning to obtain a de-identified labeled sample set. Perform compliance and integrity checks on the anonymized labeled sample set; Using labeled samples that have been verified for compliance and completeness as input, and based on high-quality dataset classification standards and industry norms, general knowledge datasets and specialized knowledge datasets are formed. The general knowledge dataset and the specialized knowledge dataset were evaluated based on data completeness, annotation accuracy, category balance, working condition coverage, and target model adaptability. The datasets that meet the evaluation requirements are packaged to obtain high-quality industrial datasets corresponding to the selected application scenarios.

2. The method according to claim 1, characterized in that, The automotive industry scenario data includes equipment data, production line data, factory data, enterprise data, and supply chain ecosystem data.

3. The method according to claim 1, characterized in that, The method for preliminary cleaning of the automotive industry scenario data is as follows: S31. Data Asset Inventory and Cataloging: Register the automotive industry scenario data as assets, establish a data catalog list, and record the data source, collection time window, channel identifier, data scale and storage format of each batch of automotive industry scenario data to form a traceable data list. S32. Eliminate completely invalid samples: Perform an integrity pre-check on each record in the data catalog list and eliminate completely invalid samples. The completely invalid samples include, but are not limited to: empty frame data with all load values ​​being empty, fixed value records with all zero or full range generated when the acquisition channel is invalid or the sensor is offline, and records that cannot be parsed normally due to data index disorder or header information damage caused by storage abnormalities. S33. Data deduplication: Deduplicating duplicate samples stored repeatedly under the same sensor channel or the same acquisition timestamp; the duplicate sample refers to two samples with completely identical content, or two samples with adjacent timestamps and Euclidean distance or cosine similarity between feature vectors that is lower than a preset threshold. S34. Coarse-grained filtering of extreme abnormal data: The records are filtered. If the data in a record exceeds the value range corresponding to the data or the industry standard threshold, the record is deleted and candidate valid data is obtained. S35. Initial screening and labeling of missing samples: The missing rate of fields in the candidate valid data is statistically analyzed, and the candidate valid data with a missing rate exceeding the preset threshold are judged as severely incomplete samples and removed. For candidate valid data with a missing rate below a preset threshold, mark the location and missing pattern of the missing fields; S36. File integrity and readability verification: Perform file header verification, verification and comparison, and format specification checks on unstructured or semi-structured data stored in file form among the candidate valid data. Eliminate candidate valid data that fails the verification or check to obtain complete industrial data with retained valid information.

4. The method according to claim 3, characterized in that, The method for preprocessing the complete industrial data is as follows: S41. Format standardization processing: Perform unified format conversion and field mapping on the complete industrial data according to the preset automotive general data specifications to obtain general storage format data, and establish a mapping relationship table between the original fields and standard fields; S42. Refined processing of missing values: For complete industrial data marked with the location and pattern of missing fields after processing in S41, the missing value processing strategy is adaptively selected to fill in the missing values ​​according to the data type, missing rate and missing pattern of the missing fields. S43. Noise Smoothing and Signal Correction: Based on the signal type and noise characteristics, a differentiated filtering strategy is used to smooth and correct the complete industrial data processed by S42. S44, Multi-source data spatiotemporal alignment: Performs synchronization in the time dimension and coordinate registration in the spatial dimension on the complete industrial data after processing by S43; S45. Numerical Normalization and Standardization: For numerical characteristics of different dimensions and orders of magnitude in the complete industrial data after S4 processing, dimensionless transformation is performed while maintaining their physical meaning and distribution pattern. S46. Data Sharding and Sampling Equalization Preprocessing: Based on the requirements of downstream model training, the data transformed in S45 is organized and resampled to obtain a complete industrial dataset with a uniform format and a preset equalization ratio for the number of samples in each category.

5. The method according to claim 1, 2, or 3, characterized in that, The method for performing industrial data contextual annotation on the complete industrial dataset is as follows: S71. Construct a tagging system that integrates expert experience: Based on the automotive industry's design specifications, maintenance manuals, fault mode libraries, and process standards, and combined with expert experience, construct a multi-dimensional hierarchical tagging system for specific business scenarios. The multi-dimensional hierarchical tagging system includes equipment-level tags, process-level tags, quality-level tags, operating condition tags, and behavioral semantic tags. Each level is further refined into four levels of labelable atomic tags according to specific application goals, and the association relationship and mutual exclusion constraint rules between tags are established. At the same time, each tag is accompanied by a tagging description document, which clarifies the tag's business definition, judgment basis, tagging boundary, and typical positive and negative examples. S72. Precise annotation based on human-machine collaboration: For tags with high requirements for complex semantic understanding and professional judgment, a human-machine collaborative annotation mode is adopted to annotate complete industrial data; S73. Large-scale automated labeling based on semi-supervised and weakly supervised learning: For label items with large data volume, relatively clear labeling rules, and formalizable expert knowledge, a combination of semi-supervised learning and weakly supervised learning is used to perform large-scale automated labeling of complete industrial data. S74. Cross-modal joint annotation of multimodal data: For complex scenarios where complete industrial data simultaneously contains multiple modalities, a cross-modal joint annotation strategy is adopted to establish a consistency and complementarity association method between labels of different modalities to annotate the complete industrial data; the cross-modal joint annotation strategy uses business semantic labels as anchors and adapts corresponding annotation tools and annotation methods to different modalities respectively; the annotation results of each modality are associated through a unified sample ID and timestamp to form cross-modal label-aligned annotation samples; S75. Cross-source information association: Vertically associate labeled samples with equipment numbers, process parameter formulas, and quality inspection results in the production and manufacturing stages to establish a full-chain label traceability chain.

6. The method according to claim 1, 2, or 3, characterized in that, The method for secondary verification and cleaning of the labeled sample set is as follows: S81. Labeling error screening and removal: Systematically detect errors in the labeled samples in the labeled sample set, including hard error screening based on rule engine, automatically detect and mark labeled samples with logical conflicts, out-of-bounds labels and violations of business constraints; The annotation results are cross-validated using a validation model. Annotated samples whose predicted labels differ from human labels and whose confidence differences exceed a preset threshold are marked as suspected errors. An anomaly detection method using isolated forests is used to identify anomalous annotation samples whose annotation patterns deviate significantly from the mainstream distribution. Annotated samples marked as suspected errors and anomalous annotation samples are pushed to the manual review interface for annotation experts to make judgments one by one. S82. Annotation Consistency Verification: The annotation samples processed in S81 undergo multi-dimensional verification of data annotation consistency and stability. This consistency verification includes: Inter-annotator consistency assessment: Selecting annotation samples of the same type independently completed by different annotators from the same batch, calculating the Kappa coefficient of the annotation results to measure the consistency of different annotators' judgments on the same semantic label. When the consistency coefficient is lower than a preset threshold, the batch of annotation samples is re-annotated; Intra-annotator consistency verification: Selecting a certain proportion of annotation samples, re-annotating them after a period of time, calculating the consistency ratio of the annotation results before and after by the same annotator. If the consistency ratio is lower than a set threshold, the relevant annotation samples are re-annotated; Cross-modal consistency verification: Cross-comparing the labels of the same annotation object in different modal data to ensure semantic consistency across modal labels, and reviewing inconsistent annotation samples. S83. Low-quality labeled sample filtering: The quality of labeled samples is comprehensively scored and graded. The quality score of each labeled sample is calculated based on the labeling confidence, labeler proficiency, labeling time and labeling consistency. Low-quality samples with a quality score below the preset threshold are filtered out or returned for relabeling. At the same time, based on the labeling interaction behavior log recorded by the system during labeling, labeled samples with abnormal behavior during the labeling process are detected and removed. S84. Overall Data Balance Verification and Repair: Perform global-level distribution statistical analysis on the labeled samples processed by S81~S83, and repair the labeled samples based on the analysis results. The sample distribution, label distribution, and cross-distribution of each dimension of the labeled sample set all meet the corresponding preset balance requirements.

7. The method according to claim 1, characterized in that, The method for protecting the privacy of the labeled sample set after secondary verification and cleaning is as follows: S91. Sensitive Information Identification and Classification: Sensitivity assessment of each data field in the labeled sample after secondary verification and cleaning. Based on automotive industry data security regulations and the company's own data classification and grading standards, each data field is divided into different sensitivity levels. Corresponding desensitization strategies and processing strengths are formulated for each sensitivity level data field, and sensitivity level marking is completed in the data asset catalog. S92. Numerical Sensitive Data Perturbation Processing Based on Differential Privacy: For continuous numerical fields identified as important data levels, a differential privacy mechanism is used for data perturbation processing. Specifically, this includes: setting a privacy budget parameter ε for each numerical field, determining the noise scale parameter of the required Laplace mechanism based on the field sensitivity, calculating the amount of random noise added to the true value of the continuous numerical field, so that the processed value satisfies (ε,δ)-differential privacy guarantee. S93. Identity information desensitization based on de-identification and anonymization: For the identifier fields and quasi-identifier combinations that can identify vehicles, equipment and individuals in the labeled sample set of secondary verification and cleaning, hierarchical de-identification and anonymization processing is adopted.

8. A method for applying the high-quality industrial dataset constructed by the method of claim 1, comprising the following steps: The basic model of the automotive industry is trained using the general knowledge dataset in the high-quality dataset, and then the trained basic model of the automotive industry is trained and fine-tuned using the specialized knowledge dataset to obtain a special model for specific business scenarios. Based on the basic model of the automotive industry and the special model produced for specific business scenarios, an industrial intelligent agent is built by overlaying task planning, memory storage and tool calling modules. After being containerized and encapsulated, it is deployed in the edge or cloud inference environment and realizes real-time perception, inference and decision-making by connecting with business systems through API.

9. A method for applying the high-quality industrial dataset constructed by the method of claim 1, comprising the following steps: The basic model of the automotive industry is trained using the general knowledge dataset in the high-quality dataset, and then the trained basic model of the automotive industry is trained and fine-tuned using the specialized knowledge dataset to obtain a special model for specific business scenarios. The trained automotive industry basic model and the specific business scenario-based model are used as a cognitive engine to generate decision reports for the selected application scenarios with confidence scores. A lightweight discriminant model is used to perform real-time cross-validation of the decision report to ensure its authenticity. Based on the basic model of the automotive industry and the special model produced for specific business scenarios, an industrial intelligent agent is built by overlaying task planning, memory storage and tool calling modules. After being containerized and encapsulated, it is deployed in the edge or cloud inference environment and realizes real-time perception, inference and decision-making by connecting with business systems through API.

10. A method for applying the high-quality industrial dataset constructed by the method of claim 1, comprising the following steps: The basic model of the automotive industry is trained using the general knowledge dataset in the high-quality dataset, and then the trained basic model of the automotive industry is trained and fine-tuned using the specialized knowledge dataset to obtain a special model for specific business scenarios. The trained automotive industry basic model and the specific business scenario-based model are used as a cognitive engine to generate decision reports for the selected application scenarios with confidence scores. For equipment control and parameter setting in the automotive field, an unbypassable rule firewall is set up to physically verify the feasibility of candidate commands. Based on the basic model of the automotive industry and the special model produced for specific business scenarios, an industrial intelligent agent is built by overlaying task planning, memory storage and tool calling modules. After being containerized and encapsulated, it is deployed in the edge or cloud inference environment and realizes real-time perception, inference and decision-making by connecting with business systems through API.

Images