Full-link data management method and device, computer equipment and storage medium

Abstract

The invention relates to the field of data management, and discloses a full-link data management method and device, computer equipment and a storage medium. According to the full-link data management method, a vehicle repair corpus classification template is designed according to an application scene and a service demand, an acquisition scheme is formulated based on the classification template, vehicle diagnosis and maintenance data are acquired, and the data are managed. The method comprises the following steps: acquiring data of a vehicle, performing first preprocessing on the data to obtain standardized vehicle diagnosis and maintenance data, auditing the standardized vehicle diagnosis and maintenance data to obtain standard vehicle diagnosis and maintenance data, performing second preprocessing on the standard vehicle diagnosis and maintenance data to obtain data meeting requirements, storing the data in a database, and generating a vehicle maintenance corpus. According to the full-link data management method, data collection, cleaning, auditing and storage are covered, a full-link data management process is constructed, duties of all links are clearly divided, the operation process is clear and orderly, efficient development and standard execution of data management work are ensured, and the overall efficiency, quality and usability of data processing are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of data management technology, and in particular to a full-link data management method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the widespread application of large language models (such as GPT, BERT, etc.), data has become a core element in model development and application. In the large model training stage, high-quality data directly affects the accuracy, generalization ability and actual performance of the model, while in the model application stage, data is also an important support for achieving precise reasoning and intelligent output. It can be seen that data collection, development, management and application run through the entire life cycle of the large model, which is the key guarantee for improving model effects and implementing scenarios.

[0003] However, the data tools currently on the market mainly focus on the function of loading local data into vector databases, usually providing basic data storage and retrieval capabilities, but have the following shortcomings:

[0004] Lack of data collection task management: Existing tools fail to cover functions such as task allocation, source management, and collection standard formulation in the data collection stage, making it difficult to ensure the efficiency and quality of data collection.

[0005] Lack of data development and cleaning functions: Most data tools lack a normalized and standardized data processing mechanism, and are unable to efficiently pre-process, clean, and review raw data, making it difficult to generate high-quality data that can be used for large model training.

[0006] Insufficient corpus management and application: Current tools can usually only achieve simple data loading and storage, lack flexible management of structured data and corpus classification templates, and cannot meet the multi-scenario application requirements of data, such as intelligent retrieval of data supply and model reasoning interface calls.

[0007] Weak coordination and collaboration capabilities: Data tools lack unified management and process collaboration for the entire data lifecycle, and are unable to effectively coordinate the entire process from data collection to storage and then to application output.

[0008] Therefore, in response to the above problems, it is urgent to develop a full-link data management method and tool to achieve unified management of data collection, development, cleaning, storage and application, connect all links of data from source to application, and solve the defects of data tools in task management, quality control, corpus classification management and application docking, so as to provide high-quality data support for the research and development and practical application of large models, and further enhance their industry competitiveness and application effects. Summary of the invention

[0009] In view of this, the embodiments of the present application provide a full-link data management method, device, computer equipment and storage medium, which can effectively solve the problems in the prior art such as imperfect data management process, lack of systematic collection, cleaning, review and application mechanism, resulting in low data quality, insufficient standardization, and low efficiency of storage and external application.

[0010] In a first aspect, an embodiment of the present application provides a full-link data management method, including:

[0011] Design the classification template of auto repair corpus according to application scenarios and business needs;

[0012] Formulate a collection plan based on the auto repair corpus classification template to obtain vehicle diagnosis and maintenance data;

[0013] Performing a first preprocessing on the vehicle diagnosis and maintenance data to obtain standardized vehicle diagnosis and maintenance data;

[0014] Reviewing the standardized vehicle diagnosis and maintenance data to obtain standard vehicle diagnosis and maintenance data;

[0015] The standard vehicle diagnosis and maintenance data is preprocessed for the second time to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and store them in a database to generate an automobile repair corpus.

[0016] In some embodiments, the design of the auto repair corpus classification template according to the application scenario and business requirements includes:

[0017] Determining the target corpus content based on the application scenario;

[0018] According to the target corpus content, the corpus categories are divided, and corresponding classification standards are formulated for each corpus category to form the auto repair corpus classification template.

[0019] In some embodiments, formulating a collection plan based on the auto repair corpus classification template to obtain vehicle diagnosis and maintenance data includes:

[0020] Analyze and determine the collection source of the vehicle diagnosis and maintenance data according to the auto repair corpus classification template;

[0021] Evaluate the collection sources, formulate collection scope and collection standards, and generate collection tasks;

[0022] The collection tasks are distributed to corresponding devices for execution, so as to obtain the vehicle diagnosis and maintenance data from different collection sources.

[0023] In some embodiments, the first preprocessing of the vehicle diagnosis and maintenance data to obtain standardized vehicle diagnosis and maintenance data includes:

[0024] Based on a preset cleaning template, the vehicle diagnosis and maintenance data is formatted to obtain the vehicle diagnosis and maintenance data in a unified format;

[0025] The vehicle diagnosis and maintenance data in a unified format are input into a preset language model, and the standardized vehicle diagnosis and maintenance data are generated according to the format requirements and content requirements of the auto repair corpus classification template.

[0026] In some embodiments, the reviewing of the standardized vehicle diagnosis and maintenance data to obtain the standard vehicle diagnosis and maintenance data includes:

[0027] Verify the accuracy and completeness of the standardized vehicle diagnostic and maintenance data based on predetermined audit standards;

[0028] The standardized vehicle diagnosis and maintenance data that does not meet the audit standards is corrected or supplemented until the standardized vehicle diagnosis and maintenance data meets the audit standards, thereby generating the standard vehicle diagnosis and maintenance data.

[0029] In some embodiments, the second preprocessing of the standard vehicle diagnosis and maintenance data to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and storing it in a database to generate an auto repair corpus includes:

[0030] Vectorizing the standard vehicle diagnosis and maintenance data based on a vector model to obtain structured standard vehicle diagnosis and maintenance data;

[0031] According to a predetermined storage rule, the structured standard vehicle diagnosis and maintenance data is stored in a database to obtain the auto repair corpus.

[0032] In some embodiments, after the standard vehicle diagnosis and maintenance data is preprocessed for the second time to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and stores it in the database to generate the auto repair corpus, the method further includes:

[0033] Based on the user's query request, the query data input by the user is vectorized to generate a query vector;

[0034] Retrieving the standard vehicle diagnosis and maintenance data having a high similarity to the query vector from the auto repair corpus using a vector similarity algorithm;

[0035] According to the search results, the standard vehicle diagnosis and maintenance data with the highest matching degree is determined, and the standard vehicle diagnosis and maintenance data is output through a preset interface.

[0036] In a second aspect, an embodiment of the present application provides a full-link data management device, including:

[0037] Template design module, used to design auto repair corpus classification templates according to application scenarios and business needs;

[0038] A data acquisition module, used to formulate a collection plan based on the auto repair corpus classification template to acquire vehicle diagnosis and maintenance data;

[0039] A first preprocessing module, used for performing a first preprocessing on the vehicle diagnosis and maintenance data to obtain standardized vehicle diagnosis and maintenance data;

[0040] A data review module, used for reviewing the standardized vehicle diagnosis and maintenance data to obtain standard vehicle diagnosis and maintenance data;

[0041] The auto repair corpus generation module is used to perform a second preprocessing on the standard vehicle diagnosis and maintenance data to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and store them in the database to generate an auto repair corpus.

[0042] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the full-link data management method of the first aspect above.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, and when the computer program is executed on a processor, the full-link data management method of the above-mentioned first aspect is implemented.

[0044] The embodiments of the present application have the following beneficial effects:

[0045] The present application discloses a full-link data management method, device, computer equipment and storage medium. By designing a classification template for auto repair corpus according to application scenarios and business needs, the data is standardized and classified. A collection plan is formulated to efficiently obtain vehicle diagnosis and maintenance data. The vehicle diagnosis and maintenance data is formatted and standardized through the first preprocessing to solve the problems of inconsistent original data formats and redundancy. The accuracy and completeness of standardized vehicle diagnosis and maintenance data are ensured through review to obtain standard vehicle diagnosis and maintenance data. The standard vehicle diagnosis and maintenance data is vectorized and structured through the second preprocessing to meet the storage and retrieval requirements, and stored in the database, and finally a high-quality auto repair corpus is generated. The present solution starts from the full link of data management, including collection, cleaning, review, and warehousing, and realizes full-link data management. The method of the present application effectively solves the problems of lack of data collection task management, insufficient data standardization, and inconvenient external application of corpus, and improves the quality and availability of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 An application scenario diagram of a full-link data management method according to an embodiment of the present application is shown;

[0048] Figure 2 A flow chart of a full-link data management method according to an embodiment of the present application is shown;

[0049] Figure 3 A structural schematic diagram of a full-link data management device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0051] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0052] Hereinafter, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0053] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.

[0054] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0055] Considering the imperfect data management process in the existing technology, the lack of systematic collection, cleaning, review and application mechanisms, resulting in low data quality, insufficient standardization, and low efficiency in storage and external application. Therefore, a full-link data management method is proposed, which designs a classification template for auto repair corpus, formulates a collection plan, performs data cleaning and preprocessing on vehicle diagnosis and maintenance data, performs data quality review on standardized vehicle diagnosis and maintenance data, obtains vehicle diagnosis and maintenance data to be stored that meets the requirements, and stores the vehicle diagnosis and maintenance data that meets the requirements in the database, and finally realizes efficient management and external application of data, thereby realizing full-link data management of collection tasks, cleaning tasks, review tasks and warehousing tasks. On this basis, through the task dispatching process and automated workflow, it ensures that the division of responsibilities in each link is clear and the operation process is efficient and coordinated, which further improves the execution efficiency and management accuracy of data management and optimizes the quality and availability of data.

[0056] The present application provides a full-link data management method, which can be applied in the following situations: Figure 1 In the application environment, specifically, the full-link data management method of the present application is applied in a computer system, the system includes: Figure 1 The client and server shown in the figure communicate with each other through the network. The client, also known as the user end, refers to a program corresponding to the server that provides local services to the client. The client can be installed on, but not limited to, various personal computers, laptops, smart phones, tablet computers, and portable diagnostic devices (e.g., diagnostic boxes, etc.). The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0057] Figure 2 A flow chart of a full-link data management method according to an embodiment of the present application is shown.

[0058] Exemplarily, the full-link data management method includes the following steps:

[0059] Step S100, designing an auto repair corpus classification template according to application scenarios and business requirements.

[0060] Exemplarily, the application scenario refers to the specific usage environment and functional requirements that the auto repair corpus data is to serve, for example, including but not limited to: fault diagnosis scenarios, maintenance guidance scenarios, circuit detection scenarios, and accessories query scenarios. Business requirements are specific data requirements based on application scenarios. For example, including but not limited to: providing a complete corpus containing core content such as fault code definitions, fault code symptoms, maintenance instructions, circuit inspections, circuit detection, component testing, and maintenance guides. For example, the corpus needs to be accurate, complete, and standardized to meet the requirements of the maintenance guidance and diagnosis system, and the data needs to have a unified structure and format to facilitate rapid system calls and retrieval. The auto repair classification template refers to the data classification and content structure standards designed according to the application scenario and business needs, which are used to guide data collection, management, and application. For example, fault code corpus template, circuit diagram corpus template, and accessories description corpus template.

[0061] In one embodiment, in step S100, a classification template for auto repair corpus is designed according to application scenarios and business requirements, including:

[0062] The target corpus content is determined based on the application scenario.

[0063] For example, application scenarios may include: Fault diagnosis scenario: When the user enters a fault code, the system needs to return an explanation of the fault code and a repair plan to assist maintenance personnel in quickly locating the problem and troubleshooting. Maintenance guidance scenario: Provide maintenance engineers or intelligent maintenance assistants with standardized maintenance manual content, including circuit detection, component testing, and maintenance steps. Accessories query scenario: Provide accessory descriptions, parameters, models, and replacement guidance to ensure accurate selection of accessories during maintenance. According to the target corpus content, divide the corpus into categories, and formulate corresponding classification standards for each corpus category to form a classification template for auto repair corpus.

[0064] After determining the application scenario, determine what core information needs to be provided in the scenario, that is, the target corpus content, based on the application scenario, for example: Fault code scenario: fault code definition, fault symptoms, circuit detection, component test, maintenance guide, etc.; Circuit detection scenario: circuit diagram, signal test, fault point troubleshooting method, etc. Component test scenario: component name, specification, test standard and replacement steps. Accessory scenario: accessory name, accessory number, compatible vehicle model, parameter specifications, etc.

[0065] After determining the target corpus content, the data needs to be scientifically classified and organized into different corpus categories. Each category is divided based on the usage function and content attributes of the corpus. For example:

[0066] Fault code corpus: fault codes and their associated definitions, symptoms, detection methods, and repair guides.

[0067] Circuit diagram corpus: circuit structure diagrams, test points and signal detection methods.

[0068] Parts corpus: Parts specifications, test methods, installation and replacement procedures.

[0069] Maintenance manual corpus: standardized maintenance steps and processes.

[0070] Furthermore, the classification standard is a unified definition of the content structure, field requirements and format specifications for each corpus category. This ensures the integrity, consistency and availability of the data. For example, the classification standard includes: Field definition: the specific fields that each corpus category needs to include and their meanings. Format standard: the format in which data is stored (for example, JSON, XML or table structure). Content specification: the filling requirements of data content, such as required items, character length, terminology uniformity, etc.

[0071] After the target content requirements, corpus classification and classification standards are completed, the auto repair corpus classification template is finally formed. For example, the corpus classification template for fault codes includes: fault code, brand, model, year, circuit inspection, circuit test, component test, maintenance guide, etc. The auto repair corpus classification template is a standardized data framework designed for vehicle diagnosis and maintenance data, with clear classification structure and field requirements, providing a unified standard for data collection, cleaning, review and storage.

[0072] Step S200, formulate a collection plan based on the auto repair corpus classification template to obtain vehicle diagnosis and maintenance data.

[0073] Exemplarily, the collection plan refers to the development of a plan and execution process for acquiring vehicle diagnostic and maintenance data based on the content standards and data requirements defined in the auto repair corpus classification template. For example, the collection plan includes the following key elements: (1) Data sources: public data, cooperative channel data, offline data, purchase data, etc. (2) Collection content scope: fault code definition, fault phenomenon, circuit diagram, maintenance process, etc. Specify the coverage of the data, including dimensions such as brand, model, year, and system components. (3) Collection standards: define the format requirements of the data and establish data quality standards, for example: data fields must be complete, data content must be accurate, and unambiguous. (4) Task execution process: decompose the collection task into multiple subtasks and assign them to collection devices. Set the time node and priority for task completion to ensure the orderly progress of the collection work.

[0074] Data collection is performed by data collectors or automated tools according to the data sources and collection standards defined in the collection plan. For example: downloading vehicle maintenance manuals and fault code tables from a network database. Obtaining fault diagnosis records for a certain brand of vehicle through cooperative channels. Scanning, digitizing, and organizing offline paper documents into structured data.

[0075] The vehicle diagnosis and maintenance data finally obtained include: Fault code related information: Fault code number, definition, fault phenomenon. Circuit detection method: Circuit diagram, test steps and parameter standards. Component test information: Component test standards, detection methods and replacement instructions. Maintenance guidance information: Troubleshooting steps and solutions in the maintenance manual, etc.

[0076] In one embodiment, in step S200, a collection plan is formulated based on the auto repair corpus classification template to obtain vehicle diagnosis and maintenance data, including:

[0077] According to the classification template of auto repair corpus, analyze and determine the collection sources of vehicle diagnosis and maintenance data.

[0078] Demonstratively, under the guidance of the designed auto repair corpus classification template, we must first analyze and determine the data collection source, that is, the channel for obtaining vehicle diagnosis and maintenance data. Data collection sources can be divided into the following categories: (1) Public data: maintenance information publicly available on the Internet, such as technical documents on the official website of automobile manufacturers, online maintenance manuals, industry standard databases, etc. (2) Cooperation channel data: Cooperate with automobile manufacturers, maintenance organizations, and third-party data suppliers to obtain high-quality vehicle diagnosis and maintenance data. (3) Offline data: manuals provided by repair shops and technicians, paper maintenance records, etc., need to be digitized. (4) Purchased data: Purchase vehicle fault diagnosis and maintenance data sets through professional data suppliers.

[0079] For example, for the "vehicle fault code diagnosis" scenario, the collection sources may include: technical manuals published on the official website of the automobile manufacturer, fault code diagnosis records obtained in cooperation with repair shops, historical fault repair data provided by data suppliers, etc.

[0080] Evaluate the collection sources, establish the collection scope and collection standards, and generate collection tasks.

[0081] Specifically, reliable data sources are screened out by conducting feasibility and quality assessments on the identified sources of collection. Evaluation indicators include: Data accuracy: whether the source is credible and whether the data content is authoritative. Data integrity: whether the source can provide all the required information (e.g., fault codes, detection methods, repair guides, etc.). Data accessibility: whether the source is easy to access, for example, whether permissions, cooperation agreements, or payment fees are required. Data timeliness: whether the source data is updated in a timely manner and meets the current vehicle model and technical requirements.

[0082] Secondly, clarify the scope of data that needs to be collected, including: brand and model, production year, data category: fault code data, circuit detection data, component test data, maintenance manual data, etc. Then, formulate collection standards, which are used to standardize the format and quality of collected data, including: Data format: unified in JSON, XML or CSV format for subsequent processing. Content specification: Ensure field integrity, for example, the fault code must include "definition, symptoms, detection steps", etc. Data quality requirements: Ensure that the data is non-redundant, error-free, and meets accuracy and availability standards.

[0083] Finally, based on the collection source, collection scope and collection standards, generate specific collection tasks, including: Task objectives: what data to collect. Data source: specify the data source. Data standard: format, field content and quality requirements. Execution method: whether the task is executed automatically or requires human participation.

[0084] Distribute the collection tasks to the corresponding equipment for execution to obtain vehicle diagnosis and maintenance data from different collection sources.

[0085] Demonstratively, the collection tasks are distributed to the corresponding collection devices or tools to perform data collection. Task distribution methods include: Automated tools: For example, crawler programs, API call scripts, which are used to automatically collect data from network databases or cooperative channels. Manual operation: Data collectors manually obtain data from offline files and maintenance records and enter them into the system. Hybrid method: Some data are automatically collected, and some require manual verification and supplementation. Through task execution, vehicle diagnosis and maintenance data that meets the template requirements are obtained from different collection sources to provide support for subsequent data processing and storage.

[0086] Step S300, performing a first preprocessing on the vehicle diagnosis and maintenance data to obtain standardized vehicle diagnosis and maintenance data.

[0087] Exemplarily, the first preprocessing mainly solves the format, content and quality problems existing in the vehicle diagnosis and maintenance data, making the data standardized and consistent, laying a good foundation for subsequent review, storage and application. For example, the following operations are performed on the collected vehicle diagnosis and maintenance data: (1) Data formatting: Convert the vehicle diagnosis and maintenance data into a unified standard format (for example, JSON, XML, CSV, etc.). Deal with the problem of inconsistent data formats from different sources. (2) Data content standardization: Clean and organize the data content, remove redundant information, irrelevant content or erroneous data. Unify field naming, terminology standards and unit representation to ensure that the data content is standardized and consistent. (3) Data screening and deduplication: Deduplication is performed on the collected data, and duplicate content is deleted. Filter out data that meets the requirements of the auto repair corpus classification template, and eliminate irrelevant or non-compliant parts. (4) Field completion and verification: Complete the missing fields or mark them as pending. Verify the integrity and initial accuracy of the data to ensure that all necessary fields have values.

[0088] After the first preprocessing, the generated standardized vehicle diagnosis and maintenance data has the following characteristics:

[0089] Unified format: All data uses a consistent data format, such as JSON or XML.

[0090] Content standardization: Field names, terminology standards, and unit expressions have all been standardized, and the data content is clear and unambiguous.

[0091] Improved integrity: Missing parts of the data have been completed or marked to ensure data integrity and availability.

[0092] Redundant data cleaning: Duplicate and invalid data are eliminated, making the data more concise.

[0093] In one embodiment, in step S300, the vehicle diagnosis and maintenance data is preprocessed for the first time to obtain standardized vehicle diagnosis and maintenance data, including:

[0094] Based on the preset cleaning template, the vehicle diagnosis and maintenance data are formatted to obtain vehicle diagnosis and maintenance data in a unified format.

[0095] Exemplarily, the cleaning template is a pre-set data processing specification and standard rules used to guide the formatting of data and content normalization. Cleaning templates are usually designed by business experts and model experts based on application requirements and corpus templates, including: data field requirements (e.g., fault code, fault definition, repair steps, etc.), data format standards (e.g., JSON, XML, CSV), content specification requirements (e.g., terminology standardization, unit unification, mandatory item settings, etc.).

[0096] Standardizing the data format of the collected vehicle diagnosis and maintenance data according to the cleaning template can solve the problems of inconsistent data format, incomplete or redundant fields, and inconsistent terminology and units. Then, the vehicle diagnosis and maintenance data with unified format is input into the preset language model, and the standardized vehicle diagnosis and maintenance data is generated according to the format requirements and content requirements of the auto repair corpus classification template.

[0097] The vehicle diagnosis and maintenance data with unified format is input into the preset language model, and the standardized vehicle diagnosis and maintenance data is generated according to the format requirements and content requirements of the auto repair corpus classification template.

[0098] For example, a language model refers to a large-scale pre-trained natural language processing model (for example, GPT, BERT, etc.), which automatically generates or supplements structured and standardized data content based on input data and preset rules. The auto repair corpus classification template defines the format and content requirements of vehicle diagnosis and maintenance data as a guiding standard for the output results of the language model. The formatted data is passed as input to the language model. According to the requirements of the auto repair corpus classification template, the language model automatically completes, normalizes, and organizes the data content to generate standardized vehicle diagnosis and maintenance data that meets the standards.

[0099] For example, the input parameters may be fault code and maintenance manual, and the language model outputs the following parameters based on the format and content requirements of the auto repair classification template: fault code, brand, model, year, circuit inspection, circuit test, component test, and maintenance guide.

[0100] Furthermore, after formatting and automatic language model generation, the standardized vehicle diagnosis and maintenance data finally obtained has the following characteristics:

[0101] Unified format: All data meets the format requirements of the template and is suitable for system storage and call.

[0102] Content completeness: Missing fields (e.g., fault symptoms, repair instructions, etc.) are automatically completed and standardized.

[0103] Standardized output: Conforms to the requirements of the auto repair corpus classification template, and the content is standardized and accurate.

[0104] Strong applicability: It provides high-quality data support for subsequent review, storage and application (for example, intelligent diagnosis and maintenance guidance systems).

[0105] Step S400, reviewing the standardized vehicle diagnosis and maintenance data to obtain standard vehicle diagnosis and maintenance data.

[0106] For example, the specific operations for reviewing standardized vehicle diagnosis and maintenance data include the following aspects: (1) Field integrity check: Ensure that each piece of data contains all the required fields required by the template, such as fault code, fault definition, fault symptom, circuit detection method, component test standard, and maintenance guide; (2) Content accuracy verification: Compare whether the data content meets the actual business standards. For example: Is the definition of the fault code consistent with the information in the industry standard or maintenance manual? Do the fault symptoms, detection steps, and maintenance guide have correct logic and operation instructions? For example: In the original data, the fault definition of "P0123" is "circuit fault". The review found that the content is too general and needs to be corrected to a specific definition. (3) Data consistency check: Unify terms, formats, and units: Ensure that all data uses standard terms, units, and formats. (4) Data compliance check: Ensure that the data source is reliable and meets business specifications, especially third-party data or manually entered data, which needs to be verified twice. (5) Review feedback and correction: Correct or supplement the data problems found during the review process. If some data cannot be corrected, it needs to be marked as abnormal and not included in the standard data set.

[0107] In one embodiment, in step S400, the standardized vehicle diagnosis and maintenance data is reviewed to obtain standard vehicle diagnosis and maintenance data, including:

[0108] Based on the predetermined audit standards, the accuracy and completeness of the standardized vehicle diagnostic and maintenance data are verified; the standardized vehicle diagnostic and maintenance data that does not meet the audit standards are corrected or supplemented until the standardized vehicle diagnostic and maintenance data meets the audit standards and standard vehicle diagnostic and maintenance data are generated.

[0109] Among them, the audit standard is a data quality assessment rule set in advance, which is used to verify the standardized data and ensure that the data meets the quality requirements. Through automated verification tools or manual verification, the standardized vehicle diagnosis and maintenance data is verified item by item to check whether it meets the audit standards. If problems are found during the audit process, the following processing can be performed: (1) Correct the incorrect content: Correct the inaccurate or non-standard field content. For example: Replace the incorrect fault code definition with the correct standard definition. (2) Supplement missing content: Supplement and improve the missing fields or content. For example: Supplement the specific description of the "fault symptoms" field, such as "unstable idle speed, weak acceleration". (3) Re-verification: Re-verify the corrected or supplemented data to ensure that it meets the audit standards. (4) Iteration of the correction process: If the corrected or supplemented data still does not fully meet the standards, repeat the correction process until the standardized vehicle diagnosis and maintenance data passes the audit and generates standard vehicle diagnosis and maintenance data.

[0110] Step S500 , pre-processing the standard vehicle diagnosis and maintenance data for the second time, obtaining the vehicle diagnosis and maintenance data to be stored that meets the requirements and storing it in a database to generate an automobile repair corpus.

[0111] Demonstratively, the second preprocessing is to perform deep processing and structural optimization of standard vehicle diagnosis and maintenance data, such as vectorization, data compression, deduplication, field supplementation, format standardization and other operations, so as to make the data more standardized and structured, and meet the design requirements of database storage, and generate corresponding vector representations to improve data retrievability and consistency.

[0112] Subsequently, the data to be stored that meets the requirements after the second preprocessing is imported into the database according to the predetermined storage rules. These storage rules may include field structure definition, index creation, data format standardization, and storage method selection, for example, relational databases are used to store field data, and vector databases are used to store vector data.

[0113] Finally, these data are stored in the database to form the auto repair corpus, a standardized and structured vehicle diagnosis and maintenance data set. The auto repair corpus can not only support fast data retrieval, but also provide high-quality data support for intelligent diagnosis systems, fault analysis platforms, etc., to meet the actual application needs of vehicle maintenance, fault troubleshooting, etc.

[0114] In one embodiment, in step S500, the standard vehicle diagnosis and maintenance data is preprocessed for the second time to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and store them in the database to generate the auto repair corpus, including:

[0115] The standard vehicle diagnosis and maintenance data are vectorized based on the vector model to obtain structured standard vehicle diagnosis and maintenance data.

[0116] Exemplarily, the vector model is a commonly used model in deep learning, which is used to convert unstructured data into a computable vector representation. In data processing, common tabular data (for example, fields in a database) is called structured data, which has a fixed format and is easy to process and store. Unstructured data (for example, text, pictures, audio, video, etc.) cannot be directly applied to traditional structured processing methods due to its complex form. Therefore, it is necessary to convert these data into a form suitable for calculation and analysis, a process called vectorization.

[0117] Vectorization is the process of converting unstructured data into multi-dimensional floating point arrays, which can be used to represent the semantics and features of the data. For example, the vectorized array may be in the form of [0.3333, 0.1224, 0.3222, ...], where each value represents the measure of the semantic features of the data in the corresponding dimension. The computational model used by vectorization is called a vector model, also called an embedding model, which extracts semantic information from data through deep learning algorithms and generates high-dimensional vector representations.

[0118] During the vectorization process, standard vehicle diagnostic and maintenance data (e.g., fault codes, fault definitions, maintenance guides, etc. in the form of text, images, audio, video, etc.) are usually vectorized in a block-by-block manner. For example, an article is divided into multiple blocks by paragraphs, and each block is vectorized independently to generate multiple sets of corresponding floating-point vectors. This processing method can preserve the semantic features of the data and facilitate subsequent storage, retrieval, and analysis.

[0119] According to the predetermined storage rules, the structured standard vehicle diagnosis and maintenance data are stored in the database to obtain the auto repair corpus.

[0120] Demonstratively, standard vehicle diagnosis and maintenance data are stored in the database after vectorization and structured processing. The data is converted into a multi-dimensional floating point form (vector representation), which highly abstracts the semantic characteristics of the data. Unstructured content such as fault codes, fault definitions and maintenance guides are stored in the form of high-dimensional floating point vectors, which is different from the text content stored in traditional databases. After the data is stored in the database, an auto repair corpus is formed, which contains standardized and structured vehicle diagnosis and maintenance data, providing support for semantic retrieval and intelligent applications.

[0121] It is worth noting that the content stored in the database is not visible text or fields in the traditional sense, but high-dimensional vector data. The fault code definition and its corresponding content are converted into floating-point numbers and stored in the database. Therefore, the data in the database is invisible and only exists in the form of a huge number of floating-point numbers. These vector data do not directly correspond to the original field content, but are multi-dimensional abstract representations of its semantic features.

[0122] In one embodiment, after the standard vehicle diagnosis and maintenance data is preprocessed for the second time to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and stores it in the database to generate the auto repair corpus, the method further includes:

[0123] Based on the user's query request, the query data input by the user is vectorized to generate a query vector.

[0124] Exemplarily, when a user initiates a query request, the query data entered by the user, such as fault description, keywords, fault code, etc., is vectorized, that is, the query content is converted into a set of numerical vectors, that is, a set of multi-dimensional floating-point arrays, through a preset vector model. This vector representation can capture the semantic information of the query content, allowing the system to understand the deep meaning of the query content.

[0125] Using the vector similarity algorithm, standard vehicle diagnosis and maintenance data with high similarity to the query vector is retrieved in the auto repair corpus.

[0126] Exemplarily, since the standard vehicle diagnosis and maintenance data stored in the auto repair corpus has been vectorized, each block of data corresponds to a high-dimensional floating point vector. Therefore, through a vector similarity algorithm (e.g., cosine similarity or Euclidean distance), the similarity between the query vector and each stored vector can be calculated in the auto repair corpus, and the block vectors with the highest similarity can be retrieved.

[0127] According to the search results, the standard vehicle diagnosis and maintenance data with the highest matching degree is determined, and the standard vehicle diagnosis and maintenance data is output through a preset interface.

[0128] Specifically, by sorting the similarities of the retrieved block vectors, the vector blocks with the highest matching degree are determined. These vector blocks correspond to the original content in the auto repair corpus. The vector blocks can be restored to readable vehicle diagnosis and maintenance data through mapping, and then a large model (for example, a deep learning model, an AI model, etc.) is called to organize and process the vehicle diagnosis and maintenance data. Then, the generated more logical, coherent and professional output results are returned or displayed to the user through the API interface (i.e., application programming interface).

[0129] It is worth noting that throughout the data management process, this application has achieved efficient and accurate data management through automated task dispatching and workflow mechanisms. By setting the completion time of each workflow and tracking the progress in real time, it ensures the orderly connection between the processes, significantly improving the execution efficiency of the data management link.

[0130] The full-link data management method of the embodiment of the present application, by combining application scenarios and business needs, designs an auto repair corpus classification template, completes the standardized classification of data, and formulates a collection plan to obtain vehicle diagnosis and maintenance data; by performing the first preprocessing of the collected data, the data is formatted and normalized to solve the problems of inconsistent original data formats and redundancy; the accuracy and completeness of the data are verified through the audit link to obtain standard vehicle diagnosis and maintenance data; further through the second preprocessing, the standard data is vectorized and structured to meet the requirements of storage and retrieval, and finally stored in the database to build a high-quality auto repair corpus. This solution revolves around the full-link process of data management, covering data collection, cleaning, review and warehousing. The method of the present application realizes closed-loop management of data. The quality and availability of data have been greatly improved.

[0131] This application also proposes a full-link data management device, such as Figure 3 As shown, the full-link data management device includes:

[0132] The template design module 31 is used to design the classification template of the auto repair corpus according to the application scenario and business requirements;

[0133] The data acquisition module 32 is used to formulate a collection plan based on the auto repair corpus classification template to obtain vehicle diagnosis and maintenance data;

[0134] A first preprocessing module 33 is used to perform a first preprocessing on the vehicle diagnosis and maintenance data to obtain standardized vehicle diagnosis and maintenance data;

[0135] The data review module 34 is used to review the standardized vehicle diagnosis and maintenance data to obtain standard vehicle diagnosis and maintenance data;

[0136] The auto repair corpus generation module 35 is used to perform a second preprocessing on the standard vehicle diagnosis and maintenance data to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and store them in the database to generate an auto repair corpus.

[0137] It can be understood that the device of this embodiment corresponds to the method of the above embodiment, and the options in the above embodiment are also applicable to this embodiment, so they will not be described repeatedly here.

[0138] The present application also provides a computer device. Exemplarily, the computer device includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the device to execute the functions of each module in the above-mentioned full-link data management method or the above-mentioned full-link data management device.

[0139] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU) and a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application.

[0140] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.

[0141] The present application also provides a computer-readable storage medium for storing the computer program used in the above-mentioned computer device. For example, the computer-readable storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0142] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0144] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application.

[0145] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A full-link data management method, characterized in that: The method comprises: Design the classification template of auto repair corpus according to application scenarios and business needs; Formulate a collection plan based on the auto repair corpus classification template to obtain vehicle diagnosis and maintenance data; Performing a first preprocessing on the vehicle diagnosis and maintenance data to obtain standardized vehicle diagnosis and maintenance data; Reviewing the standardized vehicle diagnosis and maintenance data to obtain standard vehicle diagnosis and maintenance data; The standard vehicle diagnosis and maintenance data is preprocessed for the second time to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and store them in a database to generate an automobile repair corpus.

2. The full-link data management method according to claim 1, characterized in that: According to the application scenario and business requirements, the auto repair corpus classification template is designed, including: Determining the target corpus content based on the application scenario; According to the target corpus content, the corpus categories are divided, and corresponding classification standards are formulated for each corpus category to form the auto repair corpus classification template.

3. The full-link data management method according to claim 1, characterized in that: The step of formulating a collection plan based on the auto repair corpus classification template to obtain vehicle diagnosis and maintenance data includes: Analyze and determine the collection source of the vehicle diagnosis and maintenance data according to the auto repair corpus classification template; Evaluate the collection sources, formulate collection scope and collection standards, and generate collection tasks; The collection tasks are distributed to corresponding devices for execution, so as to obtain the vehicle diagnosis and maintenance data from different collection sources.

4. The full-link data management method according to claim 1, characterized in that: The first preprocessing of the vehicle diagnosis and maintenance data to obtain standardized vehicle diagnosis and maintenance data includes: Based on a preset cleaning template, the vehicle diagnosis and maintenance data is formatted to obtain the vehicle diagnosis and maintenance data in a unified format; The vehicle diagnosis and maintenance data in a unified format are input into a preset language model, and the standardized vehicle diagnosis and maintenance data are generated according to the format requirements and content requirements of the auto repair corpus classification template.

5. The full-link data management method according to claim 1, characterized in that: The standardized vehicle diagnosis and maintenance data is reviewed to obtain standard vehicle diagnosis and maintenance data, including: Verify the accuracy and completeness of the standardized vehicle diagnostic and maintenance data based on predetermined audit standards; The standardized vehicle diagnosis and maintenance data that does not meet the audit standards is corrected or supplemented until the standardized vehicle diagnosis and maintenance data meets the audit standards, thereby generating the standard vehicle diagnosis and maintenance data.

6. The full-link data management method according to claim 1, characterized in that: The method of performing a second preprocessing on the standard vehicle diagnosis and maintenance data to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and storing the data in a database to generate an automobile repair corpus includes: Vectorizing the standard vehicle diagnosis and maintenance data based on a vector model to obtain structured standard vehicle diagnosis and maintenance data; According to a predetermined storage rule, the structured standard vehicle diagnosis and maintenance data is stored in a database to obtain the auto repair corpus.

7. The full-link data management method according to claim 1, characterized in that: After the standard vehicle diagnosis and maintenance data is preprocessed for the second time to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and stores it in the database to generate the auto repair corpus, the method further includes: Based on the user's query request, the query data input by the user is vectorized to generate a query vector; Retrieving the standard vehicle diagnosis and maintenance data having a high similarity to the query vector from the auto repair corpus using a vector similarity algorithm; According to the search results, the standard vehicle diagnosis and maintenance data with the highest matching degree is determined, and the standard vehicle diagnosis and maintenance data is output through a preset interface.

8. A full-link data management device, characterized in that: The device comprises: Template design module, used to design auto repair corpus classification templates according to application scenarios and business needs; A data acquisition module, used to formulate a collection plan based on the auto repair corpus classification template to acquire vehicle diagnosis and maintenance data; A first preprocessing module, used for performing a first preprocessing on the vehicle diagnosis and maintenance data to obtain standardized vehicle diagnosis and maintenance data; A data review module, used for reviewing the standardized vehicle diagnosis and maintenance data to obtain standard vehicle diagnosis and maintenance data; The auto repair corpus generation module is used to perform a second preprocessing on the standard vehicle diagnosis and maintenance data to obtain the vehicle diagnosis and maintenance data to be stored that meets the requirements and store them in the database to generate an auto repair corpus.

9. A computer device, characterized in that: The computer device includes a processor and a memory, the memory stores a computer program, and the processor is used to execute the computer program to implement the full-link data management method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: It stores a computer program, which, when executed on a processor, implements the full-link data management method according to any one of claims 1-7.

Images