Data quality detection method, device, storage medium, and program product

Abstract

The application provides a data quality detection method, device, storage medium and program product, and relates to the technical field of Internet of Vehicles. The data quality detection method is applied to a cloud end and comprises the following steps: receiving target data sent by a vehicle end; performing data quality detection on the target data according to at least one preset check rule to obtain data quality detection results corresponding to each check rule, wherein each check rule represents a data feature when corresponding links in an entire link from vehicle end data generation to cloud end data reception are free of faults; and determining the data quality of the target data according to the data quality detection results corresponding to each check rule. The application is used to improve the accuracy of data quality detection.

Description

Technical Field

[0001] This application relates to the field of vehicle networking technology, and in particular to a data quality detection method, device, storage medium and program product. Background Technology

[0002] With the development of vehicle-to-everything (V2X) technology, more and more vehicle-related applications are deployed in the cloud, such as autonomous driving model training and remote vehicle diagnostics. The operation of these applications depends on the reliability of the data uploaded from the vehicle to the cloud. However, during data transmission, factors such as network interference, protocol parsing errors, and vehicle controller malfunctions can lead to data loss, content discrepancies, or timestamp asynchronization, directly affecting the reliability of cloud data and consequently the reliability of applications.

[0003] In related technologies, data consistency is detected by comparing data sent from the vehicle and data received from the cloud, and the missing rate of data received from the cloud is calculated to detect data integrity. The reliability of data quality is then detected through data consistency and integrity.

[0004] However, when testing the reliability of data quality using the above methods, there are instances where the data quality test results are inaccurate. Summary of the Invention

[0005] This application provides data quality detection methods, devices, storage media, and program products to improve the accuracy of data quality detection.

[0006] In a first aspect, embodiments of this application provide a data quality detection method, applied in the cloud, comprising:

[0007] Receive target data sent by the vehicle terminal;

[0008] According to at least one preset verification rule, the target data is subjected to data quality detection, and the data quality detection results corresponding to each verification rule are obtained. At least one verification rule represents the data characteristics when there is no fault in the corresponding link in the entire link from the generation of vehicle data to the reception of cloud data.

[0009] The data quality of the target data is determined based on the data quality detection results corresponding to each verification rule.

[0010] In one possible implementation, the data quality of the target data is determined based on the data quality detection results corresponding to each verification rule, including:

[0011] If the data quality test results corresponding to each verification rule indicate that there is no fault in the corresponding link, it is determined that the data quality of the target data meets the data quality requirements.

[0012] If the data quality test result corresponding to at least one verification rule indicates that there is a fault in the corresponding link, determine whether the target data can be repaired based on the fault information corresponding to the fault.

[0013] If the target data is repairable, perform data repair on the target data and determine that the data quality of the target data meets the data quality requirements;

[0014] If the target data is irreparable, the data quality of the target data is determined to be non-compliant with data quality requirements.

[0015] In one possible implementation, determining whether the target data is repairable based on the fault information corresponding to the fault includes:

[0016] Based on the fault information corresponding to the fault, query the data cleaning template library to see if there is a data cleaning template corresponding to the fault information.

[0017] If a data cleaning template corresponding to the fault information exists, it can be determined that the target data is repairable.

[0018] If no data cleaning template corresponding to the fault information exists, the target data is determined to be unrepairable.

[0019] Correspondingly, data repair is performed on the target data, including: applying the cleaning template corresponding to the fault information to repair the target data.

[0020] In one possible implementation, it also includes:

[0021] If the data quality test result corresponding to at least one verification rule indicates that there is a fault in the corresponding link, the target data and the fault information corresponding to the target data will be stored in the structured fault database.

[0022] Based on the fault information, cluster analysis is performed on the data in the structured fault database to obtain the data cleaning templates corresponding to the fault information, and the fault information and data cleaning templates are stored in the data cleaning template library.

[0023] In one possible implementation, the validation rules are described using a structured query language;

[0024] And / or, according to at least one preset verification rule, perform data quality detection on the target data and obtain the data quality detection results corresponding to each verification rule, including: applying a distributed computing engine to scan the target data for data quality detection according to at least one preset verification rule and obtain the data quality detection results corresponding to each verification rule.

[0025] In one possible implementation, the verification rules are obtained in the following way:

[0026] Using test data, control at least one link in the link from vehicle-side data generation to cloud-side data reception to ensure that there is a fault, and receive the cloud data corresponding to the test data in the cloud.

[0027] Extract data features from cloud data and generate verification rules based on data features and fault information.

[0028] In one possible implementation, the verification rules can be obtained in the following way:

[0029] Obtain the fault information uploaded by the user and the corresponding data features, and generate verification rules based on the fault information and the corresponding data features.

[0030] Secondly, embodiments of this application provide a data quality detection device applied in the cloud, comprising:

[0031] The receiving module is used to receive target data sent by the vehicle.

[0032] The detection module is used to perform data quality detection on the target data according to at least one preset verification rule, and obtain the data quality detection results corresponding to each verification rule. At least one verification rule represents the data characteristics when there is no fault in the corresponding link in the entire link from the generation of vehicle-side data to the reception of cloud-side data.

[0033] The determination module is used to determine the data quality of the target data based on the data quality detection results corresponding to each verification rule.

[0034] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0035] The memory stores the instructions that the computer executes;

[0036] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0038] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed, implements the first aspect and / or various possible implementations of the first aspect.

[0039] The data quality detection method, device, storage medium, and program product provided in this application verify the target data sent by the receiving end by using at least one verification rule. Each verification rule represents the data characteristics when there are no faults in the corresponding link of the entire link from data generation at the vehicle end to data reception at the cloud end. This achieves full-link, multi-level quality detection from data generation to data reception. Based on the data quality detection results corresponding to each verification rule, the data quality of the target data is determined, enabling more comprehensive data quality detection and improving the accuracy of data monitoring. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0041] Figure 1 A schematic diagram of the data quality detection process provided in this application embodiment;

[0042] Figure 2 This is a schematic diagram of the data quality inspection result determination process provided in the embodiments of this application;

[0043] Figure 3 This is a schematic diagram of the data quality detection device provided in the embodiments of this application;

[0044] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0045] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0046] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0047] Data reliability is a prerequisite for cloud service reliability. Current technologies only assess data quality at the vehicle-to-cloud communication level, meaning they can only identify data quality issues caused by communication errors between the vehicle and the cloud. However, data transmission from the vehicle to the cloud involves multiple stages, including data generation, processing, transmission, and reception. Current technologies do not cover the entire data chain, leading to inaccurate data quality assessments. For example, if data is generated at the vehicle but malfunctions during processing, resulting in data errors, while vehicle-to-cloud data transmission and cloud-to-cloud reception are normal, current technologies may judge the data as consistent and complete, deeming it reliable. However, the data quality may actually be unreliable due to errors in the processing stage, resulting in inaccurate data quality assessment.

[0048] The data quality inspection method provided in this application performs data quality inspection on target data by using verification rules that represent the data characteristics of corresponding links in the entire link from vehicle-side data generation to cloud-side data reception, thereby achieving full-link and multi-layer coverage of data quality inspection and making data quality inspection more accurate and comprehensive.

[0049] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0050] Figure 1 This is a schematic diagram of the data quality detection process provided in an embodiment of this application. Figure 1 As shown, when applied in the cloud, the method includes:

[0051] S101, Receive target data sent by the vehicle terminal.

[0052] Specifically, an application for processing target data is deployed in the cloud. The vehicle uses a sending module to encapsulate the raw data using protocols and transmit it over the network. The cloud uses a receiving module to receive the target data corresponding to the raw data.

[0053] S102. According to at least one preset verification rule, perform data quality detection on the target data and obtain the data quality detection results corresponding to each verification rule. At least one verification rule represents the data characteristics when there is no fault in the corresponding link in the entire link from the generation of vehicle-side data to the reception of cloud-side data.

[0054] The process from vehicle-side data generation to cloud-side data reception involves multiple stages, including data generation, data processing, data transmission, and data reception. A concrete example is the process of uploading battery pack voltage data from the vehicle to the cloud, which involves the following steps: Stage 1: The Battery Management System (BMS) collects the voltage of the cells in the battery pack. For example, if the battery pack contains 64 cells, the cell voltages are collected sequentially and transmitted to the Microcontroller Unit (MCU) via a bus. Stage 2: The MCU arranges and packages the multiple cell voltages into a voltage data packet and sends it to the System-on-Chip (SoC). Stage 3: The SoC receives the voltage data packet, adds up the cell voltages to obtain the battery pack voltage, and sends it to the T-box. Stage 4: The T-box encrypts and packages the battery pack voltage and sends it to the receiving module in the cloud via a network protocol. Stage 5: The receiving module in the cloud receives the data packet, parses it, and obtains the battery pack voltage data.

[0055] In the complete processing and transmission chain described above, any link may fail, causing the target data received by the cloud to fail to meet the data quality requirements. If the target data that does not meet the data quality requirements is input into the cloud application, it will reduce the reliability of the cloud application.

[0056] This application's embodiments derive verification rules based on data characteristics assuming no faults in the corresponding links throughout the entire data chain from vehicle-side data generation to cloud-side data reception. These verification rules are then used to perform data quality checks on the target data, achieving comprehensive data quality inspection across the entire chain from generation to processing and transmission, ensuring thorough and complete data quality monitoring. Furthermore, by verifying whether the target data conforms to the data characteristics assuming no faults in the corresponding links, it can be determined whether a fault has occurred in that link, enabling rapid fault attribution and location.

[0057] Data characteristics are those related to specific processes, such as data increasing, decreasing, or data falling within a certain range, etc.

[0058] S103. Determine the data quality of the target data based on the data quality detection results corresponding to each verification rule.

[0059] Specifically, if the data quality test result indicates that the target data conforms to the data characteristics of a faultless link in the entire chain from vehicle-side data generation to cloud-side data reception, then the data quality of the target data is determined to meet the data quality requirements; otherwise, it does not meet the data quality requirements.

[0060] In one optional implementation, when the target data does not conform to the data characteristics of a fault-free link in the entire chain from vehicle-side data generation to cloud-side data reception, fault information of the target data is determined. Based on the fault information, it is determined whether the target data can be repaired. If it can be repaired, the target data is repaired, and the repaired data is determined to meet the data quality requirements. For target data that cannot be repaired, it is filtered to prevent it from being input into the cloud application and causing abnormal operation of the cloud application.

[0061] The data quality detection method provided in this application uses verification rules that represent data characteristics when there are no faults in the corresponding links of the entire link from vehicle-side data generation to cloud-side data reception to perform data quality detection on target data. This enables full-link, all-round data quality detection, avoids blind spots and dead zones in data quality detection, and improves the accuracy of data quality detection.

[0062] Figure 2 This is a schematic diagram illustrating the data quality inspection result determination process provided in an embodiment of this application. Figure 2 As shown, in one possible implementation, the data quality of the target data is determined based on the data quality detection results corresponding to each verification rule, including:

[0063] S201. Obtain the data quality detection results corresponding to each verification rule.

[0064] Specifically, the data quality test results indicate that the target data conforms to the data characteristics corresponding to the verification rules, meaning that there is no fault in the corresponding link; or, the data quality test results indicate that the target data does not conform to the data characteristics corresponding to the verification rules, meaning that there is a fault in the corresponding link.

[0065] S202. Determine whether the data quality detection results corresponding to each verification rule indicate that there is no fault in the corresponding link.

[0066] If yes, proceed to step S203; otherwise, proceed to step S204.

[0067] If yes, it indicates that there are no faults in the entire link, the target data is accurate, and it meets the data quality requirements. If no, it indicates that there is at least one fault in the entire link, causing the target data to be inaccurate, and further assessment is needed to determine whether the target data can be repaired.

[0068] S203. Ensure that the data quality of the target data meets the data quality requirements.

[0069] Using target data that meets data quality requirements as the data source for cloud applications ensures the reliability of cloud applications.

[0070] S204. Determine whether the target data is repairable.

[0071] If yes, proceed to step S205; otherwise, proceed to step S206.

[0072] S205. Perform data repair on the target data and determine that the data quality of the target data meets the data quality requirements.

[0073] Specifically, data cleaning templates are used to repair the target data, resulting in repaired data that meets data quality requirements. This repaired data then serves as the data source for cloud applications, ensuring the reliability of those applications.

[0074] S206. The data quality of the target data is determined to be non-compliant with data quality requirements.

[0075] Target data that does not meet data quality requirements will be filtered out to prevent it from causing abnormal operation of cloud applications.

[0076] The data quality detection method provided in this application determines whether target data meets data quality requirements based on the data quality detection results corresponding to each verification rule. For target data whose data quality detection results indicate no faults, the target data is determined to meet the data quality requirements. For target data with faults, it is further determined whether the target data can be repaired. If it can be repaired, the target data is repaired, and the repaired data is determined to meet the data quality requirements. If it cannot be repaired, the data quality of the target data is determined to be non-compliant with the data quality requirements. This method accurately determines the data quality of target data, ensuring the availability and reliability of data input into cloud applications.

[0077] In one possible implementation, determining whether the target data is repairable based on the fault information corresponding to the fault includes:

[0078] Based on the fault information, query the data cleaning template library to see if a data cleaning template corresponding to the fault information exists. If a data cleaning template corresponding to the fault information exists, determine that the target data can be repaired. If no data cleaning template corresponding to the fault information exists, determine that the target data cannot be repaired. Correspondingly, perform data repair on the target data, including applying the cleaning template corresponding to the fault information to repair the target data.

[0079] The cleaning template corresponds to the fault information and is used to correct data errors caused by faults in the target data. For example, if the fault information indicates that the data is arranged in a disordered order during data transmission, and does not conform to the preset arrangement order, the cleaning template can be set to the correct arrangement order. Referring to the cleaning template, the data is rearranged to achieve data repair.

[0080] The cleaning templates are stored in the cleaning template library, which is dynamically expandable. Cleaning templates can be obtained by performing cluster analysis on fault data or through a human-computer interaction interface and stored in the cleaning template library.

[0081] This application embodiment queries the data cleaning template library to determine whether a cleaning template corresponding to the fault information exists, determines whether the target data is repairable, and uses the cleaning template to repair the repairable target data, thereby realizing automated repair of faulty data and accurate screening of unrepairable data, ensuring the reliability of input data for cloud applications.

[0082] In one possible implementation, it also includes:

[0083] If the data quality detection result corresponding to at least one verification rule indicates that there is a fault in the corresponding link, the target data and the fault information corresponding to the target data are stored in the structured fault database; based on the fault information, cluster analysis is performed on the data in the structured fault database to obtain the data cleaning template corresponding to the fault information, and the fault information and the data cleaning template are stored in the data cleaning template library.

[0084] A structured fault database is a database that stores data according to a predefined data structure and relational model, including target data and corresponding fault information. This ensures efficient data storage, retrieval, and consistency, and improves the efficiency of data analysis and processing within the structured fault database.

[0085] Cluster analysis of data in a structured fault database can reveal common characteristics, enabling developers to quickly pinpoint the specific cause of a fault and achieve efficient fault management. For example, a structured fault database stores at least one target data entry corresponding to a controller fault. By performing cluster analysis on all target data entries corresponding to controller faults, common characteristics can be summarized. These common characteristics can then be used to further pinpoint the specific cause of the controller fault, achieving efficient fault attribution and management.

[0086] Optionally, the structured database also contains vehicle model information corresponding to the target data. By clustering the data based on the vehicle model information, the correlation between fault data features can be analyzed from the perspective of the whole vehicle, and then the correlation between faults can be analyzed, which helps to quickly locate and manage faults.

[0087] After identifying the common characteristics and causes of the faulty data, we can determine how to correct it, which means defining a data cleaning template. This ensures that the corrected data meets data quality requirements, and the data cleaning template is stored in a data cleaning template library. When faulty data is received again, the data cleaning template can be used to repair the data and improve its quality.

[0088] The data quality detection method provided in this application stores faulty target data in a structured fault database. Cluster analysis is performed on the faulty data in the structured database to quickly attribute the cause of the fault and obtain the corresponding data cleaning template, thus realizing a closed loop of "detection-treatment-accumulation" and fundamentally improving data quality.

[0089] In one possible implementation, the verification rules are described using Structured Query Language (SQL); and / or, according to at least one preset verification rule, data quality detection is performed on the target data to obtain data quality detection results corresponding to each verification rule, including: applying a distributed computing engine to scan the target data for data quality detection according to at least one preset verification rule to obtain data quality detection results corresponding to each verification rule.

[0090] SQL is a standardized programming language specifically designed for managing and manipulating relational databases. Its core functionalities include not only querying data but also inserting, updating, and deleting data, as well as creating and modifying database structures and managing access permissions. Developing validation rules based on SQL can lower the technical barrier to entry, reuse cloud data warehouse computing resources, and reduce operational costs.

[0091] This application's embodiments leverage the parallel processing capabilities of a distributed computing engine to overcome the bottleneck of single-node computing, supporting efficient scanning of massive amounts of data. The distributed engine transforms verification rules into tasks that can be executed in parallel. After completing a full data scan in the distributed environment, the engine generates detection results separately for each verification rule, ultimately summarizing the execution status of all rules to form a complete data quality report. This approach balances processing efficiency and detection granularity, making it well-suited for comprehensive and efficient data quality detection in big data scenarios.

[0092] The data quality detection method provided in this application embodiment is based on SQL-developed verification rules, which can lower the technical threshold, reuse cloud data warehouse computing resources, reduce operation and maintenance costs, and improve the efficiency of data quality detection by combining SQL with distributed computing engine technology, enabling comprehensive and efficient data quality detection of massive amounts of data.

[0093] In one possible implementation, the verification rules are obtained in the following way:

[0094] Using test data, control at least one link in the link from vehicle-side data generation to cloud-side data reception to ensure that there is a fault. Receive cloud data corresponding to the test data in the cloud; extract the data characteristics of the cloud data; and generate verification rules based on the data characteristics and fault information.

[0095] Specifically, data goes through multiple processing and transmission stages from generation to cloud reception. Using the controlled variable method, faults are injected into at least one stage, such as simulating network latency or protocol errors, to ensure that other stages function normally. After receiving cloud data, the cloud data is analyzed to extract data characteristics, and verification rules are generated based on the data characteristics and fault information.

[0096] In one possible implementation, test data is used to ensure that all links in the entire chain are processed correctly. The cloud data corresponding to the test data is received in the cloud, the data features of the cloud data are extracted, and verification rules are generated based on the data features.

[0097] The data quality detection method provided in this application can accurately obtain verification rules by injecting faults and observing the data characteristics of cloud data, thereby making the verification rules highly reliable.

[0098] In one possible implementation, the verification rules can be obtained in the following way:

[0099] Obtain the fault information uploaded by the user and the corresponding data features, and generate verification rules based on the fault information and the corresponding data features.

[0100] Specifically, the user first identifies the cause of the vehicle malfunction based on the symptoms and its impact on the data. After determining the cause, the user identifies the malfunction information and uploads the malfunction information and the corresponding data features. Based on the malfunction information, the user generates verification rules, which are used to verify whether the target data contains the malfunction corresponding to the verification rules.

[0101] The data quality verification method provided in this application can generate verification rules based on the fault information uploaded by the user and the data characteristics corresponding to the fault information, thereby improving the flexibility of verification rule generation.

[0102] Figure 3 This is a schematic diagram of the data quality detection device provided in the embodiments of this application, as shown below. Figure 3 As shown, the data quality detection device 30 provided in this embodiment includes:

[0103] The receiving module 301 is used to receive target data sent by the vehicle.

[0104] The detection module 302 is used to perform data quality detection on the target data according to at least one preset verification rule, and obtain the data quality detection results corresponding to each verification rule. At least one verification rule represents the data characteristics when there is no fault in the corresponding link in the entire link from the generation of data at the vehicle end to the reception of data at the cloud end.

[0105] The determination module 303 is used to determine the data quality of the target data based on the data quality detection results corresponding to each verification rule.

[0106] In one possible implementation, the determining module 303 is specifically used for:

[0107] If the data quality test results corresponding to each verification rule indicate that there is no fault in the corresponding link, it is determined that the data quality of the target data meets the data quality requirements.

[0108] If the data quality test result corresponding to at least one verification rule indicates that there is a fault in the corresponding link, determine whether the target data can be repaired based on the fault information corresponding to the fault.

[0109] If the target data is repairable, perform data repair on the target data and determine that the data quality of the target data meets the data quality requirements;

[0110] If the target data is irreparable, the data quality of the target data is determined to be non-compliant with data quality requirements.

[0111] In one possible implementation, the determining module 303 is specifically used for:

[0112] Based on the fault information corresponding to the fault, query the data cleaning template library to see if there is a data cleaning template corresponding to the fault information.

[0113] If a data cleaning template corresponding to the fault information exists, it can be determined that the target data is repairable.

[0114] If no data cleaning template corresponding to the fault information exists, the target data is determined to be unrepairable.

[0115] Correspondingly, data repair is performed on the target data, including: applying the cleaning template corresponding to the fault information to repair the target data.

[0116] In one possible implementation, the determining module 303 is further configured to:

[0117] If the data quality test result corresponding to at least one verification rule indicates that there is a fault in the corresponding link, the target data and the fault information corresponding to the target data will be stored in the structured fault database.

[0118] Based on the fault information, cluster analysis is performed on the data in the structured fault database to obtain the data cleaning templates corresponding to the fault information, and the fault information and data cleaning templates are stored in the data cleaning template library.

[0119] In one possible implementation, the validation rules are described using the Structured Query Language (SQL).

[0120] And / or, the detection module 302 is specifically used to: apply a distributed computing engine to scan the target data for data quality detection according to at least one preset verification rule, and obtain the data quality detection results corresponding to each verification rule.

[0121] In one possible implementation, the verification rules are obtained in the following way:

[0122] Using test data, control at least one link in the link from vehicle-side data generation to cloud-side data reception to send a fault, and receive the corresponding cloud data in the cloud.

[0123] Extract data features from cloud data and generate verification rules based on data features and fault information.

[0124] In one possible implementation, the verification rules can be obtained in the following way:

[0125] Obtain the fault information uploaded by the user and the corresponding data features, and generate verification rules based on the fault information and the corresponding data features.

[0126] The data quality detection device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0127] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 40 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the electronic device 40 further includes a communication interface 403. The processor 401, memory 402, and communication interface 403 are connected via a communication bus 404.

[0128] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0129] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0130] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0131] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0132] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0133] This application also provides a computer program product, including a computer program that, when executed, implements the above-described method.

[0134] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed, implement the above-described method.

[0135] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0136] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0137] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0138] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0140] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0142] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A data quality inspection method, characterized in that, Applied to the cloud, including: Receive target data sent by the vehicle terminal; According to at least one preset verification rule, the target data is subjected to data quality detection to obtain the data quality detection results corresponding to each of the verification rules. The at least one verification rule represents the data characteristics when there is no fault in the corresponding link in the entire link from the generation of vehicle-side data to the reception of cloud-side data. The data quality of the target data is determined based on the data quality detection results corresponding to each of the aforementioned verification rules.

2. The data quality detection method according to claim 1, characterized in that, The step of determining the data quality of the target data based on the data quality detection results corresponding to each of the aforementioned verification rules includes: If the data quality detection results corresponding to each of the aforementioned verification rules indicate that there is no fault in the corresponding link, it is determined that the data quality of the target data meets the data quality requirements. If the data quality test result corresponding to at least one verification rule indicates that there is a fault in the corresponding link, determine whether the target data can be repaired based on the fault information corresponding to the fault. If the target data is repairable, perform data repair on the target data and determine that the data quality of the target data meets the data quality requirements; If the target data is unrepairable, the data quality of the target data is determined to be non-compliant with data quality requirements.

3. The data quality detection method according to claim 2, characterized in that, The step of determining whether the target data is repairable based on the fault information corresponding to the fault includes: Based on the fault information corresponding to the fault, query the data cleaning template library to see if there is a data cleaning template corresponding to the fault information; If a data cleaning template corresponding to the fault information exists, it is determined that the target data can be repaired; If no data cleaning template corresponding to the fault information exists, the target data is determined to be unrepairable. Correspondingly, the data repair of the target data includes: applying the cleaning template corresponding to the fault information to repair the target data.

4. The data quality detection method according to claim 2 or 3, characterized in that, Also includes: If the data quality detection result corresponding to at least one verification rule indicates that there is a fault in the corresponding link, the target data and the fault information corresponding to the target data are stored in the structured fault database. Based on the fault information, cluster analysis is performed on the data in the structured fault database to obtain the data cleaning template corresponding to the fault information, and the fault information and the data cleaning template are stored in the data cleaning template library.

5. The data quality detection method according to any one of claims 1 to 3, characterized in that, The validation rules are described using Structured Query Language; And / or, the step of performing data quality detection on the target data according to at least one preset verification rule to obtain data quality detection results corresponding to each of the verification rules includes: applying a distributed computing engine to scan the target data for data quality detection according to at least one preset verification rule to obtain data quality detection results corresponding to each of the verification rules.

6. The data quality detection method according to any one of claims 1 to 3, characterized in that, The verification rules are obtained in the following way: Using test data, control at least one link in the link from vehicle-side data generation to cloud-side data reception to ensure that there is a fault, and receive the cloud data corresponding to the test data in the cloud. Extract the data features from the cloud data, and generate the verification rules based on the data features and fault information.

7. The data quality detection method according to any one of claims 1 to 3, characterized in that, The verification rules can be obtained in the following ways: Obtain the fault information uploaded by the user and the corresponding data features, and generate the verification rules based on the fault information and the corresponding data features.

8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed, are used to implement the method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, It includes a computer program that, when executed, implements the method of any one of claims 1 to 7.

Images