DATA QUALITY CONTROL SYSTEM
The system addresses manual inefficiencies in data quality control by using generative AI to automate data classification and anomaly detection, ensuring reliable data management and real-time notifications, thus enhancing data integrity and decision-making.
Patent Information
- Application Number
- TR202502738
- Authority / Receiving Office
- TR · TR
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-03-06
- Publication Date
- 2026-10-21
AI Technical Summary
Existing data quality control processes rely heavily on manual intervention, leading to inefficiencies and errors, especially as data sources grow beyond manageable sizes, necessitating advanced automation to ensure accurate and reliable data monitoring and reporting.
A system utilizing generative artificial intelligence to automatically control data quality, classify data based on criticality levels, and provide real-time notifications of anomalies, leveraging a server to manage data quality rules and perform checks without human intervention.
Enables efficient, automated data quality management, reducing human error and ensuring timely notifications of data anomalies, thereby maintaining data integrity and enabling informed decision-making.
Abstract
Description
1 TARIFF DATA QUALITY CONTROL SYSTEM Technical Area This invention addresses the problem of data sources growing to sizes that cannot be manually controlled over time. When it arrives, the data quality automatically improves without any human intervention. The report should be presented based on measurements, and the user should be informed of any possible anomalies or data corruption. It relates to a system that enables notifications based on criticality levels. Previous Technique Data quality is crucial for making successful decisions and conducting efficient business in today's data-driven world. It is critical for managing processes. Accurate and reliable data is essential. While increasing the organization's efficiency, incorrect or erroneous data leads to faulty analyses and 15 This can lead to serious risks by disrupting strategies. Therefore, monitoring data quality is crucial. Monitoring and improving it is of great importance. However, data quality control, It faces various challenges. Existing solutions often require human intervention. It relies on manual processes based on data analysis and operator predictions. It is based on rules that include a margin of error. This situation sometimes results in missing 20 This can lead to problems such as misdefined rules and incorrect data evaluations. Data More advanced technologies are needed to make quality control processes more effective. Automation solutions are needed. Therefore, the proliferation of data sources will eventually become impossible to control manually. When it reaches certain dimensions, data quality automatically improves without any human intervention. The report should be presented based on measurements, and the user should be informed of any possible anomalies or data corruption. 2 We need a system that allows notifications to be made based on the level of criticality. It seems that it was heard. United States patent number US2024241867, which falls under the known state of the art. in the document, the method and system of providing data quality as a service 5 It is mentioned that the invention in question involves data quality processes within an organization. An API-based service is offered to integrate and automate this. The service is able to retrieve data from heterogeneous data sources and is independent of data field type. It has the ability to provide data quality metrics. A subscribing application, Data Quality (DQ) 10 using templates from a customized rule engine. You can write the controls as code and based on predefined timelines. It can generate metrics. The service's analysis, visualization, and reporting capabilities utilize data. Stakeholders such as managers, application developers, and risk assessors need data It enables stakeholders to obtain actionable insights about quality. DQ 15, which is created by the service, includes accuracy, completeness, and timeliness. By using these metrics, we can identify data quality issues and reduce the risk of data usage. It can implement preventive and corrective actions to reduce [the problem]. Artificial Intelligence (AI) / Machine Learning (ML) modeling techniques, analyzing data quality, data to identify anomalies, measure data quality metrics, and assess overall data quality. It is used to generate the score. 20 Brief Description of the Invention The aim of this invention is to address the issue of proliferating data sources that cannot be manually controlled over time. When it reaches certain dimensions, data quality automatically improves to 25% without any human intervention. The report should be presented based on measurements, and the user should be informed of any possible anomalies or data corruption. a system developed to enable notification based on criticality level The goal is to implement the system. 3 Detailed Description of the Invention The "Data Quality Control System" developed to achieve the purpose of this invention is attached. This is shown in the figure; 5 Figure 1 shows a schematic view of the system that is the subject of the invention. The parts shown in the figure are individually numbered, and the corresponding numbers correspond to these numbers. given below. 10 1. System 2. Database 3. Server The proliferation of data sources eventually reaches sizes that cannot be manually controlled. When it arrives, the data quality automatically improves without any human intervention. The report should be presented based on measurements, and the user should be informed of any possible anomalies or data corruption. The invention was developed to enable notification based on the level of criticality. subject system (1); 20 -the quality control rules generated and the rules checked afterwards at least one database structured to ensure that the results are recorded. (2), -Relevant data is collected from its sources and subjected to control processes at regular intervals. preparation, inclusion of incoming data in the necessary definitions, data lifecycle 25 the inclusion of generative artificial intelligence in the cycle and the initiation of the initial control flow with the model, data patterns, sample data, and all other possible information that provides insight into the data. By transmitting the information, it is automatically determined which quality control rules apply to which data. 4 decisions and recommendations on where to run the project and the quality of the matched data. at least one server configured to enable the operation of the controls (3) It includes. The database (2) in the system (1) which is the subject of the invention, any communication protocol 5 to communicate with the server (3) and exchange data using It is being structured. The server (3) in the system (1) which is the subject of the invention, uses any communication protocol 10 to communicate with the database (2) and exchange data using It is configured. The server (3) retrieves data source access information from the relevant sources. to ensure that the data is received and the connection to the relevant source is made. is configured. The server (3) follows the connection to the data source or stream. performing checks on data and datasets and verifying that the relevant data has not been previously verified. Depending on whether it is defined or not, it is labeled as newborn, child or mature with 15 It is configured to enable classification. Server (3), previously the unidentified data being recorded as "newborn" It is configured to provide the content of the received data productively. The server (3) Defining it as artificial intelligence (Generative AI), data generated by generative artificial intelligence. 20 to ensure that quality control recommendations and rules are kept on record is configured. The server (3) is configured with the identified data quality control rules. performing non-comparative quality control checks and keeping records of the results. It is configured to provide the server (3), which already has the received dataset registered. And in cases where the dataset is defined as newborn, the data is defined as child. 25 when there is newborn data, with identification and non-comparative controls. to enable the first comparative checks to be carried out together It is structured. The server (3) controls data exchanges with generative artificial intelligence. by checking if new controls can be added and if so, new ones... to ensure that the rules are included in the existing defined rules It is configured. The server (3) is defined as the child of the received dataset. in these situations, non-comparative and comparative data defined for it. by performing checks and the deviations in the results determined by generative artificial intelligence. Quality is 5, categorized into three segments: green, yellow, or red, according to their range. It is configured to ensure that the result is reported to the relevant parties. Server (3), Each data point in the obtained mature definition is compared with short and long windowing. to enable quality control to be performed with the acquired data set It is being structured. Industrial Applicability Thanks to the system (1) that is the subject of the invention, the increasing data sources can be manually controlled over time. When it reaches unmanageable levels, it automatically stops working without any human intervention. a report is presented by measuring data quality and in case of possible anomalies or data contamination 15 The system ensures that the user receives notifications based on the criticality level. Around these basic concepts, the subject of the invention is "Data Quality Control System (1)". It is possible to develop a wide variety of related applications, and the invention described herein It cannot be limited to examples; it is essentially as stated in the claims. 20
Claims
6 REQUESTS 1. The proliferation of data sources eventually reaches sizes that cannot be manually controlled. When it arrives, the data quality automatically improves without any human intervention. a report should be presented based on measurements, and in case of possible anomalies or data contamination, 5 Enables notifications to the user based on the criticality level; - after the quality control rules and regulations of the generated data are checked at least structured to ensure that the resulting outcomes are recorded. containing a database (2) and -Relevant data is collected from its sources and subjected to control processes at regular intervals. 10 the preparation of the incoming data, including the necessary definitions, and the data itself. initiating the lifecycle and starting the initial control flow, productive artificial intelligence model analyzes data patterns, sample data, and information about the data. by transmitting all other possible information, it automatically determines which data is for which. Decisions and recommendations on which quality control rules will be implemented are 15. to ensure that the data is retrieved and matched and quality control checks are run. a system characterized by having at least one server (3) configured for (1).
2. To communicate with the server (3) using any communication protocol and database (2) configured to carry out data exchange and 20 A system like the one in Claim 1, characterized (1).
3. Communicate with the database (2) using any communication protocol. and characterized by the server (3) configured to carry out data exchange. A system like the one in Request 1 or 2 (1). 25 4. Obtaining data source access information from the relevant sources and sending it to the relevant source. server configured to enable connection operations 7 (3) like any of the above-mentioned claims characterized by system (1).
5. For data and datasets that arrive after connecting to a data source or stream. the control is performed and the relevant data has been previously defined 5 Since it is not defined, with labels such as newborn, child or mature characterized by the server (3) configured to enable its classification. a system like any of the above-mentioned requests (1).
6. Data that was not previously defined is registered as "newborn". characterized by the server (3) configured to ensure its retention a system like any of the above requests (1).
7. The content of the collected data is processed using generative artificial intelligence (GAI). 15 recommendations for defining and quality control of data generated with generative artificial intelligence. and a server configured to ensure that the rules are kept on record. (3) like any of the above-mentioned claims characterized by system (1).
8. Quality that is not compared with the identified data quality control rules. 20 to ensure that checks are carried out and the results are recorded any of the above requests characterized by the configured server (3) a system like one of them (1).
9. The dataset received is already registered and defined as new born data. 25 In cases where there is a set, the data should be defined as a child and not compared. When controls and newborn data are available, the first comparison is made together. server (3) configured to enable the performance of checks a system like any of the above characterized demands (1). 8 10. New control mechanisms are implemented by monitoring data changes with generative artificial intelligence. Checking the eligibility criteria for additions and, if necessary, applying new rules. server configured to ensure inclusion in defined rules (3) like any of the above-mentioned claims characterized by system (1). 5 11. In cases where the acquired dataset is defined as a child, it must be defined as follows: by performing non-comparative and comparative data checks green, according to the deviation ranges determined by generative artificial intelligence, The quality result is 10 by categorizing it into three segments, either yellow or red. with the server (3) configured to ensure that it is reported to the relevant parties a system like any of the above characterized demands (1).
12. Short and long windowing for each data point in the received mature definition. 15 quality control checks with each data set obtained through comparisons characterized by the server (3) configured to enable its implementation. a system like any of the above-mentioned requests (1).
