data shape confidence

Abstract

A system determines a shape of incoming data and, based on the shape, determines whether the data should be passed to support answering a query. If the shape of the data indicates that the data is insufficient or even potentially misleading, the system can gate the data or prompt for more.

Description

Technical Field

[0001] This disclosure relates to data shape confidence. Background Technology

[0002] Entities seeking to perform data analysis (such as companies) typically submit a "query" to a data source (such as a server or device). The data source responds with data, which the entity can then use to perform its analysis.

[0003] For example, in the healthcare field, companies may seek to determine patient adherence to medication plans (e.g., whether the patient has been administered the prescribed medication according to the planned schedule / the extent of patient adherence to the medication plan). Some medication devices are now capable of recording data about usage. For example, inhalers can be equipped to record when they are used, how much medication the user receives, etc. This data can be synchronized with the user's mobile device. Summary of the Invention

[0004] Some embodiments of this disclosure can be illustrated as a method. The method includes receiving a query from a query source. The method also includes receiving a dataset. The method further includes obtaining an expected data shape. The method also includes determining the data shape of the dataset. The method further includes comparing the data shape with the expected data shape. The method further includes determining a confidence factor based on the comparison between the data shape and the expected data shape. The method further includes detecting that the confidence factor is above a confidence threshold. The method also includes sending a response to the query source (in response to the detection) based on the dataset in response to the query.

[0005] Some embodiments of this disclosure may also be shown as computer program products including a computer-readable storage medium having program instructions embodied therein, which can be executed by a computer to cause the computer to perform the methods discussed above.

[0006] Some embodiments of this disclosure can be shown as a system. The system may include a memory and a central processing unit (CPU). The CPU may be configured to execute instructions to perform the methods described above.

[0007] The above description is not intended to depict every illustrated embodiment or implementation of this disclosure. Attached Figure Description

[0008] The accompanying drawings included in this application are incorporated in and form a part of the specification. They illustrate embodiments of the present disclosure and, together with the specification, serve to explain the principles of the disclosure. The drawings are merely illustrative of certain embodiments and are not intended to limit the scope of the disclosure. As the following detailed description proceeds, and when reference is made to the accompanying drawings, in which like reference numerals denote like parts, and wherein:

[0009] Figure 1 It is a data-shape-based advanced query response method consistent with several embodiments of this disclosure.

[0010] Figure 2 This is a detailed data shape model generation and comparison method consistent with several embodiments of this disclosure.

[0011] Figure 3 It is an advanced confidence assessment method based on data shape, consistent with several embodiments of this disclosure.

[0012] Figure 4 This is a block diagram illustrating an example of the expected data shape consistent with several embodiments of this disclosure.

[0013] Figure 5 A high-level block diagram of an example computer system that can be used to implement embodiments of the present disclosure is shown.

[0014] While the invention may have various modifications and substitutions, its details have been shown by way of example in the accompanying drawings and will be described in detail. However, it should be understood that the purpose is not to limit the invention to the specific embodiments described. Rather, the invention covers all modifications, equivalents, and substitutions falling within the spirit and scope of the invention. Detailed Implementation

[0015] This disclosure relates to systems and methods for determining confidence in the shape of data. More specifically, it relates to a system for receiving data, determining the shape of the data, determining a desired data shape based on a query, and determining a confidence rating for the received data.

[0016] In this disclosure, reference is made to one or more “queries”. As used herein, a “query” can refer to a request for information. Queries may be received from external sources, such as via a network. For example, in some embodiments, a compliance query may be received, requesting data about a patient’s compliance with a medication plan.

[0017] Systems and methods consistent with this disclosure can receive datasets from any number of sources and determine a response to a query based on the received dataset(s). To determine the response, example systems may determine the expected data shape based on the query, the received datasets, industry standards, combinations thereof, etc.

[0018] As used in this article, the “shape” of a dataset refers to the dataset’s values ​​and metadata, such as, for example, the type of data described by the data (“data fields”), the number of columns and / or rows of data, the integrity of the data, multiple supporting references, etc. In other words, the data shape describes several attributes of the dataset, such as data fields (e.g., “patient name”, “drug name”, “time of drug administration”, etc.), the values ​​contained within the data fields (e.g., “John Doe”, “acetaminophen”, “10:26 A.M”, etc.), the size of the dataset, etc. Figure 4 The example dataset is described below. The values ​​in the received dataset are categorized by data fields.

[0019] In response to queries, multiple data sources can use various methods to provide data in various states. Data sources can include, for example, applications running on a user's mobile device, external databases, etc. For instance, a first external data source might provide a first dataset when connected to the internet (e.g., an application running on a user's mobile device might attempt daily synchronization, but if it is disconnected, synchronization may not be possible for an extended period; upon reconnection, the application could send a dataset including all previously unsent data). Furthermore, the first data source might provide data only on a limited number of data fields. Meanwhile, a second data source might periodically provide a second dataset, which includes data on different data fields. Overlap is also possible; a third data source might also periodically provide its own dataset, which includes some data fields covered by the first data source dataset and some data fields covered by the second data source dataset, etc.

[0020] The shape of one or more received datasets can be compared to the expected data shape to determine whether the dataset(s) will be helpful or useful in responding to a query. In other words, comparing the expected data shape for a complete response to a hypothetical query with the shape of the received dataset allows the system to determine whether the received dataset will provide a relevant response to the query. For example, if a query requests the quantity of items sold in a store, the received dataset describing the store's expenses (wages, rent / mortgage, utility bills, etc.) will have a different data shape than the expected response shape, allowing the system to determine that the received dataset is not a useful response to the query.

[0021] Figure 1 This is a high-level query response method 100 based on data shape, consistent with several embodiments of this disclosure. Method 100 includes receiving a query at operation 102. Operation 102 may include, for example, receiving a request for data on a topic such as patient adherence to a medication plan. This information may be received from an external source such as a client server.

[0022] Method 100 also includes acquiring data at operation 104. Operation 104 may include, for example, sending a request to a device and receiving data as a response. An example of operation 104 is receiving a dataset from an application (such as a drug tracker app) running on a user's mobile device. In some embodiments, the data in the dataset may be labeled to identify data fields represented in the dataset. For example, the dataset may indicate that it includes information identifying the patient, the administered drug, the time(s) of administration(s), etc. In some embodiments, operation 104 may include identifying data fields in the dataset (e.g., if the dataset is not explicitly labeled). In some embodiments, data may be received from the same source as the query (e.g., the client may submit a query to the system performing method 100 about the client's own data).

[0023] Method 100 also includes determining the expected data shape at operation 106. As used herein, “data shape” refers to attributes of the data, such as data type (e.g., data format or file type), a list of data fields included in the data (e.g., user ID, drug ID, administration event, etc.), the size of the dataset (e.g., the number of rows / columns, file size, etc.), and / or the values ​​of some data (e.g., the drug ID is “acetaminophen” or a numeric code corresponding to acetaminophen). Operation 106 may include, for example, predicting the expected shape of the dataset that would be an appropriate response to a query. For example, a query about adherence to a acetaminophen medication plan for patient John Doe. The expected shape of the data responding to such a query could indicate that the data should include fields describing the patient’s identity, drug identity, and the amount of drug administered. The shape could also indicate that the value of the patient ID field should be (or a reference to) “John Doe” (because data about drug administration to different patients may, at best, be irrelevant, and at worst, be positively misleading), and the value of the drug ID field should be “acetaminophen”.

[0024] The expected shape of the data can be determined based on the received query, the received data, existing industry standards, and / or a combination thereof, as referenced below. Figure 2 Further detailed description. In some embodiments, operation 106 may include receiving the desired data shape from an external source (which may be the source of the query, the source of the data, or a different source) or loading one from a storage device. Figure 4 The text describes the expected shape of the instanced data.

[0025] Method 100 also includes determining a confidence factor at operation 108. Operation 108 may include, for example, comparing the shape of the received data with the expected shape of the data. Typically, a confidence factor indicates whether the received dataset includes data fields and values ​​that the expected shape indicates it should contain. In other words, it represents the degree of confidence in the responsiveness of the data (e.g., the degree to which the data is sufficient as a complete answer to a query). For example, a high confidence factor indicates that the data is more likely to satisfy a response to a query, while a low confidence factor may indicate that the data may be incomplete or that more data is needed.

[0026] The specific method for determining the confidence factor can vary depending on the implementation and usage. In some embodiments, the confidence factor can be a simple ratio of data fields in the received data to data fields that constitute the expected data shape. In some embodiments, the confidence factor may further take into account unwanted data included (e.g., this could result in a confidence factor penalty if a portion of the received data is irrelevant).

[0027] Method 100 also includes submitting a response to the query at operation 110. Operation 110 may include, for example, sending data received at operation 104 to the source of the query received at operation 102 (or, in some use cases, to a different source identified by the query). In some embodiments, operation 110 may also include sending a confidence factor. This may be particularly advantageous for certain query sources (e.g., customers) as it allows the query source to utilize the data more appropriately. For example, even if a customer has a relatively low confidence level requirement (being included in the query), the customer may benefit from knowing that data included in a particular response has a particularly high confidence rating. In some embodiments, operation 110 may only send data whose shape matches the expected shape. For example, data sent in response to a drug compliance query may be expected to include data fields such as drug identification data, patient identification data, and drug administration data. If the received data includes the expected data fields but also includes irrelevant data fields, such as, for example, the patient's doctor's appointment date, operation 110 may include removing the irrelevant data before submitting the relevant data in response to the query.

[0028] Figure 2 This is a more detailed method 200 for determining and comparing the expected data shape and the received data shape, consistent with several embodiments of this disclosure. Method 200 includes receiving a query at operation 202. Operation 202 may include, for example, receiving a request for data on a topic such as patient adherence to a medication plan. This information may be received from an external source such as a client server.

[0029] Method 200 also includes determining the expected data fields for the response to the query at operation 204. Operation 204 may include, for example, analyzing the query and selecting one or more data fields from a predetermined generalized list. Data fields can be selected such that the dataset containing each selected data field can form an aggregate response to the query. As an example, a response to a “medication compliance” query might require a “patient identifier” field (to allow the client to verify that the data is associated with the correct patient), a drug ID field (to allow the client to verify that the patient was given the correct medication), and a dosage field (to allow the client to verify that the medication was administered at the appropriate dosage).

[0030] In some embodiments, data fields may be selected based on pre-programmed behavior. For example, operation 204 may include analyzing the query and determining which fields should be included in the response based on the query. In some embodiments, data fields may be based on historical data. For example, operation 204 may include comparing the query with past queries and selecting fields based on responses to the most similar previous queries. In some embodiments, data fields may be selected via manual user review. For example, operation 204 may include a user of the system performing method 200 examining the query and (via one or more user input devices) entering a list of data fields that should be included in the response to the query. Combinations of the above are also considered (e.g., operation 204 may include manual user review and automated analysis of the query).

[0031] In some embodiments, the system may receive a query, identify the general industry of the query (e.g., healthcare), select a pre-existing generalized shape based on the industry, and then refine the generalized shape into the desired shape.

[0032] Method 200 also includes determining, at operation 206, the expected data value of the response to the query. Operation 206 may include, for example, analyzing the query and determining whether the value of any data field determined at operation 204 can be predicted (and if so, predicting that value). For example, if the query concerns adherence to a medication plan for John Doe, and operation 204 identifies at least the “Patient ID” field, then operation 206 may include determining that the value of the “Patient ID” field should include (or decode to, etc.) “John Doe”. Similarly, if the query indicates that the medication plan is for the administration of acetaminophen, operation 206 may include determining that the value of the “Drug ID” field should include “acetaminophen”. However, not all data fields have expected values; for example, the “dose timestamp” data field may not have a specific expected value (unless the query indicates that the dose should be administered within a specific time window), but the field itself can still be expected to be included regardless of its value. In some use cases, no data field will have an expected value.

[0033] The expected fields and values ​​determined in operations 204 and 206 can respectively form the “expected shape” of the response to the query. Method 200 also includes comparing the shape of the received data with the expected shape of the data at operation 208. Operation 208 may include, for example, determining whether the received dataset is missing any expected data fields, or whether the value of any expected data field is “incorrect” (different from the expected value).

[0034] Operation 208 may also include determining whether any included data fields are “irrelevant” (e.g., a patient’s height may be irrelevant to their adherence to a medication plan). Including irrelevant data may result in unnecessary privacy risks, resource (e.g., bandwidth) consumption, or even potentially obscure the requested data; if information about the first patient (even along with appropriate data about the second patient) is submitted in response to an inquiry about a second patient, the client may mistakenly interpret it as relevant to the second patient.

[0035] Method 200 also includes determining a confidence factor in operation 210. Operation 210 may include, for example, compiling information obtained from the comparison in operation 208 into numbers (such as values ​​from 0 to 1). A confidence factor can represent the confidence that the received data forms a complete response to the query. In other words, a high confidence factor can indicate that the received data includes most (if not all) of the expected data fields, and that the values ​​of those data fields are their expected values.

[0036] When determining confidence factors, different expected fields and values ​​can have different “weights.” For example, expected patient values ​​(e.g., patient names such as “John Doe”) might have a significant weight, causing incorrect patient IDs to result in a severe penalty to the confidence factor (because the received data would likely constitute a poor response to the query, as it involves entirely different patients). However, in the case of a medication adherence plan query, deviations from expected dosage timing can have a relatively small weight because the confidence in the validity of the data as a response to the query is unaffected. In other words, patients who fail to adhere to the plan do not affect the confidence rating; the confidence rating describes how accurately the data answers the query. It does not describe the response to the query itself.

[0037] Confidence factors can be used to make various decisions about what to do with the received data (e.g., whether to find out additional data or send a reply), as referenced below. Figure 3 More detailed description. Depending on the example / query, the confidence factor may be included in the response to the query.

[0038] Figure 3This is an advanced data shape-based confidence assessment method 300 consistent with several embodiments of this disclosure. Method 300 includes receiving a confidence factor at operation 302. Operation 302 may include, for example, determining a confidence factor describing that the received dataset will answer a received query (such as through...). Figure 2 Method 200).

[0039] Method 300 further includes determining at operation 304 whether the confidence factor is above a threshold. Operation 304 may include, for example, comparing the confidence factor to a preset threshold (e.g., 0.8, 0.9). The threshold may be determined based on a query, which may explicitly specify the threshold or may utilize a default threshold. In some embodiments, the query (or setting) may imply that a threshold comparison is not required, in which case operation 304 may be skipped (e.g., the client may want to collect all query responses as long as it knows which responses may be insufficient; this could further enable the client to make its own determination about which data to trust). Typically, operation 304 determines whether the received dataset will constitute a sufficient response to the query.

[0040] If the confidence factor is above a threshold (304 "Yes"), or if 304 is skipped, method 300 further includes sending a response to the query at operation 312. Operation 312 may include, for example, forwarding the original (unmodified) dataset to the client or other external entity. In some embodiments, operation 312 may include modifying the received data to suit a particular format and / or file type. In some embodiments, the confidence factor may be included in the response.

[0041] If the confidence factor is not above the threshold (304 "No"), method 300 further includes attempting to obtain additional data at operation 306. Operation 306 may include, for example, waiting for the user device to synchronize, sending a request for additional data to an external entity (such as a physician's office or a database), etc.

[0042] In some embodiments, operation 306 may include selecting a supplementary data acquisition method based on data or a query. For example, the received query may instruct the system performing method 300 to utilize a “backup” data source (such as the patient’s provider’s server). Operation 306 may also depend on the missing data. For example, if a missing data field is likely to be received via periodic device synchronization (such as a medication administration event field), operation 306 may more likely include waiting for device synchronization. As another example, if a missing data field is unlikely to be received via periodic device synchronization, operation 306 may send a request to the auxiliary source instead of waiting for synchronization. In some embodiments, operation 306 may include sending a request or reminder to the user device (e.g., prompting the user of the user device to synchronize the device or perform another action, such as administering medication).

[0043] Method 300 also includes receiving additional data at operation 308. Operation 308 may include receiving a supplementary dataset, for example, via device synchronization or in response to a request made as part of operation 306. The additional data may be examined to determine whether it includes previously lost data fields and / or values ​​(such as those identified via operation 208). In some embodiments, the additional data may be combined with previously received data to form a new dataset, and then the integrity of the new dataset may be checked (e.g., the shape of the new dataset may be compared to the expected shape of the response).

[0044] Method 300 also includes modifying the confidence factor at operation 310. Operation 310 can be performed in a manner substantially similar to operation 210 of method 200, as referenced. Figure 2 Operation 310 may include, for example, compiling a number (such as a value from 0 to 1) indicating confidence that the received data and additional data form a complete response to the query. In other words, a high confidence factor may indicate that the data includes, if not all, at least most of, the expected data fields, and that the values ​​of those data fields are their expected values.

[0045] Using the modified confidence factor, method 300 can loop back to operation 304, comparing the modified confidence factor with a threshold. Operations 304-310 can be repeated until the confidence factor meets the threshold (at which point a response is sent via operation 312). In some embodiments, an upper limit can be implemented on the number of loops, set by the user of the system executing method 300, or outlined in a query.

[0046] Figure 4 This is a block diagram illustrating an example expected data shape 401 according to several embodiments of the present disclosure. Figure 4 Describe a “generalized” data shape 400 that includes multiple different data fields. The generalized shape 400 may be an industry standard, or it may evolve over time based on past queries and / or responses to those queries.

[0047] Expected shape 401 includes data fields and / or values ​​that are determined to be particularly relevant to the query. Note that in some cases, expected shape 401 may be identical to generalized shape 400. However, expected shape 401 may also include data fields / values ​​not found in generalized shape 400. Similarly, expected shape 401 may exclude any data fields or values ​​from generalized shape 400. In general, expected shape 401 can typically be a subset of generalized shape 400.

[0048] exist Figure 4In the example depicted, the generalized shape 400 is a "healthcare" related shape (including common healthcare data fields such as patient ID 402, weight field 418, etc.). Figure 4 The example expected shape 401 depicted includes data fields and / or values ​​identified as being related to responses to queries regarding patient medication plan compliance.

[0049] The expected shape 401 includes a patient field 402. The value of the patient field 402 may include a patient ID, such as a name (e.g., "John Doe"), a patient number, etc. The expected shape 401 also includes a drug field 404. The value of the drug field 404 may include one or more drug IDs. For example, the value of drug 1 405 could be "acetaminophen," and the value of drug 2 406 could be "ibuprofen," etc.

[0050] The intended shape 401 also includes a drug administration field 408. The drug administration field 408 may include a value identifying when the drug is administered to the patient identified in the patient field 402. The drug administration field 408 includes its own data fields for each administration (dose 1 409 and dose 2 410). Dose 1 409 and dose 2 410 may include specific data including the amount of drug administered / estimated to be absorbed or bioavailable.

[0051] The generalized shape 400 includes all fields of the intended shape 401. The generalized shape 400 also includes a provider field 412, whose value can identify the provider (e.g., a physician) associated with the patient identified in the patient field 402. The generalized shape 400 also includes a prescription field 414, which can include values ​​identifying a prescription associated with the patient. The prescription field 414 is not necessarily related to compliance inquiries, as the medication in question may be an “over-the-counter” medication, or the inquiry may state its own parameters outside of a prescription.

[0052] The generalized shape 400 also includes a blood pressure field 416 (whose value can identify the patient's blood pressure), a weight field 418 (whose value can identify the patient's weight), and a height field 420 (whose value can identify the patient's height). As those skilled in the art will understand, these may not be relevant to drug compliance inquiries (except in rare special cases), in which case they may be omitted from the intended shape 401.

[0053] Now for reference Figure 5This illustration shows a high-level block diagram of an example computer system 500 that can be configured to perform various aspects of the present disclosure, including, for example, methods 100, 200, and 300. According to embodiments of the present disclosure, the example computer system 500 can be used to implement one or more of the methods or modules described herein, as well as any associated functions or operations (e.g., using one or more processor circuits or a computer processor). In some embodiments, the main components of the computer system 500 may include one or more CPUs 502, a memory subsystem 508, a terminal interface 516, a storage interface 518, an I / O (input / output) device interface 520, and a network interface 522, all of which may be directly or indirectly communicatively coupled to enable inter-component communication via a memory bus 506, an I / O bus 514, and an I / O bus interface unit 512.

[0054] Computer system 500 may include one or more general-purpose programmable central processing units (CPUs) 502, some or all of which may include one or more cores 504A, 504B, 504C, and 504D, collectively referred to herein as CPU 502. In some embodiments, computer system 500 may include a typical multiple processors of a relatively large system; however, in other embodiments, computer system 500 may alternatively be a single-CPU system. Each CPU 502 may execute instructions stored in a memory subsystem 508 on CPU core 504 and may include one or more levels of on-board cache.

[0055] In some embodiments, memory subsystem 508 may include random access semiconductor memory, storage devices, or storage media (volatile or non-volatile) for storing data and programs. In some embodiments, memory subsystem 508 may represent the entire virtual memory of computer system 500 and may also include virtual memory coupled to computer system 500 or other computer systems connected via a network. Memory subsystem 508 may conceptually be a single monolithic entity, but in some embodiments, memory subsystem 508 may be a more complex arrangement, such as a hierarchy of caches and other memory devices. For example, memory may reside in multi-level caches, and these caches may be further functionally partitioned such that one cache holds instructions while another cache holds non-instruction data used by one or more processors. Memory may also be distributed and associated with different CPUs or sets of CPUs, as is known in any of the various so-called Non-Uniform Memory Access (NUMA) computer architectures. In some embodiments, main memory or memory subsystem 804 may include control and flow elements for memory used by CPU 502. This may include memory controller 510.

[0056] Although the memory bus 506 is Figure 5 While shown as a single bus structure providing a direct communication path between CPU 502, memory subsystem 508, and I / O bus interface 512, in some embodiments, memory bus 506 may include multiple different buses or communication paths, which may be arranged in any of a variety of forms, such as hierarchical point-to-point links, star or mesh configurations, multi-layer buses, parallel and redundant paths, or any other suitable type of configuration. Furthermore, although I / O bus interface 512 and I / O bus 514 are shown as a single corresponding unit, in some embodiments, computer system 500 may include multiple I / O bus interface units 512, multiple I / O buses 514, or both. Additionally, although multiple I / O interface units are shown separating I / O bus 514 from various communication paths to various I / O devices, in other embodiments, some or all I / O devices may be directly connected to one or more system I / O buses.

[0057] In some embodiments, computer system 500 may be a multi-user mainframe computer system, a single-user system, a server computer, or a similar device that has little or no direct user interface but receives requests from other computer systems (clients). Furthermore, in some embodiments, computer system 500 may be implemented as a desktop computer, portable computer, laptop or notebook computer, tablet computer, pocket computer, telephone, smartphone, mobile device, or any other suitable type of electronic device.

[0058] Notice, Figure 5 This description aims to depict representative major components of an exemplary computer system 500. However, in some embodiments, the various components may have more... Figure 5 The greater or lesser complexity represented therein can exist differently from... Figure 5 The components shown, or other components, and the number, type, and configuration of these components may vary.

[0059] This invention can be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0060] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or recessed structures on which instructions are recorded, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0061] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the respective computing / processing device.

[0062] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including local area networks (LANs) or wide area networks (WANs), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing state information from the computer-readable program instructions.

[0063] This document describes aspects of the invention with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0064] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0065] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0066] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing one or more specified logical functions. In some alternative embodiments, the functions indicated in the blocks may occur in a non-consecutive order. For example, two blocks shown consecutively may actually be performed as a single step, executed simultaneously or substantially simultaneously with partial or complete time overlap, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0067] Various embodiments of this disclosure have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to explain the principles of the embodiments, their practical application, or improvements to existing technologies in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for data shape confidence, comprising: receiving a query from a query source; receiving a data set; comparing the query to historical queries; obtaining an expected data shape based on the comparison of the query to the historical queries; determining a data shape of the data set; comparing the data shape to the expected data shape; determining a confidence factor based on the comparison of the data shape to the expected data shape; detecting that the confidence factor is above a confidence threshold; in response to the detection, selecting the data set; and sending a reply to the query source in response to the query, the reply based on the data set, wherein the obtaining comprises: identifying an industry based on the query; selecting a generalized data shape based on the industry, the generalized data shape being an industry standard or evolving over time based on the historical queries and historical replies corresponding to the historical queries; and determining the expected data shape based on the generalized data shape.

2. The method of claim 1, wherein the obtaining further comprises determining one or more expected data fields of the data set based on the query, wherein the expected data shape is further based on the one or more expected data fields.

3. The method of claim 1, wherein the expected data shape consists of the generalized data shape.

4. The method of claim 1, further comprising: receiving a second query from a second query source; receiving an additional data set; determining a second data shape based on the additional data set; obtaining a second expected data shape; comparing the second data shape to the second expected data shape; determining a second confidence factor based on the comparison of the second data shape to the second expected data shape; detecting that the second confidence factor is below a second confidence threshold; and obtaining further data.

5. The method of claim 4, further comprising: determining a third confidence factor based on the additional data set and the further data; detecting that the third confidence factor is above the second confidence threshold; and in response to detecting that the third confidence factor is above the second confidence threshold, sending a second reply to the second query source in response to the second query, the second reply based on the additional data set and the further data.

6. A system for data shape confidence, comprising: a memory; and a central processing unit (CPU) coupled to the memory, the CPU configured to implement the method of any of claims 1-5.

7. A computer program product, comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to implement the method of any of claims 1-5. ​ ​ ​

Images