Prediction system for bias reduction of physical system parameters and method of implementation

Abstract

This disclosure discloses a prediction system and implementation method for reducing bias in physical system parameters, including a processor for receiving training data of user activities; receiving bias criteria; determining a set of model parameters for a machine learning model, including: (1) applying the machine learning model to the training data; (2) generating a model prediction error; (3) generating a data selection vector based on the model prediction error to identify non-outlier target variables; (4) generating a non-outlier dataset using the data selection vector; (5) determining updated model parameters based on the non-outlier dataset; and (6) repeating steps (1)-(5) until the review performance termination criteria are met; training classifier model parameters for an outlier classifier machine learning model; applying the outlier classifier machine learning model to activity-related data to determine non-outlier activity-related data; and applying the machine learning model to the non-outlier activity-related data to predict future activity-related attributes of user activities.

Description

[0001] Priority requirements

[0002] This application claims priority to U.S. Provisional Application No. 62 / 902,074, filed September 18, 2019, entitled “COMPUTER-BASED SYSTEMS, COMPPUTING COMPONENTS AND COMPUTING OBJECTS CONFIGURED TO IMPLEMENT DYNAMICOUTLIER BIAS REDUCTION IN MACHINE LEARNING MODELS”, which is incorporated herein by reference in its entirety.

[0003] Copyright Notice

[0004] This patent document contains copyrighted material in parts of its disclosure. The copyright holder does not object to any fax reproduction of the patent document or patent disclosure as if it were present in a Patent and Trademark Office document or record, but otherwise reserves all copyright rights. The following notice applies to the software and data described below, as well as the accompanying drawings that form part of this document: Copyright, Hartford Steam Boiler Inspection and Insurance Company, All Rights Reserved. Technical Field

[0005] This disclosure generally relates to computer-based systems, computing components, and computing objects configured to achieve bias reduction improvements in machine learning models. Background Technology

[0006] Machine learning models can include one or more computers or processing devices to form predictions or determinations based on patterns and inferences learned from sample / training data. Bias in the selection of sample / training data can propagate to the predictions and determinations of the machine learning model. Summary of the Invention

[0007] Embodiments of this disclosure include a method for a machine learning model for dynamic outlier bias reduction. The method includes receiving, by at least one processor, a training dataset of a target variable representing at least one activity-related attribute of at least one user activity; receiving, by at least one processor, at least one bias criterion for determining one or more outliers; and determining a set of model parameters for the machine learning model by the at least one processor including: (1) applying the machine learning model with the initial set of model parameters to the training dataset to determine a set of model predictions; (2) generating an error set of data element errors by the at least one processor by comparing the set of model predictions with corresponding actual values ​​in the training dataset; (3) generating a data selection vector by the at least one processor to identify non-outlier target variables at least in part based on the error set of data element errors and at least one bias criterion; (4) generating a non-outlier dataset by the at least one processor using the data selection vector on the training dataset; (5) determining an updated set of model parameters for the machine learning model based on the non-outlier dataset by the at least one processor; and (6) repeating steps (1)-(5) by the at least one processor. As an iteration, until at least one review performance termination criterion is met, thereby obtaining a set of model parameters for the machine learning model as updated model parameters, thereby regenerating a set of predicted values, an error set, a data selection vector, and a non-outlier dataset using the updated set of model parameters as the initial set of model parameters in each iteration; training a set of classifier model parameters for an outlier classifier machine learning model by at least one processor based at least in part on the training dataset and the data selection vector to obtain a trained outlier classifier machine learning model configured to identify at least one outlier data element; applying the trained outlier classifier machine learning model to a dataset of activity-related data for at least one user activity by at least one processor to determine: i) a set of outlier activity-related data in the dataset of activity-related data, and ii) a set of non-outlier activity-related data in the dataset of activity-related data; and applying the machine learning model to the set of non-outlier activity-related data elements by at least one processor to predict future activity-related attributes associated with at least one user activity.

[0008] Embodiments of this disclosure include a system for a machine learning model for dynamic outlier bias reduction. The system includes at least one processor communicating with a non-transitory computer-readable storage medium having software instructions stored thereon, wherein the software instructions, when executed, cause the at least one processor to perform the following steps: receiving a training dataset of a target variable representing at least one activity-related attribute of at least one user activity; receiving at least one bias criterion for determining one or more outliers; determining a set of model parameters for a machine learning model includes: (1) applying a machine learning model with an initial set of model parameters to the training dataset to determine a set of model predictions; (2) generating an error set of data element errors by comparing the set of model predictions with corresponding actual values ​​in the training dataset; (3) generating a data selection vector to identify non-outlier target variables, at least in part based on the error set of data element errors and at least one bias criterion; (4) generating a non-outlier dataset using the data selection vector on the training dataset; (5) determining an updated set of model parameters for the machine learning model based on the non-outlier dataset; and (6) repeating the steps. (1)-(5) are iterations until at least one review performance termination criterion is met, thereby obtaining the set of model parameters of the machine learning model as the updated model parameters, thereby regenerating the set of predicted values, the set of errors, the data selection vector and the non-outlier dataset using the updated set of model parameters as the initial set of model parameters in each iteration; training the set of classifier model parameters of the outlier classifier machine learning model based at least in part on the training dataset and the data selection vector to obtain a trained outlier classifier machine learning model configured to identify at least one outlier data element; applying the trained outlier classifier machine learning model to a dataset of activity-related data of at least one user activity to determine: i) the set of outlier activity-related data in the dataset of activity-related data, and ii) the set of non-outlier activity-related data in the dataset of activity-related data; and applying the machine learning model to the set of non-outlier activity-related data elements to predict future activity-related attributes related to at least one user activity.

[0009] The systems and methods of embodiments of this disclosure further include: applying a data selection vector to a training dataset by at least one processor to determine an outlier training dataset; training at least one outlier-specific model parameter of at least one outlier-specific machine learning model using the outlier training dataset to predict outlier data values; and using the outlier-specific machine learning model to predict outlier activity-related data values ​​of an outlier activity-related dataset by at least one processor.

[0010] The systems and methods of embodiments of this disclosure further include: training generalized model parameters of a generalized machine learning model using a training dataset by at least one processor to predict data values; using the generalized machine learning model by at least one processor to predict outlier activity-related data values ​​of an outlier activity-related dataset; and using the generalized machine learning model by at least one processor to predict activity-related data values.

[0011] The systems and methods of embodiments of this disclosure further include: applying a data selection vector to a training dataset by at least one processor to determine an outlier training dataset; training outlier-specific model parameters of an outlier-specific machine learning model using the outlier training dataset by at least one processor to predict outlier data values; training generalized model parameters of a generalized machine learning model using the training dataset by at least one processor to predict data values; predicting outlier activity-related data values ​​of an outlier activity-related dataset by at least one processor using an outlier-specific machine learning model; and predicting activity-related data values ​​by at least one processor using an outlier-specific machine learning model.

[0012] The systems and methods of embodiments of this disclosure further include: training generalized model parameters of a generalized machine learning model using a training dataset by at least one processor to predict data values; using the generalized machine learning model by at least one processor to predict activity-related data values ​​of an activity-related dataset; using an outlier classifier machine learning model by at least one processor to identify outlier activity-related data values; and removing outlier activity-related data values ​​by at least one processor.

[0013] Systems and methods according to embodiments of this disclosure, wherein the training dataset includes at least one active-related property of concrete compressive strength as a function of concrete composition and concrete curing exposure.

[0014] Systems and methods according to embodiments of this disclosure, wherein the training dataset includes at least one activity-related attribute of energy usage data as a function of household environmental conditions and lighting conditions.

[0015] The systems and methods of embodiments of this disclosure further include: receiving an application programming interface (API) request by at least one processor to generate a prediction having at least one data element; instantiating at least one cloud computing resource by at least one processor to schedule the execution of a machine learning model; using the machine learning model according to the execution schedule to predict at least one activity-related data element value of at least one data element; and returning at least one activity-related data element value to a computing device associated with the API request by at least one processor.

[0016] The systems and methods of embodiments of this disclosure include a training dataset comprising at least one activity-related attribute of three-dimensional patient images of a medical dataset; and a machine learning model configured to predict activity-related data values ​​comprising two or more physically based rendering parameters based on the medical dataset.

[0017] The systems and methods of this disclosure include a training dataset comprising at least one activity-related attribute of simulated control results of electronic machine commands; and a machine learning model configured to predict activity-related data values ​​including control commands for the electronic machine.

[0018] The systems and methods of embodiments of this disclosure further include: dividing an activity-related data set into multiple activity-related data subsets by at least one processor; determining an ensemble model for each activity-related data subset by at least one processor; wherein the machine learning model includes an ensemble of models; wherein each ensemble model includes a random combination of models from the ensemble of models; predicting ensemble-specific activity-related data values ​​by at least one processor using each ensemble model individually; determining the error of each ensemble model based on the ensemble-specific activity-related data values ​​and known values ​​by at least one processor; and selecting the ensemble model with the highest performance based on the lowest error by at least one processor. Attached Figure Description

[0019] Various embodiments of this disclosure can be further explained with reference to the accompanying drawings, in which the same structures are represented by the same numbers in several views. The drawings shown are not necessarily drawn to scale, but rather the emphasis is generally placed on illustrating the principles of this disclosure. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for teaching those skilled in the art to employ one or more illustrative embodiments in various ways.

[0020] Figures 1-14B One or more schematic flowcharts illustrating some exemplary aspects of at least some embodiments of this disclosure, screenshots of certain computer-based architectures and / or various dedicated graphical user interfaces are shown.

[0021] Figure 15 A graph showing the model error as a function of the error acceptance value for an example use case according to one or more embodiments of the present disclosure is illustrated. Detailed Implementation

[0022] This document discloses various detailed embodiments of the present disclosure in conjunction with the accompanying drawings; however, it should be understood that the disclosed embodiments are merely illustrative. Furthermore, each example given in conjunction with the various embodiments of the present disclosure is intended to be illustrative and not restrictive.

[0023] Throughout this specification, unless the context clearly indicates otherwise, the following terms have the meaning explicitly associated with this document. The phrases “in one embodiment” and “in some embodiments” as used herein do not necessarily refer to the same(s) embodiments(s), but may refer to the same(s) embodiments(s). Furthermore, the phrases “in another embodiment” and “in some other embodiments” as used herein do not necessarily refer to different embodiments, but may refer to different embodiments. Therefore, as described below, various embodiments can be readily combined without departing from the scope or spirit of this disclosure.

[0024] The term "based on" is not exclusive and allows for basing on additional factors not described unless the context clearly specifies otherwise. Additionally, throughout the specification, the meanings of "a," "an," and "the" include plural references. The meaning of "in..." includes both "in..." and "on...".

[0025] It should be understood that at least one aspect / function of the various embodiments described herein can be performed in real time and / or dynamically. As used herein, the term "real time" refers to an event / action that can occur instantaneously or nearly instantaneously in time while another event / action has already occurred. For example, "real-time processing," "real-time computation," and "real-time execution" all relate to the execution of computation during the actual time during which relevant physical processing (e.g., a user interacting with an application on a mobile device) occurs, so that the results of the computation can be used to guide the physical processing.

[0026] As used herein, the terms “dynamically” and “automatically”, and their logical and / or linguistic related and / or derived terms, indicate that certain events and / or actions can be triggered and / or occur without any human intervention. In some embodiments, events and / or actions according to this disclosure can be real-time and / or based on a predetermined periodicity of at least one of the following: nanoseconds, nanoseconds, milliseconds, milliseconds, seconds, seconds, minutes, minutes, hours, hours, days, days, weeks, months, etc.

[0027] In some embodiments, exemplary inventive, specially programmed computing systems with associated devices are configured to operate in a distributed network environment, communicate with each other via one or more suitable data communication networks (e.g., the Internet, satellite, etc.), and utilize one or more suitable data communication protocols / modes, such as, but not limited to, IPX / SPX, X.25, AX.25, AppleTalk™, TCP / IP (e.g., HTTP), Near Field Communication (NFC), RFID, Narrowband Internet of Things (NB-IoT), 3G, 4G, 5G, GSM, GPRS, WiFi, WiMax, CDMA, satellite, ZigBee, and other suitable communication modes. In some embodiments, NFC may refer to a short-range wireless communication technology, wherein an NFC-enabled device is moved to proximity by "swiping," "bumping," "tapping," or otherwise to communicate.

[0028] The material disclosed herein may be implemented as software or firmware or a combination thereof, or as instructions stored on a machine-readable medium that can be read and executed by one or more processors. A machine-readable medium may include any medium and / or mechanism for storing or transmitting information in a machine-readable (e.g., computing device) form. For example, a machine-readable medium may include read-only memory (ROM); random access memory (RAM); disk storage media; optical storage media; flash memory devices; electrical, optical, acoustic, or other forms of propagation signals (e.g., carrier waves, infrared signals, digital signals, etc.) and others.

[0029] As used herein, the terms “computer engine” and “engine” identify at least one software component and / or a combination of at least one software component and at least one hardware component that are designed / programmed / configured to manage / control other software and / or hardware components (such as libraries, software development kits (SDKs), objects, etc.).

[0030] Examples of hardware components may include processors, microprocessors, circuits, circuit elements (e.g., transistors, resistors, capacitors, inductors, etc.), integrated circuits, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), logic gates, registers, semiconductor devices, chipsets, microchips, chipsets, and so on. In some embodiments, one or more processors may be implemented as complex instruction set computer (CISC) or reduced instruction set computer (RISC) processors; x86 instruction set compatible processors, multi-core or any other microprocessor or central processing unit (CPU). In various embodiments, one or more processors may be one or more dual-core processors, one or more dual-core mobile processors, and so on.

[0031] Examples of software may include software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, application programming interfaces (APIs), instruction sets, computational code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof. The determination of whether to use hardware and / or software components to implement an embodiment can vary depending on many factors, such as desired computational speed, power levels, thermal tolerance, processing cycle budget, input data rate, output data rate, memory resources, data bus speed, and other design or performance limitations.

[0032] One or more aspects of at least one embodiment can be implemented by representative instructions stored on a machine-readable medium, representing various logic within a processor, which, when read by a machine, cause the machine to manufacture the logic to perform the techniques described herein. This representation, referred to as an "IP core," can be stored on a tangible machine-readable medium and provided to various customers or manufacturing facilities for loading into manufacturing machines that produce logic or processors. It should be noted that the various embodiments described herein can, of course, be implemented using any suitable hardware and / or computational software language (e.g., C++, Objective-C, Swift, Java, JavaScript, Python, Perl, QT, etc.).

[0033] In some embodiments, one or more exemplary inventive computer-based devices of this disclosure may include, or be partially or wholly incorporated into, at least one personal computer (PC), laptop computer, ultra-laptop computer, tablet computer, touchpad, portable computer, handheld computer, PDA, personal digital assistant (PDA), cellular phone, cellular phone / PDA combination, television, smart device (e.g., smartphone, smart tablet or smart TV), mobile internet device (MID), messaging device, data communication device, etc.

[0034] As used herein, the term "server" should be understood to refer to a point of service that provides processing, database, and communication facilities. By way of example and not limitation, the term "server" can refer to a single physical processor with associated communication, data storage, and database facilities, or it can refer to a networked or clustered complex of processors and associated network and storage devices, as well as the operating software supporting the services provided by the server and one or more database systems and application software. A cloud server is an example.

[0035] In some embodiments, as detailed herein, one or more exemplary inventive computer-based systems of this disclosure may acquire, manipulate, transmit, store, transform, generate, and / or output any suitable form of digital object and / or data unit (e.g., from within and / or outside a particular application) such as, but not limited to, files, contacts, tasks, emails, tweets, maps, entire applications (e.g., calculators), etc. In some embodiments, as detailed herein, one or more exemplary inventive computer-based systems of this disclosure can span one or more implementations across a variety of computer platforms, such as, but not limited to: (1) AmigaOS, AmigaOS 4, (2) FreeBSD, NetBSD, OpenBSD, (3) Linux, (4) Microsoft Windows, (5) OpenVMS, (6) OS X (Mac OS), (7) OS / 2, (8) Solaris, (9) Tru64 UNIX, (10) VM, (11) Android, (12) Bada, (13) BlackBerry OS, (14) Firefox OS, (15) iOS, (16) Embedded Linux, (17) Palm OS, (18) Symbian, (19) Tizen, (20) WebOS, (21) Windows Mobile, (22) Windows Phone, (23) Adobe AIR, (24) Adobe Flash, (25) Adobe Shockwave, (26) Wireless Binary Runtime Environment (BREW), (27) Cocoa (API), (28) Cocoa Touch, (29) JavaPlatforms, (30) JavaFX, (31) JavaFX Mobile, (32) Microsoft XNA, (33) Mono, (34) Mozilla Prism, XUL and XULRunner, (35) .NET Framework, (36) Silverlight, (37) Open Web Platform, (38) Oracle Database, (39) Qt, (40) SAP NetWeaver, (41) Smartface, (42) Vexi and (43) WindowsRuntime.

[0036] In some embodiments, the exemplary inventive computer-based systems and / or exemplary inventive computer-based devices of this disclosure may be configured to utilize hardwired circuitry systems that can be used in place of or in combination with software instructions to implement features consistent with the principles of this disclosure. Therefore, implementations consistent with the principles of this disclosure are not limited to any particular combination of hardware circuitry systems and software. For example, various embodiments may be implemented as software components in many different ways, such as, but not limited to, standalone software packages, combinations of software packages, or it may be a software package incorporated as a “tool” into a larger software product.

[0037] For example, exemplary software specifically programmed according to one or more principles of this disclosure may be downloaded from a network (e.g., a website) as a standalone product or as a plug-in package for installation in existing software applications. For example, exemplary software specifically programmed according to one or more principles of this disclosure may also be available as a client-server software application or as a web-enabled software application. For example, exemplary software specifically programmed according to one or more principles of this disclosure may also be implemented as a software package installed on a hardware device.

[0038] In some embodiments, the exemplary inventive computer-based systems / platforms, exemplary inventive computer-based devices, and / or exemplary inventive computer-based components of this disclosure can be configured to handle multiple concurrent users, which can be, but are not limited to, at least 100 (e.g., but not limited to 100-999), at least 1,000 (e.g., but not limited to 1,000-9,999), at least 10,000 (e.g., but not limited to 10,000-99,999), or at least 100,000 (e.g., but not limited to 100,000-99,999). The following quantities are allowed: at least 1,000-999,999, at least 1,000,000 (e.g., but not limited to 1,000,000-9,999,999), at least 10,000,000 (e.g., but not limited to 10,000,000-99,999,999), at least 100,000,000 (e.g., but not limited to 100,000,000-999,999,999), and at least 1,000,000,000 (e.g., but not limited to 1,000,000,000-10,000,000).

[0039] In some embodiments, the exemplary inventive computer-based systems and / or exemplary inventive computer-based devices of this disclosure can be configured to output to different, specially programmed graphical user interface implementations of this disclosure (e.g., desktop, web applications, etc.). In various embodiments of this disclosure, the final output can be displayed on a display screen, which can be, but is not limited to, a computer screen, a mobile device screen, etc. In various embodiments, the display can be a holographic display. In various embodiments, the display can be a transparent surface capable of receiving visual projections. Such projections can convey various forms of information, images, and / or objects. For example, such projections can be visual overlays for mobile augmented reality (MAR) applications.

[0040] As used herein, the terms “cloud,” “Internet cloud,” “cloud computing,” “cloud architecture,” and similar terms correspond to at least one of the following: (1) a large number of computers connected via a real-time communication network (e.g., the Internet); (2) providing the ability to run programs or applications simultaneously on multiple connected computers (e.g., physical machines, virtual machines (VMs)); (3) a network-based service that appears to be provided by real server hardware but is actually provided by virtual hardware (e.g., virtual servers) and is simulated by software running on one or more real machines (e.g., thus allowing for instantaneous movement and scaling (or scaling down) without affecting the end user).

[0041] In some embodiments, the exemplary inventive computer-based systems and / or exemplary inventive computer-based devices of this disclosure may be configured to securely store and / or transmit data using one or more cryptographic techniques (e.g., private / public key pairs, Triple Data Encryption Standard (3DES), block cipher algorithms (e.g., IDEA, RC2, RC5, CAST, and Skipjack), and cryptographic hash algorithms (e.g., MD5, RIPEMD-160, RTR0, SHA-1, SHA-2, Tiger (TTH), WHIRLPOOL, RNGs).

[0042] Of course, the above examples are illustrative rather than restrictive.

[0043] As used herein, the term "user" should have the meaning of at least one user. In some embodiments, the terms "user," "subscriber," "consumer," or "customer" should be understood to refer to a user of one or more applications as described herein and / or a consumer of data provided by a data provider. By way of example and not limitation, the terms "user" or "subscriber" may refer to a person who receives data provided by a data or service provider over the Internet in a browser session, or may refer to an automated software application that receives, stores, or processes data.

[0044] Figure 1 A block diagram of an exemplary computer-based system 100 for bias reduction in machine learning according to one or more embodiments of the present disclosure is depicted. However, practicing one or more embodiments may not require all of these components, and the arrangement and type of components may be varied without departing from the spirit or scope of the various embodiments of the present disclosure. In some embodiments, exemplary inventive computing devices and / or exemplary inventive computing components of the exemplary computer-based system 100 may be configured to manage a large number of members and / or concurrent transactions, as detailed herein. In some embodiments, the exemplary computer-based system / platform 100 may be based on a scalable computer and / or network architecture that incorporates various strategies for evaluating data, caching, searching, and / or database connection pooling, including Dynamic Outlier Bias Reduction (DOBR) as described in the embodiments herein. An example of a scalable architecture is an architecture capable of operating multiple servers.

[0045] In some embodiments, reference Figure 1Members 102-104 (e.g., clients) of the exemplary computer-based system 100 may actually include any computing device capable of receiving and sending messages to and from another computing device (such as servers 106 and 107), and to each other, via a network such as network 105 (e.g., a cloud network). In some embodiments, member devices 102-104 may be personal computers, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, etc. In some embodiments, one or more member devices within member devices 102-104 may include computing devices typically connected using wireless communication media such as cellular phones, smartphones, pagers, walkie-talkies, radio frequency (RF) devices, infrared (IR) devices, CBs, integrated devices combining one or more of the aforementioned devices, or virtually any mobile computing device. In some embodiments, one or more member devices within member devices 102-104 may be devices capable of connecting using wired or wireless communication media such as PDAs, POCKET PCs, wearable computers, laptops, tablets, desktop computers, netbooks, video game devices, pagers, smartphones, ultra-mobile personal computers (UMPCs), and / or any other device equipped with wired or wireless communication media for communication via wired and / or wireless communication media (e.g., NFC, RFID, NB-IoT, 3G, 4G, 5G, GSM, GPRS, WiFi, WiMax, CDMA, satellite, ZigBee, etc.). In some embodiments, one or more member devices within member devices 102-104 may include devices capable of running one or more applications, such as internet browsers, mobile applications, voice calls, video games, video conferencing, and email. In some embodiments, one or more member devices within member devices 102-104 may be configured to receive and send web pages, etc. In some embodiments, the exemplary specially programmed browser applications of this disclosure can be configured to receive and display graphics, text, multimedia, etc., employing virtually any web-based language, including but not limited to Standard Generalized Markup Language (SMGL), such as Hypertext Markup Language (HTML), Wireless Application Protocol (WAP), and Handheld Device Markup Language (HDML), such as Wireless Markup Language (WML), WMLScript, XML, JavaScript, etc. In some embodiments, the member devices within member devices 102-104 can be specially programmed using Java, .Net, QT, C, C++, and / or other suitable programming languages. In some embodiments, one or more member devices within member devices 102-104 can be specially programmed to include or execute applications to perform various possible tasks, such as, but not limited to, messaging functions, browsing, searching, playing, streaming, or displaying various forms of content, including locally stored or uploaded messages, images and / or videos and / or games.

[0046] In some embodiments, the exemplary network 105 may provide network access, data transmission, and / or other services to any computing device coupled thereto. In some embodiments, the exemplary network 105 may include and implement at least one dedicated network architecture, which may be at least partially based on one or more standards defined by, for example, but not limited to, the Global System for Mobile Communications (GSM) Association, the Internet Engineering Task Force (IETF), and the Global Microwave Access Interoperability (WiMAX) Forum. In some embodiments, the exemplary network 105 may implement one or more of the GSM architecture, the General Packet Radio Service (GPRS) architecture, the Universal Mobile Telecommunications System (UMTS) architecture, and UMTS evolution known as Long Term Evolution (LTE). In some embodiments, the exemplary network 105 may include and implement the WiMAX architecture defined by the WiMAX Forum as an alternative or in combination with one or more of the above. In some embodiments, and optionally, in conjunction with any embodiments described above or below, the exemplary network 105 may also include at least one of, for example, a Local Area Network (LAN), a Wide Area Network (WAN), the Internet, a Virtual LAN (VLAN), an Enterprise LAN, a Layer 3 Virtual Private Network (VPN), an Enterprise IP network, or any combination thereof. In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, communication via at least one computer network of the exemplary network 105 may be based at least in part on a variety of communication modes, such as, but not limited to, NFC, RFID, Narrowband Internet of Things (NB-IoT), ZigBee, 3G, 4G, 5G, GSM, GPRS, WiFi, WiMax, CDMA, satellite, and any combination thereof. In some embodiments, the exemplary network 105 may also include a mass storage device, such as a network-attached storage device (NAS), a storage area network (SAN), a content delivery network (CDN), or other forms of computer or machine-readable media.

[0047] In some embodiments, exemplary server 106 or exemplary server 107 may be a web server (or a series of servers) running a network operating system, examples of which may include, but are not limited to, Microsoft Windows Server, Novell NetWare, or Linux. In some embodiments, exemplary server 106 or exemplary server 107 may be used for and / or provide cloud and / or network computing. Although not explicitly stated in the text... Figure 1As shown, but in some embodiments, exemplary server 106 or exemplary server 107 may connect to external systems, such as email, SMS messaging, text messaging, advertising content providers, etc. Any feature of exemplary server 106 may also be implemented in exemplary server 107, and vice versa.

[0048] In some embodiments, one or more of the exemplary servers 106 and 107 may be specifically programmed, in non-limiting examples, to perform as an authentication server, search server, email server, social networking service server, SMS server, IM server, MMS server, exchange server, photo sharing service server, advertising delivery server, financial / banking related service server, travel service server, or any similar service-based server suitable for the users of member computing devices 101-104.

[0049] In some embodiments, and optionally in conjunction with any of the embodiments described above or below, for example, one or more exemplary computing member devices 102-104, exemplary server 106, and / or exemplary server 107 may include specially programmed software modules that can be configured to send, process, and receive information using scripting languages, remote procedure calls, email, tweets, short message service (SMS), multimedia messaging service (MMS), instant messaging (IM), Internet Relay Chat (IRC), mIRC, Jabber, application programming interfaces, simple object access protocol (SOAP) methods, generic object request broker architecture (CORBA), HTTP (Hypertext Transfer Protocol), REST (Representative State Transfer), or any combination thereof.

[0050] Figure 2A block diagram of another exemplary computer-based system / platform 200 according to one or more embodiments of the present disclosure is depicted. However, practicing one or more embodiments may not require all of these components, and the arrangement and type of components may be varied without departing from the spirit or scope of the various embodiments of the present disclosure. In some embodiments, each of the illustrated member computing devices 202a, 202b to 202n includes at least a computer-readable medium, such as random access memory (RAM) 208 coupled to processor 210 or flash memory. In some embodiments, processor 210 may execute computer-executable program instructions stored in memory 208. In some embodiments, processor 210 may include a microprocessor, ASIC, and / or state machine. In some embodiments, processor 210 may include or be able to communicate with a medium, such as a computer-readable medium storing instructions that, when executed by processor 210, may cause processor 210 to perform one or more steps described herein. In some embodiments, examples of computer-readable media may include, but are not limited to, electronic, optical, magnetic, or other storage or transmission devices capable of providing computer-readable instructions to a processor such as processor 210 of client 202a. In some embodiments, other examples of suitable media may include, but are not limited to, floppy disks, CD-ROMs, DVDs, magnetic disks, memory chips, ROMs, RAMs, ASICs, configured processors, all optical media, all magnetic tapes or other magnetic media, or any other media from which a computer processor can read instructions. Furthermore, various other forms of computer-readable media may transmit or carry instructions to or from a computer (including routers, private or public networks, or other transmission devices or channels) both wired and wirelessly. In some embodiments, instructions may include code from any computer programming language, including, for example, C, C++, Visual Basic, Java, Python, Perl, JavaScript, etc.

[0051] In some embodiments, member computing devices 202a to 202n may further include multiple external or internal devices, such as a mouse, CD-ROM, DVD, physical or virtual keyboard, monitor, or other input or output devices. In some embodiments, examples of member computing devices 202a to 202n (e.g., clients) may be any type of processor-based platform connected to network 206, such as, but not limited to, personal computers, digital assistants, personal digital assistants, smartphones, pagers, digital tablets, laptops, internet-connected appliances, and other processor-based devices. In some embodiments, member computing devices 202a to 202n may be specifically programmed with one or more applications according to one or more principles / methods detailed herein. In some embodiments, member computing devices 202a to 202n may run on any operating system (such as Microsoft™, Windows™, and / or Linux) capable of supporting browsers or browser-enabled applications. In some embodiments, the illustrated member computing devices 202a to 202n may include, for example, a personal computer executing browser applications (such as Microsoft's Internet Explorer™, Apple Computer's Safari™, Mozilla Firefox, and / or Opera). In some embodiments, through member computing client devices 202a to 202n, users 212a to 212n can communicate with each other and / or with other systems and / or devices coupled to network 206 via exemplary network 206. Figure 2 As shown, exemplary server devices 204 and 213 may also be coupled to network 206. In some embodiments, one or more member computing devices 202a to 202n may be mobile clients.

[0052] In some embodiments, at least one of the exemplary databases 207 and 215 can be any type of database, including databases managed by a database management system (DBMS). In some embodiments, the database managed by the exemplary DBMS can be specifically programmed to control the organization, storage, management, and / or retrieval of data in the respective database. In some embodiments, the database managed by the exemplary DBMS can be specifically programmed to provide the ability to query, back up and replicate, enforce rules, provide security, compute, perform change and access logging, and / or automate optimization. In some embodiments, the database managed by the exemplary DBMS can be selected from Oracle Database, IBM DB2, Adaptive Server Enterprise, FileMaker, Microsoft Access, Microsoft SQL Server, MySQL, PostgreSQL, and NoSQL implementations. In some embodiments, the database managed by the exemplary DBMS can be specifically programmed to define each respective schema of each database in the exemplary DBMS according to a specific database model of this disclosure, which may include a hierarchical model, a network model, a relational model, an object model, or some other suitable organization that can produce one or more applicable data structures, including fields, records, files, and / or objects. In some embodiments, the database managed by the exemplary DBMS can be specifically programmed to include metadata about the stored data.

[0053] In some embodiments, the exemplary inventive computer-based systems / platforms, exemplary inventive computer-based devices and / or exemplary inventive computer-based components of this disclosure may be specifically configured to operate in a cloud computing / architecture, such as, but not limited to, Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and / or Software as a Service (SaaS). Figure 3 and Figure 4 The illustration shows a schematic diagram of an exemplary implementation of a cloud computing / (one or more) architecture, wherein the exemplary inventive computer-based systems / platforms, exemplary inventive computer-based devices and / or exemplary inventive computer-based components of this disclosure may be specifically configured to operate.

[0054] In exemplary embodiments of the computer-based systems and / or devices of the present invention, Dynamic Outlier Bias Reduction (DOBR) can be used to improve the accuracy and understanding of generalized linear models, particularly for benchmarking studies. However, it is a method applicable to a variety of analytical models having one or more independent variables and one dependent variable. This disclosure and the embodiments therein illustrate the inventive application of DOBR in improving the accuracy of predictions by machine learning models.

[0055] In this embodiment, DOBR is not a predictive model. Instead, in this embodiment, it is an add-on method to the predictive or interpretive model that can improve the accuracy of the model's predictions. In this embodiment, DOBR identifies outliers based on the difference between the target variable provided by the data and the value calculated by the model. As outliers are identified based on predetermined selection criteria, the outlier-related data records and the dependent variable generated by the model are removed from the analysis. After these records are permanently removed, further analysis can continue. However, in other embodiments of the exemplary inventive systems and methods, in each model iteration, the outlier identification process includes the entire dataset, such that all records are examined for outliers using the predictive model of the last iteration as defined by its computational parameters. Thus, exemplary embodiments of the present invention reduce bias in machine learning models by, for example, including the entire dataset in each iteration to reduce the propagation of selection bias in the training data. Therefore, machine learning models can be trained and implemented more accurately and efficiently to improve the operation of machine learning systems.

[0056] Figure 5 The illustration shows a block diagram of an exemplary inventive bias reduction system in machine learning according to one or more embodiments of the present disclosure.

[0057] In some embodiments, the bias reduction system 300 may include components for dynamic outlier bias reduction (DOBR) in a dataset analyzed by, for example, a machine learning engine. In some embodiments, DOBR provides an iterative process to remove outlier records that conform to a predefined criterion. This criterion is a user-defined error acceptance value, expressed as a percentage. It refers to how much error the user is willing to potentially accept in the model based on their insights and other analytical results described later in this discussion. A value of 100% indicates that all errors are accepted and no records are removed in the DOBR process. If 0% is selected, then all records are removed. Generally, for industrial applications, error acceptance values ​​in the range of 80% to 95% have been observed.

[0058] In some embodiments, a user can interact with the bias reduction system 300 to manage error acceptance values ​​via user input device 308, view results via display device 312, and engage in other user interactions using display device 312 and user input device 308. Based on the error acceptance values, the bias reduction system 300 can analyze dataset 311 received in database 310 or other storage devices communicating with the bias reduction system 300. The bias reduction system 300 can receive dataset 311 via database 310 or other storage devices and make predictions using one or more machine learning models with dynamic outlier bias reduction to improve accuracy and efficiency.

[0059] In some embodiments, the bias reduction system 300 includes a combination of hardware and software components, including, for example, storage devices and memory equipment, caches, buffers, buses, input / output (I / O) interfaces, processors, controllers, networking and communication devices, operating systems, kernels, device drivers, and other components. In some embodiments, the processor 307 communicates with multiple other components to perform the functions of those other components. In some embodiments, each component schedules time on the processor 307 to perform its component functions, but in some embodiments, each component is scheduled to one or more processors in the processor 307's processing system. In other embodiments, each component has its own processor contained therein.

[0060] In some embodiments, components of the bias reduction system 300 may include, for example, a DOBR engine 301 that communicates with the model index 302 and model library 303, regressor parameter library 305, classifier parameter library 304 and DOBR filter 306, and other possible components. Each component may include a combination of hardware and software to implement the component's functionality, such as, for example, memory and storage devices, processing devices, communication devices, input / output (I / O) interfaces, controllers, networking and communication devices, operating systems, kernels, device drivers, instruction sets, and other components.

[0061] In some embodiments, the DOBR engine 301 includes a model engine for instantiating and executing machine learning models. The DOBR engine 301 can access models for instantiation in the model library 303 using model index 302. For example, the model library 303 may include a library of machine learning models that can be selectively accessed and instantiated for use by engines such as the DOBR engine 301. In some embodiments, the model library 303 may include machine learning models such as, for example, support vector machines (SVM), linear regressors, lasso models, decision tree regressors, decision tree classifiers, random forest regressors, random forest classifiers, K-neighbor regressors, K-neighbor classifiers, gradient boosting regressors, gradient boosting classifiers, and other possible classifiers and regressors. For example, the model library 303 can import models according to the following example pseudocode 1:

[0062] Pseudocode 1

[0063]

[0064] However, in some embodiments, to facilitate access to the machine learning model library in model library 303, DOBR engine 301 may employ model index 302, which indexes each model to a model identifier for use by DOBR engine 301 as a function. For example, models including linear regression, XGBoost regression, support vector regression, Lasso, K-neighbor regression, Bagging regression, gradient boosting regression, random forest regression, decision tree regression, and other regression and classification models can be indexed by numeric identifiers and labeled by name. For example, pseudocode 2 below illustrates an example of the model index code used by model index 302.

[0065] Pseudocode 2

[0066]

[0067] Other embodiments of the pseudocode for the expected model library 303 and model index 302. In some embodiments, software instructions are stored in the memory of the corresponding model library 303 or model index 302 and buffered in a cache for provision to the processor 307. In some embodiments, the DOBR engine 301 can utilize the model index 302 by accessing or calling the index via communication and / or I / O devices, using the index to call the model as a function from the model library 303 via communication and / or I / O devices.

[0068] In some embodiments, to facilitate the optimization and customization of models invoked by the DOBR engine 301, the bias reduction system 300 may record model parameters in, for example, memory or storage devices such as hard disk drives, solid-state drives, random access memory (RAM), flash memory, and other storage devices and memory equipment. For example, regression parameters may be recorded and adjusted in a regressor parameter library 305. Therefore, the regressor parameter library 305 may include storage and communication hardware configured with sufficient memory and bandwidth to, for example, store, adjust, and transmit multiple parameters for multiple regressors in real time. For example, for each regression machine learning model instantiated by the DOBR engine 301, the corresponding parameters may be initialized and updated in the regressor parameter library 305. In some embodiments, a user may establish an initial parameter set via a user input device 308. However, in some embodiments, the initial parameter set may be predetermined or randomly generated. When instantiating a regression machine learning model, the DOBR engine 301 may associate a model, such as that identified in the model index 302, with the parameter set in the regressor parameter library 305. For example, the DOBR engine 301 may invoke the parameter set based on, for example, an identifier (ID) associated with a given regression model. For example, the regressor parameter library 305 can identify the parameters of each regression model in a manner similar to the following pseudocode 3:

[0069] Pseudocode 3

[0070]

[0071]

[0072]

[0073]

[0074] Similarly, in some embodiments, classifier parameters can be recorded and adjusted in a classifier parameter library 304. Therefore, the classifier parameter library 304 may include storage and communication hardware configured with sufficient memory and bandwidth to, for example, store, adjust, and transmit multiple parameters for multiple regressors in real time. For example, for each classification machine learning model instantiated by the DOBR engine 301, the corresponding parameters can be initialized and updated in the regressor parameter library 305. In some embodiments, a user can establish an initial parameter set via a user input device 308. However, in some embodiments, the initial parameter set can be predetermined. When instantiating a regression machine learning model, the DOBR engine 301 can associate the model, as identified in the model index 302, with the parameter set in the regressor parameter library 305. For example, the DOBR engine 301 can recall the parameter set based on, for example, an identifier (ID) number associated with a given regression model. For example, the regressor parameter library 305 can identify the parameters of each regression model similar to the following pseudocode 4:

[0075] Pseudocode 4

[0076]

[0077] In some embodiments, by invoking and receiving a set of models from model library 303 and corresponding parameters from regressor parameter library 305 and / or classifier parameter library 304 via model index 302, DOBR engine 301 can load one or more instantiated and initialized models into, for example, a cache or buffer of DOBR engine 301. In some embodiments, dataset 311 can then be loaded from database 310 into the same or different cache or buffer or other storage device of DOBR engine 301. Processor 307 or the processor in DOBR engine 301 can then execute each model to transform dataset 311 into corresponding predictions of activity-related data values, for example, based on certain input attributes associated with the activity to characterize the outcome or parameters of the activity. Examples include electrical energy use in home and / or commercial environments, concrete compressive strength in various applications and formulations, object or image recognition, speech recognition, or other machine learning applications. For example, DOBR engine 301 can model electrical energy use based on dataset 311 of factors such as historical energy use, time of year, time of day, location, etc. The DOBR engine 301 can call the regressor set from the model library 303 via the model index 302 connected to the bus of the DOBR engine 301. The DOBR engine 301 can then call the parameter file or log associated with the regressor used for appliance energy use estimation from the regressor parameter library 305 connected to the bus of the DOBR engine 301. The DOBR engine 301 can then use the processor 307 to predict future energy consumption based on the model and model parameters, time and date, location, or other factors and combinations thereof.

[0078] Similarly, for example, DOBR engine 301 can model the compressive strength of concrete based on a dataset 311 containing factors such as concrete material, time of year, time of day, location, humidity, curing time, and age. DOBR engine 301 can call a set of regressors from a model library 303 via a model index 302 connected to the bus of DOBR engine 301. DOBR engine 301 can then call parameter files or logs associated with the regressors used for estimating concrete compressive strength from a regressor parameter library 305 connected to the bus of DOBR engine 301. DOBR engine 301 can then use processor 307 to predict future concrete compressive strength using models and model parameters based on specific concrete formulations, time and date, location, or other factors and combinations thereof.

[0079] As another example, the DOBR engine 301 may be performing speech recognition based on a dataset 311 of utterances, real transcripts, and other factors. The DOBR engine 301 can call a set of classifiers from a model library 303 based on a model index 302 connected to the bus of the DOBR engine 301. The DOBR engine 301 can then call parameter files or logs associated with the classifiers used for speech recognition from a classifier parameter library 304 connected to the bus of the DOBR engine 301. The DOBR engine 301 can then utilize the processor 307 to predict the transcripts of the recorded speech data based on models and model parameters used for one or more utterance sets.

[0080] As another example, the DOBR engine 301 can automatically predict the rendering settings of medical images based on a dataset 311 of settings for multiple rendering parameters across imaging and / or visualization, as well as other factors, as described in U.S. Patent No. 10,339,695. The DOBR engine 301 can call a set of classifiers from a model library 303 via a model index 302 connected to the bus of the DOBR engine 301. The DOBR engine 301 can then call parameter files or logs associated with the classifiers used for the rendering settings from a classifier parameter library 304 connected to the bus of the DOBR engine 301. The DOBR engine 301 can then utilize the processor 307 to predict rendering setting data based on models and model parameters for one or more sets of medical datasets.

[0081] As another example, the DOBR engine 301 can perform robot control of a machine based on a dataset 311 of machine control command results, simulation results of the machine control commands, and other factors, as described in U.S. Patent No. 10,317,854. The DOBR engine 301 can call a set of regression models from a model library 303 via a model index 302 connected to the bus of the DOBR engine 301. The DOBR engine 301 can then call parameter files or logs associated with the regression models used for robot control from a regressor parameter library 305 connected to the bus of the DOBR engine 301. The DOBR engine 301 can then utilize the processor 307 to predict the success or failure of a particular control command based on the model and model parameters of the simulation of the control command set, environmental information, sensor data, and / or commands.

[0082] In some embodiments, the bias reduction system 300 can implement machine learning models in a cloud environment, such as a cloud service for remote users. Such cloud services can be designed to support a large number of users and a variety of algorithms and problem scales, including those mentioned above, as well as other potential models, datasets, and parameter fine-tuning specific to user use cases, as described in U.S. Patent No. 10,452,992. In one embodiment, multiple programming interfaces (such as application programming interfaces (APIs)) can be defined by the service implementing the bias reduction system 300, guiding non-expert users to get started with machine learning best practices relatively quickly, without requiring users to spend significant time and effort fine-tuning models or learning advanced statistical or artificial intelligence techniques. For example, these interfaces can allow non-experts to rely on default settings or parameters for various aspects of the process of building, training, and using machine learning models, where default values ​​are derived from one or more sets of parameters from classifier parameter libraries 304 and / or regressor parameter libraries 305 for similar models used by individual users. Default settings or parameters can serve as a starting point for training a customized machine learning model for the user using the user's dataset via the DOBR engine 301 and optimizer 306. Simultaneously, users can customize parameters or settings they wish to use for various types of machine learning tasks, such as input record processing, feature processing, model building, execution, and evaluation. In at least some embodiments, in addition to or instead of using predefined libraries that implement various types of machine learning tasks, the cloud service bias reduction system 300 can have the service's scalable built-in capabilities, for example, by registering custom functions with the service. Depending on the business needs or goals of the clients implementing such custom modules or functions, these modules may be shared with other users of the service in some cases, while in other cases, the use of the custom modules may be limited to their implementer / owner.

[0083] In some embodiments, whether implemented as a cloud service, a local or remote system, or in any other system architecture, the bias reduction system 300 may include models in model library 303 that enable ensemble methods for training and implementing machine learning models, as described in U.S. Patent No. 9,646,262. This approach may be useful for applications performing data analysis using electronic datasets of electronic activity data. In some embodiments, database 310 may include one or more structured or unstructured data sources. In some embodiments, the unsupervised learning module is configured to assemble unstructured datasets into organized datasets, for example, using multiple unsupervised learning techniques in model ensembles from model library 303. For example, the unsupervised learning module is configured to assemble unstructured datasets into multiple versions of organized datasets, while in some embodiments, the supervised learning module is configured to generate one or more machine learning ensembles based on each of the multiple versions of organized datasets and, after training each model in each ensemble using DOBR engine 301 and optimizer 306, determine, for example, which machine learning ensemble exhibits the highest predictive performance based on model error.

[0084] An example of the DOBR engine 301 instructions used to control the hardware to make predictions based on dataset 311 is depicted in the following pseudocode 5:

[0085] Pseudocode 5

[0086]

[0087]

[0088]

[0089]

[0090]

[0091] However, in some embodiments, outliers in dataset 311 may reduce the accuracy of the implemented model, thereby increasing the number of training iterations required to achieve an accurate set of parameters for a given model in a given application. To improve accuracy and efficiency, DOBR engine 301 may include DOBR filter 301b to dynamically test the errors of data points in the dataset to identify outliers. Therefore, outliers can be removed to provide a more accurate or representative dataset 311. In some embodiments, DOBR filter 301b may provide an iterative mechanism for removing outlier data points that conform to predefined criteria (e.g., as described above and, for example, user-defined error acceptance values ​​provided by the user via user input device 308). In some embodiments, the user-defined error acceptance values ​​are expressed as percentages, where, for example, a value of 100% indicates that all errors are accepted and filter 301b does not remove any data points, while, for example, a value of 0% results in all data points being removed. In some embodiments, filter 301b may be configured to have error acceptance values ​​in the range of, for example, approximately 80% and approximately 95%. For example, filter 301b may be configured to perform the function depicted in pseudocode 6 below:

[0092] Pseudocode 6

[0093]

[0094] In some embodiments, the DOBR filter 301b works in conjunction with the optimizer 306, which is configured to determine the error and optimize the parameters of each model in the regressor parameter library 305 and the classifier parameter library 304. Therefore, in some embodiments, the optimizer 306 may determine the model and pass the error to the filter 301b of the DOBR engine 301. Thus, in some embodiments, the optimizer 306 may include, for example, a storage device and / or memory device with sufficient memory capacity and bandwidth, and a communication device to receive the dataset 311 and model the predictions and determine, for example, outliers, convergence, error, absolute error, and other error metrics. For example, the optimizer 306 may be configured to perform the functions depicted in pseudocode 7 below:

[0095] Pseudocode 7

[0096]

[0097] In some embodiments, the bias reduction system 300 then, by reducing outliers that would otherwise bias the prediction, can return machine learning model predictions, outlier analysis, prediction convergence, and other data generated by the DOBR engine to the user 301 in a more accurate and efficient manner via, for example, a display 312.

[0098] Figure 6A flowchart illustrating an exemplary inventive method according to one or more embodiments of the present disclosure is shown.

[0099] DOBR, such as DOBR Engine 301 and Filter 301b mentioned above, provides iterative processing to remove outlier records that conform to predefined criteria. This criterion is a user-defined error acceptance value, expressed as a percentage. It refers to how much error the user is willing to potentially accept in the model based on their insights and other analytical results described later in this discussion. A value of 100% means all errors are accepted, and no records are removed in the DOBR processing. If 0% is selected, then all records are removed. Generally, for industrial applications, error acceptance values ​​in the range of 80% to 95% have been observed.

[0100] However, in some embodiments, it should also be noted that if the dataset does not contain outliers, then DOBR provides no value. In practice, analysts rarely possess this knowledge before using the dataset. As will be shown later in this discussion, embodiments of the DOBR method can also determine the percentage of the dataset representing outliers in the model. This pre-analysis step can help set the correct error acceptance values ​​or determine whether outliers actually exist.

[0101] The following steps outline the basic DOBR method applied to the complete dataset.

[0102] Pre-analysis: In the example, we first select an error acceptance criterion, for example, we select... = 80%. (How to determine this value from the data will be demonstrated after explaining the DOBR method). Then, the error acceptance criterion C(∝) is defined according to, for example, Equation 1 below:

[0103]

[0104] in C is the error acceptance criterion, and C is a function of the error acceptance criterion. It is a comparison function. It is the value of the data record. It is a predicted value, and That is the target value.

[0105] Other functional relationships can be used to set C(α), but the percentile function is an intuitive guide to understanding why a model includes or excludes certain data records, such as Equation 2 below:

[0106]

[0107] in It is the percentile function. It is the index of the record entries, and It represents the number of recorded entries.

[0108] Since the DOBR process is iterative, a convergence criterion is also defined in the embodiments, which is set to 0.5% in this discussion.

[0109] In an embodiment, given a dataset 404. Solution Model 408 and error acceptance criteria 424 can achieve DOBR to reduce the number of training models. Bias of 408. In some embodiments, solution model 408 is implemented by a model engine, including, for example, processing devices and memory and / or storage devices. According to an embodiment, an exemplary method is used to process the solution model according to, for example, Equation 3 below. 408 is applied to the complete input dataset. Calculate the model coefficients for all records in 404. and model estimation 410:

[0110]

[0111] Where 0 represents the initial state, and This refers to the input record.

[0112] Then, according to the illustrative embodiment, the total error function 418 calculates the total error of the initial model according to, for example, the following equation 4. :

[0113]

[0114] in It is the total error of the initial model, and This represents the initial value.

[0115] Then, according to the illustrative embodiment, the error function 412 calculates the model error according to, for example, the following equation 5:

[0116]

[0117] in It is the recorded error of the prediction, and This indicates the iteration for recording the selection.

[0118] Then, according to the illustrative embodiment, the error function 412 calculates the new data record selection vector according to, for example, Equation 6 below. :

[0119]

[0120] in It records the selection vector.

[0121] Then, according to the illustrative embodiment, the data record selector 414 calculates the non-outlier data records to be included in the model calculation by selecting only the records whose record selection vector is equal to 1, for example, according to the following equation 7:

[0122]

[0123] in It is an index whose references include the DOBR set of records that are non-outlier values.

[0124] Then, according to the illustrative embodiment, model 408 with the latest coefficient 402 calculates new predicted values ​​420 and model coefficients 402 from data record 416 selected from DOBR according to, for example, the following equation 8:

[0125]

[0126] Then, according to the illustrative embodiment, model 408, using the new model coefficients, computes new predicted values ​​420 for the complete dataset. This step reproduces the computation of predicted values ​​420 for the records selected for DOBR in the formal steps, but in practice, the new model can be applied only to the records removed by DOBR, for example, according to Equation 9 below:

[0127]

[0128] Then, according to the illustrative embodiment, the total error function 418 calculates the total model error according to, for example, the following equation 10:

[0129]

[0130] in That is the target output.

[0131] Then, according to the illustrative embodiment, the convergence test 424 tests the convergence of the model according to, for example, the following equation 11:

[0132]

[0133] in It is a convergence criterion of 422, such as, for example, 0.5%.

[0134] In some embodiments, if, for example, the percentage error is less than, for example, 0.5%, then convergence test 424 may terminate the iterative process. Otherwise, the process may return to the initial dataset 404. Each of the above steps may then be performed and convergence criterion 422 may be retested. This process is repeated until convergence test 424 falls below convergence criterion 424.

[0135] Figure 7This is a diagram illustrating an example of the relationship between model error and error acceptance criteria in another exemplary computer-based machine learning model with reduced bias according to one or more embodiments of the present disclosure.

[0136] because These are the input parameters for DOBR, and the model results can vary based on the chosen values. Therefore, in this embodiment, a data-driven process is documented to demonstrate which values ​​are important. In the practical application of DOBR development, its selection (yet) lacks a theoretical basis. However, in practice, the relationship between model error and ∝ can produce slope changes that significantly reduce the impact of outliers. Figure 1 The graph shown is a result of a nonlinear regression 402 calculation related to a power generation benchmark according to an embodiment of the present invention.

[0137] In the embodiment, the general shape of the curve is predetermined, that is, it always follows a certain shape. The maximum error begins when = 100%, and when When the value is 0%, the model error is zero. Figure 7 In the middle, pay attention to the slope of the curve. = Around 85% variation. And for all smaller The slope is almost constant. Changes in the slope at this point indicate that the model's variability remains unchanged after removing data records, or in other words, there are no outliers at these acceptable error levels. Above [above] At 85%, at least two slopes show significant slope changes, indicating that some parts of the dataset contain behaviors or phenomena not considered in the model. This visual test can help set an appropriate error acceptance level and also determine whether DOBR is truly necessary. Figure 7 If the slope of the median remains unchanged, then the model satisfactorily explains the variability of the observed data. There are no model outliers and DOBR is not required.

[0138] In simulation studies that add a specific percentage of additional variability to the dataset, similar Figure 6 The curve shows an initial steep slope that intersects with a smaller slope near the error acceptance value programmed into the simulation. However, in practice, when outliers are observed, the slope typically transitions gradually to a constant slope, indicating that more than one type of variability is not being considered in the model.

[0139] Calculating an appropriate error acceptance value is a necessary part of using DOBR, and it also visually demonstrates the magnitude and severity of the impact of outliers on the model results. This step is documented. The choice of DOBR can be justified if the impact of outliers is judged to be minimal compared to the model predictions for outlier data.

[0140] In some embodiments, and model error and The value can be used as a metric for identifying the best-performing model or model ensemble for a specific scenario. Because the linearity can vary across different datasets, the exact linearity of the data and model... Values ​​can alter model performance. Therefore, model error as a function of the error acceptance level can be used to determine the extent to which a given model can explain data variability by having model errors that indicate a greater or lesser tolerance to data variability in order to form accurate predictions. For example, the precision and accuracy of model predictions can be fine-tuned by selecting models and / or model parameters that exhibit, for example, low model error for high error acceptance values, to select models that are more tolerant of outlier data.

[0141] In some embodiments, model selection can be automated by employing, for example, rule-based programming and / or machine learning models to identify the best performing model for the dataset based on a balance between model error and error acceptance criteria. Thus, a model can be automatically selected to optimally account for outliers in the dataset. For example, model errors can be compared across models for one or more error acceptance values, and the model with the lowest model error is automatically selected to generate predictions.

[0142] Therefore, the DOBR machine learning techniques according to various aspects of this disclosure provide more efficient model training and visibility into improvements to data and model behavior on individual datasets. Consequently, in fields such as artificial intelligence, data analytics, business intelligence, and others, machine learning models can be experimented with more effectively and efficiently on various types of data. Model performance can then be evaluated more efficiently to determine the optimal model for the application and data type. For example, artificial intelligence applications can be improved by using models selected and trained using DOBR tailored to the type of intelligence generated. Similarly, business intelligence and data analytics, as well as other applications such as physical behavior prediction, content recommendation, resource usage prediction, natural language processing, and other machine learning applications, can be improved by using DOBR to fine-tune model parameters and select models based on outlier characteristics and model errors in response to outliers.

[0143] Figure 8 This is a graph illustrating an example of the relationship between model error and error acceptance criteria in another exemplary computer-based machine learning model with reduced bias according to one or more embodiments of the present disclosure.

[0144] As an example of a DOBR implementation on the dataset, we use the Concrete Compressive Strength Dataset 504, downloaded from the Machine Learning Data Repository at the University of California, Irvine. This dataset contains 1030 observations, records, or instances with eight independent variables. The first seven terms describe the concrete composition, with the age given in days: cement content, high-efficiency water-reducing agent, blast furnace slag, coarse aggregate, fly ash, fine aggregate, water, and age.

[0145] The output variable is the concrete compressive strength measured in megapascals (MPa). For comparison, 1 MPa ≈ 145 psi. The linear regression model is constructed based on, for example, Equation 12 below:

[0146] Equation 12 ,

[0147] in These are the coefficients calculated from the linear regression model. There are 8 observed values ​​for each variable, and i is the variable index.

[0148] Figure 8 This was constructed by running a linear regression model 504 as a function of the DOBR error acceptance percentage α from 100% to 60%. From α = 100% to approximately α = 95%, the model error decreases rapidly, as shown in regression 506, then the error as a function of α decreases at a slightly slower rate until α = 85%. From this point onward, α decreases at a constant rate, as shown in regression 508. The point where the error begins to decrease at a constant rate is where the influence of outliers in the model calculation was ignored. In this case, the chosen point is α = 85%.

[0149] In this embodiment, DOBR is then modified and rerun with ∝=92.5% to determine the best model for fitting the non-outlier data. Figure 9 and Figure 10 Showing the use of the complete dataset 512 ( Figure 9 ) and DOBR version ( Figure 10 The results of these calculations show that outliers were identified and removed from the computation. Outlier 516, marked with a red cross, was calculated from the non-outlier model. These two figures are shown separately. Figure 9 and Figure 10 The diagonals 510 and 514 show the actual and predicted target values, depicting the equality. The complete dataset calculation ( Figure 9 This shows how outliers affect the results. DOBR modified graph ( Figure 10 The diagram shows how to remove bias by dividing the non-outlier 518 and the obvious outlier data point group 516, which may require further investigation, by diagonal line 514.

[0150] Figure 9 This is a graph illustrating an example of the relationship between baseline compressive strength and predicted compressive strength of a computer-based machine learning model with reduced bias, according to one or more embodiments of the present disclosure.

[0151] Figure 10 This is a graph illustrating an example of the relationship between compressive strength and predicted compressive strength of another exemplary computer-based machine learning model with reduced bias according to one or more embodiments of the present disclosure.

[0152] The identification of outliers and the patterns they sometimes form in the types of graphs described above are useful for the additional benefits of the DOBR method in industrial applications. Outliers can form patterns or groups that are simply unobservable by other methods. This information is created by simply using DOBR with a model provided by an analyst. No additional information or assumptions are required. In practice, the set of outliers defined by DOBR can provide useful information to improve, provide insights, or validate the underlying model.

[0153] Figure 11 This is a block diagram of another exemplary computer-based system for making machine learning predictions using DOBR, according to one or more embodiments of the present disclosure.

[0154] In embodiments of the present invention, the machine learning process starts from a dataset. Initially, the dataset consists of n independent variables, m records of length, and an array of target variables. Composition. In an embodiment, the dataset is used to train the machine learning model. According to, for example, Equation 13 below, the model is divided into two randomly selected subsets of predetermined size: one for training the model and the other for testing its prediction accuracy.

[0155]

[0156] in It is the independent variable of the dataset. A subset of, and It is the independent variable of the dataset. A subset of.

[0157] Regarding this discussion, The 70% / 30% split is used for training (n records) and testing (j records) (e.g., 70% of the records are for training and 30% are for testing), but any suitable split can be used, such as 50% / 50%, 60% / 40%, 80% / 20%, 90% / 10%, 95% / 5%, or other suitable training / test splits. The trained machine learning model L calculates the prediction of the set of target variables. To conduct tests, as expressed in, for example, Equation 14 below:

[0158]

[0159] In the illustrative embodiment, the model accuracy is then measured as a norm, for example... It can have, for example, the following forms:

[0160]

[0161] In the illustrative embodiment, we can directly measure outliers in the training and testing environments because we have both input and output variables. Generally, outliers in model predictions... Large deviations from the actual target variable values, such as the model L's inability to transform specific input values ​​into predicted values ​​near the known target variable, are due to the model's inability to map these values. These recorded input data contain the influence of factors and / or phenomena that the model cannot map to as they are given by the target variable. Including these records in the dataset may bias the results, as the model coefficients are calculated under the assumption that all data records are equally valid.

[0162] In some embodiments, the above refers, for example, to the above. Figure 6 The DOBR processing is suitable for a given dataset where analysts want to fit the best model to the data by removing outliers that adversely bias the results. It improves the model's predictive accuracy by restricting the model solution to a subset of the initial dataset after outlier removal. In an illustrative example, the DOBR-assisted solution has two outputs:

[0163] a) The set of x-values, model parameters, and model solutions of the model description data, and

[0164] b) The set of x-values, model parameters, and model solutions for data not described by the model.

[0165] Therefore, in addition to computing more accurate models for limited datasets, in this embodiment, DOBR also provides outlier datasets that can be further investigated relative to a given model to understand the reasons for high model prediction errors.

[0166] In the illustrative embodiment of the machine learning framework shown earlier in this section, the predictive model is computed based on the training data, and this model is used alone in the testing phase. Because by design, the testing phase may not use the target value to identify outliers, therefore the above reference... Figure 6The DOBR method described may not be applicable. However, exemplary aspects of the DOBR method may not be used above: such as the outlier-non-outlier classification ability suggested by the DOBR output mentioned above.

[0167] To describe the DOBR in the machine learning application of embodiments of the present invention, the dataset can be divided into two randomly selected parts: one for training and one for testing. During the training phase, both the independent and target variables are preserved, but during testing, the target variable is hidden, and the independent variables are used to predict the target variable. The known values ​​of the target variable are only used to measure the model's prediction error.

[0168] In this embodiment, a training dataset with n records is given. 604, Machine Learning Model 608 and error acceptance criteria 622 can implement DOBR to reduce the training time of machine learning models. 608 bias. In some embodiments, the machine learning model 608 is implemented by a model engine, including, for example, processing devices and memory and / or storage devices. According to an embodiment, an exemplary method model is configured to process the machine learning model according to, for example, Equation 16 below. 608 is applied to the complete input dataset. All records estimated for 604 606:

[0169]

[0170] Where 0 indicates the initial state, and This refers to the input record.

[0171] Then, according to the illustrative embodiment, the total error function 618 calculates the total error of the initial model according to, for example, the following equation 17. :

[0172]

[0173] in It is the total error of the initial model.

[0174] Then, according to the illustrative embodiment, the error function 612 calculates the model error according to, for example, the following equation 18:

[0175]

[0176] Where E is the predicted recording error, and k represents the iteration.

[0177] Then, according to the illustrative embodiment, the error function 612 calculates the new data record selection vector according to, for example, the following equation 19:

[0178]

[0179] in It records the selection vector.

[0180] Then, according to the illustrative embodiment, the data record selector 614 calculates the non-outlier data records to be included in the model calculation by selecting only the records whose record selection vector is equal to 1, for example, according to the following equation 20:

[0181]

[0182] in It is an index that references the DOBR set of records that are non-outlier values.

[0183] Then, according to the illustrative embodiment, the machine learning module 608 with the latest coefficient 602 calculates a new prediction value 620 for the complete training set 604 using the data records selected by DOBR according to, for example, the following equation 21:

[0184]

[0185] Then, according to the illustrative embodiment, the total error function 618 calculates the total model error according to, for example, the following equation 22:

[0186]

[0187] Then, according to the illustrative embodiment, the convergence test 624 tests the convergence of the model according to, for example, the following equation 23:

[0188]

[0189] in The convergence criterion is 622, such as, for example, 0.5%.

[0190] In some embodiments, if, for example, the percentage error is less than, for example, 0.5%, then convergence test 624 may terminate the iterative process. Otherwise, the process may return to the training dataset 604.

[0191] In some embodiments, the DOBR iterative process measures the extent to which the model can predict itself, rather than its accuracy relative to the test dataset. The goal here is to test the model's ability to predict the target variable, systematically removing records with large biases to improve the model's ability to focus on the majority of data that it predicts relatively well. This process must be performed on the same dataset. If outliers are identified in the test set, then removing records from the training set makes no sense. This process is fundamental to the DOBR method because the records removed in previous iterations are re-inputted after a new model (new model parameters) is computed. This process requires the use of the same dataset.

[0192] In this embodiment, the iterative process is performed after the learning model is defined. Based on the problem to be solved, in this embodiment, the user selects a machine learning algorithm and then determines the specific hyperparameters to "fine-tune" or configure the model. These parameters can be selected using standard techniques such as cross-validation or simply by plotting the test error as a function of a specific range of user-provided parameters. The specific values ​​used optimize prediction accuracy versus computation time while ensuring that the model is neither underfitting nor overfitting. Several powerful tools can help with this process, but user experience and intuition are also valuable benefits in selecting the optimal model hyperparameters. The examples discussed below use specific models and associated hyperparameters.

[0193] Error acceptance and model error plots are calculated from this step by applying a sequence of error acceptance values ​​and tabulating or plotting the results. These plots identify outliers in the dataset because their error contribution is slightly greater than the error contribution of data records suitable for the model. Also in practice, these plots can show more than one type of variation not explained by the model. The slope can vary as it converges to the slope of the model. These variations can help investigate the nature of additional data encoding behavior that the model cannot explain. Records occupying different slope intervals can be identified, and further investigation of them can provide insights that help build more robust models.

[0194] In this embodiment, during training, as described above, two models have been computed:

[0195] Model 1

[0196] ,

[0197] in It is a reference model, used as the basis for improving measurement accuracy; and

[0198] Model 2

[0199] ,

[0200] in It is the DOBR basic model, built from convergent outlier review records, and applied to non-outlier data. Training will be conducted on [the platform].

[0201] In the embodiment, the errors associated with Model 1 and Model 2 are, for example, respectively

[0202] and .

[0203] Therefore, in the embodiments, the basic model This suggests it might be a better predictor for non-outlier records. However, the test dataset is unreviewed and contains both non-outliers and outliers. Therefore, it's uncertain whether applying a custom model with non-outliers to unreviewed test data will produce better results than... Better predictive models. However, in many cases, it can be observed that... Statistically equal to or greater than .

[0204] In non-machine learning applications where the goal is to compute the best predictive model for a given dataset, the DOBR model computed from selected (non-outlier) records will always produce a lower model error because the identified outlier records are omitted. In the limiting case where there are no outliers, the DOBR model error equals the total model error because the dataset is identical.

[0205] However, in machine learning applications, the goal might be to develop a model using a subset of available data (training) and then measure its predictive accuracy on another subset (testing). In some implementations, however, the DOBR method removes outliers from the model in each iteration before computing the model parameters. In machine learning model development, this can be done during the training phase, but by definition, target values ​​in the test phase can only be used to measure the model's predictive accuracy, without the high-level knowledge of outliers. This observation implies that the standard DOBR method can generalize by leveraging more DOBR model information computed during the training phase.

[0206] Figure 11 This is a block diagram of another exemplary computer-based system for machine learning with reduced bias, according to one or more embodiments of the present disclosure.

[0207] In this embodiment, during training, as described above, the following information is generated: DOBR is the training dataset value selected for non-outlier values. DOBR is a data selection vector for training non-outlier values. DOBR is the training dataset value selected for non-outlier values. And DOBR is used to select vectors for training data on outliers. .

[0208] In this embodiment, DOBR classifies the training data into two mutually exclusive subsets. Furthermore, we have corresponding selection vectors, which, for example, provide a binary (non-outlier or outlier) classification value for each record in the training dataset according to Equation 24 below:

[0209]

[0210] In the embodiment, the complete set of training data attributes The classification generated by DOBR Used for building / training machine learning models such as classifiers stored in model library 303. The model was applied to the test dataset. This classifies test data records into outliers or non-outliers based on knowledge built upon the training dataset DOBR. For example, a classifier machine learning model. This can be achieved according to the following equation 25:

[0211]

[0212] Therefore, in the embodiments, Two test prediction datasets were generated respectively; and ,in Alternatively, the above information creates several possible "complete dataset" prediction models for analyzing the test dataset. In some embodiments, the three models that show the greatest prediction improvement across the entire dataset are:

[0213]

[0214] In some embodiments, for Machine learning model 608 is based on non-outlier data. Define and apply to DOBR test classification data. To predict non-outlier test values, the same process is performed on outlier data. In this embodiment, the aim of this combination is to use the most accurate predictive model and its corresponding dataset. In other words, the model is tested on the overall predictive accuracy of the non-outlier and outlier models, respectively, applied to their respective datasets defined using DOBR classification.

[0215] In some embodiments, for Machine learning models 608 from training data Defined, and also applied to DOBR test classification data. The model uses The model utilizes extensive knowledge to predict target values ​​for outliers and non-outliers x-values ​​as defined by DOBR. The aim of this model is to test the prediction accuracy of a fully trained model applied individually to both the non-outlier and outlier datasets for DOBR classification.

[0216] In some embodiments, the third model It is a hybrid model that combines the predictive properties of the first two methods. This model will be tested... Model 608 trained on the total training and The predictive benefits (if any) of connecting specific models trained on DOBR (which classifies outliers in the training set of datasets applied to their respective classifications) are also considered. Additional hybrid models exist that can be explored in further research.

[0217] In each of these three models and other embodiments, non-outlier and outlier records classified using DOBR are used to predict the complete test dataset. The DOBR method improves the overall predictive accuracy of machine learning models tested using these models. However, the main benefit of DOBR is identifying and removing outliers from the model, and then computing the best model predictor based on the remaining non-outliers. Furthermore, by definition, an outlier defined by DOBR is a record containing variations that are not adequately described in the current variables (or features) of a given machine learning model.

[0218] In some embodiments, analysts have three options based on the computed outlier and non-outlier datasets. In one embodiment, the first option is to apply the base model. And DOBR is not applied. This is a data-driven strategy when risk acceptance has a nearly linear relationship with the model error curve. In an embodiment, the second option is to apply one or more models: , or And combine, for example, the average results. In an embodiment, a third option is to develop predictions only for non-outlier records and further study the outlier data to develop modeling strategies for that specific new dataset—for example, changing the machine learning model or adding variables to account for unexplained variations, etc.

[0219] Regarding option 3, there are several methods for computing non-outlier datasets; two possible options are mentioned here. One reason for the relatively large number of possibilities may be due to the non-linearity of machine learning models in many applications. Generally speaking, This inequality may be due to the complexity of many machine learning models. For example, the equation applies to linear regression, but is not a general rule for machine learning models.

[0220] In this embodiment, regarding outlier prediction, the DOBR method was not initially designed to improve predictions on the complete dataset. By design, the method converges to the optimal set of outliers based on the provided model and dataset. The remaining data and model computations provide improved accuracy, but offer no guidance on how to predict outliers. The implicit decision is to apply different models to the outlier dataset, reflecting unique data variations not present in the non-outlier model.

[0221] In this embodiment, two models are defined to test the prediction accuracy of non-outlier datasets—removing outliers from the analysis. The first choice of the non-outlier dataset is to classify the DOBR vectors according to, for example, Model 6 below. Applied to reference model :

[0222]

[0223] In this embodiment, the reference model uses the complete training data to define the model based on the dataset. Make predictions. Then, based on the knowledge of the DOBR method obtained from the training dataset, apply classification vectors to remove outliers from the predictions. This model applies DOBR to the most general or broadest domain models.

[0224] In an embodiment, the second model, according to, for example, Model 7 below, applies DOBR to the records selected only by the classification model in the narrowest or "precise" manner using the DOBR model created from the training phase based on the non-outlier training data:

[0225]

[0226] There are other models that can be derived from the analytical formulas developed in this study, and depending on the problem, they can have significant potential for predictive improvement. However, the model used here... and This is the limiting case, representing the broadest and narrowest version in terms of training domain utilization and model definition.

[0227] In this embodiment, to test the prediction accuracy of the DOBR development models (such as Models 3-7) defined above, we use As a model , and The basis for comparison (of Models 3, 4, and 5 respectively). For and (Model 6 and Model 7 respectively), model prediction on non-outlier datasets, the more basic ones are Therefore, in the embodiments, the error can be determined according to, for example, the following equations 26, 27, and 28:

[0228]

[0229] In the following examples of illustrative embodiments, the prediction accuracy of DOBR is measured by how much (if any), Error for non-outlier datasets The improved measurement is the basic error adjusted relative to outliers. The reduction is described below based on the example results.

[0230] In some embodiments of the exemplary inventive DOBR-improved machine learning example, seven machine learning regression models can be used to test the accuracy of five previously defined models: linear regression, k-nearest neighbors, LASSO, support vectors, decision trees, bagging, and random forests. These machine learning regression models are examples of a wide range of model constructions. Additional or alternative models, such as neural networks, clustering, ensemble models, and combinations thereof, are also envisioned.

[0231] Linear regression is a method that allows analysts to understand the technical / treatment-related implications of coefficients (or model parameters). Analysts must provide a treatment model represented by an equation and determine the coefficients by minimizing the error between the predicted and target values ​​provided by the data.

[0232] LASSO, short for "Minimum Absolute Shrinkage and Selection Operator," is a regression correlation method in which an additional term is added to the objective function. This term is the sum of the absolute values ​​of the regression coefficients and is minimized based on user-provided parameters. The purpose of this additional term is to penalize increases in the coefficient values ​​of variables (or features). Minimization retains only the principal coefficients and can help reduce the unexplained effects of variable (or feature) covariance or collinearity.

[0233] Decision tree regression mimics human thinking and is intuitive and easy to interpret. The model uses a decision tree construction that logically shows how the value of x produces the target variable. Analysts set specific parameters in training / testing machine learning exercises, such as the maximum depth and minimum number of samples for each leaf.

[0234] Random forest regression is built upon the decision tree approach. Just as a forest is made up of trees, a random forest regression model is made up of groups of decision trees. Analysts define the forest structure by providing estimators (the number of trees in the forest), parameters similar to the maximum tree depth of a decision tree, leaf properties, and technical parameters related to how to calculate and apply model errors.

[0235] k-NN stands for k-nearest neighbor method, where predicted values ​​are computed based on the k nearest neighbors in the x (or feature) domain. The choice of the distance metric and the specific number of nearest neighbors to use are the main parameters analysts set when fine-tuning the model to make predictions on a given dataset. It is a simple and straightforward approach that works well for both regression and classification predictions.

[0236] Support Vector Regression (SVR) is a general machine learning method with several variations. Regression means fitting a model to data, and optimization typically involves minimizing the error between the predictor and target variables. With SVR, the error criterion is generalized to, for example, that if the error is less than a certain value "ε", then we say "close enough" and only errors greater than "ε" are measured and optimized. In addition to this property, the method allows data to be transformed into a nonlinear domain with a standard or, in some cases, a user-defined transformation function or kernel. Multidimensional data structures are used when the goal is to compute robust predictions—rather than modeling the data according to the traditional spirit of regression.

[0237] Bagging regression computes predictive estimates by using a random subset of samples with replacements. A decision tree (by default) prediction of the target variable is computed for each random sample. The final ensemble prediction can be computed in several ways—the average is one example. The main machine learning variables are the number of estimators in each ensemble, the number of variables (or features), the number of samples drawn to train each estimator, and the selection / replacement guidelines. Compared to other methods such as decision tree regression, this approach reduces variance.

[0238] Classifier Model This is an illustrative example because it applies DOBR non-outlier / outlier classification and training set x-values ​​to define non-outliers and outliers in the test dataset. This is a crucial step in DOBR machine learning applications, as it transfers knowledge of outliers from the training set to the test or production dataset. If incorrect classifications exist, the DOBR method's utility in improving machine learning prediction accuracy cannot be realized.

[0239] The classification accuracy of decision tree, k-NN, random forest, and bagging classifier models was tested. The bagging and random forest models were selected and fine-tuned to produce correct error acceptance scores for non-outliers. More detailed classification analysis could suggest alternative models. Even though classification accuracy is crucial, extensive classifier analysis is beyond the scope of this initial discussion.

[0240] Figure 12This is a graph illustrating an example of the relationship between model error and error acceptance criteria for some exemplary computer-based machine learning models with reduced bias used to predict concrete strength according to one or more embodiments of this disclosure.

[0241] The first example uses the same dataset as the reference concrete compressive strength mentioned above, with DOBR applied to the full dataset. As a brief recap, this dataset contains concrete compressive strength as a function of its composition and exposure, defined by eight quantitative input variables. This dataset contains 1,030 records or instances and can be found in the UC Irvine Machine Learning Repository archive.

[0242] The machine learning training exercise splits this dataset into a 70%:30% split, where model fine-tuning is performed on the training dataset (70%) and predictions are measured on the test dataset (30%).

[0243] The fine-tuning results of seven machine learning models in predicting the compressive strength of concrete are given in Table 1 below.

[0244] Table 1

[0245]

[0246] Default model parameters are not displayed (e.g., for Python 3.6) because they do not add information to the results. In this example, fine-tuning is an exercise in selecting parameters that minimize the error on the training and test datasets, using mean squared error as a metric. More complex algorithms could be applied, but this simple approach is used simply to ensure that the results do not overfit or underfit any dataset error.

[0247] In this embodiment, to apply DOBR, the percentage of data with excessive error (if any) is determined. In this embodiment, the machine learning model is applied to record a sequence of error acceptance scores corresponding to the model's error. This is done only for the training dataset, as the test dataset is only used to measure the predictive accuracy of the machine learning model. The percentage of data included in the model, "error acceptance," refers to the total amount of model error that a user is willing to accept, and also indicates the portion of data that the model fully describes.

[0248] In this embodiment, the error acceptance percentage sequence ranges from 100% to 60% in increments of 2.

[0249] Figure 13 This is a graph illustrating an example of the relationship between model error and error acceptance criteria for some exemplary computer-based machine learning models with reduced bias for predicting energy use according to one or more embodiments of the present disclosure.

[0250] The second example contains data on appliance energy usage, as well as home environment and lighting conditions, sampled every 10 minutes for 4.5 months. It consists of 29 attributes: 28 input variables and 1 output (target variable) and 19,735 records: the dataset and documentation can be found in the UC Irvine Machine Learning Repository archive.

[0251] Similarly, in the embodiments, the model fine-tuning results for the seven machine learning models in predicting electrical energy use are given in Table 2 below.

[0252] Table 2

[0253]

[0254] In this embodiment, default model parameters are not shown (e.g., for Python 3.6) because they do not add information to the results. Fine-tuning is an exercise in selecting parameters that minimize the error on the training and test datasets, using mean squared error as a metric. More complex algorithms could be applied, but a simple approach was used simply to ensure that the results do not overfit or underfit any dataset error.

[0255] In this embodiment, to apply DOBR, the percentage of data with excessive error (if any) is determined. In this embodiment, the machine learning model is applied to record a sequence of error acceptance scores for the corresponding model errors. This is done only for the training dataset, as the test dataset is only used to measure the predictive accuracy of the machine learning model. The percentage of data included in the model, "error acceptance," refers to the total amount of model error that the user is willing to accept, and also indicates the portion of data that the model fully describes.

[0256] In this embodiment, the error acceptance percentage sequence ranges from 100% to 60% in increments of 2.

[0257] Figure 12 and Figure 13 This partially demonstrates the ability of machine learning models to adapt to highly variable data. The closer the line is to linear (straight), the greater the model's ability to fully describe data transformed into fewer outliers (if any). The linear behavior of several models applied to specific data suggests that they can almost completely and adequately describe the entire training dataset. The non-linearity of the results on the energy dataset indicates the existence of a significant proportion of data records where the model produces inaccurate predictions or outliers.

[0258] According to embodiments of the invention, for each curve in the concrete data graph including, for example, linear regression 530, LASSO 540, decision tree regression 522, random forest regression 528, k-neighbor regression 524, support vector regression (SVR) 520, and bagging regression 526, and for each curve in the energy use data graph including, for example, linear regression 730, LASSO 740, decision tree regression 722, random forest regression 728, k-neighbor regression 724, support vector regression (SVR) 720, and bagging regression 726, a straight line defined by a low error acceptance percentage can be extrapolated to determine the error acceptance value at the beginning of the outlier portion. This process can be automated, but in practice, it can be performed manually to ensure that the selected error acceptance value reflects the analyst's judgment.

[0259] Extrapolation practice and the choice of the percentage of error acceptance are relatively simple procedures, but they are of great significance. They indicate how well the proposed model fits the data. The two's complement of the error acceptance value is the percentage of outliers in the dataset, i.e., the percentage of records where the model failed to make relatively accurate predictions. This is crucial information for choosing a machine learning (or any model) for a given dataset and practical application. Table 3 shows the error acceptance values ​​for each pattern selection for two example datasets.

[0260] Table 3

[0261]

[0262] In this embodiment, the prediction accuracy of the values ​​selected by DOBR is compared only to the reference model. This is the basic utility of DOBR, as the method itself does not provide any specific information about improving prediction accuracy on the entire dataset. Therefore, DOBR analysis presents analysts with a potential trade-off: better predictive power for a portion of the dataset, but no information for outlier records. The problem addressed in this section is to what extent, if any, the results of DOBR selection are more accurate than the predictions of the corresponding reference model test data.

[0263] The reference error is calculated for the complete dataset. The adjusted reference error value used for comparison with the non-outlier dataset is calculated by multiplying the complete reference error by the error acceptance value. For example, if the reference error is 10.0 and the error acceptance value is 80%, then the adjusted reference error is 10 x 80%, or 8.0. This explanation utilizes the definition of "error acceptance." For example, if the non-outlier data is calculated based on 80% of the data, then 80% of the total error should still be retained in the non-outlier data. This is the definition of error acceptance.

[0264] The results of measuring the prediction accuracy performance of the DOBR-selected outlier values ​​are presented in Tables 4 and 5 below, corresponding to, for example, the concrete strength dataset and the energy dataset. The reference error is calculated by multiplying the actual error by a percentage. The point estimates are used for calculation. The Random Forest classifier was not applied to the concrete dataset because it was judged that it would not change the conclusion about the improvement in DOBR prediction accuracy.

[0265] For all the statistics below, the results show the mean ± 95% confidence interval from a subset of training and test data selected from 100 random trials. In some examples in the table below, the support vector results are computed with fewer iterations (5 or 10) to manage computation time.

[0266] Table 4

[0267]

[0268] Table 5

[0269]

[0270]

[0271] Table 4 shows that using DOBR to select records resulted in minimal improvement in predictions, even with any apparent enhancement. This result is not surprising; in fact, it is based on... Figure 12 The error acceptance shown is as expected, based on the shape of the model error curve.

[0272] As expected, based on Figure 13 Table 5 indicates that the predictions selected by DOBR from the reference model values ​​of the bagging and random forest classifiers show considerable improvement, as detailed below. Figure 14A and Figure 14B DOBR model The maximum improvement indicates that removing outliers before model learning, plus DOBR classification, provides better results than using DOBR classification alone for the full (non-DOBR) model. The differences in improvement between models demonstrate the importance of model selection. While this decision rests with the analyst, comparing predictive accuracy through model comparisons is meaningful. Model runtime and several other factors are also important, and this study was not designed or intended to suggest the feasibility of one model being superior to another.

[0273] The conclusions in Table 5 are clear and statistically significant. However, considering the possibility of outlier bias, such as... Figure 13As shown in the figure, a machine learning model using the DOBR method can provide better prediction accuracy for non-outlier records compared to a machine learning model using a model without DOBR. Therefore, an exemplary inventive computing system including a machine learning model with DOBR has improved accuracy and reduced error when making predictions, thereby improving the performance and efficiency of the implemented model. However, this improvement may come at a cost: for identified outliers, there may be no predicted value or consideration. In embodiments, how outlier records are modeled can vary depending on the application.

[0274] Table 6 shows the prediction accuracy results for training / test sampling of the concrete compressive strength dataset using the bagged classifier. The random forest classifier was not applied to this dataset. The table shows the root mean square error at the 95% confidence level between 100 randomly selected test data points from the training and test datasets (see Equation 15).

[0275] Table 6

[0276]

[0277] Linear regression and LASSO produce the largest errors in the underlying or reference model. However, The predictive accuracy produced by the model is statistically the same as all other models except for the decision tree. In this case, the decision tree model produces the best predictive accuracy, and none of the models except for linear regression and LASSO appear to improve with the addition of DOBR.

[0278] Table 7 shows the increase (+) or decrease (-) in the prediction accuracy of the DOBR model relative to the reference model for each case, for example, the DOBR model: bagged classifier's prediction accuracy performance for concrete compressive strength.

[0279] Table 7

[0280]

[0281] These results are not surprising, as the model error versus error acceptance curves for linear regression and LASSO are graphs with maximum nonlinearity, and the other curves are almost linear, indicating that these models adequately predict the target variable and outlier analysis is unnecessary. This is the message conveyed in Table 7. The model outputs for the predicted concrete compressive strength are given in Appendix A, which is attached to this paper.

[0282] Looking at the energy consumption prediction error results in Table 8, we see different scenarios involving, for example, the prediction errors of bagged and random forest classifiers for appliance energy consumption. The bagged, linear regression, and LASSO models have the largest reference prediction errors, while the random forest model has the smallest. The DOBR model errors in the right three columns indicate that, in many cases, the DOBR model produces predictions with higher accuracy than the reference model.

[0283] Table 8

[0284]

[0285]

[0286] It is worth noting that the bagged reference model has the largest reference error value, but its DOBR-enhanced model results are generally within the same statistical range as other models. Also for practical reasons, the support vector model was only run for 10 iterations. This explains the increased uncertainty in its model results.

[0287] Detailed results of the improvements are shown in Table 9, relating to the accuracy performance of appliance energy consumption predictions, for example, the DOBR model. Note that for most machine learning models, at least one DOBR model produces some improvement in prediction accuracy. However, relatively large differences exist, so no conclusive results are available regarding the predictability improvements produced by DOBR. Looking at the model error versus error acceptance curves for energy data, all plots show that the random forest and decision tree models exhibit nonlinear behavior with minimal curvature. Moreover, it appears that the models, particularly the random forest, can adequately model this variation based on the results shown here. Model outputs for predicting energy use are given in Appendix B, which is attached to this paper.

[0288] Table 9

[0289]

[0290]

[0291] Figure 14A and Figure 14B The illustration shows a graph of non-outliers and outlier distributions in a classifier model of an exemplary inventive exemplary computer-based system having a DOBR classifier, according to one or more embodiments of the present disclosure.

[0292] The concrete dataset is relatively small, so a data plot can provide visual insight, but since DOBR is of little value in this case, plotting this dataset does not improve our understanding of how DOBR works. However, for the energy dataset prediction, DOBR does produce some significant prediction improvements. But its relatively large size (13,814 training records, 5,921 test records) makes a straightforward scatter plot visualization difficult to interpret. Scatter plots with a large number of points (such as...) Figure 9 and Figure 10 This may obscure any details. The error improvement results given in Table 3 are the sum for the non-outlier dataset, but the question remains how the DOBR method and classification model produce these results.

[0293] In an embodiment, to address this issue, the error distributions of the two model representations can be analyzed: the outlier dataset and the non-outlier dataset. Random Forest Classifier Figure 14A )and Bagging Sorter ( Figure 14B In the embodiments, the non-outlier error is designed to be smaller than the outlier error, but the exemplary inventive DOBR model and classification processing are constructed only from the training data, so the test dataset may contain previously unseen information. Therefore, the model and classification computation may be inaccurate, and the extent of the classification error can be visualized in these graphs. This work was performed for linear and bagging regression models because these two approaches have the largest and smallest improvement benefits given in Table 5, respectively.

[0294] For the purpose of discussion, the reference error value is... Figure 14A and Figure 14B Both figures highlight this. The top set of arrows indicates that 80% of the non-outlier error values ​​are less than 1,000, which means 20% of the error values ​​are >1,000. The lower set of arrows also indicates that for the outlier distribution, approximately 20% of the outliers have an error <1,000 or 80% have an error >1,000 – this should represent the outlier error. Without knowing the percentage of error acceptance beforehand, we cannot accurately calculate the accuracy of the classification treatment, but the figure above does show that even with misclassification, most values ​​are still correctly classified.

[0295] Figure 15 The illustration shows a graph of model error as a function of error acceptance value for an exemplary use case of an inventive exemplary computer-based system according to one or more embodiments of the present disclosure, having a DOBR-trained machine learning model for predicting non-productive time in drilling.

[0296] Offshore drilling operations present unique challenges for the oil and gas industry. In addition to observable logistical and environmental risks from weather and ocean depth, there are hidden downhole risks associated with operating in high-temperature, high-pressure, and vibrating environments. Drilling schedules are tight, and delays due to downhole equipment failures (non-production time or NPT) can result in significant revenue losses.

[0297] To aid in NPT management, machine learning models were constructed to help predict future downtime events, so that these estimated delays could be incorporated into contractual terms that set drilling targets. Historical events can be viewed, including: borehole distance [feet], hole size [inches], tool size [inches], location pressure severity, maximum dogleg [degrees / 100 feet], vibration severity category, curvature category, and NPT (hr).

[0298] Linear, XGBoost, Gradient Boosting, and Random Forest Regression models were applied to downhole equipment fault data, and model prediction accuracy was measured using an 80 / 20 training / test split. Hyperband was used to fine-tune the model and related parameter values, as shown in Table 10 below.

[0299] Table 10

[0300]

[0301] The classification function used to transfer the outlier information computed by DOBR to the test dataset can be a random forest model, where the number of estimators is, for example, equal to 5. This fine-tuning is also performed during the training phase of the analysis. The metric for parameter selection is to calculate the percentage of correctly classified elements in the training set and compare it to the model's error acceptance value.

[0302] Linear regression is included in this analysis because it is the only model whose coefficients can provide engineering insights to help identify additional best practice refinements. From a predictive standpoint, other models are more robust but offer less insight.

[0303] As discussed in this specification, there are several DOBR-related models that can be used to construct basic DOBR processing. In this example, three models are given: M represents the given overshoot model.

[0304] Use DOBR to select inline and outline values ​​in the training and testing datasets:

[0305] Pseudocode 8

[0306]

[0307] in and These are the DOBR in-values ​​calculated from the training set. and These are the DOBR outlier values ​​calculated from the training set. and These are the in-class and out-of-class values ​​in the test dataset, calculated from the DOBR classification model, respectively. This is the result of a non-DOBR computational model that uses the DOBR classification model to classify values ​​as inliers and outliers, where I represents the inliers as defined by DOBR. Value assigned And the outlier values ​​defined for DOBR will Assigned to .

[0308] From these subsets, the three DOBR models are:

[0309]

[0310] The error acceptance percentage and model error curve generated by running the above overshoot model are as follows: Figure 15 The curves shown are important characteristics of these curves, not their error values. Generally, the more linear the slope of a given curve is over the domain (0, 100%), the smaller the impact of outliers. For offshore downhole equipment failure data, the curves are linear with an error acceptance rate of approximately 80%, followed by various nonlinear slopes. The following table (Table 11) shows the defined error acceptance thresholds for DOBR analysis when the slope is analyzed as a function of the error acceptance value.

[0311] Table 11

[0312]

[0313] All models were run using the computed hyperparameters and assigned error acceptance values. Model outputs for predicting NPT are given in Appendix C, which is attached to this paper, and the error results are listed in Table 12 below:

[0314] Table 12

[0315]

[0316] Now that we have the non-DOBR model and the three DOBR models, we can choose which model to use for future forecasting in production. In summary, the linear model offers the lowest forecast accuracy, and DOBR model #1 or #2 offers the best forecast accuracy. Analysts can then balance these accuracy figures with other practical considerations, such as computation time, to select the model to apply to future forecasts.

[0317] While DOBR is used to train and implement machine learning models for predicting concrete compressive strain and energy, other applications are also expected.

[0318] For example, image rendering and visualization can leverage machine learning models to automatically predict and implement rendering parameters based on, for example, medical data, as described in U.S. Patent No. 10,339,695. DOBR can be used to train and implement machine learning models for content-based rendering. A medical dataset representing a patient's three-dimensional region can be used as input data. Using DOBR, outliers can be removed from the training medical dataset, allowing the machine learning model to be trained on non-outlier data according to the DOBR technique described above. The machine learning model is trained through deep learning on non-outlier data from the training medical dataset to extract features from the medical dataset and output values ​​of two or more physically based rendering parameters based on the input of the medical dataset. In some embodiments, the physically based rendering parameters are controls for consistent data processing, lighting design, viewing design, material proprietaryness, or internal renderer properties. The physically based renderer uses the output values ​​generated by the application to render a realistic image of the patient's three-dimensional region.

[0319] In another example application of DOBR for training and implementing machine learning models, the aforementioned DOBR technique can be used to train a machine learning model to generate control commands for machine output control commands, as described in U.S. Patent No. 10,317,854. In such an example, a simulator can simulate the machine's working operations based on the control commands. The simulator can generate a complete dataset for training the machine learning model by simulating the machine's physical actions based on the control commands. Such a dataset can be processed using DOBR iterations to ensure that any outlier simulations are removed when training the model parameters, including working operation data, control command data, and machine data used as input for each simulation.

[0320] In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, the exemplary DOBR machine learning model may be based at least in part on Monte Carlo methods of computational algorithms (e.g., Sollovay-Strassen type algorithms, Baillie-PSW type algorithms, Miller-Rabin type algorithms, and / or Schreier-Sims type algorithms), which may take into account historical quality data of the desired non-outlier data. In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, the exemplary DOBR machine learning model may be continuously trained by applying, for example but not limited to, at least one machine learning technique (such as, but not limited to, decision trees, boosting, support vector machines, neural networks, nearest neighbor algorithms, Naive Bayes, bagging, random forests, etc.) to collected and / or compiled sensor data (e.g., various types of visual data about the physical / visual appearance of the environment and / or goods). In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, the exemplary neural network technique may be, but not limited to, feedforward neural networks, radial basis function networks, recurrent neural networks, convolutional networks (e.g., U-net) or other suitable networks. In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, an exemplary implementation of a neural network may be performed as follows:

[0321] i) Define the neural network architecture / model.

[0322] ii) Transmit the input data to the exemplary neural network model.

[0323] iii) Incrementally train the exemplary model,

[0324] iv) Determine the accuracy of a specific number of time steps.

[0325] v) Apply the example trained model to process the newly received input data.

[0326] vi) Optionally and in parallel, the exemplary trained model may be continued to be trained at predetermined periods.

[0327] In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, the exemplary trained neural network model may specify the neural network at least by means of a neural network topology, a series of activation functions, and connection weights. For example, the topology of the neural network may include the configuration of the nodes of the neural network and the connections between these nodes. In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, the exemplary trained neural network model may also be specified to include other parameters, including but not limited to bias values / functions and / or aggregation functions. For example, the activation function of a node may be a step function, a sine function, a continuous or piecewise linear function, a sigmoid function, a hyperbolic tangent function, or other types of mathematical functions representing a threshold at which a node is activated. In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, the exemplary aggregation function may be a mathematical function that combines the input signal with the nodes (e.g., summation, product, etc.). In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, the output of the exemplary aggregation function may be used as the input of the exemplary activation function. In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, bias may be a constant value or function that can be used by aggregation functions and / or activation functions to make a node more or less activated.

[0328] In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, exemplary connection data for each connection in the exemplary neural network may include at least one of node pairs or connection weights. For example, if the exemplary neural network includes connections from node N1 to node N2, the exemplary connection data for the connections may include node pairs.<N1,N2> In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, connection weights may be numerical quantities that affect whether and / or how the output of N1 is modified before the input at N2 is applied. In the example of a recurrent network, nodes may have connections to themselves (e.g., connection data may include node pairs).<N1,N1> ).

[0329] In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, the exemplary trained neural network model may also include category identifiers (IDs) and fitness data. For example, each category ID may indicate which of several categories (e.g., cargo loss categories) the model is classified into. For example, fitness data may indicate how well the exemplary trained neural network model models the input sensor dataset. For example, fitness data may include fitness values ​​determined based on an evaluation of the model's fitness function. For example, an exemplary fitness function may be an objective function based on the frequency and / or magnitude of errors produced by testing the exemplary trained neural network model on the input sensor dataset. As a simple example, suppose the input sensor dataset consists of ten rows, the input sensor dataset includes two columns denoted as A and B, and the exemplary trained neural network model will output a predicted value of B given an input value A. In this example, testing the exemplary trained neural network model may include each of the ten values ​​of A input from the input sensor dataset, comparing the predicted value of B with the corresponding actual value of B from the input sensor dataset, and determining whether the two predicted and actual values ​​of B are different and / or by how much. To illustrate, if a particular neural network correctly predicts the B value in nine out of ten rows, the exemplary fitness function could assign a fitness value of 9 / 10 = 0.9 to the corresponding model. It should be understood that the preceding examples are for illustrative purposes only and should not be considered limiting. In some embodiments, the exemplary fitness function may be based on factors independent of error frequency or error rate, such as the number of input nodes, node layers, hidden layers, connections, computational complexity, etc.

[0330] In some embodiments, and optionally, in conjunction with any of the embodiments described above or below, this disclosure may utilize at least one of the following aspects:

[0331] US Patent Serial No. 8195484, titled "Insurance product, rating system and method";

[0332] US Patent Serial No. 8548833, titled "Insurance product, rating system and method";

[0333] US Patent Serial No. 8554588, titled "Insurance product, rating system and method";

[0334] US Patent Serial No. 8554589, titled "Insurance product, rating system and method";

[0335] US Patent Serial No. 8595036, titled "Insurance product, rating system and method";

[0336] US Patent Serial No. 8676610, titled "Insurance product, rating system and method";

[0337] US Patent Serial No. 8719059, titled "Insurance product, rating system and method";

[0338] US Patent Serial No. 8812331, entitled "Insurance product, rating and credit enhancement system and method for insuring project savings".

[0339] At least some aspects of this disclosure will now be described with reference to the following numbered clauses:

[0340] Clause 1. A method comprising:

[0341] The processor receives a training dataset of a target variable representing at least one activity-related attribute of at least one user activity;

[0342] The at least one processor receives at least one bias criterion for determining one or more outliers;

[0343] The at least one processor determines the set of model parameters for the machine learning model, including:

[0344] (1) The at least one processor applies a machine learning model with an initial set of model parameters to a training dataset to determine a set of model predictions;

[0345] (2) The processor generates an error set of data element errors by comparing the set of model predictions with the corresponding actual values ​​in the training dataset;

[0346] (3) A data selection vector is generated by the at least one processor to identify non-outlier target variables based at least in part on an error set of data element errors and at least one bias criterion;

[0347] (4) The processor generates a non-outlier dataset on the training dataset using a data selection vector;

[0348] (5) The at least one processor determines the updated set of model parameters for the machine learning model based on the non-outlier dataset; and

[0349] (6) Steps (1)-(5) are repeated by the at least one processor as an iteration until at least one review performance termination criterion is met, thereby obtaining the set of model parameters of the machine learning model as updated model parameters. Thus, in each iteration, the updated set of model parameters is used as the initial set of model parameters to regenerate the set of predicted values, the set of errors, the data selection vector and the non-outlier dataset.

[0350] The at least one processor trains a set of classifier model parameters for an outlier classifier machine learning model based at least in part on a training dataset and a data selection vector to obtain a trained outlier classifier machine learning model configured to identify at least one outlier data element.

[0351] The at least one processor applies a trained outlier classifier machine learning model to a dataset of activity-related data for at least one user activity to determine:

[0352] i) Outliers in the activity-related data dataset.

[0353] ii) The non-outlier set of activity-related data in the activity-related data dataset; and

[0354] The at least one processor applies a machine learning model to a set of non-outlier activity-related data elements to predict future activity-related attributes associated with at least one user activity.

[0355] Clause 2. A system comprising:

[0356] At least one processor communicating with a non-transitory computer-readable storage medium having software instructions stored thereon, wherein the software instructions, when executed, cause the at least one processor to perform the following steps:

[0357] Receive a training dataset containing a target variable representing at least one activity-related attribute of at least one user activity;

[0358] Receive at least one bias criterion used to determine one or more outliers;

[0359] Determine the set of model parameters for the machine learning model, including:

[0360] (1) Apply the machine learning model with the initial set of model parameters to the training dataset to determine the set of model predictions;

[0361] (2) By comparing the set of model predictions with the corresponding actual values ​​in the training dataset, an error set of data element errors is generated;

[0362] (3) Generate a data selection vector to identify non-outlier target variables, based at least in part on the error set of data element errors and at least one bias criterion;

[0363] (4) Use data selection vectors on the training dataset to generate a dataset of non-outliers;

[0364] (5) Determine the updated set of model parameters for the machine learning model based on the non-outlier dataset; and

[0365] (6) Repeat steps (1)-(5) as iterations until at least one review performance termination criterion is met, thereby obtaining the set of model parameters of the machine learning model as the updated model parameters. Thus, in each iteration, the updated set of model parameters is used as the initial set of model parameters to regenerate the set of predicted values, the set of errors, the data selection vector and the non-outlier dataset.

[0366] A set of classifier model parameters is trained, at least in part, based on a training dataset and a data selection vector, to obtain a trained outlier classifier machine learning model configured to identify at least one outlier data element.

[0367] A trained outlier classifier machine learning model is applied to a dataset of activity-related data for at least one user activity to determine:

[0368] i) Outliers in the activity-related data dataset.

[0369] ii) The non-outlier set of activity-related data in the activity-related data dataset; and

[0370] Machine learning models are applied to a set of non-outlier activity-related data elements to predict future activity-related attributes associated with at least one user activity.

[0371] Clause 3. The systems and methods of Clauses 1 and / or 2 also include:

[0372] The at least one processor applies a data selection vector to the training dataset to determine the outlier training dataset.

[0373] The at least one processor trains at least one outlier-specific model parameter of at least one outlier-specific machine learning model using an outlier training dataset to predict outlier data values; and

[0374] The processor uses an outlier-specific machine learning model to predict outlier activity-related data values ​​for the outlier activity-related dataset.

[0375] Clause 4. The systems and methods of Clauses 1 and / or 2 also include:

[0376] The at least one processor trains the generalized model parameters of the generalized machine learning model using the training dataset to predict data values;

[0377] The at least one processor uses a generalized machine learning model to predict outlier activity-related data values ​​in the outlier activity-related dataset; and

[0378] The at least one processor uses a generalized machine learning model to predict activity-related data values.

[0379] Clause 5. The systems and methods of Clauses 1 and / or 2 also include:

[0380] The at least one processor applies a data selection vector to the training dataset to determine the outlier training dataset.

[0381] The outlier-specific model parameters of the outlier-specific machine learning model are trained by the at least one processor using the outlier training dataset to predict outlier data values;

[0382] The at least one processor trains the generalized model parameters of the generalized machine learning model using the training dataset to predict data values;

[0383] The at least one processor uses an outlier-specific machine learning model to predict outlier activity-related data values ​​in an outlier activity-related dataset; and

[0384] The at least one processor uses an outlier-specific machine learning model to predict activity-related data values.

[0385] Clause 6. The systems and methods of Clauses 1 and / or 2 also include:

[0386] The at least one processor trains the generalized model parameters of the generalized machine learning model using the training dataset to predict data values;

[0387] The at least one processor uses a generalized machine learning model to predict activity-related data values ​​from the activity-related dataset;

[0388] The at least one processor uses an outlier classifier machine learning model to identify outlier activity-related data values; and

[0389] The outlier activity-related data values ​​are removed by the at least one processor.

[0390] Clause 7. Systems and methods of Clauses 1 and / or 2, wherein the training dataset includes at least one active-related property of concrete compressive strength as a function of concrete composition and concrete curing exposure.

[0391] Clause 8. Systems and methods of Clauses 1 and / or 2, wherein the training dataset includes at least one activity-related attribute of energy use data as a function of household environmental conditions and lighting conditions.

[0392] Clause 9. The systems and methods of Clauses 1 and / or 2 also include:

[0393] The at least one processor receives an application programming interface (API) request to generate a prediction having at least one data element; and

[0394] At least one cloud computing resource is instantiated by the at least one processor to schedule the execution of a machine learning model;

[0395] The at least one processor uses a machine learning model to predict at least one activity-related data element value for at least one data element based on the execution schedule; and

[0396] The at least one processor returns at least one activity-related data element value to the computing device associated with the API request.

[0397] Clause 10. Systems and methods of Clauses 1 and / or 2, wherein the training dataset comprises at least one activity-related attribute of three-dimensional patient images from a medical dataset; and

[0398] The machine learning model is configured to predict activity-related data values ​​based on a medical dataset, including two or more physically based rendering parameters.

[0399] Clause 11. Systems and methods of Clauses 1 and / or 2, wherein the training dataset includes at least one activity-related attribute of the simulated control results of electronic machine commands; and

[0400] The machine learning model is configured to predict activity-related data values, including control commands for electronic machines.

[0401] Clause 12. The systems and methods of Clauses 1 and / or 2 also include:

[0402] The at least one processor divides the activity-related data set into multiple activity-related data subsets;

[0403] The at least one processor determines an integration model for each of the multiple activity-related data subsets;

[0404] Machine learning models include model ensembles;

[0405] Each ensemble model comprises a random combination of models from an ensemble of models;

[0406] Each ensemble model is individually used by the at least one processor to predict ensemble-specific activity-related data values;

[0407] The error of each ensemble model is determined by the at least one processor based on ensemble-specific activity-related data values ​​and known values; and

[0408] The processor selects the highest-performing integrated model based on the lowest error.

[0409] While one or more embodiments of this disclosure have been described, it should be understood that these embodiments are illustrative only and not restrictive, and many modifications will be apparent to those skilled in the art, including the various embodiments of the inventive methods, inventive systems / platforms, and inventive devices described herein, which can be used in any combination thereof. Furthermore, the steps can be performed in any desired order (and any desired steps can be added and / or eliminated).

Claims

1. A computer-based method for predicting bias reduction in physical system parameters, comprising: A training dataset representing at least one target physical property selected from the following is received by at least one processor: The compressive strength value of a concrete sample is a function of the concrete composition parameters measured by the sensor and the concrete curing exposure conditions. Building energy consumption as a function of the measured environmental conditions and lighting status; Image-derived quantitative parameters extracted from 3D patient medical images; as well as The activity-related data values ​​of the electronic machine's control commands, which are measured as a function of the machine's state; The at least one processor receives at least one bias criterion for determining one or more outliers; Determining a data selection vector by the at least one processor to select non-outlier data elements of the training dataset includes: (1) The at least one processor applies a reference machine learning model with an initial set of model parameters to a training dataset to determine a set of model predictions for the measured physical property; (2) An error set is generated by the at least one processor by comparing the model predictions with the corresponding measurements of the physical properties; (3) The data selection vector is generated by the at least one processor, based at least in part on the error set and the at least one bias criterion; (4) The at least one processor generates a non-outlier dataset on the training dataset using the data selection vector; (5) The at least one processor determines the updated model parameters of the reference machine learning model based on the non-outlier dataset; and (6) Steps (1)-(5) are repeated by the at least one processor until at least one review performance termination criterion is met; The outlier dataset and the non-outlier dataset are generated by the at least one processor based at least in part on the data selection vector; The at least one processor trains a set of classifier model parameters for a classifier machine learning model, at least in part, based on the outlier dataset, to obtain a trained classifier machine learning model trained to configure the identification of anomalous measurements in the data of the physical property. The at least one processor trains a set of non-outlier model parameters of a base machine learning model, at least in part, based on the non-outlier dataset, to obtain a trained base machine learning model trained to configure the prediction of values ​​of the physical properties; and The at least one processor outputs the trained classifier machine learning model and the trained base machine learning model.

2. The method of claim 1, further comprising: The at least one processor applies a data selection vector to the training dataset to determine the outlier training dataset. The at least one processor trains at least one outlier-specific model parameter of at least one outlier-specific machine learning model using an outlier training dataset to predict outlier data values ​​of the physical property. as well as The at least one processor uses an outlier-specific machine learning model to predict outliers of the physical properties of the outlier dataset.

3. The method of claim 1, further comprising: The at least one processor trains the generalized model parameters of a generalized machine learning model using a training dataset to predict the data values ​​of the measured physical properties; The at least one processor uses a generalized machine learning model to predict outliers of the measured physical properties in the outlier dataset; as well as The at least one processor uses a generalized machine learning model to predict the value of the measured physical property.

4. The method of claim 1, further comprising: The at least one processor applies a data selection vector to the training dataset to determine the outlier training dataset. The outlier-specific model parameters of the outlier-specific machine learning model are trained by the at least one processor using an outlier training dataset to predict outlier data values ​​of the measured physical property; The at least one processor trains the generalized model parameters of a generalized machine learning model using a training dataset to predict the value of the measured physical property; The at least one processor uses an outlier-specific machine learning model to predict outlier data values ​​for the measured physical properties of the outlier dataset; as well as The outlier of the measured physical property is predicted by the at least one processor using an outlier-specific machine learning model.

5. The method of claim 1, further comprising: The at least one processor trains the generalized model parameters of a generalized machine learning model using a training dataset to predict the data values ​​of the measured physical properties; The at least one processor uses a generalized machine learning model to predict the value of the measured physical property; The at least one processor uses an outlier classifier machine learning model to identify outliers of the measured physical property; as well as Outlier values ​​of the measured physical properties are removed by the at least one processor.

6. The method of claim 1, wherein the training dataset includes at least one active-related property of concrete compressive strength as a function of concrete composition and concrete curing exposure.

7. The method of claim 1, wherein the training dataset includes at least one activity-related attribute of energy use data as a function of household environmental conditions and lighting conditions.

8. The method of claim 1, further comprising: The at least one processor receives an application programming interface (API) request to generate a prediction having at least one data element; as well as At least one cloud computing resource is instantiated by the at least one processor to schedule the execution of a machine learning model; The at least one processor uses a machine learning model to predict at least one activity-related data element value for at least one data element based on the execution schedule; as well as The at least one processor returns at least one activity-related data element value to the computing device associated with the API request.

9. The method of claim 1, wherein the training dataset includes the measured physical properties, the measured physical properties comprising quantitative parameters extracted from three-dimensional patient images of a medical dataset; and The machine learning model is configured to predict the values ​​of the physical properties, which include two or more physically based rendering parameters, based on a medical dataset.

10. The method of claim 1, wherein the training dataset includes the measured physical properties, the measured physical properties including simulated control results of electro-machine commands; and in, The machine learning model is configured to predict the values ​​of the measured physical properties, including control commands for electronic machines.

11. The method of claim 1, further comprising: The at least one processor divides the dataset of measured physical properties into multiple subsets; The integration model is determined for each subset by the at least one processor; Machine learning models include model ensembles; Each ensemble model comprises a random combination of models from an ensemble of models; The at least one processor individually utilizes each ensemble model to predict ensemble-specific data values ​​for the measured physical properties; The error of each ensemble model is determined by the at least one processor based on ensemble-specific data values ​​and known measurements; as well as The at least one processor selects the highest-performing integrated model based on the lowest error.

12. A bias reduction prediction system for physical system parameters, comprising: At least one processor communicating with a non-transitory computer-readable storage medium having software instructions stored thereon, wherein the software instructions, when executed, cause the at least one processor to perform the following steps: Receive a training dataset representing a target variable that is selected from at least one of the following: The compressive strength value of a concrete sample is a function of the concrete composition parameters measured by the sensor and the concrete curing exposure conditions. Building energy consumption as a function of the measured environmental conditions and lighting status; Image-derived quantitative parameters extracted from 3D patient medical images; as well as The activity-related data values ​​of the electronic machine's control commands, which are measured as a function of the machine's state; Receive at least one bias criterion used to determine one or more outliers; The data selection vector is determined by the following items to select non-outlier data elements from the training dataset: (1) Apply a reference machine learning model to the training dataset to determine a set of model predictions for the measured physical properties; (2) By comparing the model predictions with the corresponding measured values ​​of the physical properties, an error set of data element errors is generated; (3) The data selection vector is generated at least in part based on the error set and at least one bias criterion; (4) Use data selection vectors on the training dataset to generate a non-outlier dataset; (5) Determine the updated model parameters of the reference machine learning model based on the non-outlier dataset; as well as (6) Repeat steps (1)-(5) until at least one review performance termination criterion is met; Outlier and non-outlier datasets are generated, at least in part, based on the data selection vectors. The classifier model parameters are trained, at least in part, on the outlier dataset to identify anomalous measurements in the physical property data. The parameters of the non-outlier model are trained, at least in part, on the non-outlier dataset to predict the value of the physical property; as well as Output the trained classifier machine learning model and the trained base machine learning model.

13. The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform the following steps: Apply data selection vectors to the training dataset to identify outliers in the training dataset; At least one outlier-specific model parameter is used to train at least one outlier-specific machine learning model using an outlier training dataset to predict outlier data values ​​for the physical property; and Outlier-specific machine learning models are used to predict outliers of the physical properties of outlier datasets.

14. The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform the following steps: The generalized model parameters of the generalized machine learning model are trained using the training dataset to predict the data values ​​of the measured physical properties; The generalized machine learning model is used to predict the values ​​of the measured physical properties in the outlier dataset; and A generalized machine learning model is used to predict the value of the measured physical property.

15. The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform the following steps: Apply data selection vectors to the training dataset to identify outliers in the training dataset; Outlier-specific model parameters are used to train an outlier-specific machine learning model using an outlier training dataset to predict outlier data values ​​for the measured physical property. The generalized model parameters of the generalized machine learning model are trained using the training dataset to predict the values ​​of the measured physical properties; Outlier-specific machine learning models are used to predict outlier data values ​​for measured physical properties of the outlier dataset; as well as Outlier-specific machine learning models are used to predict outlier data values ​​for the measured physical properties.

16. The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform the following steps: The generalized model parameters of the generalized machine learning model are trained using the training dataset to predict the data values ​​of the measured physical properties; The generalized machine learning model is used to predict the value of the measured physical property; An outlier classifier machine learning model is used to identify outliers in the measured physical properties. as well as Remove outlier data values ​​from the measured physical properties.

17. The system of claim 12, wherein the training dataset includes at least one active-related property of concrete compressive strength as a function of concrete composition and concrete curing exposure.

18. The system of claim 12, wherein the training dataset includes at least one activity-related attribute of energy usage data as a function of home environmental conditions and lighting conditions.

19. The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform the following steps: Receive application programming interface (API) requests to generate predictions with at least one data element; as well as Instantiate at least one cloud computing resource to schedule the execution of a machine learning model; Based on the executed schedule, a machine learning model is used to predict the value of at least one activity-related data element for at least one data element; and Return at least one activity-related data element value to the computing device associated with the API request.

20. The system of claim 12, wherein the software instructions, when executed, further cause the at least one processor to perform the following steps: The dataset of measured physical properties is divided into multiple subsets; Determine the ensemble model for each subset; Machine learning models include model ensembles; Each ensemble model comprises a random combination of models from an ensemble of models; Each ensemble model is used individually to predict data values ​​for the measured physical properties specific to the ensemble; The error of each ensemble model is determined based on ensemble-specific data values ​​and known measurements; and The ensemble model with the lowest error is selected based on its performance.

Images