Machine learning based incident evaluation and prioritization methods and systems

Abstract

Machine learning based incident evaluation and prioritization methods and systems are provided. A priority assessment system may apply incident data comprising incident reports to one or more machine learning models to infer a report priority category for at least a first incident report of the incident data based on textual data from the first incident report. The one or more machine learning models may further infer at least one incident classification for the first incident report based on the textual data from the first incident report, and the priority assessment system generating a tagged first incident report based on the first incident report and the at least one incident classification. The tagged first incident report may be selectively conveyed to an incident alert system or used to supplement monitored tracking data based on the report priority category. The incident classification may include an inferred keyword and or a severity score.

Description

MACHINE LEARNING BASED INCIDENT EVALUATION AND PRIORITIZATION METHODS AND SYSTEMSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This patent application is an International Patent Application claiming priority to, and the benefit of, U.S. Provisional Patent Application No. 63 / 510,044, titled “MACHINE LEARNING BASED INCIDENT EVALUATION AND PRIORITIZATION METHODS AND SYSTEMS” filed on June 23, 2023, which is incorporated herein by reference in its entirety.SUMMARY

[0002] This summary is intended to introduce a selection of concepts in a simplified form that are further described below in the detailed description section of this disclosure. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in isolation to determine the scope of the claimed subject matter.

[0003] Embodiments of the present disclosure, at least in part, assess incident data originating from diverse sources using an event assessment system that comprises a machine- leaming-based priority assessment system. An incident report may refer to a report generated to document a non-compliance of a vehicle component with one or more vehicle standards and / or a problem with a vehicle component that can result in an undesired level or occurrence of issues including degradation, maintenance, and / or repair. From the incident data, the machine learning based priority assessment system may use natural language processing to infer information that may be used to identify clusters of incidents of sufficient safety concern to potentially warrant a recall notice. In some embodiments, the priority assessment system may include a first machine learning model that is trained to perform a natural language text categorization task to analyze text and assign a pre-defined category to an incident report based on the content of the text. For example, the report categorization model (operating as a text classifier) may infer from the natural language text of an incident report whether to categorize the incident report into one of three report priority categories, such as, for example, an escalation category, a monitoring category, or a disregard category. In some embodiments, the priority assessment system may include a second machine learning model trained to performan incident classification task based on the natural language text of an incident report. This second machine learning model may generate at least one incident classification for an incident report based on inferring keywords from a set pre-defined keywords from the natural language text that are associated with potential elevated-priority anomalies, and / or a severity score. In some embodiments, an incident may be tagged based on the incident classification and selectively applied to either an incident alter system, or combined into monitored tracking data with other incident reports and tracked. By processing incident data originating from diverse sources using the machine learning model(s), computing systems used for evaluating and / or recommending recall notices may operate substantially more efficiently by focusing their processing power and resources on tagged indecent reports of the incident data that have been already vetted and found to represent an elevated-priority anomaly or other issues suitable to be addressed that could warrant issuing a recall notice, and labeled with keywords and / or a severity score describing the documented incident. The embodiments described herein can thus improve the overall efficiency, effectiveness, and operability of technology that generates recall notices, improving produce quality, product longevity, and user-experience, among other benefits.BRIEF DESCRIPTION OF THE DRAWING

[0004] The embodiments presented in this disclosure related to systems, methods, and technologies for motoring incident reports, identifying recall-related issues, and generating and issuing vehicle recalls are described in detail below with reference to the attached drawing figures, which illustrate non-limiting examples of the disclosed subject matter, wherein:

[0005] FIG. 1 is a data flow diagram of an operating environment suitable for a priority assessment system, in accordance with embodiments of the present disclosure;

[0006] FIG. 2 is a data flow diagram for a machine learning model based priority assessment system, in accordance with embodiments of the present disclosure;

[0007] FIG. 3 is a flow chart illustrating an example method for a machine learning model based priority assessment system, in accordance with embodiments of the present disclosure;

[0008] FIG. 4 is a diagram illustrating an example computing environment suitable for supporting the operations and functions described herein, in accordance with embodiments of the present disclosure; and

[0009] FIG. 5 is a diagram illustrating an example cloud-based computing environment, in accordance with embodiments of the present disclosure.DETAILED DESCRIPTION

[0010] A vehicle manufacturer may issue a recall notice to consumers when a potential issue affecting a vehicle is detected based on analysis of reported incidences. A reported incident may include a report detailing a premature component wear, excess service requirements, degraded or improperly manufactured materials, component degradation, improper labeling, and / or other issues that may adversely affect a driver or passengers, and / or can impact vehicle operation, functionality, and / or longevity. Submissions documenting such incidences may include data that originates from a plurality of sources such as from vehicle dealer service centers, vehicle owners, and / or government agencies. Submissions documenting such incidences may also vary with respect to the vocabulary used, level of detail, level of completeness and / or level of technical accuracy, of the language used to describe the incident. Moreover, raw incident data submissions may be collected in bulk by a vehicle manufacturer into a common data store that comprises incident data across a range of different vehicle makes, models, and years. These factors make it challenging to evaluate incidents and trends that may be used to identify potential safety and / or compliance issues or other anomalies that may warrant the issuance of a recall notice.

[0011] Embodiments of the present disclosure, at least in part, relate to system and methods for analyzing vehicle incident data to identify potential anomalies, issues, concerns, and undesired performance and operation of components and systems that can undesirably impact vehicle operation, longevity, and user-experience, among other things. Incident data submitted to document events and / or issues with vehicle components may originate from a plurality of sources (e.g., vehicle dealer service centers, vehicle owners, and / or government agencies) and may be received in a form that lacks any standard structure with respect to documenting the details of an incident. For example, incident data submitted by consumers may describe similar incidences using varying degrees of precision, accuracy, and / or different terminology depending on the degree of technical sophistication and knowledge of the consumer submitting the data. Incident data submitted by a government agency may use terminology and include information that reflects regulatory language. Incident data submitted by a vehicle dealer service center may include more technically accurate data and includespecialized information such as diagnostic trouble codes (DTC) obtained directly from a vehicle’s diagnostic system. That said, vehicle dealer service center may describe incidents or vehicle conditions using industry and / or local jargon that may vary from dealer-to-dealer, or vary by geographic region. Moreover, the incident data is continuously generated such that a volume of incident submissions from the various sources aggregated into an incident data pool (e.g., stored to a data store) reviewed by a vehicle manufacturer can often be on the order of hundreds of submissions per day or more. The resulting incident data pool thus represents a large, non-homogenous, aggregation of data that lacks a standard report format, terminology, and / or incident details, and that is challenging to efficiently process using existing computing technologies to identify potential vehicle issues, concerns, and / or problems suitable for addressing with a recall.

[0012] Embodiments of the present disclosure, at least in part, assess incident data originating from diverse and / or variable sources using an event assessment system that comprises a machine learning based priority assessment system. From the incident data, the machine learning based priority assessment system may use natural language processing to infer information that may be used to identify clusters of incidents of sufficient overall impact to performance, longevity, and / or user-experience to potentially warrant a recall notice in connection with a particular manufactured vehicle. As such, the event assessment system may function as an early indication system to notify a vehicle manufacturer when the incident data indicates a recall could be warranted. In some embodiments, the incident data received by the priority assessment system may include a plurality of incident reports, each incident report representing a distinct instance of a raw incident data submission (e.g., from a consumer, dealer, and / or government agency). As the term is used herein, an incident report may refer to a report generated to document a potential non-compliance of a vehicle component with one or more vehicle standards, and / or a defect in a vehicle component that poses a potential issue that should be addressed rather than allowed to continue, both of which may be referred to as an elevated-priority anomaly. An incident report may be produced by one of many different stakeholders to describe an incident with a vehicle that may represent or be associated with an elevated-priority anomaly. The priority assessment system may include one or more machine learning models that infer a report categorization and an incident severity classification for respective individual incident reports received by the system.

[0013] In some embodiments, the priority assessment system may include a first machine learning model that is trained to perform a natural language text categorization task.This first machine learning model may be referred to as a report categorization model. Using natural language processing (herein “NLP”), a machine learning model trained as a text classifier can automatically analyze text and assign a pre-defined category to an incident report based on the content of the text. For example, the report categorization model (operating as a text classifier) may infer from the natural language text of an incident report whether to categorize the incident report into one of three report priority categories, such as, for example, an escalation category, a monitoring category, or a disregard category.

[0014] In some embodiments, the report categorization model may comprise a recurrent neural network (RNN), a deep neural network (DNN), and / or other neural network architecture, trained to perform text classification. In some embodiments, the report categorization model may comprise a sequence processing model (e.g., a Long Short Term Memory (LSTM) model or Bidirectional Long Short Term Memory (Bi-LSTM) model) trained to perform text classification.

[0015] In some embodiments, an incident report may be replete with details, but the model may infer that those details may not align with a potential elevated-priority anomaly, so that the report categorization model may assign that incident report to the disregard category. Alternatively, the report categorization model may be trained to recognize when the text of an incident report unambiguously aligns with a potential performance -related defect, and infer that such an incident report should be classified into the escalation category. Other incident reports may be classified by the report categorization model into a monitoring category. Incident reports classified as the monitoring category may not rise to the level of the escalation category, but contain information that may merely suggest, rather than clearly relate to, a potential performance -related defect. Incident reports assigned to the monitoring category may be aggregated and stored as monitored tracking data (e.g., in a database, data store, or similar storage).

[0016] In some embodiments, the priority assessment system may further comprise a second machine learning model trained to perform an incident classification task based on the natural language text of an incident report that has been categorized into either the monitoring category or the escalate category by the report categorization model. This second machine learning model may be referred to as an incident classification model. Using natural language processing (hereinafter “NLP”), the incident classification model can infer classification keywords and / or assign a severity score to incidents that represent potential elevated-priority anomalies based on the content of the text. For example, the incident classification model mayinfer that the text of an incident report is describing an overheating, control, stability, impact, component failure or degradation, and / or other incidences, and assign a corresponding severity score to the incident report. The incident classification model may function as a multiclassification model, meaning that the incident classification model may recognize and infer when the textual content of an incident report is describing more than one type of incident.

[0017] In some embodiments, the incident classification model may comprise a convolutional neural network (CNN), a deep neural network (DNN), and / or other neural network architecture. In some embodiments, the incident classification model is trained as a phrase-matching model. That is, the incident classification model may be trained to correlate text and phrases from an incident report to a defined set of keywords that are associated with potential elevated-priority anomalies. The keywords may be words describing or otherwise indicative of an incident characteristic. For example, words or phrases in the natural language of an incident report that refer to fire, burn, flame, charred, scorched, blaze, glowing, smoke, may all be phrases that are matched by the incident classification model to the predefine keyword “burning”. The incident classification model may be trained to match hundreds of such keywords to phrases appearing in incident reports. In some embodiments, the incident classification model may be trained by a vehicle manufacturer to match words or phrases to keywords based on industry wide incident report data rather than just incident reports received on vehicles of that vehicle manufacturer. That is, even if the vehicle manufacture has never received an incident report describing any form of burning for one of its own vehicles, the incident classification model may none-the-less learn to match phrases related to burning to associated keywords based on incidents reported for other manufacturers. Moreover, synthetic training data may be used to train the incident classification model to correlate words or phrases to keywords for incidents that are exceedingly rare (or not know to have every actually occurred) and that may not be represented in training data derived from real-world incident data.

[0018] Based on the keywords matched to an incident report, the incident classification model may assign a severity score to the incident report. For example, in some embodiments, the incident classification model may correlate keywords with corresponding severity values so that the severity score assigned to an incident report may be generated based on a function of the severity values of the key words matched to the incident report. The severity score may further, or instead, be computed as a function of the frequency of keyword appearance for a particular incident report. In some embodiments, the priority assessment system may tag anincident report with an indication of the keywords and / or severity score inferred by the incident classification model.

[0019] For incident reports categorized as escalate (e.g., by the report categorization model), the incident report, tagged with the keywords and / or severity score, may be transmitted or otherwise conveyed to an incident alert system where it may be queued and for further analysis, aggregation, and tracking to determine if a recall should be issued. For example, the incident alert system may sort a database of received tagged incident reports by severity, keywords, and / or other criteria to determine when to trigger a specific investigation into a particular category of incident that may represent an elevated-priority anomaly that could warrant issuing a recall notice.

[0020] For incident reports categorized as monitor (e.g., by the report categorization model), the priority assessment system may store the tagged incident reports as monitored tracking data to a monitored incident data store. The priority assessment system may apply a pattern tracking algorithm (e.g., such as a clustering algorithm or other grouping algorithm) to track patterns of incident reports stored as monitored report data to identify one or more sets of incident reports that in combination may represent a potential elevated-priority anomaly or other issue that results in degradation, undesired performance characteristics, and / or undesired user-experience. For example, a clustering algorithm may be applied by the priority assessment system to cluster tagged incident reports based on keywords and / or severity scores against vehicle grouping such a such as make, model, model year, mileage, region of use, date of reports (and / or other criteria of data available from an incident report) that identify particular grouping of vehicles. For example a cluster may indicate that incident reports tagged with the same keyword(s) appear prominently in reports for a particular population of vehicles of the same model and band of mileage. A cluster of tagged incident reports exceeding a threshold size (or other threshold pattern criteria) may indicate a relationship of information between reports that represent a potential elevated-priority anomaly that constitutes an issue for escalation to the incident alert system. In some embodiments, when a cluster of tagged incident reports reaches a threshold cluster size, the priority assessment system may aggregate the data from the tagged incident reports belonging to that cluster, and reapply the apply data to the incident classification model to generate keyword classifications and / or a severity score to the cluster, tag the cluster with an indication of the keywords and / or severity score inferred by the incident classification mode, and forward the tagged cluster to the incident alert system where it may be queued and for further analysis, aggregation, and tracking to determine if a recallshould be issued. In other words, the priority assessment system may track the monitored report data from incident reports initially categorized in the monitor category, and when a pattern detected by clustering (or other grouping algorithm) meets a triggering threshold, those incident reports of a cluster may be essentially treated as a compound incident report. Such a compound incident report may then be further processed as an incident report assigned to the escalate category.

[0021] Although the report categorization model and incident classification model have been described separately described above as distinct models, it should be understood that in some embodiments, the priority assessment system may comprise a machine learning model that integrates the functions of both the report categorization model and incident classification model described above.

[0022] As explained in greater detail below, in some embodiments, the priority assessment system may comprise an incident data pre-processor that performs one or more preprocessing functions to condition the incident data prior to processing by the report categorization model and / or incident classification model. For example, an incident data preprocessor may filter out stop words, null words, and / or punctuation from incident reports. In some embodiment, the incident data preprocessor may include a code extractor to extract potentially meaningful data from coded entries of an incident report. The code extractor may identify a code from an incident report and augment the incident report with a natural language tag corresponding to the code.

[0023] By applying incident data to priority assessment system that comprises machine learning based natural language processing to perform report categorization and incident classification tasks, the embodiment presented herein provide for a technology that may efficiently process and assess incident data originating from diverse sources, mitigating variations in the vocabulary used, level of detail, level of completeness and / or level of technical accuracy, of the language used to describe the incident.

[0024] Turning now to FIG. 1, an example configuration of an operating environment 100 in which some implementations of the present disclosure can be employed is provided, in accordance with embodiments herein. It should be understood that this and other arrangements described herein are set forth only as examples. Operating environment 100 is but one example of a suitable telecommunications network and is not intended to suggest any limitation as to the scope of use or functionality of the embodiments disclosed herein. Neither should the operating environment be interpreted as having any dependency or requirement relating to anyone or combination of components illustrated. Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, and the like) can be used in addition to or instead of those shown, and some elements can be omitted altogether for the sake of clarity. Further, many of the elements described herein are functional entities that can be implemented as discrete or distributed components or in conj unction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities are be carried out by hardware, firmware, and / or software. For instance, in some embodiments, some functions are carried out by a processor executing instructions stored in memory as further described with reference to FIG. 4, and / or within a cloud computing environment as further described with respect to FIG. 5.

[0025] It should be understood that operating environment 100 shown in FIG. 1 is an example of one suitable operating environment. Among other components not shown, operating environment 100 includes a user device, such as user device 102, network 104, a data store 106, and one or more servers 108. Each of the components shown in FIG. 1 can be implemented via any type of computing device, such as one or more of computing device 400 described in connection to FIG. 4, or within a cloud computing environment 500 as further described with respect to FIG. 5, for example. These components communicate with each other via network 104, which can be wired, wireless, or both.

[0026] It should be understood that any number of user devices, servers, and other components are employed within operating environment 100 within the scope of the present disclosure. Each component comprises a single device or multiple devices cooperating in a distributed environment. Network 104 can include multiple networks, or a network of networks, but is shown in simple form so as not to obscure aspects of the present disclosure. By way of example, network 104 can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks such as the Internet, and / or one or more private networks. Where network 104 includes a wireless telecommunications network, components such as a base station, a communications tower, or even access points (as well as other components) to provide wireless connectivity. Networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet. Accordingly, network 104 is not described in significant detail.

[0027] The user device 102 can be any type of computing device capable of being operated by a user. For example, in some implementations, user device 102 may comprise the type of computing device described in relation to FIG. 4. By way of example and not limitation,a user device may be embodied as a personal computer (PC), a laptop computer, a mobile device, a smartphone, a tablet computer, a smart watch, a wearable computer, a headset, an augmented reality device, a personal digital assistant (PDA), a handheld communications device, a gaming device or system, an entertainment system, a vehicle computer system, an embedded system controller, a remote control, an appliance, a consumer electronic device, a workstation, any combination of these delineated devices, and / or any other suitable device. Similarly, server 108 can be any type of computing device capable of executing server applications 109 that may provide one or more services to user device 102 via the network 109.

[0028] In some embodiments, the one or more server applications 109 executed on the one or more servers 108 may implement the priority assessment system 110 for assessing and prioritizing vehicle incident data as disclosed herein. That is, the priority assessment system 110 essentially applies machine learning models to perform a triage task to assess priority levels to events captures in incident reports to determine a most effective order in which to deal with them. In some embodiments, the user device 102 may execute at least one client application 103 for accessing and controlling the priority assessment system 110 implemented by the one or more server applications 109. In some implementations, the client application 103 comprises a web application, which can run in a web browser, and could be hosted at least partially on the server-side of environment 100 (such as by server applications 109). It is therefore contemplated herein that "application" should be interpreted broadly. The user device 102 and / or servers 108 can include one or more processors, and one or more computer-readable media to implement the functions of the priority assessment system described herein. The computer-readable media may comprise non-transient tangible media that includes computer- readable instructions executable by the one or more processors. The instructions are embodied by one or more applications, such as application 103 and / or server applications 109 shown in FIG. 1 . In some embodiments, one or more functions of the server applications 109 discussed herein are implemented on the user device 102, on the server(s) 108, or in a distributed form across both the user device 102 and server(s) 108. As illustrated in FIG. 1, data store 106 may store incident data 120 that comprises an aggregation of incident report data that may include data such as, but not limited to, regulatory agency incident data 121, dealer service center incident data 122, and / or consumer incident data 123.

[0029] Turning now to FIG. 2, a data flow diagram illustrating the priority assessment system 110 is shown, in accordance with embodiments of the present disclosure. As shown in FIG. 2, the priority assessment system 110 may include an incident data preprocessor 210, oneor more machine learning models that may include a report categorization model 220 and an incident classification model 230. In some embodiments, the tasks and functions performed by the report categorization model 220 and the incident classification model 230 may be integrated together into a single machine learning model. The priority assessment system 110 may further include, or otherwise be coupled to monitored incident data store 250. As shown in FIG. 2, the priority assessment system 110 may receive incident data 120 (e.g., from the data store 106), wherein the incident data 120 comprises one or more incident reports 205.

[0030] In some embodiments, the incident data preprocessor 210 may condition the incident data 120 to facilitate more efficient processing of the incident data 120 by the report categorization model 220 and incident classification model 230. For example, in some embodiments, one or more of stop words, null words, and / or punctuation may be removed from incident reports 205 of the incident data 120 by the incident data preprocessor 210 using at least one text filter algorithm 212. As previously discussed, stop words are words that do not convey any meaning by themselves (e.g., “a”, “an”, "the” and the like) and are not considered of use for natural language processing tasks performed by the report categorization model 220 and / or incident classification model 230. Null words may include, for example, random character strings or uncorrectable misspellings, and adjectives or adverbs, or other verbiage that may not be useful for discerning potential keywords.

[0031] In embodiments, the incident data preprocessor 210 may comprise a code extractor 214 that functions to extract potentially meaningful data from coded entries of an incident report 205 that may not be recognizable by natural language processing. For example, in some embodiments, an incident report 205 from the incident data 120 (such as dealer incident data 122 originating from a vehicle dealer service center or other repair shop) may comprise diagnostic trouble codes (hereinafter “DTCs”) generated by vehicle on-board computer diagnostic systems and / or other diagnostic system used by automotive technicians. In some embodiments, the code extractor 214 may identify a DTC from an incident report 205 and augment the incident report 205 with a natural language tag corresponding to the DTC (e.g., by inserting text and / or tagging the incident report 205 with a label readable by the report categorization model 220 and / or incident classification model 230). In some embodiments, incident data preprocessor 210 may be coupled to and / or otherwise retrieve trending data 218. When a code is recognized and extracted from an incident report by the code extractor 214, the incident data preprocessor 210 may further augment the incident report 205 with trending data 218 associated with the extracted codes (e.g., using a natural language description) prior toevaluation by the report categorization model 220 and / or incident classification model 230. In some embodiments, other encoded data may be similarly extracted by the code extractor 214 and used to augment an incident report 205 using a natural language description recognizable by natural language processing. An incident report from a regulatory agency, for example, may include coded information. As an example, an incident report 205 from the U.S. National Highway Traffic Safety Administration may include Crash Outcome Data Evaluation System (CODES) encodings, that may be extracted by the code extractor 214 and incorporated into the incident report 205 using natural language text (e.g., either by labeling and / or incorporating additional text into the report).

[0032] In embodiments, the report categorization model 220 is trained to perform a natural language text categorization task using natural language processing (NLP). For example, the report categorization model 220 may infer from the natural language text of an incident report whether to categorize an incident report 205 into the escalation category, the monitoring category, or the disregard category. In some embodiments, the report categorization model 220 may comprise a recurrent neural network (RNN), a deep neural network (DNN), and / or other neural network architecture, trained to perform text classification. In some embodiments, the report categorization model may comprise a sequence processing model (e.g., a Long Short Term Memory (LSTM) model or Bidirectional Long Short Term Memory (Bi-LSTM) model) trained to perform text classification.

[0033] In some embodiments, the natural language text of an incident report 205 may be tokenized (e.g., broken down into simpler words and / or then converted into a sequence or array of integers). For example, the priority assessment system 110 and / or the report categorization model 220 may comprise a tokenizing algorithm trained to recognize text particularly relevant in the context of incidences relevant to safety and / or recall notices, and disregard text that is not. The tokenized representation of the incident report 205 may then by processed by the report categorization model 220 to infer and assign a report category to the incident report 205. Based on the tokenize representation, the report categorization model 220 may determine that an incident report contains no actionable information of relevance to the task of identifying potential elevated-priority anomalies and assign the incident to the disregard category. For example, the information included from the incident report 205 may be too sparse for the report categorization model 220 to reach an inference. In some embodiments, the report categorization model 220 may compute a confidence score for each report categorization inference it predicts, and an when an inference fails to meet a baselineconfidence threshold for either of the escalation or monitor categories, then the report categorization model 220 may automatically assign it to the disregard category. In some embodiments, disregarded incident reports may be stored to a data store for archival purposes.

[0034] In some embodiments, an incident report 205 may be replete with details, but the model 220 may infer that those details do not align with a potential defect or degradation or other performance-impacting issue. In such a case, the report categorization model 220 may assign that incident report to the disregard category. Alternatively, the report categorization model 220 may be trained to recognize when the text of an incident report 205 unambiguously aligns with a potential elevated-priority anomaly, and infer that such an incident report should be classified into the escalation category. An incident report classified into the escalation category may be aggregated with other escalation category incident reports 205 and forwarded to an incident alert system 260 for further evaluation.

[0035] Other incident reports 205 may be classified by the report categorization model 220 into a monitoring category. Incident reports 205 classified into the monitoring category may contain information that merely suggests, rather than clearly relates to, a potential elevated-priority anomaly. For example, the report categorization model 220 may detect from the language of the textual data in an incident report 205 a hesitancy or lack of certainty in the details provided, or detect incomplete information and / or contradicting information. In some embodiments, the report categorization model 220 may classify an incident report 205 into the monitoring category when it computes a low confidence for assigning the incident report 205 to the escalation category (e.g., a confidence score that does not reach at least a confidence threshold for assigning it to the escalation category), but a sufficient confidence (e.g., greater than a low confident threshold) to not place the incident report 205 into the disregard category.

[0036] In some instances, the report categorization model 220 may recognize from textual information data that an incident report 205 may comprises incomplete information that merely suggest a potential elevated-priority anomaly, but that if combined with information from other incident reports, may more fully represent a potential elevated-priority anomaly, and therefore categorize the incident report 205 in the monitoring category. In some embodiments, as further discussed below, incident reports 205 assigned to the monitoring category may be aggregated and stored as monitored tracking data 255 to a monitored incident data store 250 (e.g., in a database, data store, or similar storage).

[0037] In some embodiments, the report categorization model 220 may output a representation of a report categorization (shown at 221) in the form of an embedding and / or avector representation. The report categorization model 220, for example, may output an inferred report categorization in the form of a one -hot vector where each element of the vector comprises a binary indication of whether the incident report 205 has been assigned to a particular category. For example, the report categorization model 220 may generate an output of

[0001] for an incident report 205 categorized as disregard, an output of

[0010] for an incident report 205 categorized as monitor, and an output of

[0100] for an incident report 205 categorized as escalate.

[0038] In some embodiments, the report categorization model 220 may be trained to perform text classification through supervised learning. For example, during training, the report categorization model 220 may be fed training data comprising incident reports prelabeled with ground truth (e.g., incident reports that have been reviewed by a human labeler and tagged as belonging to a category of disregard, monitor or escalate). During training, the report categorization model 220 may be adjusted using a loss feedback that indicates when the report categorization model 220 infers a category to a training incident report that matches that report’s ground truth labeling.

[0039] As discussed above, the one or more machine learning models of the forecasting triage logic 110 may further comprise an incident classification model 230 that is trained to perform an incident classification task based on the natural language text of an incident report 205. In some embodiments, incident classification model 230 infers an incident classification for incident reports 205 that have been categorized as monitoring or escalate by the report categorization model 220 (e.g., and may disregard incident reports 205 in the disregard category). Using natural language processing (NLP), the incident classification model 230 can classify and assign a severity score to incidents reports 205 that represent potential elevatedpriority anomalies based on the content of the text of the incident report 205. For example, the incident classification model 230 may infer that the text of an incident report 205 is describing an incident such as (but not limited to) an automobile fire, a vehicle rollover, a break failure, and / or other incidences, and assign a corresponding severity score to the incident report 205. The incident classification model 230 may function as a multi-classification model, meaning that the incident classification model 230 may recognize and infer when the textual content of an incident report 205 is describing more than one type of incident and accordingly tag the incident report 205 with multiple incident classifications.

[0040] In some embodiments, the incident classification model 230 may comprise a convolutional neural network (CNN), a deep neural network (DNN), and / or other neuralnetwork architecture. In some embodiments, the incident classification model 230 is trained as a phrase-matching model. That is, the incident classification model 230 may be trained to correlate text and phrases from an incident report 205 to a pre-defined set of keywords that are associated with potential elevated-priority anomalies. The keywords may be words describing or otherwise indicative of an incident characteristic. For example, words or phrases in the natural language of an incident report 205 that refer to charring may be matched by the incident classification model to the predefine keyword “burning”. Other example keywords may refer to incidences that include, but are not limited to, engine temperature control, break degradation, loss of traction, exterior lighting, steering malfunction, battery malfunction, and / or other classification(s) of performance and / or conformance issues that are relevant to issuing recall warnings. In some embodiments, the incident classification model 230 may match more than one keyword to a particular phrase of an incident report 205. The incident classification model 230 may be trained to match hundreds of such pre-defined keywords to phrases appearing in incident reports 205.

[0041] Based on the keywords matched to an incident report, the incident classification model 230 may assign a severity score to the incident report 205. For example, in some embodiments, the incident classification model 230 may correlate detected keywords with corresponding severity values so that the severity score assigned to an incident report may be generated based on a function of the severity values of keywords matched to the incident report 205. The severity score may further, or instead, be computed as a function of the frequency of keyword appearance for a particular incident report 205. For example, for an incident report 205 describing multiple details that are matched to the keyword “fire”, the incident classification model 230 may apply a multiplier to adjust the severity score based on the number of times the keyword “fire” was matched to terminology in the incident report 205. In some embodiments, the priority assessment system 1 10 may tag an incident report 205 with an indication of the keywords and / or severity score inferred by the incident classification model 230.

[0042] For incident reports 205 categorized as escalate (e.g., by the report categorization model 220), the tagged escalate incident reports 236 (e.g., tagged with the keywords and / or severity score by the incident classification model 230) may be conveyed to the incident alert system 260 where it may be flagged for further analysis, aggregation, and tracking to determine if a recall should be issued and / or used as input to a recall notice recommendation algorithm. For example, the incident alert system 260 may sort a database ofincident reports 205 that have been tagged by the incident classification model 230 to indicate severity, keywords, and / or other criteria that may be used to determine when to trigger a specific investigation into a particular category of incident that may represent an elevatedpriority anomaly that could warrant issuing a recall notice.

[0043] For incident reports 205 categorized as monitor (e.g., by the report categorization model 220), the tagged monitor incident reports 234 (e.g., tagged with the keywords and / or severity score by the incident classification model 230) may be collected in a monitored incident data store 250 as monitored tracking data 255. In some embodiments, the priority assessment system 110 may apply a pattern tracking algorithm 252 (e.g., such as a clustering algorithm or other grouping algorithm) to track patterns of incident reports stored as monitored report data 255 to identify one or more sets of incident reports that in combination may represent a potential elevated-priority anomaly. For example, a pattern tracking algorithm 252 may be applied to cluster tagged incident reports 205 in the monitored tracking data 255 based on their respective assigned keywords and / or severity scores, against vehicle grouping information such a such as, but not limited to, make, model, model year, mileage, manufacturing plant, region of use, date of reports, and / or other data available from an incident report 205 that may be used identify a particular grouping of vehicles.

[0044] For example a cluster of incident reports 205 detected by the pattern tracking algorithm 252 may indicate that monitor incident reports 236 tagged with the same keyword(s) appear prominently in reports for a particular population of vehicles (e.g., such as vehicles of the same model and mileage range). A cluster of tagged incident reports exceeding a threshold size may indicate a relationship of information between reports that represents a potential elevated-priority anomaly that constitutes an issue for escalation to the incident alert system. In some embodiments, when a cluster of tagged incident reports in the monitored report data 255 reaches a threshold cluster size, the priority assessment system 1 10 may aggregate the data from the tagged incident reports belonging to that cluster, and reapply the incident reports as a set of data to the incident classification model 230 to generate keyword classifications and / or a severity score to the cluster, tag the cluster with an indication of the keywords and / or severity score inferred by the incident classification mode, and forward the tagged cluster to the incident alert system 260, where it may be queued and for further analysis, aggregation, and tracking to determine if a recall should be issued. In other words, the priority assessment system 110, using the pattern tracking algorithm 252, may track the monitored report data 255, and when a pattern detected by clustering (or other grouping technique) meets a triggering threshold, thoseincident reports of a cluster may be essentially treated as a compound incident report. Such a compound incident report may then be further processed in the same manner as an incident report assigned to the escalate category.

[0045] Turning now to FIG. 3, a flowchart illustrating a method 300 for machine learning based priority assessment of incidence reports is provided, in accordance with embodiments of the present disclosure. Method 300 comprises a method for prioritizing incident reports. It should be understood that the features and elements described herein with respect to the method 300 of FIG. 3 can be used in conjunction with, in combination with, or substituted for elements of, any of the other embodiments discussed herein and vice versa. Further, it should be understood that the functions, structures, and other descriptions of elements for embodiments described in FIG. 3 can apply to like or similarly-named or described elements across any of the figures and / or embodiments described herein and vice versa. In some embodiments, elements of method 300 are implemented utilizing elements of the priority assessment system 110 disclosed herein, or other processing device implementing the present disclosure. The method 300 shown in FIG. 3 includes blocks B310 to B318, but is not limited to this selection of elements.

[0046] The method 300 at block 310 includes receiving incident data comprising one or more incident reports from at least one data store. The data store (e.g., such as data store 106) may store incident data that comprises an aggregation of incident report data that may include data such as, but not limited to, regulatory agency incident data 121, dealer service center incident data 122, and / or consumer incident data 123. As the term is used herein, an incident report may refer to a report generated to document a non-compliance of a vehicle component with one or more vehicle standards and / or a defect in a vehicle component that results in degradation or performance -related issues and / or results in an undesirable user experience. An incident report may be produced by one of many different stakeholders to describe an incident with a vehicle that may represent or be associated with an elevated-priority anomaly.

[0047] The method 300 may apply the incident data to one or more machine learning models that infer a report categorization and an incident severity classification for respective individual incident reports received by the system. For example, the method 300 at block 312 includes applying the incident data to one or more machine learning models to infer a report priority category for at least a first incident report of the one or more incident reports as a natural language text categorization task, and infer at least one incident classification for thefirst incident report based on a pre-defined set of keywords that are associated with at least one elevated-priority anomaly.

[0048] The one or more machine learning models may infer at least one keyword indicative of an incident characteristic from the textual data and the method further comprises generating a severity score based on the at least one keyword, wherein the tagged first incident report is tagged based on the at least one keyword and the severity score. The one or more machine learning models comprise a first machine learning model trained to infer the report priority category as a natural language text categorization task. For example, the first machine learning model may be trained (e.g., in a supervised training process) using a set of labeled ground truth incident report training data to determine the report priority category of the first incident report. During training, the report categorization model may be fed training data comprising pre-label incident report (e.g., incident reports that have been reviewed by a human labeler and tagged as belonging to a category of disregard, monitor, or escalate) and adjusted using a loss feedback that indicated when the report categorization model inferred a category for a training incident report that matched that reports labeling.

[0049] As discussed above, may the first machine learning model may be implemented based on the report categorization model 220 discussed above. In some embodiments, the first machine learning model comprises at least one of a recurrent neural network (RNN), a deep neural network (DNN), a sequence processing model, a Long Short Term Memory (LSTM) model, or Bidirectional Long Short Term Memory (Bi-LSTM) model. The one or more machine learning models may comprise a second machine learning model trained as a phrasematching model to infer the at least one incident classification based on a pre-defined set of keywords that are associated with at least one elevated-priority anomaly (e.g., issues that may involve a component failure, degradation, or undesired performance-related characteristic or user-experience). For example, the second machine learning model may be implemented based on the incident classification model 230 discussed above. Such an incident classification model may comprise a convolutional neural network (CNN), a deep neural network (DNN), and / or other neural network architecture.

[0050] The method 300 at block 314 includes generating a tagged first incident report based on the first incident report and the at least one incident classification. In some embodiments, the second machine learning model may be trained to correlate text and phrases from an incident report to a defined set of keywords that are associated with potential elevatedpriority anomalies. Based on the keywords matched to an incident report, the incidentclassification model may assign a severity score to the incident report. For example, in some embodiments, the incident classification model may correlate keywords with corresponding severity values so that the severity score assigned to an incident report may be generated based on a function of the severity values of the key words matched to the incident report. The severity score may further, or instead, be computed as a function of the frequency of keyword appearance for a particular incident report. The first incident report may thus be tagged with at least one incident classification, such as keyword(s) and / or a severity score, to reflect a type of potential elevated-priority anomaly that may be described by the text of the first incident report.

[0051] The method 300 at block 316 includes conveying the tagged first incident report to an incident alert system, when the report priority category for the first incident report comprises a first priority category. Incident reports provided to the incident alert system represent reports thus are those that may provide information to initiate or support a further investigation into a particular category of incident that may represent an elevated-priority anomaly that could warrant issuing a recall notice. For example, the incident alert system may sort a database of received tagged incident reports by severity, keywords, and / or other criteria to determine when to trigger a specific investigation to determine if a recall should be issued. By processing the incident data using the machine learning model(s) of embodiment described herein, an incident alert system may operate substantially more efficiently by focusing processing on tagged indecent reports of the incident data that have been already vetted and found to represent an elevated-priority anomaly that could warrant issuing a recall notice, and labeled with keywords and / or a severity score describing the documented incident.

[0052] The method 300 at block 318 includes adding the tagged first incident report to a data store of monitored tracking data, when the report priority category for the first incident report comprises a second priority category. Thus, in some embodiments, the first priority category may represent a highest report priority category indicating incident reports that should be escalated for further analysis as a potential elevated-priority anomaly, and the second priority may represent a lesser report priority category indicating incident reports that do not by themselves sufficiently describe a potential elevated-priority anomaly- but may when considered in combination with information from other incident reports. In some embodiments, the method may further include discarding the first incident report when the one or more machine learning models infer the report priority category as a third priority category rather than the first or second priority candidates. For example, an incident report comprisinginformation too sparse to categorize in the first or second priority category, or comprising details that do not align with a potential elevated-priority anomaly, may be categorized into the third priority category and disregarded. In some embodiments, disregarded incident reports may be stored to a data store for archival purposes.

[0053] In some embodiments, the method may further include tracking the monitored tracking data using a pattern tracking algorithm to identify a set of tagged incident reports meeting a threshold pattern criteria and conveying the set of tagged incident reports to the incident alert system. For example, the method may include defining a first cluster including a set of tagged incident reports from the monitored tracking data based on the at least one incident classification, and conveying the set of tagged incident reports to the incident alert system based on the first cluster meeting a threshold pattern criteria. The method may include applying a text filter to the incident data to remove at least one of stop words and null words from the one or more incident reports prior to application of the incident data to the one or more machine learning models. In some embodiments, the method may include applying a code extractor to the first incident report to extract one or more coded entries and augment the first incident report with a natural language tag corresponding to the one or more coded entries. In such an embodiment, the method may further include retrieving trending data corresponding to the one or more coded entries and further augment the first incident report based on the trending data.

[0054] Turning now to FIG. 4, one exemplary operating environment for implementing aspects of the technology described herein is shown and designated generally as computing device 400. For example, in some embodiments, one or more parts of the priority assessment system 110 can be implemented using computing device 400. Computing device 400 is just one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the technology described herein. Neither should the computing device 400 be interpreted as having any dependency or requirement relating to any one or combination of components illustrated.

[0055] The technology described herein can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions such as program components, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program components, including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks or implements particular abstract data types. Embodiments of thetechnology described herein can be practiced in a variety of system configurations, including vehicles, industrial machinery, robots, consumer electronics, general-purpose computers, and specialty computing devices. Embodiments of the technology described herein can also be practiced in distributed computing environments where tasks are performed by remoteprocessing devices that are linked through a communications network. For example, the computing device 400 may comprise a computing device integrated within a vehicle, industrial machinery, robot, and / or other mobile or stationary systems.

[0056] With continued reference to FIG. 4, computing device 400 includes a bus 410 that directly or indirectly couples the following devices: at least one memory 412, one or more processors 414, one or more presentation components 416, input / output (I / O) ports 418, I / O components 420, an illustrative power supply 422, and a radio(s) 424. In some embodiments, the bus 410 comprises a vehicle bus that interconnect components inside a vehicle.

[0057] Bus 410 represents one or more busses (such as an address bus, data bus, or combination thereof). Although the various blocks of FIG. 4 are shown with lines for the sake of clarity, it should be understood that one or more of the functions of the components can be distributed between components. For example, a presentation component 416 such as a display device can also be considered an I / O component 420. The diagram of FIG. 4 is merely illustrative of an exemplary computing device that can be used in connection with one or more aspects of the technology described herein. Distinction is not made between such categories as "workstation," "server," "laptop," "tablet," "smart phone" or "handheld device," as all are contemplated within the scope of FIG. 4 and refer to "computer" or "computing device."

[0058] In some embodiments, one or more aspects of the priority assessment system 110, as described in any of the examples of this disclosure, may be implemented at least in part using code executed by the one or more processors(s) 414 and / or other components of computing device 400. In some embodiments, the one or more processors(s) 414 may include one or more central processing units (CPUs) 430 and / or one or more graphics processing units (GPUs) 432. In some embodiments, one or more aspects of priority assessment system 110 (e.g., such as the report categorization model 220 and / or incident classification model 230) may be implemented at least in part using a neural network inference engine executed on the one or more processors(s) 414.

[0059] Computing device 400 typically includes a variety of computer-readable media. Computer-readable media can be any available non-transient media that can be accessed by computing device 400 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data.

[0060] Computer storage media includes non-transient RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices. Computer storage media and computer-readable media do not comprise a propagated data signal or signals per se.

[0061] Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer- readable media.

[0062] Memory 412 includes computer- storage media in the form of volatile and / or nonvolatile memory. Memory 412 may be removable, non-removable, or a combination thereof. Exemplary memory includes solid-state memory, hard drives, optical-disc drives, etc. Memory(s) 412 may include any type of tangible medium that is capable of storing information, such as a database. A database may include any collection of records, data, and / or other information. In some embodiments, monitored tracking data 255 may be stored to the memory 412.

[0063] Computing device 400 includes one or more processors 414 that read data from various entities such as bus 410, memory 412 or RO components 420. One or more presentation components 416 present data indications to a person or other device. For example, presentation components 416 may include an in-cabin annunciator or human machine interface (HMI) which may display information generated by the priority assessment system 110. Example presentation components 416 include a display device, speaker, printing component, vibrating component, etc. VO ports 418 allow computing device 400 to be logically coupledto other devices including I / O components 420, some of which may be built in computing device 400. In some embodiment, I / O ports 418 may comprise at least one network interface to establish a network connection between the priority assessment system 110, data store 106, and / or user device 102. Illustrative I / O components 420 include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, display device, a controller (such as a keyboard, and a mouse), a natural user interface (NUI) (such as touch interaction, pen (or stylus) gesture, and gaze detection), and the like.

[0064] Radio(s) 424 represents a radio that facilitates communication with a wireless telecommunications network. For example, radio(s) 424 may comprise a wireless network interface used to establish communications with one or more of server(s) 108, data store 106, or other network provided service via the network 104. Illustrative wireless telecommunications technologies include CDMA, GPRS, TDMA, GSM, and the like. Radio 424 might additionally or alternatively facilitate other types of wireless communications including Wi-Fi, WiMAX, LTE, and / or other VoIP communications. As can be appreciated, in various embodiments, radio(s) 424 can be configured to support multiple technologies and / or multiple radios can be utilized to support multiple technologies.

[0065] The computing device 400, in some embodiments, is equipped with or connected to sensors such as, but not limited to, temperature sensors, pressure sensors, airflow sensors, air mass sensors, process flow sensors, electrical current and / or voltage sensors, image sensors, cameras, depth cameras, such as stereoscopic camera systems, infrared camera systems, RGB camera systems, and combinations of these, which may be used for generating sensor data for input to the adaptive component enthalpy models discussed herein. Additionally, the computing device 400, in some embodiments, is equipped with accelerometers or gyroscopes that enable detection of motion. The output of the accelerometers or gyroscopes can be provided to the display of the computing device 400 to render immersive augmented reality or virtual reality.

[0066] FIG. 5 is a diagram illustrating a cloud-based computing environment 500 for implementing one or more aspects of the priority assessment system 110 with respect to any of the embodiments discussed herein. Cloud-based computing environment 500 comprises one or more controllers 510 that each comprises one or more processors and memory, each programmed to execute code to implement at least part of the priority assessment system 110 (as shown in FIG. 2). In some embodiments, the one or more controllers 510 comprise server components of a data center. The controllers 510 may be configured to establish a cloud basedcomputing platform executing aspects of the priority assessment system 110. For example, in some embodiments, one or more operations of the priority assessment system 110, report categorization model 220, incident classification model 230 and / or pattern tracking algorithm 252, are virtualized network services running on a cluster of worker nodes 520 established on the controllers 510. For example, the cluster of worker nodes 520 can include one or more pods 522 (such as Kubernetes (K8s) pods) orchestrated onto the worker nodes 520 to realize one or more containerized applications 524 to implement the priority assessment system 110, report categorization model 220, incident classification model 230 and / or pattern tracking algorithm 252. The priority assessment system 110 may thus be at least partially implemented by the containerized applications 524. In some embodiments, incident data 120 may be provided via network 104 to the controllers 510 implementing the priority assessment system 110. In some embodiments the cluster of worker nodes 520 includes one or more one or more data store persistent volumes 530 that implement the data store 106 storing the incident data 120, and / or data store 250 storing the monitored tracking data 255.EXAMPLE CLAUSES

[0067] Clause 1 : A system for prioritizing incident reports, the system comprising: one or more processors coupled to at least one memory, the one or more processors configured to perform operations comprising: applying incident data comprising one or more incident reports to one or more machine learning models to infer a report priority category for at least a first incident report of the incident data based on textual data from the first incident report; applying the incident data to the one or more machine learning models to infer at least one incident classification for the first incident report based on the textual data from the first incident report, and tagging the first incident report based on the at least one incident classification to generate a tagged first incident report; conveying the tagged first incident report to an incident alert system when the one or more machine learning models infer the report priority category as a first priority category; adding the tagged first incident report to monitored tracking data when the one or more machine learning models infer the report priority category as a second priority category; and tracking the monitored tracking data using a pattern tracking algorithm to identify a set of tagged incident reports meeting a threshold pattern criteria and conveying the set of tagged incident reports to the incident alert system.

[0068] Clause 2: The system of clause 1, the operations further comprising: discarding the first incident report when the one or more machine learning models infer the report priority category as a third priority category.

[0069] Clause 3: The system of any of clauses 1-2, wherein the one or more machine learning models infer at least one keyword indicative of an incident characteristic from the textual data, the operations further comprising: generating a severity score based on the at least one keyword, wherein the tagged first incident report is tagged based on the at least one keyword and the severity score.

[0070] Clause 4: The system of any of clauses 1-3, wherein the one or more machine learning models comprise a first machine learning model trained to infer the report priority category as a natural language text categorization task.

[0071] Clause 5: The system of clause 4, wherein the first machine learning model is trained using a set of labeled ground truth incident report training data to determine the report priority category of the first incident report.

[0072] Clause 6: The system of any of clauses 4-5, wherein the first machine learning model comprises at least one of a recurrent neural network (RNN), a deep neural network (DNN), a sequence processing model, a Long Short Term Memory (LSTM) model, or Bidirectional Long Short Term Memory (Bi-LSTM) model.

[0073] Clause 7: The system of any of clauses 1 to 6, wherein the one or more machine learning models comprise a second machine learning model trained as a phrase-matching model to infer the at least one incident classification based on a pre-defined set of keywords that are associated with at least one elevated-priority anomaly.

[0074] Clause 8: The system of any of clauses 1 to 7, the operations further comprising: applying a text filter to the incident data to remove at least one of stop words and null words from the one or more incident reports prior to application of the incident data to the one or more machine learning models.

[0075] Clause 9: The system of any of clauses 1 to 8, the operations further comprising: applying a code extractor to the first incident report to extract one or more coded entries and augment the first incident report with a natural language tag corresponding to the one or more coded entries.

[0076] Clause 10: The system of clause 9, the operations further comprising: retrieving trending data corresponding to the one or more coded entries and further augment the first incident report based on the trending data.

[0077] Clause 11: The system of any of clauses 1 to 10, the operations further comprising: defining a first cluster from the monitored tracking data based on the at least oneincident classification; and conveying the set of tagged incident reports to the incident alert system based on a cluster size of the first cluster exceeding the threshold pattern criteria.

[0078] Clause 12: A priority assessment system for prioritizing incident reports, the system comprising one or more processors configured to: receive incident data comprising one or more incident reports from at least one data store; apply the incident data to one or more machine learning models to infer a report priority category and at least one incident classification for at least a first incident report of the one or more incident reports, based at least on textual data from the first incident report; generate a tagged first incident report based on the first incident report and the at least one incident classification; when the report priority category for the first incident report comprises a first priority category, convey the tagged first incident report to an incident alert system; and when the report priority category for the first incident report comprises a second priority category, add the tagged first incident report to a data store of monitored tracking data.

[0079] Clause 13: The system of clause 12, wherein the one or more machine learning models comprise a first machine learning model trained to infer the report priority category as a natural language text categorization task, and wherein the one or more machine learning models comprise a second machine learning model trained as a phrase-matching model to infer the at least one incident classification based on a pre-defined set of keywords that are associated with at least one elevated-priority anomaly.

[0080] Clause 14: The system of any of clauses 12 to 13, wherein the one or more machine learning models infer at least one keyword indicative of an incident characteristic from the textual data, the one or more processors are further to: generate a severity score based on the at least one keyword, wherein the tagged first incident report is tagged based on the at least one keyword and the severity score.

[0081] Clause 15: The system of any of clauses 12 to 14, wherein the one or more processors are further to: track the monitored tracking data using a pattern tracking algorithm to cluster tagged incident reports of the monitored tracking data based on the at least one incident classification.

[0082] Clause 16: The system of clause 15, wherein the one or more processors are further configured to: convey a set of tagged incident reports to the incident alert system based on a cluster size of a first cluster associated with the set of tagged incident reports exceeding a threshold pattern criteria.

[0083] Clause 17: The system of any of clauses 12 to 16, wherein the one or more processors are further configured to: apply a code extractor to the first incident report to extract one or more coded entries and augment the first incident report with a natural language tag corresponding to the one or more coded entries.

[0084] Clause 18: The system of clause 17, wherein the one or more processors are further configured to: retrieve trending data corresponding to the one or more coded entries and further augment the first incident report based on the trending data.

[0085] Clause 19: A method for prioritizing incident reports, the method comprising: receiving incident data comprising one or more incident reports from at least one data store; applying the incident data to one or more machine learning models to infer a report priority category for at least a first incident report of the one or more incident reports as a natural language text categorization task, and infer at least one incident classification for the first incident report based on a pre-defined set of keywords that are associated with at least one elevated-priority anomaly; generating a tagged first incident report based on the first incident report and the at least one incident classification; when the report priority category for the first incident report comprises a first priority category, conveying the tagged first incident report to an incident alert system; and when the report priority category for the first incident report comprises a second priority category, adding the tagged first incident report to a data store of monitored tracking data.

[0086] Clause 20: The method of clause 19, further comprising: defining a first cluster including a set of tagged incident reports from the monitored tracking data based on the at least one incident classification; and conveying the set of tagged incident reports to the incident alert system based on the first cluster meeting a threshold pattern criteria.

[0087] In various alternative embodiments, system and / or device elements, method steps, or example implementations described throughout this disclosure can be implemented at least in part using one or more computer systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs) or similar devices comprising a processor coupled to a memory and executing code to realize that elements, processes, or examples, said code stored on a non-transient hardware data storage device. Therefore, other embodiments of the present disclosure can include elements comprising program instructions resident on computer readable media which when implemented by such computer systems, enable them to implement the embodiments described herein. As used herein, the terms "computer readable media" and "computer storage media" refer to tangible memory storage devices having non-transient physical forms and include both volatile and nonvolatile, removable and nonremovable media. Such non-transient physical forms can include computer memory devices, such as but not limited to: punch cards, magnetic disk or tape, or other magnetic storage devices, any optical data storage system, flash read only memory (ROM), non-volatile ROM, programmable ROM (PROM), erasable-programmable ROM (E-PROM), Electrically erasable programmable ROM (EEPROM), random access memory (RAM), CD-ROM, digital versatile disks (DVD), or any other form of permanent, semi-permanent, or temporary memory storage system of device having a physical, tangible form. By way of example, and not limitation, computer-readable media can comprise computer storage media and communication media. Computer storage media and computer readable media do not comprise a propagated data signal. Program instructions include, but are not limited to, computer executable instructions executed by computer system processors and hardware description languages such as Very High Speed Integrated Circuit (VHSIC) Hardware Description Language (VHDL).

[0088] Many different arrangements of the various components depicted, as well as components not shown, are possible without departing from the scope of the claims below. Embodiments in this disclosure are described with the intent to be illustrative rather than restrictive. Alternative embodiments will become apparent to readers of this disclosure after and because of reading it. Alternative means of implementing the aforementioned can be completed without departing from the scope of the claims below. Certain features and subcombinations are of utility and can be employed without reference to other features and subcombinations and are contemplated within the scope of the claims.

Claims

CLAIMSWhat is claimed is:

1. A system for prioritizing incident reports, the system comprising: one or more processors (414) coupled to at least one memory (412), the one or more processors (414) configured to perform operations comprising: applying incident data (120) comprising one or more incident reports to one or more machine learning models (220, 230) to infer a report priority category for at least a first incident report (205) of the incident data (120) based on textual data from the first incident report (205); applying the incident data (120) to the one or more machine learning models (220, 230) to infer at least one incident classification for the first incident report (205) based on the textual data from the first incident report (205), and tagging the first incident report (205) based on the at least one incident classification to generate a tagged first incident report (234, 236); conveying the tagged first incident report (234, 236) to an incident alert system (260) when the one or more machine learning models (220, 230) infer the report priority category as a first priority category; adding the tagged first incident report (234, 236) to monitored tracking data (255) when the one or more machine learning models (220, 230) infer the report priority category as a second priority category; and tracking the monitored tracking data (255) using a pattern tracking algorithm (252) to identify a set of tagged incident reports meeting a threshold pattern criteria and conveying the set of tagged incident reports to the incident alert system (260).

2. The system of claim 1, the operations further comprising: discarding the first incident report (205) when the one or more machine learning models (220, 230) infer the report priority category as a third priority category.

3. The system of claim 1, wherein the one or more machine learning models (220, 230) infer at least one keyword indicative of an incident characteristic from the textual data, the operations further comprising:generating a severity score based on the at least one keyword, wherein the tagged first incident report is tagged based on the at least one keyword and the severity score.

4. The system of claim 1, wherein the one or more machine learning models (220, 230) comprise a first machine learning model (220) trained to infer the report priority category as a natural language text categorization task.

5. The system of claim 4, wherein the first machine learning model (220) is trained using a set of labeled ground truth incident report training data to determine the report priority category of the first incident report (205).

6. The system of claim 4, wherein the first machine learning model (220) comprises at least one of a recurrent neural network (RNN), a deep neural network (DNN), a sequence processing model, a Long Short Term Memory (LSTM) model, or Bidirectional Long Short Term Memory (Bi-LSTM) model.

7. The system of claim 1, wherein the one or more machine learning models (220, 230) comprise a second machine learning model (230) trained as a phrasematching model to infer the at least one incident classification based on a pre-defined set of keywords that are associated with at least one elevated-priority anomaly.

8. The system of claim 1 , the operations further comprising: applying a text filter (212) to the incident data (120) to remove at least one of stop words and null words from at least a first incident report (205) prior to application of the incident data (120) to the one or more machine learning models (220, 230).

9. The system of claim 1 , the operations further comprising: applying a code extractor (214) to the first incident report (205) to extract one or more coded entries and augment the first incident report (205) with a natural language tag corresponding to the one or more coded entries.

10. The system of claim 9, the operations further comprising: retrieving trending data (218) corresponding to the one or more coded entries and further augment the first incident report (205) based on the trending data (218).

11. The system of claim 1 , the operations further comprising: defining a first cluster from the monitored tracking data (255) based on the at least one incident classification; and conveying the set of tagged incident reports to the incident alert system (260) based on a cluster size of the first cluster exceeding the threshold pattern criteria.

12. A priority assessment system for prioritizing incident reports, the system comprising one or more processors (414) configured to: receive incident data (120) comprising one or more incident reports (205) from at least one data store (106); apply the incident data (120) to one or more machine learning models (220, 230) to infer a report priority category and at least one incident classification for at least a first incident report (205) of the one or more incident reports (205), based at least on textual data from the first incident report (205); generate a tagged first incident report (234, 236) based on the first incident report (205) and the at least one incident classification; when the report priority category for the first incident report (205) comprises a first priority category, convey the tagged first incident report (236) to an incident alert system (260); and when the report priority category for the first incident report (205) comprises a second priority category, add the tagged first incident report (234) to a data store (250) of monitored tracking data (255).

13. The system of claim 12, wherein the one or more machine learning models (220, 230) comprise a first machine learning model (220) trained to infer the report priority category as a natural language text categorization task, and wherein the one or more machine learning models (220, 230) comprise a second machine learning model (230) trained as a phrase-matching model to infer the at least one incident classification based on a predefined set of keywords that are associated with at least one elevated-priority anomaly.

14. The system of claim 12, wherein the one or more machine learning models (220, 230) infer at least one keyword indicative of an incident characteristic from the textual data, the one or more processors (414) are further to:generate a severity score based on the at least one keyword, wherein the tagged first incident report (234, 236) is tagged based on the at least one keyword and the severity score.

15. The system of claim 12, wherein the one or more processors (414) are further to: track the monitored tracking data (255) using a pattern tracking algorithm (252) to cluster tagged incident reports of the monitored tracking data (255) based on the at least one incident classification.

16. The system of claim 15, wherein the one or more processors (414) are further configured to: convey a set of tagged incident reports to the incident alert system (260) based on a cluster size of a first cluster associated with the set of tagged incident reports exceeding a threshold pattern criteria.

17. The system of claim 12, wherein the one or more processors (414) are further configured to: apply a code extractor (214) to the first incident report (205) to extract one or more coded entries and augment the first incident report (205) with a natural language tag corresponding to the one or more coded entries.

18. The system of claim 17, wherein the one or more processors (414) are further configured to: retrieve trending data (218) corresponding to the one or more coded entries and further augment the first incident report (205) based on the trending data (218).

19. A method for prioritizing incident reports, the method comprising: receiving incident data (120) comprising one or more incident reports (205) from at least one data store (106); applying the incident data (120) to one or more machine learning models (220, 230) to infer a report priority category for at least a first incident report (205) of the one or more incident reports as a natural language text categorization task, and infer at least one incident classification for the first incident report (205) based on a pre-defined set of keywords that are associated with at least one elevated-priority anomaly; generating a tagged first incident report (234, 236) based on the first incident report (205) and the at least one incident classification;when the report priority category for the first incident report (205) comprises a first priority category, conveying the tagged first incident report (236) to an incident alert system (260); and when the report priority category for the first incident report (205) comprises a second priority category, adding the tagged first incident report (234) to a data store (250) of monitored tracking data (255).

20. The method of claim 19, further comprising: defining a first cluster including a set of tagged incident reports from the monitored tracking data (255) based on the at least one incident classification; and conveying the set of tagged incident reports to the incident alert system (260) based on the first cluster meeting a threshold pattern criteria.

Images