Automatic classification of emotion recognition

Abstract

A system and method for detecting psychological emotions in natural language content using a rule-based engine includes receiving the natural language content as text input, searching for matches between linguistic rules for a given emotion cognition and components of the natural language content, activating a matching linguistic rule, where an instance of the linguistic rule has a human dimension, evaluating the human dimension of the matched rule, scoring each human dimension to obtain a profile of dimension scores for the given emotion cognition, aggregating dimensions in the obtained profile of dimension scores to obtain an intensity indication of the given emotion cognition, and displaying the natural language content in a manner relating the matching linguistic rules to the given emotion cognition and the respective intensity indication of the given emotion cognition.

Description

Technical Field

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority from non-provisional application No. 17 / 589,512 filed on January 31, 2022, claims the benefit of priority from provisional application No. 63 / 163,618 filed on March 19, 2021, claims the benefit of priority from provisional application No. 63 / 163,621 filed on March 19, 2021, claims the benefit of priority from provisional application No. 63 / 162,987 filed on March 18, 2021, and claims the benefit of priority from provisional application No. 63 / 143,730 filed on January 29, 2021, the entire contents of all of which are incorporated herein by reference.

[0002] [1] The present disclosure relates generally to monitoring emotions, and more specifically to classifying and tracking the intensity of emotional perception in natural language content.

Background Art

[0003] [2] Affective computing is the study and development of systems and devices that can recognize, interpret, process, and simulate human emotions. Affective computing is an interdisciplinary field that brings together linguistics, statistics, human factors, and computer science. One aspect of affective computing is enabling bots and other computer applications to intelligently respond to natural human emotional feedback. In the case of text, affective computing includes sentiment detection from text. Often, a form of sentiment detection known as sentiment analysis is used to classify text as positive, negative, or neutral. Almost every large company that develops computer software and hardware, as well as university research and some newly established companies, have development projects that include some form of tool for sentiment analysis. The popularity of sentiment analysis stems from the need to better understand sentiment in responses to news media and various customer comments, which are widespread in social media, customer product comments, and interactions with chatbots.

[0004] [3] In practice, sentiment analysis is the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective emotional cognitive states and subjective information. Sentiment analysis has been accelerated in part by the availability of large datasets of human dialogue obtained from sources such as various social media platforms, recorded conversations, and other outlets for text representation. However, sentiment analysis must deal with the evolutionary nature of natural language. For example, sentiment analysis must deal with subtle differences or nuances of meaning between words, or even entire phrases. Some phrases may appear to express the same idea, but they may indicate differences in sentiment. Sentiment analysis must deal with words or phrases that may have different meanings depending on the context.

[0005] [4] Despite the remarkable recent advances in the techniques used for natural language processing, sentiment analysis suffers from the drawbacks of being typically manually classified and therefore subjective, tied to training sets. In particular, annotation of large training datasets of text is performed manually. Training sets by manual classification methods tend to be slow, poorly performing, and expensive. Furthermore, financially competitive methods for acquiring evaluators, primarily Mechanical Turk, have the drawback of being incentivized by non-native speakers to rush through classification tasks, and attempts to mitigate these effects are limited to cumbersome ID scanning and unreliable IP address filtering, resulting in low-quality, inconsistent results. Detecting subtle sentiment is difficult and error-prone.

[0006] [5] Moreover, the training sets used for machine learning do not bring about creativity in language. Humans have an obviously limitless capacity to generate new sentences that have never been written or spoken before, through speech and writing. Indeed, the essential property of language is precisely this capacity for such creativity, to which current technological systems are incapable of effectively adapting.

[0007] [6] State-of-the-art machine learning models for natural language processing (BERT (Bidirectional Encoder Representation from Transformers), GPT2 (Generative Pre-trained Transformer 2), GPT3) are seemingly robust tools that can hold a great deal in memory and look effectively in both directions, but they do not match the evolving nature of natural language. Statistical models rely on strongly statistical, often probabilistic components. Supervised machine learning models predict what they have seen before or relationships they have seen reappear. There are countless phrases and sentences that will be created in the future that cannot be encapsulated by sentences that have occurred before, including elements such as compound words, slang, jargon, metaphors or neologisms.

[0008] [7] In addition, sentiment analysis still suffers from semantic shortcomings, which are partly due to the lack of effective methods for measuring the intensity of sentiment and the lack of use of vetted, non-representative datasets for training purposes. Furthermore, over-reliance on lexical or token-based solutions prevents such solutions from ultimately gaining endurance when linguists and technicians grapple with the dynamic nature of language, including semantic shifts over time, obsolescence, and various other linguistic changes.

[0009] [8] Thus, sentiment analysis has several major drawbacks, namely the lack of best-reference datasets with objectively rated or labeled data for training, the limitations of n-gram-based solutions, the inability to determine genuine intensity, the difficulty in parsing hashtags, and the lack of rigorous validation of results. Furthermore, while affectivity ratings of a similar scale have been subjectively applied, there is no robust means within industry or academia to validate the classification results.

[0010] [9] Occasional attempts to quantify practical factors appear in the sociolinguistic literature (overwhelmingly through brainstormed enumerations and occasionally using n-gram selection with high token:type ratios), but these are sociolinguistic studies that focus primarily on fMRI studies of intuitive and practical examples, social or real-world experiments, intuitive analyses, as well as rigorous examination and comparison. Certain, but limited, syntax-driven theories exist within the field primarily in individual constructions such as Andrews constructions (adjective and adverbial phrases with the constructions "X knows," e.g., "God[only] knowss," "Who knows"), or, somewhat more broadly, in semantically rich and complex but verb-centered semantic analyses of syntactic grammars that focus on specific grammatical constructions for object relations, known as argument structures, which essentially deal with the verbal argument (object).

[0011]

[10] Preferably, for a word to be syntactic, the pieces of language must be interchangeable and modular. Essentially, this results in a focus not only on grammatical rules but also on common usage. Sociolinguistics focuses on pragmatics, the science of real-world usage, which analyzes the use of specific words in a broader view. Syntactic analysis, however, focuses on argument structure and verb-focused analysis of morphosenical pairings. There is a need for more efficient, more robust and more accurate detection of cognition, emotion, and emotion cognition, bridging cognitive science, sociolinguistics, and semantics.

[0012]

[11] The above “Background Art” description is intended to provide a general context for the present disclosure. The inventor’s work, as well as any description that may not be considered prior art at the time of filing, as described in this Background Art section, is not explicitly or implicitly considered prior art to the present invention. [Overview of the project]

[0013]

[0012] According to one embodiment of the present disclosure, one aspect is a method for automatically augmenting natural language content with emotion recognition by a processing circuit, the method including receiving natural language content as text input via an input device; the processing circuit searching for a match between a plurality of language rules for a given emotion recognition and components of the text input, wherein an instance of a language rule has at least one human dimension, the processing circuit activating the matched language rule and evaluating at least one human dimension of the activated matched language rule; the processing circuit scoring each human dimension to obtain an archetypal profile of the dimension score of a given emotion recognition; the processing circuit aggregating the dimensions in the obtained profile of the dimension score to obtain an intensity indication of a given emotion recognition; and relating the matched language rule to a given emotion recognition and displaying the augmented natural language content on a display in a manner that signals the respective intensity indications of the given emotion recognition.

[0014]

[0013] Furthermore, according to one embodiment of the present disclosure, a further aspect is an electronic reader which may include a touchscreen display, a processing circuit and a memory, wherein the touchscreen display is configured to display the text of an ebook, the processing circuit is configured to scan and tag the text using rules that, when triggered, detect an emotional recognition state and determine the intensity in which the emotional recognition state occurs, the processing circuit is configured to generate and display one or more sidebars for listing dynamic and emotional recognition intensity information based on the detected components of the displayed text, the touchscreen is configured to select a dynamic or emotional recognition intensity when touched at a position in the display, and the processing circuit is further configured to generate and display a color-coded highlight indicating the occurrence of the selected dynamic or emotional recognition intensity.

[0015]

[0014] Furthermore, according to one embodiment of the present disclosure, a further aspect is a system for mitigating psychological disorders, the system may include a mobile device having a processing circuit and memory, and a peripheral device having a communication device and one or more actuators, wherein the memory of the mobile device stores program instructions, and when the program instructions are executed by the processing circuit of the mobile device, the mobile device receives natural language content as text input via an input device, and the processing circuit searches for a match between a given set of language rules for emotion recognition and the components of the text input. The method includes, in doing so, having a processing circuit detect a matched language rule in order to obtain a given emotion cognition intensity indication in which an instance of a language rule has at least one human dimension, and transmitting a first activation signal that identifies the emotion cognition intensity when the given emotion cognition intensity indication reaches a negative emotion cognition intensity that exceeds a first threshold, and a peripheral device is configured to receive the transmitted first activation signal via a communication device and to activate one or more actuators to produce a sensory distraction to alleviate psychological distress.

[0016] The above general description of exemplary implementations and the following detailed description thereof are merely illustrative and not limiting to the teachings of this disclosure.

[0017]

[15] The accompanying drawings incorporated herein and forming part thereof illustrate one or more embodiments, and these embodiments are described together with the specification. The accompanying drawings are not necessarily drawn to a certain scale. Any values ​​or dimensions shown in the accompanying graphs and figures are for illustrative purposes only and may or may not represent actual or preferred values ​​or dimensions. Where applicable, some or all features may not be illustrated to aid in the description of the underlying features.

[0018]

[16] The features and advantages of the exemplary embodiments are presented in more detail in the following description with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] [Figure 1]

[17] A block diagram of a system for automatic classification of beliefs, opinions, sentiments, and emotions, according to an exemplary aspect of the present disclosure. [Figure 2]

[18] A block diagram of a computer system, according to an exemplary aspect of the present disclosure. [Figure 3]

[19] A system block diagram for automatic classification of emotions, according to an exemplary aspect of the present disclosure. [Figure 4]

[20] A diagram of a training architecture, according to an exemplary aspect of the present disclosure. [Figure 5]

[21] A diagram of an adaptive operation of an emotion classification system, according to an exemplary aspect of the present disclosure. [Figure 6]

[22] A diagram showing types of linguistic rules, according to an exemplary aspect of the present disclosure. [Figure 7]

[23] A diagram of a bottom-up stack of layers of a rule engine, according to an exemplary aspect of the present disclosure. [Figure 8]

[24] A flowchart of an operation method of a computer system, according to an exemplary aspect of the present disclosure. [Figure 9]

[25] A flowchart of steps for evaluation using linguistic rules, according to an exemplary aspect of the present disclosure. [Figure 10]

[26] A flowchart for detecting rules, according to an exemplary aspect of the present disclosure. [Figure 11]

[27] A flowchart for scoring, according to an exemplary aspect of the present disclosure. [Figure 12]

[28] A flowchart for detecting rules, according to an exemplary aspect of the present disclosure. [Figure 13]

[29] A flowchart for determining a numerical value for a token proximity rule, according to an exemplary aspect of the present disclosure. [Figure 14]

[30] A flowchart of classification, according to an exemplary aspect of the present disclosure. [Figure 15]

[31] A flowchart of hybrid multi-model learning, according to an exemplary aspect of the present disclosure. [Figure 16]

[32] A diagram illustrating an electronic reader, according to an exemplary aspect of the present disclosure. [Figure 17]

[33] A flowchart of the operation of an electronic reader, according to an exemplary aspect of the present disclosure. [Figure 18]

[34] A system flow diagram for a multimedia audiobook or a visuo-spatial data sentiment classifier, according to an exemplary aspect of the present disclosure. [Figure 19]

[35] A block diagram of a multimedia rule engine, according to an exemplary aspect of the present disclosure. [Figure 20]

[36] A flowchart of a rule discovery engine based on HUNCHes, according to an exemplary aspect of the present disclosure. [Figure 21A]

[37] A flowchart of rule discovery in audio media, according to an exemplary aspect of the present disclosure. [Figure 21B] A flowchart of rule discovery in audio media, according to an exemplary aspect of the present disclosure. [Figure 22]

[38] A graph of speech signal patterns. [Figure 23]

[39] A flowchart of a method for real-time sentiment classification in a video / audio stream, according to an exemplary aspect of the present disclosure. [Figure 24]

[40] A diagram illustrating a display device, according to an exemplary aspect of the present disclosure. [Figure 25]

[41] A system diagram of the adaptive operation of a sentiment classification system, according to an exemplary aspect of the present disclosure. [Figure 26A]

[42] A schematic diagram of an electronic bracelet, according to an exemplary aspect of the present disclosure. [Figure 26B] A schematic diagram of an electronic bracelet, according to an exemplary aspect of the present disclosure. [Figure 26C]A schematic diagram of an electronic bracelet according to an exemplary embodiment of the present disclosure. [Figure 27]

[43] A circuit diagram of an electronic bracelet according to an exemplary aspect of the present disclosure. [Figure 28]

[44] A system diagram of user interface features for sociolinguistic data according to an exemplary aspect of the present disclosure. [Figure 29]

[45] A flowchart of a sociolinguistic engine pipeline according to an exemplary aspect of the present disclosure. [Modes for carrying out the invention]

[0020]

[46] The following description with respect to the attached drawings is intended to describe various embodiments of the disclosed subject matter and is not necessarily intended to represent only one embodiment. In some cases the description includes specific details to provide an understanding of the disclosed embodiments. However, it will be apparent to those skilled in the art that the disclosed embodiments can be practiced without those specific details. In some cases well-known structures and components are shown in the form of block diagrams to avoid obscuring the concepts of the disclosed subject matter.

[0021]

[47] Any reference to “one embodiment,” “several embodiments,” or “embodiment” as used herein means that any particular element, feature, structure, or characteristic described in relation to this embodiment is included in at least one embodiment. The phrase “in one embodiment” appearing in various places herein does not necessarily refer to all of the same embodiments. In particular, conditional language as used herein, such as “can,” “may,” “might,” “may,” “may,” and “for example,” is generally intended to convey that a particular embodiment includes certain features, elements, and / or steps, and other embodiments do not include them, unless otherwise specified or understood in the context in which they are used. In addition, the articles “a” and “an” as used in this application and the appended claims should be interpreted as “one or more” or “at least one,” unless otherwise specified.

[0022]

[48] ​​Next, referring to the figures, where similar reference numbers designate equivalent or corresponding parts across several figures, the following description relates to a system and method for the automated classification of beliefs, opinions, sentiments, and emotions. The method optimally includes a language rule engine as an input layer to a probabilistic layer and a layer for determining intensity.

[0023]

[49] As mentioned above, language is constantly evolving, and even robust tools that can hold a vast amount in memory and can be viewed effectively bidirectionally (i.e., BERT) cannot match evolving natural languages. There are countless sentences to be created in the future that cannot be encapsulated by sentences that have occurred before, including elements such as compound words, slang, jargon, metaphors, or neologisms. Focusing on replaceable syntax and mining them for extractable belief, opinion, sentiment, and affect dimensions will enable more efficient, more robust, and more accurate detection.

[0024]

[50] Furthermore, in natural language processing, probability is a poor indicator of the intensity of a word, since frequency is not necessarily equal to intensity. Therefore, in the embodiments disclosed, probability is used as an indication that emotion is present in a given portion of the natural language input. The intensity of emotion is determined separately.

[0025]

[51] Various techniques may be used to analyze a text in an attempt to understand the sentiments, feelings, opinions, or beliefs that may be expressed or implied by the text. Sentiment may be defined as an attitude, thought, or judgment that is facilitated by a feeling. A related species to sentiment is an emotion, which may be defined as a strong feeling such as love, anger, joy, hatred, or fear. An emotion may include a conscious mental response that is experienced subjectively as a strong feeling and is accompanied by physiological and behavioral changes of the body. An opinion may be defined as a belief, judgment, or way of thinking about something. A belief may be anything that has been accepted, considered to be true, or held as an opinion. In this disclosure, the term “emotio-cognition” is used in its ordinary sense to describe each of the following: sentiment, opinion and belief; emotion, judgment; and feelings (affect / desire, emotional or sexual connection); interpersonal / social forces (affinity, community, connection, influence); cognitive elements (thoughts, opinions, beliefs, stances); and the intermediate sentiment space (aspirations, values, motivations, regrets).

[0026]

[52] Embodiments disclosed utilize lexical rules having a semantic syntax that are pre-scored across multiple dimensions and include building blocks for semantic sentiment logic operations. Rule search criteria make tagging easier, faster, and more empirical, thus reducing the need for GPU fine-tuning (i.e., fine-tuning pre-trained transformers such as BERT, GPT2, GPT3), matching computations, or high RAM computations. Rule suggestion via dimension pattern spotting, sentiment occurrence tracking, and sequence spotting can also save the enormous amount of resources that would be used to tag, retrain, or use resources to add epochs for accuracy increases or to add multiple models to solve ambiguity problems. Logical derivation of other sentiment phenomena via dimension aggregation / processing enables simple line-by-line search and counting instead of heavy computations, creating novel ways of computation (detection of mental health signs) without the need for new or specialized training sets, new tasks, or added layers.

[0027]

[53] Figure 1 is a block diagram of a system for automated classification of sentiment cognition according to an exemplary embodiment of the present disclosure. System 100 includes a text input 102 that receives text from various sources, including continuous data streams, social media dialogues, documents, and entire books. The text received from text input 102 undergoes a data cleaning and data normalization process 104. Various tools are available for cleaning and text normalization, which generally involve removing unwanted characters from the text data and normalizing words. The selection of which characters are unwanted is user-dependent. For example, in some cases punctuation may be unwanted. In some embodiments, certain punctuation may not be removed and may instead be used in later processing. Certain punctuation may include commas, quotation marks, and exclamation marks. Punctuation that may be removed may include the at (@) symbol, hashtag (#), dollar sign ($), percent sign (%), carrot sign (^), ampersand (&), and asterisk (*). In some embodiments, markup such as HTML tags may be removed. In some embodiments, emoticons may be left as they are. In addition, text may be converted to lowercase. The cleaned and normalized data is then preprocessed, including tokenization, part-of-speech tagging, stemming, and lemmatization.106 Tokenization divides the text into individual elements (for example, dividing the cleaned text by its whitespace). Part-of-speech tagging attaches labels to those word tokens to identify the part of speech of the word tokens. Stemming is the process of converting a word into its base form. Lemmmatization is the process of obtaining the base form of individual words. Another task may be to remove stop words. The preprocessed data may be formatted for unsupervised learning processes108, rule-based systems110, and supervised learning processes112. A typical method for formatting preprocessed words is to use Google's word2vec algorithm.

[0028]

[54] An unsupervised learning process 108 may classify data, including the classification of emotion cognition. The unsupervised learning process 108 does not require labeled data, but instead may cluster pre-processed data into classes. The pre-processed data input and the resulting classifications may be used for feature / rule suggestion 114. The suggested rules and feature selection 120 may be performed to generate future language rules.

[0029]

[55] The rule-based system 110 includes a set of linguistic rules, which may be organized by categories of emotion cognition.

[0030]

[56] Supervised learning processes 112 require labeled data. Labeling of emotion cognitions may be performed manually. Supervised learning processes 112 may be used for machine classification of emotion cognitions 118. Classification errors may be adjusted / corrected to improve subsequent classifications 122. Supervised learning processes 112 generate neural models to perform their own classifications, assigning probabilities. Neural models are also trained on the rules themselves. They locate similar co-occurrence vectors, similar POS patterns, and similar n-grams, suggesting these as latent rules / suggested features.

[0031]

[57] Aggregation and scoring 116 can be applied to the results of the unsupervised learning process 108 and the supervised learning process 112. Both the unsupervised learning process 108 and the supervised learning process 112 may output probabilities for each class (for example, using the softmax function).

[0032]

[58] In one implementation, the functions and processes of system 100 may be implemented by computer 226. Next, a hardware description of computer 226 according to an exemplary embodiment will be described with reference to Figure 2. In Figure 2, computer 226 includes a CPU 200 that performs the processes described herein. Process data and instructions may be stored in memory 202. These processes and instructions may also be stored on a storage medium disk 204 such as a hard drive (HDD) or portable storage medium, or they may be stored remotely. Furthermore, the claimed advances are not limited to the form of computer-readable medium on which the process instructions of the present invention are stored. For example, the instructions may be stored on a CD, DVD, flash memory, RAM, ROM, PROM, EPROM, EEPROM®, hard disk, or any other information processing device with which computer 226 communicates, such as a server or computer.

[0033]

[59] Furthermore, the claimed advances may be provided as utility applications, background daemons, or components of an operating system, or combinations thereof, that run with the CPU200 and an operating system such as Microsoft® Windows®, UNIX®, Oracle® Solaris, LINUX®, Apple macOS® and other systems known to those skilled in the art.

[0034]

[60] To realize the computer 226, the hardware elements can be realized by various circuit elements known to those skilled in the art. For example, the CPU 200 may be a Xenon® or Core® processor from Intel Corporation in the United States, or an Opteron® processor from AMD in the United States, or may be other processor types recognized by those skilled in the art. Alternatively, the CPU 200 may be implemented on an FPGA, ASIC, PLD, or using discrete logic circuits, as recognized by those skilled in the art. Furthermore, the CPU 200 may be implemented as multiple processors working in parallel and cooperatively to carry out the instructions of the process of the present invention as described above.

[0035]

[61] The computer 226 in Figure 2 also includes a network controller 206, such as an Intel Ethernet® PRO network interface card from Intel, Inc., USA, to interface with the network 224. As may be understood, the network 224 may be a public network such as the Internet, or a private network such as a LAN or WAN network, or any combination thereof, and may also include a PSTN or ISDN subnetwork. The network 224 may also be wired, such as an Ethernet network, or wireless, such as a cellular network, including EDGE, 3G and 4G wireless cellular systems. The wireless network may also be WiFi®, Bluetooth®, or any other wireless form of known communication.

[0036]

[62] The computer 226 further includes a display controller 208, such as an NVIDIA® GeForce® GTX or Quadro® graphics adapter from NVIDIA, Inc., USA, for interface with a display 210, such as a Hewlett Packard® HPL2445w LCD monitor. The general-purpose I / O interface 212 interfaces with a keyboard and / or mouse 214, as well as an optional touchscreen panel 216, or a haptic device on or separate from the display 210. The general-purpose I / O interface also connects to various peripherals 218, including printers and scanners such as OfficeJet® or DeskJet® from Hewlett Packard®. The I / O interface 212 may also connect to a microphone for audio input and to speakers and / or headphones for sound output. The microphone and / or headphones may be connected to the I / O interface 212 via input ports, including USB, HDMI®, or other peripheral input connections.

[0037]

[63] The general-purpose storage controller 220 connects the storage medium disk 204 to a communication bus 222 which may be an ISA, EISA, VESA, PCI, or similar for interconnecting all the components of the computer 226. A general description of the features and functions of the display 210, keyboard and / or mouse 214, as well as the display controller 208, storage controller 220, network controller 206, and general-purpose I / O interface 212 is omitted herein for the sake of brevity, as these features are well known.

[0038]

[64] Figure 3 is a system block diagram for automatic sentiment classification according to an exemplary aspect of the present disclosure. System 300 includes a multimedia classification engine 312. Multimedia can include video, audio, and text. Multimedia can include multimedia with scripts or subtitles, such as audiobooks, movies, TV shows, augmented reality, or virtual reality, which are transcribed, scanned, and fed into System 300. The transcribed media is received as input text 302. Input text 302 is processed in a preprocessing engine 304 to convert the text data into a format required for matching with rules 306, 308, 310, or for input into a machine learning model 320. In some embodiments, the preprocessed text data passes through a natural language rule (306). A rule is said to be fired when it matches text media or transcribed media. Visuospace rules 310 can include rules for visual cues. Audio speech rules 308 can include rules for speech signals.

[0039]

[65] The emotion recognition sensor (314) processes input at the sentence, paragraph, passage, scene, or chapter level, classifying each with a given emotion, cognition, sentiment, state-based or dynamic-based or trait-based tag. When a rule is triggered, an emotion is detected. The language rules also have dimension-based ratings. In some embodiments, the dimension values ​​of the rule are stored as vectors. Emotions can be inferred from the "shape" of the dimension. Emotion features include the shape of the dimension vector, the dimension value, and the difference or similarity to it from calculations derived from other vectors associated with the rule. The output of the natural language rule 306 is fed to the emotion recognition sensor 314 and the intensity rating sensor 316, and the remaining qualitative data can be converted into quantitative data for population analysis, as in scientific research or political surveys.

[0040]

[66] The intensity rating sensor (316) determines the intensity rating of each detected emotion based on the dimensions. In some embodiments, the intensity rating sensor (316) assigns an objective intensity rating based on the subcomponents of each cognitive, emotion-based, social, interpersonal, or state-based element as dimensions.

[0041]

[67] The sentiment recognition tagging engine (318) tags text data with assigned sentiment classes. The sentiment and intensity of sentiment recognition may be associated with metadata such as demographic information, online profile features, timestamps, sources, and geolocation. Aggregate sentiment recognition state results for a sample of the population and sentiment recognition state labeling are returned by the sentiment recognition tagging engine 318.

[0042]

[68] The rule discovery engine 326 can generate new rules, also called rule suggestions. A machine learning model 320 can be trained on the rules themselves. The trained machine learning model 320 can locate similar co-occurrence vectors, similar POS patterns, and similar n-grams and suggest these as potential new rules. In addition, emotion patterns, as well as dimension patterns, such as those detected in the emotion recognition sensor (314), can be used to generate new rules.

[0043]

[69] The machine learning model engine 320 may include any one or more machine learning models from among the transformer models, such as BERT, RoBERTa, support vector machines, word2vec, KNN models, long short-term memory models, and convolutional neural network models.

[0044]

[70] The statistical model engine 322 may include one or more statistical models. The statistical models may include any one or more statistical models from the following: k-means models, Bayesian models, document search models, logistic regression models, linear regression models, multinomial regression models, recommendation matrices, random forest models, and n-gram language models. Each statistical model is used as a classifier.

[0045]

[71] The vocabulary engine 324 provides vocabulary that can be used in system 300. Sources of vocabulary include NRCLex, Harvard Inquirer, MPQA, sentiwordnet, textblob, VADER, and other vocabulary not specifically listed.

[0046]

[72] In some embodiments, the aggregate sentiment cognition ensemble classifier 328 may output a final answer, such as sentiment. A random forest may be used as an ensemble classifier using one-hot coding. In another embodiment, logistic regression may be used for ensemble. In further embodiments, a neural layer may be used as the output of ensemble. The output may include the sentiment and intensity of sentiment cognition sentiment and its association with metadata such as demographic information, online profile features, timestamps, sources, and geolocation. The output may be an image or graphical representation of the sentiment cognition overview of users, brands, businesses, celebrities, organizations, topics, words, or phrases over a period of time or point in time, using a color-based geometric representation. In some cases, the output may include a report on dimensions, sentiment, dynamics, or social trends of a demographically or otherwise segmented or aggregated sample at any given time or over time. In some cases, the output may include the generation and display of a distribution of sentiment cognition states over a given period of time for single or aggregated user sentiment cognition states over time or at a given point in time.

[0047]

[73] Figure 4 is a diagram of a training architecture according to an exemplary embodiment of the present disclosure. To adapt to evolving natural language, the training architecture 400 uses a machine learning model engine 320 to train machine learning models to generate rules. The machine learning models are also trained on the rules themselves. They locate similar co-occurrence vectors, similar POS patterns, and similar n-grams and suggest these as potential new rules.

[0048]

[74] The language rule model 310 begins with a set of preprogrammed language rules that can be applied to natural language phrases and sentences. Detection can involve latent comparison of these rules, similarity recognition, and rule creation using the concatenation of rule latent elements (rule types). Rule creation can be done by identifying strings with sufficient similarity via a threshold, then creating tuples of each word with words, parts of speech, dependencies, stems, and lemmas, and matching among similar items based on index position to find cohesive parts in each tuple. The results can then be concatenated into predefined rules as needed, along with optional parts, or wildcards, or proximity calculations.

[0049]

[75] The training architecture 400 includes a text input 302 for accessing multiple natural language data items via processing circuits and memory. The multiple natural language data items may be read from a data repository or input directly as a stream of text. The text may include human speech input, text databases, documents, or captured, transcribed, and translated text resulting from other text data sources.

[0050]

[76] The training architecture 400 includes a preprocessor 304 for performing various preliminary processes typically performed on text for natural language processing. The preprocessor 304 may utilize any of the known software libraries for data normalization, tokenization, part-of-speech tagging, dependency, stemming, and lemmatization to generate a number of preprocessed natural language data items. An example of a software library is the Natural Language Toolkit (NLTK). The NLTK includes text processing libraries for classification, tokenization, stemming, tagging, and parsing. The NLTK includes the WordNet lemmatizer with lemmatization capabilities, as well as various other lemmatizers. Normalization may be performed as canonical normalization.

[0051]

[77] In some embodiments, the training architecture 300 may involve labeling a plurality of preprocessed natural language data items as the emotion and intensity of the expressed emotion. The labels and associated natural language data items may be used to train a supervised learning model.

[0052]

[78] The training architecture 400 provides a plurality of preprocessed natural language data items in parallel to the machine learning model engine 320 and the language rule model 306. The training architecture 400 trains the machine learning model engine 320 and the language rule model 306 in parallel over multiple training epochs to identify emotions and determine the intensity of emotions in the natural language data. Each training epoch of the machine learning model engine 320 may generate feature or rule suggestion 326 for subsequent training epochs of the language rule engine 306 based on decisions 412. After each training epoch of the language rule engine 306, the subsequent epochs of the machine learning model engine 320 are aggregated and scored (provided as probabilities for each class). In 414, the outputs representing the trained machine learning model engine 320 and the trained language rule model 306 are stored in non-volatile memory.

[0053]

[79] Figure 5 is a system diagram of the adaptive operation of the emotion classification system. The rule discovery engine 326 can generate new rules while presenting emotion and intensity information as output.

[0054]

[80] To assist in the operation, an example of how the sentiment classification system works is described. This example is simplified for ease of understanding. The disclosure is not limited to this example. In this example, the hashtagged text “Why am I the one who'needs'to take out the trash? #NOTANEED” is input by reading the text from a file or as a continuous stream entered into a computer system 302. The text input is processed by a preprocessing engine 304 that performs functions including tagging the input text at index positions, tokenizing the input text, and extracting the input hashtags. In some embodiments, the hashtags are components that have associated sentiments. The index positions may be integers indicating a position relative to the entire text, for example by a counter or sequential numbers generated within an array of tokenized inputs. Alternatively, the index positions may be indices such as vector positions, line numbers, or some input number that identify where in the sequence of hashtagged text the current text input occurs. The tokenization function may selectively extract punctuation marks such as “?” as tokens and remove others. In this example, the quotation marks around "needs" may be retained to indicate emphasis. Hashtags may be extracted from other text so that they may be treated as identifiers rather than expressions of emotion. In some embodiments, hashtags may be used as auxiliary indications of emotion. In this example, the hashtag may indicate that the input text is sarcastic. Hashtags may also be used to indicate emphasis, commentary / whispering, subtweets, organization, continuous expression, humor, context, emotion, marketing, or protest.

[0055]

[81] The natural language engine has the ability to search among the natural language rules 306 for one or more rules that match the input text. A rule may represent a text pattern with several required words or phrases mixed with syntax. In this example, rules such as "[why]" [...] + being verb + "the one who" + verb are pattern-matched and triggered.

[0056]

[82] The rules are grouped by emotional terms, cognitive terms, and terms that blend emotional and cognitive concepts. The emotion-cognitive sensor 314 has the function of applying an emotion-cognitive label (anger) based on the triggered rule.

[0057]

[83] The intensity rating sensor 316 has the ability to activate a dimension that, for its triggered rule, may have three dimensions with positive, negative, neutral, or null scores, such that these dimensions are, for example, respect, ego, and condemnation. The values ​​of the dimension scores are not limited to such and may include numerical values ​​within a given range. The intensity rating sensor 316 has the ability to aggregate the dimension scores to obtain an intensity score. The intensity rating sensor 316 assigns the intensity scores to emotion cognition labels and compares the intensity scores against a given threshold. The given threshold may be a decimal value such as 0.6, an integer within a range such as 6 within the range of 0 to 10, a percentage such as 60% which is common to all emotions, or it may be set for each emotion cognition label. In this example, according to the threshold, emotion intensity levels are labeled (anger - moderate). In some embodiments, if the speaker's baseline is known, the intensity scores may be relative to the speaker.

[0058]

[84] The meta-emotional cognitive aggregator 328 has the function of assessing emotional cognitive labels and combining the labels with other surrounding labels to form patterns of emotional cognitive labels (e.g., anger ____ peace).

[0059]

[85] At the next level, the dynamic 610 has the ability to pattern match patterns of emotion cognition labels to dynamic patterns. Based on the matched dynamic patterns, the dynamic 610 assigns labels, for example, forgiveness.

[0060]

[86] Figure 6 shows types of language rules according to exemplary embodiments of the present disclosure. Types of language rules for sentiment, feelings, opinions, and beliefs applied to a given portion of natural language input may include:

[0061]

[87] Rules that use part-of-speech tagging, syntax, or dependency parse, including modal and optative verbs, tense notation, morphology, conjugation, accent, as well as direct objects and proper nouns (502).

[0062]

[88] Rules for using string matching, including exact, inaccurate, masked, or wildcard (504),

[89] Rules that use distance between tokens (506),

[90] Rules for using punctuation (508),

[91] Rules that use lemmaization (510),

[92] Rules for using stemming (512),

[93] Rules for using vocabulary (514), as well as

[94] Rules to use word lookup or dictionary (516).

[0063]

[95] In some embodiments, rule components may include graphical content, including emojis, glyphs, emoticons, characters, scripts, and any other graphics. Applying language rules to a given portion of natural language input results in obtaining a numerical or boolean value for each of one or more language rules.

[0064]

[96] In some embodiments, additional language rule types are adapted to other world languages, including Japanese, Korean, Chinese, Vietnamese, and other Asian languages. Additional language rule types may include:

[0065]

[97] Rules for tagging dependencies,

[98] One or more rules for detecting invariable words,

[99] One or more rules for detecting markers,

[0100] One or more rules that detect structural narrative power, including topics, themes, and predicates.

[0101] One or more rules that represent classes of nouns that are grouped by commonalities.

[0102] One or more rules for detecting grammatical cases, including nominative and ablative cases.

[0103] One or more rules to detect the non-finite verb category of a verb,

[0104] Rules for detecting click transcriptions to capture semantic information from clicks on a mouse, touchpad, or touchscreen.

[0105] In some embodiments, language rules may incorporate OR operators for optional conditions and concatenation operators for combined conditions. In some embodiments, language rules may include rule referents that point to other parts of the rule. Rule referents may include doubling or repetition, such as adjective + noun + noun, with the condition that the noun is the preceding noun. Rule referents may include placeholders or position elements in the rule, such as position indexes in the rule[0].

[0066]

[0106] In some embodiments, the rule-directed object of a language rule may be in the form of a pattern. The pattern may include noun->noun in the case of redundancy, and noun!+noun in the case of non-redundancy.

[0067]

[0107] Language rules are grouped by type of emotion or cognition. In some embodiments, rules may have an order of importance, which can be modified to indicate higher priority for rules below it and lower priority for rules above it. Each language rule has one or more dimensions and values ​​for each dimension. In one embodiment, dimensions may include sentiment, emotion, emotion-cognitive attitude, value, social convention, way of thinking, view, manner, response, trait, belief, opinion, viewpoint, motivation, bias, state, manner, approach, dynamic, personality trait, emotional approach, emotional selection, reaction, tendency, temporary state, change of state, cognitive manner, behavioral manner, internal condition, external condition, feeling, emotion, assertion, attitude, assertive attitude, directed attitude, undirected attitude, spontaneous attitude, conscious emotion cognition, and unconscious emotion cognition. In one embodiment, dimensions may include emotional emotions such as anger, anticipation, disgust, fear, joy, sadness, surprise, and credit (from NRCLex). In another embodiment, dimensions may include, but are not limited to, ego, blame, suitability, sanctity, kindness, respect, time (future), (self) favor, approval, sincerity, vulnerability, judgment, isolation, purpose, stiffness, minimization, specificity, force, action (activity), agency, curiosity, clarity, intention, emphasis, energy, conviction, interest, involvement, shock / surprise, tension, speed, nuance, logic, paranoia, credit, distance, discrimination, and, associated with them, facets, components, and aspects of emotional cognition, such as respect (self), respect (other), objectification, attachment, empathy, and patience.

[0068]

[0108] In some embodiments, dimensions can be grouped into categories such as personality traits, beliefs, opinions, perspectives, manners, motivations, biases, states, emotional approaches / selections, manners, reactions, and interpersonal dynamics.

[0069]

[0109] The inventors found that even perfectly accurate classifications can provide extremely limited semantic information. For example, the "accuracy rate" does not always reflect useful semantic information with respect to emotion detection. Labels for emotions can become too ambiguous (attempting to force more than 25 emotions into six labels). Classifications may further lack useful subtextual information and may lack directionality toward the speaker's cognitive state. Classifications do not offer hints about why feelings exist. Sentences, phrases, or syntax, due to the complexity and depth of human emotions, may be poorly classified for specific emotions and deeper semantic values, parsed at the sentence level, forced into a single class to which multiple labels should often apply, or remain impossible to accurately understand by machine algorithms. Furthermore, while conversations can be mined for further information, argumentative textual values ​​are limited to contextual cue approaches such as named entity recognition and speaker-specific information, which yield little about emotional states.

[0070]

[0110] Despite the existence of cutting-edge technologies such as Transformers, they have historically excelled only in predictive and translation tasks, and not in semantic interpretation (partly due to their inferior overall performance in semantic tasks). Transformers are ultimately limited in their semantic capabilities due to the gap between human and machine interpretive abilities. Among these semantic limitations, particularly concerning the interpretation of immediate speech and text, is the ability to distinguish between mixed emotions, complex emotions, figurative language, and insincere emotional expressions (such as sarcasm, irony, politeness, and passive aggression). The disclosed embodiments are approaches to addressing the range and depth of human emotion and cognition.

[0071]

[0111] Figure 7 is a bottom-up stack of the rule engine layers. "Dimensions" (702) are elements of an emotional cognitive representation, as well as the feelings and thoughts behind them. In some embodiments, the system is implemented using object-oriented programming, and emotional is a class that has a set of dimensions as attributes of the class. Intensity is a method of the emotional class used to determine an intensity value based on the presence of a dimension. Intensity values ​​can be stored in a structured or unstructured database. The emotional class has archetype values ​​for each attribute / dimension. The values ​​of a dimension are small combined valences with ternary values ​​(+1 is a positive force, -1 is a negative force, 0 is a neutral force, and φ is non-existent or inapplicable, the latter two of which are not equivalent). In some embodiments, for example, the values ​​of a dimension include layers of Booleans, such as (1) true-false (neutral vs. non-neutral), (2) true-false (true vs. false), and (3) potentially null. The values ​​can be in the form of floating-point numbers or integers.

[0072]

[0112] For each data line, these dimensions receive a rating each time the rule is triggered. Therefore, for example, in the following example... The construction ("[why / how come] + be verb + pronoun + [temporal hyperbole] + "the one / s who" + verb") shows the following: (Optional word list item from the two options: "why" or "how come") be verb + pronoun (e.g., "are you", "am I", "are they", "are we", "is he", "is she") + A list of optional words for exaggerating time (e.g., "always", "never", "constantly") + Fuzzy (inaccurate string) ("the ones who", "the one who") + Almost any verb (e.g., "sits up front," "pays for everything," "brings up problems," "gets the easy job"). The rule uses those modular parts and covers a certain degree of substitution.

[0073]

[0113] The system detects a core dimension profile of a given emotion. Different rules find distinct, partial, but strongly matching configurations that represent various expressions of an emotional state. In other words, an emotion is a mental state. Fundamental emotions are pure and irrational (e.g., fear, anger, happiness, sadness). Complex emotions are more social and cognitive (e.g., sadness, depression, shame, anxiety, admiration). Complex emotions arise when cognitive and emotional states coexist, or when multiple emotions coexist. When rules are activated, they provide indications of when such emotional states are occurring.

[0074]

[0114] The more dimensions that match a particular emotion, the higher the intensity of that emotion. The intensity measure then becomes an objective measure, representing the degree of intensity. In some embodiments, if the same syntax occurs twice in the input text, the matching syntax is counted both times. When measuring intensity, the dimension scores are normalized relative to the number of words. Therefore, the intensity value is density-based.

[0075]

[0115] In an exemplary embodiment, there are 50 dimensions. In some embodiments, there are a larger number of dimensions. Each rule has a rating across the entire set of dimensions.

[0076]

[0116] Each Rule 704 includes one or more dimensions. In the rule above, one exemplary dimension is ego. Ego can be absent from a given syntax (completely irrelevant - null value), neutral (has ego components, but they are extremely even and fair), positive (self-centered, humble), or negative (praise, supplication, self-loathing, etc.). Ego is the dimension that is rated for this rule. Another dimension is curiosity. This syntax, detected by this rule, is proactively curious and therefore receives a +1 rating.

[0077]

[0117] Rule 704, when activated, indicates the detected emotion. When an emotion is detected (in the part of the sentence in which they exist, tagging is performed via index location rather than simply at the sentence or passage level, thereby allowing for greater distinction and clarity), complex emotions 706 may be identified, which typically consist of simultaneous emotions. For example, contempt is a combination of pride / humility and anger. In addition, emotion cognitive states can overlap.

[0078]

[0118] When complex and underlying emotions are properly activated, emotion patterns 708 can be assessed when emotions naturally shift to other emotions in specific situations. In addition, emotions can progress, escalate, or resolve. Emotion patterns 708 are formed by linking emotions into ordered arrays or strings of emotions (e.g., hope + expectation + surprise + sadness). These patterns can be assessed to some extent, like the Skipgram model with limited possibilities. Patterns of shifting emotions, progressing emotions, escalating emotions, and resolving emotions are stored as recognizable emotion patterns.

[0079]

[0119] Emotional patterns can be used to predict the flow of emotions, such as excitement turning into acceptance of calm, anger shifting to confusion, then to recognition, then understanding, and finally returning to love as empathy or forgiveness unfolds.

[0080]

[0120] Emotional patterns are built into Dynamic 710. Dynamic 710 is a descriptive label for a major emotional event that summarizes an emotional pattern. For example, the Dynamic for "disappointment" is the label for the emotional pattern of Hope + Expectation + Surprise + Sadness. Dynamics can occur internally within oneself, which is usually a complex emotion. Dynamics can also occur with others / interpersonally, which may be a phenomenon like forgiveness. In such cases, forgiveness is the label for the emotional pattern of Anger + Recognition + Love.

[0081]

[0121] The upper layer of Dynamic 710 may include Social Trends 712, which are meta-outcomes of strong emotional dynamics of a group, often brought about by large events or the pressure of mass movements, such as upheaval or innovation. Large events may include imminent real-world events (e.g., rage outbursts, coups, riots, pop culture or social trends, stock market events, etc., such as the stock market hoarding of GameStop stocks, the "assassination" of a public figure, the limit of violence, etc.). These social trends may also exist in smaller social groups, accompanying the limit of domestic violence, falling in love, rage outbursts, etc. Detection of social trends enables reaction / prevention or capitalization, or at least passive monitoring / identification. Such events may be correlated with certain previous such real-world events accompanied by these social trends, thereby enabling rapid reaction, prevention or capitalization if similar events are likely to be detected or predicted quickly in the future. Calculation of emotional cognitive states, dimensions, dynamics, and social trends from natural language from one or more users via natural language rules 306.

[0082]

[0122] In some embodiments, each level of emotion / cognitive flow can be predicted for these layers based on historical data stored when the classification system is being used.

[0083]

[0123] Figure 8 is a flowchart of a computer system operation according to an exemplary embodiment of the present disclosure. One embodiment relates to a process for tracking subtle differences in psychological emotions in natural language content. The method first applies linguistic rules for emotion to a natural language syntax, phrase, or sentence, each rule having one or more dimensions. The results of applying the linguistic rules are used to detect emotion and determine the intensity of emotion. A machine learning model is used to suggest new linguistic rules and thus works to augment the application of linguistic rules. The process may be carried out in a device having text and / or voice input and text, speech, and / or graphical output. A computer program includes instructions stored in a computer-readable storage medium that, when executed by a computer system such as the one in Figure 2, carries out the steps shown in the flowchart.

[0084]

[0124] In S802, the computer 226 is configured to receive natural language content as text that can be derived from text and / or speech input. Natural language content may be read from files stored in disk storage 204, from a stream of data received by the network controller 206, from text input at the keyboard 214, or from text from speech input via the microphone. In addition, input may include aggregations of data from online or offline databases and queries, taken from historical archives or corpora, or scraped from websites such as social media or user review websites. Input may relate to topics, people, brands, organizations, concepts, words, or groups of words, and may or may not be prompted. Input may include interview data transcribed or otherwise obtained from participants in surveys, market research, or academic studies. In some embodiments, input may include timestamps that can be read along with the natural language content.

[0085]

[0125] For large sections of text that may exceed the size of the input text handled by the natural language rule engine 306 or the machine learning model engine 320, one approach is to read the input using a fixed-size sliding window. In an optional step S804, the computer 226 is configured to apply a fixed-length scanning window to the natural language content. The length may be several characters. The scanning window may overlap by a certain number of characters during a series of scans. In step S806, the computer 226 is configured to evaluate the natural language content using language rules for each emotion in order to obtain language features of the human dimension of emotion. As described above, the language rules are pattern-matched to the natural language content.

[0086]

[0126] In S808, computer 226 is configured to score each human dimension as present, neutral, level, or absent as a dimension score for each matched rule.

[0087]

[0127] In S810, the computer 226 is configured to aggregate the dimension scores of each emotion in order to obtain the intensity score of each emotion. The aggregated score represents the intensity of the emotion. In some embodiments, the start and end indices in the natural language content for the emotion are determined for each particular dimension. In some embodiments, the indexed natural language content and the corresponding dimensions can be forwarded to the machine learning model engine 320. In some embodiments, the top dimension of emotion can be forwarded to the machine learning model engine 320 along with its respective numerical value (dimension score) or mapped descriptor. The emotion patterns, including the start and end indices, can be stored for an entire passage having several syntaxes, phrases, and sentences.

[0088]

[0128] In S812, the computer 226 is configured to classify natural language content into sentiment classes based on dimension scores. In some embodiments, the classification may generate probabilities for each sentiment class.

[0089]

[0129] In S814, computer 226 is configured to label the aggregated values ​​as sentiment and to determine the context of the labels, such as the pattern of sentiment labels, for natural language content.

[0090]

[0130] In S816, the computer 226 is configured to output classification and sentiment label patterns as text, speech, and / or graphical output.

[0091]

[0131] In S818, the computer 226 is configured to track the pattern of sentiment labels and their associated components in a time sequence on natural language content in order to track sentiment labels over time. Each sentiment label may be assigned an ordered index number to identify their order in the sequence. For natural language content that includes timestamps, each sentiment label may be assigned a timestamp of its associated component.

[0092]

[0132] Figure 9 is a flowchart of the steps for evaluation using language rules, according to an exemplary embodiment of this disclosure. Evaluation using multiple language rules in S806 includes the following steps:

[0093]

[0133] In S902, computer 226 is configured to detect rules using rule pattern matching.

[0134] In S904, computer 226 is configured to evaluate the human dimension of each detected rule.

[0094]

[0135] Figure 10 is a flowchart for detecting a rule according to an exemplary embodiment of the present disclosure. Detecting a rule in S902 includes the following:

[0095]

[0136] S1002 detects the presence or absence of syntax containing emotion-related components in natural language content.

[0096]

[0137] Figure 11 is a scoring flowchart according to an exemplary embodiment of the present disclosure. Scoring S808 includes the following:

[0097]

[0138] S1102, evaluate each dimension to determine the dimension score.

[0098]

[0139] Figure 12 is a flowchart for detecting a rule according to an exemplary embodiment of the present disclosure. Detecting a rule among multiple language rules in S702 includes the following:

[0099]

[0140] S1202, determine the following values.

[0100]

[0141] Part-of-speech tagging or syntax rules,

[0142] String matching rules, exact, inaccurate, masked, or wildcard.

[0143] Token proximity rule,

[0144] Punctuation rules,

[0145] Lemma rules,

[0146] Stemming rules,

[0147] Vocabulary rules, and

[0148] Word lookup or dictionary-based rules.

[0101]

[0149] Figure 13 is a flowchart for determining a token proximity rule value according to an exemplary embodiment of the present disclosure. Determining a token proximity rule value involves accessing all tokens in natural language content that have a token distance of less than n from a specified point, where n is an integer.

[0102]

[0150] Figure 14 is a flowchart of a classification according to an exemplary embodiment of the present disclosure. Computer 225 is configured to perform the classification of S812 using a machine learning method which includes supervised learning, unsupervised learning, and rule-based systems.

[0103]

[0151] Figure 15 is a flowchart of machine learning according to an exemplary embodiment of the present disclosure. The machine learning in S1402 includes the following:

[0104]

[0152] S1502, receives multiple natural language data items from a data repository.

[0153] S1504, using a preprocessing engine to generate multiple preprocessed natural language data items, normalizes and tokenizes multiple natural language data items, which may include pre-sorting data lines into positive, negative, and neutral to save computational power and time in classification.

[0154] S1506, Labeling multiple preprocessed natural language data items by expressed sentiment, emotion, opinion, or belief, and the intensity of the expressed sentiment, emotion, opinion, or belief.

[0155] S1508 provides multiple preprocessed natural language data items in parallel to an unsupervised learning engine, a rule-based engine, and a supervised learning engine.

[0156] S1510, in order to identify expressed emotions in natural language data and to determine a scalar measure as the intensity of those emotions, an unsupervised learning engine, a rule-based engine, and a supervised learning engine are trained in parallel over multiple training epochs.

[0105]

[0157] Each training epoch of the unsupervised learning engine provides features or rule suggestions to subsequent training epochs of the rule-based engine, and each training epoch of the rule-based engine provides aggregated and scoring data to subsequent epochs of both the unsupervised and supervised learning engines.

[0106]

[0158] In S1512, outputs are generated representing a trained unsupervised learning engine, a trained rule-based engine, and a trained supervised learning engine.

[0107]

[0159] In S1514, to suggest new rules for the rule-based engine, human dimensions present in natural language data are matched by matching them with existing dimension arrays, with and without wildcards or pattern skips.

[0108]

[0160] The system enables the deduction or recognition of the point and level (intensity) at which feelings translate into actions (as described in and detected within text). The system enables the recognition of dynamics within the self alone, bipersonally (one-on-one), interpersonally / multipersonally or family, or socially (community, country, region, or world). The system enables tracking the intensity of an emotion as it fluctuates across sentences, items / paragraphs, and entire passages. The system enables the identification of significant shifts in perception, such as abrupt shifts in aggregated self-image (via the ego dimension), which may indicate that a problem or significant change has occurred. The system can reveal relationships between important factors through correlation, such as the relationship between self-esteem / self-image and general optimism.

[0109]

[0161] In light of the above teachings, numerous modifications and variations are possible. Therefore, it will be understood that, within the scope of the attached claims, the present invention may be practiced in ways other than those specifically described herein.

[0110]

[0162] Therefore, the above discussion merely discloses and describes exemplary embodiments of the present invention. As those skilled in the art will understand, the present invention can be embodied in other specific forms without departing from its spirit or essential features. Accordingly, the disclosure of the present invention is illustrative and not limiting the scope of the invention or any other claims.

[0111]

[0163] Example Implementation

[0164] Electronic reading devices

[0165] Embodiments of the present invention include an electronic reader. An electronic reader may be a dedicated device (commonly called an e-book reader) incorporating dedicated firmware and a display configured to optimally display text with high clarity, or it may be a general-purpose computing device such as a tablet computer or smartphone, typically configured with text-reading software in the form of a mobile application (app). Electronic readers generally have a display screen that is 10 inches diagonal or smaller and have limited computing power and memory. In most cases, electronic readers can communicate with web services via an internet connection, typically via a Wi-Fi connection. Some electronic readers include a communication module for communication via cellular transmission.

[0112]

[0166] The system 300, which has a multimedia classification engine 312, can be run on a device with limited processing power and memory, as the majority of its processing is based on the execution of the natural language rule engine 306. The machine learning model engine 320 can be run offline on a separate computer or as a cloud service.

[0113]

[0167] Figure 16 shows an electronic reader according to an exemplary embodiment of the present disclosure. The electronic reader 1600 includes a display screen or touchscreen display 1602. When the display 1602 is displaying text from a book 1604, the display may include a scrolling function, which is displayed as a scroll bar (not shown), and a page-turning function, which is displayed as a button 1606.

[0114]

[0168] Figure 17 is a flowchart of the operation of an e-reader according to an exemplary embodiment of the present disclosure. In S1702, the system 300 may assess a written fiction or nonfiction work for emotional, cognitive, interpersonal or social dynamics, motivation, beliefs, opinions, or psychological elements using the multimedia classification engine 312. In S1704, the text in the e-book is scanned and tagged with rules that trigger identified emotional, cognitive, or other states, and the intensity in which they occur.

[0115]

[0169] In one embodiment, in S1706, the system 300 may generate and display color-coded highlights that indicate the occurrence of several emotional, cognitive, sociological, or interpersonal dynamics and / or states. In one embodiment, in S1708, the system 300 may generate and display one or more sidebars for dynamics 1610 and emotional intensity 1620. The sidebars may summarize the emotional, psychological, cognitive, sociological, or interpersonal dynamics or states occurring in the text 1604, along with any added context. In S1710, each dynamic 1612 or state 1622 may be interacted with by selection 1630 (via touch, mouse, keyboard, etc.), thereby enabling the e-reader 1600 to be presented with an example 1632 of that given dynamic or state in the text.

[0116]

[0170] Multimedia audiobook or visuospatial data sentiment classifier

[0171] System 300, which has a multimedia classification engine 312, can be run on devices with limited processing power and memory, as the majority of its processing is based on the execution of a natural language rule engine 306. System 300 can perform sentiment classification on sentences, phrases, and syntax, and can perform sentiment classification in real time as text is received. The machine learning model engine 320 can be run offline on a separate computer or in a cloud service.

[0117]

[0172] Figure 18 is a flowchart of a multimedia audiobook or visuospatial data sentiment classifier according to an exemplary embodiment of the present disclosure.

[0118]

[0173] Audiobooks, or multimedia with scripts or subtitles such as movies or TV shows, can be scanned through the system and transcribed into text.

[0119]

[0174] In 1802, the transcribed media is passed through a natural language rule engine (306) and matched against rules.

[0120]

[0175] The emotion recognition sensor (314) processes input at the sentence, paragraph, passage, scene, or chapter level, classifying each with given emotion, cognition, sentiment, state-based, dynamic-based, or social-based tags. Drifting short substrings and selected individual words, known as High Use Non-Construction Hooks (HUNCHes), which have partial dimensions, are detected and matched within the text.

[0121]

[0176] The intensity rating sensor (316) analyzes the text and assigns an objective intensity rating based on the subcomponents of each cognitive, emotional, social, interpersonal, or state-based element, known as a dimension.

[0122]

[0177] The emotion recognition tagging engine (318) tags text data with the assigned class.

[0123]

[0178] Temporal coordination (1804) performs timing-based associations between tagged classes of text information and coordinated sections of auditory or visuospatial data or signals, which are then sent to an emotion recognition tagging engine (318) for classification and tagging.

[0124]

[0179] System 300 scans for further instances of matching audio or visual patterns, both absolute and relative to the speaker's audio or visual baseline, as well as adjusted for gender, age, class, race, accent, locale, and other demographic information, and contextual information, and automatically tags them to the emotion cognition class associated with the pattern.

[0125]

[0180] Text that has been automatically tagged with emotion classes in the previous step can now be automatically flagged in the text as a match to the emotion cognition class that is now preserved by the audio or visual data, or as a high likelihood match.

[0126]

[0181] Figure 19 is a block diagram of the multimedia rule engine. Text, visuospatial, and audio data are compared, reprocessed as multimedia rule suggestions, and sent back to the natural language rule engine (310), visuospatial rule module (306), and audio speech rule module (308).

[0127]

[0182] Subsequent audio or visual inputs that exhibit similar, opposite, or partial matching in audio or visual signals and patterns, or otherwise show mathematically significant ratios or relationships, will receive tags indicating the likelihood of the emotion cognition class.

[0128]

[0183] A comparison of the similarities, differences, contradictions, and other measurements between the rules of each multimedia rule engine 312 and the suggested rules improves the rule suggestion capability and highlights the modular components of each rule.

[0129]

[0184] In one embodiment, HUNCHes are used as hooks, thereby enabling the system to analyze the surrounding text, which then suggests potential new rules back to the multimedia rule engine (312). The dimension pattern must match positive and negative dimensions or be consistent with the rule pattern.

[0130]

[0185] Specifically, HUNCHes are non-syntactic-based partial pieces (such as one or two words or partial phrases) that convey their own dimensions. By feeding these into system 300, neural model 320 becomes capable of "matching" these partial dimension patterns and using them as hooks to find indicators of emotion. For example, an anger rule is in an undiscovered state but has ego +1, irritation +1, power +1, etc. A hook (like "I won't") may only have ego +1 and power +1, but the surrounding text will be examined if it looks sufficiently like it could fit anger. To do this, it must have zero contradictory dimensions and it must have the existing dimensions of its matched version. This allows for the detection of rule substitution (if both are true) or potentially a new rule (if they are not contradictory, but the dimension profile looks sufficiently like anger). Through this technique, new angry constructions such as "I [won't / am not going to / am not gonna] take it ["one more time", "again", "anymore", "at all"]" can be discovered, and this construction can be refined into rules like "[first-person pronoun] + ([auxiliary verb] + [negation] + (("GOING") + [infinitive verb] or [future verb]) + ((direct object-pronoun))* + [word list: time phrase] + "!", where the asterisked part is optional, which gives sentences like the following, for example: I'm not going to take it anymore! I'm not going to serve you again! We will not bow anymore! For some emotions, and for some dimensions relating to a given emotion, more than one value may be acceptable. For example, anger can be rude or have subtle differences, and therefore can receive any score.

[0131]

[0186] In some embodiments, depending on the emotion and dimension, two or more values ​​may be acceptable. For example, anger can (and must) be either a neutral ego or a positive ego. In other cases, it must have a valence of 1 for the dimension. For example, anger must be powerful, anger must be distinctive. Consecutive HUNCHes may match together in an emotion dimension array. They may also not match together if the emotion has shifted.

[0132]

[0187] Figure 20 is a flowchart of a rule discovery engine based on HUNCHes, according to an exemplary embodiment of the present disclosure. The process is carried out in the rule discovery engine 326.

[0133]

[0188] In S2002, dimensions related to HUNCHes can be scored.

[0134]

[0189] In S2004, HUNCHes are matched to profiles of emotional cognitive states. For them to match to possible emotional cognitive states, they cannot have contradictory elements or dimensions with respect to that state. Non-contradictory empty dimensions can potentially be matched.

[0135]

[0190] In S2006, negative or positive dimensions should have a higher probability of matching unless otherwise flagged.

[0136]

[0191] In S2008, an emotional cognitive state is suggested.

[0137]

[0192] In S2010, new string matching and surrounding text are suggested back in the natural language rules module 306.

[0138]

[0193] In S2012, audio or visuospatial data corresponding to the new string is supplied to the corresponding multimedia rule engine 312.

[0139]

[0194] Figures 21A and 21B are flowcharts of rule discovery in audio media according to exemplary embodiments of the present disclosure.

[0140]

[0195] In S2102, audio media (or sound from visual media) is transcribed into text and input. In some cases, a closed-caption transcript or script may be input. In some embodiments, breaths, background noise, gestures, and other non-textual audio may be transcribed as annotations to capture more features of the audio media.

[0141]

[0196] In S2104, rules are applied to incoming text to detect emotions (as well as cognition and other elements).

[0142]

[0197] In S2106, the index position in the input where the rule was successfully applied is noted.

[0143]

[0198] In S2108, rule dimensions are calculated to determine intensity at the phrase, sentence, passage, and chapter levels.

[0144]

[0199] In S2110, passages are annotated with corresponding emotions (and / or other elements) as well as their respective intensities and attributes.

[0145]

[0200] In S2112, the corresponding sound wave or visuospatial data is coordinated with the index position of the word in the passage over time.

[0146]

[0201] In S2114, the actual audio characteristics and the relative relationship between the speaker's baseline audio or visuospatial profile and a given specific emotional cognitive profile of those characteristics are calculated and stored.

[0147]

[0202] In S2116, when similar, derived, opposite, or otherwise related audio fragments appear, they are matched to the corresponding emotion cognition label. Subpassages or strings that trigger the same rule for different emotions are fed to the learning algorithm. Similar sound or visuospatial fragments detected in the data are pre-tagged with suggested emotion cognition states based on existing tags in similar audio or visuospatial data. Similar fragments detected in the audio or visuospatial data are fed back to the natural language rule module 306.

[0148]

[0203] Figure 22 is a graph of speech signal patterns. In one embodiment, a speech signal pattern 2210 for text input 2202 is used to mark the start and end times (2204), emotions 2206, and text input 2202, which are indexed 2204 and emotions 2206.

[0149] Emotion Recognition Dynamic Display

[0204] Historically, certain subsets of groups have experienced various barriers to understanding emotional, cognitive, social, belief-based, interpersonal, or metacognitive, metaemotional, or decision-making cues or dynamics in spoken or written text. In particular, non-neurotypical individuals (e.g., neurodivergents) with conditions such as autism are more likely to encounter misinterpreted or missing social cues, emotions, and other dynamic elements present in language. Non-native speakers of a given language may also encounter these difficulties. When media consumed by these groups is confusing, misleading, and misleading, and when social norms are not understood and / or deviated from, it can sometimes lead to disconnection, interpersonal conflict or alienation, avoidance, and isolation. Furthermore, media enjoyment may be diminished or less likely to be discussed due to reasons of social cohesion. Entertainment companies may also experience audience loss or decline as audiences do not pick up on the emotional, interpersonal, belief-based, or cognitive dynamics at hand.

[0150]

[0205] As new situations, dynamics, emotions, cognitive states, and social aspects, trends, and dynamics develop or are introduced, analyzed annotated subtitles can be fed to machine learning algorithms for greater effectiveness. The theory of mind module can be updated with additional iterations to be adapted for a given society, gender, religion, beliefs, age, topic, or culture or subculture for accuracy.

[0151]

[0206] Training data can be partially automated by robust information that is extracted and matched to incoming media.

[0152]

[0207] Media can be created using training data, written to using annotations on previous data, collected, spliced, compromised, or mimicked.

[0153]

[0208] Training data can be auditory, visual, textual, multisensory, or a combination of all types.

[0154]

[0209] One type of training data includes the GoEmotions dataset, with the following modifications: removal of desire classes, removal of neutral labels, and instead of using neutral labels when deriving a guaranteed score of 1 - neutral probability score = guaranteed score.

[0155]

[0210] Training data will be supplied, but will not be limited, from locations where the aforementioned threshold requirements for sentiment, such as emotion, cognition, and other dimensions, are logically necessary for the case in question, or where authors themselves declare their own emotions, and / or from locations where tags are effectively provided and empirical labeling is possible.

[0156]

[0211] One embodiment of the system 300 for automated emotion classification is an emotion recognition dynamic display. The dynamic display can insert emotion subtext data in real time, thereby enabling neurodivergent users to learn and improve their social, emotional, interpersonal, and theory of mind skills as they progress.

[0157]

[0212] The natural language rule engine 306 consists of rules comprised of syntactic, frequent, and / or replaceable phrases that enable rapid determination of linguistic patterns in text information made available by subtitles. Hashtags, instead of being made available only through vocabulary, are broken down by context derived from the input, and by corpora, unigrams, and other existing n-grams, thereby providing more accurate and relevant natural language processing and sentiment analysis. Similar, partial, and opposite dimension patterns are used for sentiment detection. Rules are created to encode dimensions that can be used to detect mental health indicators in each rule-triggering syntax.

[0158]

[0213] The emotion recognition engine 314 is configured to score the subcomponents ("dimensions") of each rule, enabling the rapid and increasingly accurate recognition of motivation, influence, emotion, and cognition, as well as similar subcomponents, aggregated in distinct patterns and combinations to identify transient and persistent emotional, cognitive, belief-based, opinion-centered, social, and personality states. Emotion is inferred from the "shape" of the dimensions. Emotional features may include dimension vectors, dimension values, and differences or similarities to them derived from calculations of these parts.

[0159]

[0214] The emotion cognition tagging engine 318 is configured for tagging and tracking the development of subtext information throughout the process of subtitles and speech, using subcomponents of emotional, cognitive, belief, motivation, and opinion states. Once emotions are tagged, the aggregate emotion cognition ensemble classifier 328 tags meta-emotional states, shifts, and combinations based on emotion patterns.

[0160]

[0215] The rule discovery engine 326 is configured to identify, utilize, deduce, and infer gaps in sentiment states, tendencies, dimensional transitions, and other sentiment states and shifts to suggest potential new rules, which are then fed back to system 300. Sentiment patterns and meta-sentiment shifts, states, and combinations are inferred from gaps in patterns of sentiment, cognition, or other sentiment components. The rule discovery engine 326 is composed of affective logic used to infer missing sentiment states and shifts in the data, or to resolve semantic, cognitive, affective-cognitive, or other sentiment ambiguities. The rule discovery engine 326 is configured to control the training of the machine learning model engine 320 on the “edges” of classes, such as between close sentiment states, like grief and sadness, or sadness and anger, for more sophisticated, finer, and faster distinctions.

[0161]

[0216] Display 210 is configured to display subtext cues juxtaposed with visual and auditory (auditory tone and speech) data to enable richer information and contextual awareness. The subtext data is displayed to inform the user and viewer and to augment and / or clarify the social context and complex emotional and cognitive states depicted or described in the media.

[0162]

[0217] The rule discovery engine 326 is configured to work in conjunction with the machine learning model engine 320. As new situations, dynamics, emotions, cognitive states, and social aspects, trends, and dynamics develop or are introduced, analyzed annotated subtitles can be fed to the machine learning model engine 320. Training data can be obtained from information that is extracted and matched to incoming media. Training data can be auditory, visual, textual, multisensory, or a combination of all modes.

[0163]

[0218] In one embodiment, the training data includes the GoEmotions dataset. In this embodiment, the GoEmotions dataset is modified by removing the desire classes. Neutral labels are replaced with guaranteed scores derived from the neutral labels, such that 1 - neutral probability score = guaranteed score.

[0164]

[0219] In some embodiments, training data is obtained from sources where sentiment, such as emotion, cognition, and other dimensions, necessarily has or involves the aforementioned threshold requirement that is logically necessary for the case in question, as well as from sources in which authors themselves declare their own emotions, effectively providing tags and enabling empirical labeling.

[0165]

[0220] Figure 23 is a flowchart of a method for real-time sentiment classification in a video / audio stream, according to an exemplary embodiment of the present disclosure. The method is implemented on a display device, including a tablet computer, smartphone, or smart TV, which receives streaming audio and / or video and includes its own built-in computer along with memory.

[0166]

[0221] In S2302, a scene from a movie or television program or streaming show or a captured theatrical play or animated video source is received along with a coordinated text transcription.

[0167]

[0222] In S2304, rule matching of text data is performed by the rule-based engine 306, the emotion recognition engine 314, the intensity rating sensor 316, and the emotion recognition tagging engine 318, which tag emotional, cognitive, and other such states with intensity ratings in the text transcription.

[0168]

[0223] In S2306, the aggregate emotion cognition ensemble classifier 328 determines contextual cues based on word co-occurrence, argumentative elements, and topical elements.

[0169]

[0224] In S2308, the emotion recognition sensor 316 may, in some cases, mark individual strings or n-grams with a three-dimensional score.

[0170]

[0225] In S2310, the visuospatial rule engine 310 and the audio-speech rule engine 308 detect augmented information (AugI) and situational elements (SE) that are evident in the visual data, or tone elements that are evident in the auditory data, and input them into separate, but time-coordinated, sources of media.

[0171]

[0226] In S2312, the emotion recognition sensor 314 performs context-oriented semantic information (contextual cues) and juxtaposition (coordination and divergence, and their respective degrees) from AugI and SE data to create a context score for each scene.

[0172]

[0227] In S2314, the sentiment data enclosed in parentheses is returned inline and inserted into a text transcript for display on the display device 210, so that viewers may have more time to correctly identify the emotional, cognitive, or social elements of the media.

[0173]

[0228] Figure 24 shows a display device according to an exemplary embodiment of the present disclosure. The exemplary display device 2400 includes a display screen 2402 for displaying scenes from a movie or television program or streaming show or a captured theatrical play or animated video source, along with coordinated text transcription 2412. In this exemplary screen, parenthetical sentiment data 2414 (e.g., sentiment intensity pairs, "anger-moderate") are returned inline, inserted into the text transcript 2410, and displayed on the display device 2400. While sentiment intensity pairs are shown as pairs of terms, it should be noted that sentiment intensity pairs 2414 may be displayed in other formats, including intensity as a numerical value, or graphical symbols indicating relative quantities (low, moderate, high), such as bars of a certain relative size and / or color, or a set of colored dots of varying length. Also, sentiment intensity pairs 2414 may be displayed within a graphical container, such as inside a circular or rectangular shape, or within a shape like a comment balloon. In addition, while a single emotion intensity pair 2414 is shown, the number and / or sequence of emotion intensity pairs depends on the content of the text transcription 2412. For example, emotion intensity pairs 2414 may appear as a pattern.

[0174]

[0229] Borderline Personality Disorder Sedation Device

[0230] Figure 25 is a diagram of an emotion classification system with sensory distraction, according to an exemplary embodiment of the present disclosure.

[0175]

[0231] Borderline Personality Disorder (BPD) is a mental health disorder that affects how a person thinks and feels about themselves and others, causing problems that operate in daily life. It includes self-image issues, difficulty managing emotions and behaviors, and patterns of unstable relationships. Treatment for BPD involves learning to manage emotions that are overwhelmingly intense and may lead to self-harm. The disclosed embodiments include peripheral devices for providing feedback based on several emotions and emotional patterns, particularly the intensity of emotions.

[0176]

[0232] Patients diagnosed with Borderline Personality Disorder (BPD) may be provided with a microphone-equipped device or input device that can accept input from other text input devices such as a keyboard, a voice recorder with transcription, or a touchscreen device with a text input screen. The speech signal can be converted / transcribed into text. The text can be entered directly into the device (302). The device is preferably a portable / mobile computing device that the patient can carry. In some embodiments, the portable / mobile computing device may be a device that facilitates digital communication with cloud services.

[0177]

[0233] The text input is processed by the natural language rule module (306), which then provides an emotion label and passes the input to the emotion recognition sensor (314).

[0178]

[0234] The intensity rating sensor (316) calculates the dimensions of each input and assigns an objective intensity rating.

[0179]

[0235] In the Aggregate Strength module (2504), the moving average and aggregate score are calculated.

[0180]

[0236] In the instantaneous intensity module (2502), the instantaneous intensity score is calculated.

[0181]

[0237] When the intensity of negative emotions being performed in the aggregate intensity module (2504) reaches a sufficiently high threshold, the system may proactively prescribe sensory sedation aids (2506) via peripheral devices configured for dialectical behavior therapy (DBT) to produce sensory sedation distractions for the patient, including, but not limited to, the prescription of spicy candy, vibration of a Bluetooth bracelet, or heating of a Bluetooth bracelet.

[0182]

[0238] When any given data point in the instantaneous intensity module (2502) reaches a negative emotion intensity that exceeds the danger threshold, the system proactively activates the peripheral device 2506 to produce sensory distraction.

[0183]

[0239] Language or speech intensity monitoring is resumed via both an intensity rating sensor (316) and an emotion recognition sensor 314. When the aggregate or data point intensity reaches a first threshold, various randomized sensory sedation distractions are activated.

[0184]

[0240] If the moving average of the aggregate intensity module (510) and the rating of the instantaneous intensity module (2504) do not rise above the threshold for 30 minutes or longer, the diary card is prescribed by the patient's central unit for personal or therapeutic use to record and document the experience (2508).

[0185]

[0241] In other embodiments, the emotion classification system with sensory distractions has applications beyond BPD. In some cases, a user attempting to overcome addiction to alcohol, drugs, or compulsive behavior is monitored for emotional cognitive signs prior to or during a relapse in an aftercare rehabilitation program or sobriety living home, and text or transcribed input (302) is provided to the device. In some cases, professional communications from a worker or manager are monitored for signs of specific emotional cognitive states, such as stress or anxiety, and input to the device (302). In some cases, when the intensity of negative emotions in progress in the aggregate intensity module (2504) reaches a sufficiently high threshold, the system alerts the facility, rehabilitation program, or sobriety living home, or the employer, that the user is at risk of relapse or extreme stress or anxiety.

[0186]

[0242] Figures 26A, 26B, and 26C are schematic diagrams of an electronic bracelet according to an exemplary embodiment of the present disclosure. The electronic bracelet 2600 may take the form of a ring 2612 into which electronic components are incorporated.

[0187]

[0243] Wire: Wire 2610 is a bundle of multiple insulated wires. Each individual wire is either positive (+) or negative (-). Wire 2610 is preferably nichrome (nickel-chromium), but can be made from other wire materials.

[0188]

[0244] The bracelet 2612 may include an embedded communications chip (connected to the microprocessor chip 2620) configured for wireless communications. Wireless communications are preferably short-range communications for sending and receiving signals from a portable / mobile device. In one embodiment, the communications chip performs wireless communications using Bluetooth or Bluetooth Low Energy (BLE).

[0189]

[0245] The sliding window feature 2614 is made of glass or plastic. The sliding window 2614 exposes a dispenser 2616, which is a single aperture. The sliding window 2614 is powered by a micro solenoid stepper motor 2602 (<5 mm).

[0190]

[0246] Device 2600 is powered by an embedded microprocessor 2620. The microprocessor includes a base for wires and components using assembly board type technology.

[0191]

[0247] The micro electromagnetic eccentric motor 2622 is an actuator that generates vibrations via an unbalanced load.

[0192]

[0248] The copper plate 2608 seated on the silicone provides heat.

[0193]

[0249] The device is powered by a lithium-ion rechargeable battery 2604. The rechargeable battery 2604 has an associated charging interface 2606.

[0194]

[0250] The indicator light 2630 on band 2612 is present for pairing with a portable / mobile device.

[0195]

[0251] The Band 2612 is translucent and can change color via an LED light in response to detected emotions. This may be useful for people with BPD who are experiencing harmful tension in a severely activated state, allowing them to communicate their emotions to a caregiver or loved one.

[0196]

[0252] Another adaptation of device 2600 may be useful for calming people with PTSD during anxiety attacks and for transmitting emotional information to loved ones and caregivers when triggered.

[0197]

[0253] In the above description, any process, description, or block in the flowchart should be understood as representing a module, segment, or portion of code containing one or more executable instructions for implementing a particular logical function or step in the process, and alternative implementations are included within the scope of exemplary embodiments of the present advance, in which functions may be performed in a non-linear order, including substantially simultaneously or in reverse order, depending on the functions involved, as can be understood by those skilled in the art.

[0198]

[0254] Emotional Cognitive Profiler

[0255] In an exemplary embodiment, emotion recognition classification can be implemented as a profiler.

[0199]

[0256] Text input is either spoken or otherwise entered into Natural Language Rule 306 for analysis. The input may be about the user, another person, or a topic. In some cases, users may video capture themselves responding to randomized job prompts. In some cases, OCR or audio transcription transcribes (written and video or audio encapsulated) text information into plain text.

[0200]

[0257] The text is analyzed for sentiment content by the emotion recognition sensor 314.

[0201]

[0258] Emotional cognition sentiment is labeled by the emotion cognition tagging engine 318 after classification.

[0202]

[0259] The intensity of the sentiment is calculated via the dimensions within the intensity rating sensor 316.

[0203]

[0260] The profile is generated within the emotional cognitive profiler using personality traits, expressive traits, cognitive traits, and emotional traits, as well as values ​​and biases.

[0204]

[0261] Demographic information is associated with profile information in the emotion cognitive profiler.

[0205]

[0262] The Aggregate Emotional Cognitive Ensemble Classifier 328 makes predictions about the types of candidates that are a strong fit for a date, job, or emergency. In some cases, the user inputs information about personality traits, values, cognitive traits, emotional traits, and the types of values ​​and deciding factors that their ideal candidates might represent, such as through surveys or questionnaires.

[0206]

[0263] Recommendations for recommending a given person as a match, rating desirability, and fit are made to date-filled site users, emergency consultants, or employment managers.

[0207]

[0264] The profiles collected by the emotion recognition profiler are retained and stored for future use by the machine learning model engine 320.

[0208]

[0265] Emotional Recognition Informed Texting / Communication

[0266] In exemplary embodiments, emotion recognition classification may be implemented as a pretext application, particularly including telephone banking or text banking.

[0209]

[0267] Users submit text or transcribed input to the system for emotion recognition guidance to inform them about healthy, empathetic, positive, and socially / subculturally affiliated communication techniques and psychology.

[0210]

[0268] Depending on the circumstances, the relationship in question could be a sales relationship, customer service relationship, workplace relationship, academic relationship, counseling / coaching relationship, romantic relationship, friendship relationship, family relationship, ex- or proposed relationship, estranged relationship, relationship in crisis, acquaintance relationship, or other personal relationship.

[0211]

[0269] When unhealthy, inappropriate, insensitive, confusing, or limited-perspective responses are detected in the text of one or more users, users are prompted to develop their communication skills by learning about stronger communication approaches through games, modules, tutorials, or other teaching devices. Regardless of whether users choose to learn through games or modules, they are asked if they would like to input more context about the situation.

[0212]

[0270] Users are asked if they want to try communicating again.

[0213]

[0271] If a user chooses to add context, they can answer a series of questions to help assess their emotional cognitive state and emotional or social situation. If a user declines to provide additional context, they can choose to be asked further questions to assess their own emotional cognitive state.

[0214]

[0272] In some cases, users can choose from a list of concerns, disputes, problems, and wishes regarding communication or relationships.

[0215]

[0273] Users are given the opportunity to change their communication techniques and input new communications that incorporate their new knowledge and awareness or reflect their altered emotional cognitive states.

[0216]

[0274] The user will be prompted to start fresh.

[0217]

[0275] Dynamic / Adaptive Embodiments

[0276] Spontaneous text or transcribed user input is entered into a computing device, app, gaming console, phone, or tablet.

[0218]

[0277] The text goes to the preprocessing engine 304, is analyzed by natural language rules 306, and then by the multimedia classification engine 312.

[0219]

[0278] Explicitly stated (explicit) and subtextually detected (implicit) emotional cognition states are assessed by the emotional cognition sensor 314, and the degree of the mental state is scored by the intensity rating sensor 316.

[0220]

[0279] User input is received as a label by the emotion recognition tagging engine 318.

[0221]

[0280] The system provides users with customized, interactive, argumentative experiences, offering crafted, selected, generated, edited, or otherwise transformed responses that are adapted to the user's emotional cognitive state. In some cases, the discourse may involve two or more computer-generated and / or guided bots or characters. In some cases, the discourse may take place within conventional, virtual reality, or augmented reality video games. In some cases, the discourse may involve one or more additional human agents. In some cases, the discourse may involve therapeutic bots that provide psychotherapy, counseling, crisis, or sobriety help to the user or group. In some cases, the discourse may involve virtual friends or assistants that provide companionship or assistance to the user. In some cases, the discourse may involve one or more automated vending or customer support bots.

[0222]

[0281] In some cases, environmental controls such as complexity, difficulty, volume, speed, color scheme, arbitration, plotline, minigames and / or side quests, options, questions, characters, warnings, commentary, dialogue, abilities or selections offered, as well as other such customizations and custom activities, will adapt to the user's response to the most recent aggregate bot-driven language response.

[0223]

[0282] Past responses from one or more users are stored in the database, paired with their respective recurring prompts, for future reference during current and future user sessions.

[0224]

[0283] Sociolinguistics

[0284] assignment:

[0285] Historically, sociolinguistics studies the close-up attitudes, beliefs, opinions, and sentiments of individuals, classes of individuals, groups of people, races, genders, socioeconomic groups, and other groups of people through observation and / or analysis of spoken, written, or gestured words.

[0225]

[0286] Sociolinguistics is a subfield of semantics that assesses texts in relation to social factors, including subcultures, race, gender, sex, subclasses of people, and regional differences. Sociolinguistics has long studied the context of language, including cultural norms, collaboration, expectations, identity issues, interpersonal dialogue, and social context.

[0226]

[0287] Researchers in the humanities and social sciences, as well as natural language processing (NLP) technicians in the computer science field, conduct analyses of spontaneous and crafted text and speech. Historically, these researchers and experts had to develop computer code to parse, clean, tag, annotate, and analyze words in order to gain insights.

[0227]

[0288] These shortcomings have alienated researchers from the humanities and social sciences (psychology, sociology, applied behavioral science, anthropology, communication, rhetoric, women's studies, ethnology, and political science, among others), depriving society of rich commentary on everyday language use and its significance. Furthermore, the general lack of integrated computational solutions for sociolinguistic and cognitive language assessment is hindering the acceleration of social progress in an era where text and speech communication is becoming increasingly fast, critical, and central to social progress, equity, and understanding.

[0228]

[0289] Furthermore, natural language, cognitive, and sociolinguistic pipelines themselves have generally required unsuitable solutions that result in difficult installations, reliance on and constraints of pre-built data flows, or difficulties and delays in collaboration.

[0229]

[0290] Traditional approach:

[0291] Traditional attempts to automate sociolinguistic research and analysis of digital, scanned, transcribed, or translated texts have required a set of multiple tools, each requiring technical expertise for classification and integration. Available corpora often suffer from limitations in scope, accessibility, and size, and additional corpora are difficult and expensive to manage. Few user-friendly scraping engines exist, offering only basic parsing and therefore failing to incorporate granular linguistic natural language tools that enable the classification of human sentiment, beliefs, opinions, or feelings, centered on observer or tagging insights in native environments, supervised learning (often with Bayesian probabilities), and unsupervised learning (such as neural networks).

[0230]

[0292] Existing solutions do not incorporate state-of-the-art social research tools that cover the entire research pipeline, from top-level research design, survey skip pattern design, volunteer recruitment, consent form incorporation, polling, surveys, interviews, or data contribution, data ingestion and intake, research supervision, and query generation. Similarly, existing solutions do not offer a suite of modern custom statistical analysis tools spanning linguistic, computational, aggregation, cloud integration, and machine learning training, testing, and usage.

[0231]

[0293] Existing solutions also fail to provide a seamless way to download, distribute, collect, and comment on survey and academic datasets, statistical sets, associated code, and result papers. Finally, existing solutions do not allow users to contribute datasets back to the general public or academic peers for further research.

[0232]

[0294] method

[0295] As technologies advance that expand the scope of sociolinguistic research and amplify humanities and science research outside the field of linguistics by incorporating textual, auditory, visual, historical, subcultural, collective, and geographical metadata for the advancement of research and society, this sociolinguistic pipeline is necessary and in demand.

[0233]

[0296] Currently, sociolinguistics is conducted on controlled, narrower datasets. Existing solutions are overly basic, lacking in granularity, and present high technical barriers to progress, thus creating a need for computational tools for professional or academic sociolinguists. The very few user-friendly solutions available for sociolinguists tend to be incomplete pipelines, only allowing basic management, simple search and most frequent phrase calculation, and export.

[0234]

[0297] This pipeline is novel and useful for continuous, broad computational linguistic analysis of various topics pertaining to public opinion management, commercial use in advertising and marketing assessment, viral topic tracking, propaganda and "fake news" detection, and related rating.

[0235]

[0298] In general, the performance of data sets often varies depending on the type and task. Existing solutions provide POS tagging, n-grams, collocations, and frequency and relative frequency, enabling scraping and basic linguistic, terminology or linguistic analysis. However, existing solutions do not provide robust, discipline-driven sociolinguistic tools, centralized academic, research and non-profit driven tools for sociolinguistic research in humanities and sciences, nor research design and development tools throughout the entire life cycle of research and investigation.

[0236]

[0299] This method also includes optional geolocation tools, word and terminology comparison tools, discourse analysis and social dialogue tools, as well as sociolinguistic sentiment analysis. When juxtaposed over time, linguistic shifts that indicate collective value detection and attitude change are also novel and can be used for social issue and social value tracking.

[0237]

[0300] Advocacy groups, non-profit organizations, and academic researchers currently do not have a comprehensive sociolinguistic pipeline to adequately address these needs, nor digital means for end-to-end design and development of robust sociolinguistic short-term, near-sighted, and long-term research. Public order can benefit from the digital application of sociolinguistic pipelines to other disciplines, and commercial brands can also benefit from messaging tracking and analysis, market research, product development, focus group testing, and automatic sociolinguistic sentiment analysis over time.

[0238]

[0301] Evaluation uses can also be applied to this pipeline, including national security analysis, cruelty monitoring by law enforcement or police, psychological and social behavioral research, social program evaluation, legal analysis, fair and effective employment practice, and crisis management.

[0239]

[0302] Finally, mental and physical health care, as well as vaccines, drugs, and therapist and physician care, are in great need of a sociolinguistic pipeline for comprehensive, integrated research analysis for large-scale assessment of healthcare issues, adverse drug reactions, and disease surveillance. Financial markets are currently vulnerable to movements driven by mass social media expression, and should benefit from means to assess, track and monitor social media expression through a technical lens.

[0240]

[0303] Method Features

[0304] The sociolinguistic engine provides the following features.

[0241] Incorporation of automated semantic tools available for sentiment analysis, including positive-negative-neutral (PNN) polarity-based measurement and specific emotion detection with multiple emotion labels.

[0242] A seamless data processing flow that can be hierarchically set up by a user to fully automate a custom pipeline for text data processing.

[0243] Incorporation of implicit meaning tools that establish, detect and enable representation and prediction of potential subtext inference information, word embeddings, sentence embeddings, cognitive linguistic features, structural semantic mapping for deductive and inductive inference, and markers.

[0244] User-definable measurable dimensions including, but not limited to, linguistic features and morphological, phonemic, lexical, sentence-level and paragraph-level structural components with user-assigned weights for algorithm development.

[0245] Arbitrary, built-in threshold determination and weighting for categorizing specific, predefined, or hierarchical semantic information.

[0246] Selectable and tunable machine learning integration, including seamless deployment of 0 to n models for supervised, semi-supervised, and unsupervised learning on a dataset.

[0247] A GUI (Graphical User Interface) guided wizard that allows non-technical users to make research design selections for setting parameters for observers, annotators, subjects, and evaluators in research projects for subjective labeling of text or transcribed data.

[0248] Calculation of dataset statistics mean, moving average, and other methods for central tendency, user-defined themes, historical sentiment and sentiment scores linked to corpora or exploration queries, implicit semantic scores per user-defined dimension for user-defined themes, corpora or exploration queries, and statistics on other user-defined syntactic, phonological, morphological, cognitive linguistic, sociolinguistic, and semantic linguistic dimensions of text.

[0249] Intensity: Our classification model is the only one we are aware of that classifies emotion, implied meaning, or lexical frame intensity instead of probabilities (classifications) provided as surrogates for intensity.

[0250]

[0305] Model

[0306] This invention describes a sociolinguistic pipeline that enables researchers from the disciplines and subdisciplines of the social sciences, computer sciences, and humanities to analyze spoken and written language obtained online and offline, without requiring any computer programming knowledge, and to create analyzable datasets that are analyzed by machine learning and procedural code, thereby providing insights to users.

[0251]

[0307] Researchers, non-profit organizations, and other users can use sociolinguistic engines to deeply analyze speech and text corpora. Texts can be time-sliced ​​for tracking over time or constrained for time-limited analysis.

[0252]

[0308] Preprocessing includes part-of-speech tagging (with the option to create POS grams), n-gram generation, frequency and relative frequency calculation, named entity recognition, collocations, skip grams, bootstrapped n-grams, bootstrapped other vocabulary, and other NLP data preparation functions.

[0253]

[0309] Technical details

[0310] The core sociolinguistic engine system flow incorporates the following essential and optional customized steps:

[0254] I. The system begins by saving, ingesting, scraping, and otherwise cataloging text (including, but not limited to, audio or video spoken language ["speech"], plain text, marked-up text and emojis, annotated, hyperlinked or augmented text, transcribed or translated text) posted, generated, responded to, digitized, transcribed, and resident via and within the Internet. The selection is scraped, cleaned, and placed in structured and unstructured databases (respectively) for storage.

[0255] II. On the other hand, the sociolinguistic engine undertakes continuous corpus collection and aggregation, including, but not limited to, social media posts, forum posts, Q&A site responses, internet comments, advertising copy, educational copy, written or spoken works of art, and legal texts. Where applicable, automated tagging begins with continuous data formatting, indexing, and processing / enriching, including, but not limited to, the memory and tracking of beliefs, opinions, feelings, and stances of a given individual or group regarding and triggered by various themes and topics in a continuous sentiment analysis development. The data is then indexed for future exploration.

[0256] III. Next, administrators receive options for developing sentiment analysis on the dataset, enabling the data to be processed in terms of emotions, sentiments, beliefs, and opinions by employing semantic analysis tools and methods, including, but not limited to, neural networks, random forest algorithms, clustering algorithms, principal component algorithms, matrix decomposition algorithms, Bayesian classification algorithms, rule-based engines, and search engines.

[0257] IV. Once automatic tagging is complete, users can access a graphical user interface on a personal computer, smartphone, tablet, or mobile handheld device to create and manage surveys, create new data, or tag data stored throughout the survey.

[0258] V. Users can access a graphical user interface to create new cognitive linguistic component tagging rules that can be applied to existing and future data.

[0259] VI. A user can access a graphical user interface for creating query-driven custom views on data to answer research questions including, but not limited to, insights on academic and scientific research questions, business intelligence, marketing intelligence, stock market analysis, and political campaigns.

[0260] VII. Finally, surveys, tagged data, and new user-generated corpora and sub-corpora can be incorporated into a data store to allow new queries to be performed against the data store.

[0261] VIII. Similarly, new user-created automatic tagging rules, query-driven custom views, visual representations of data, and machine learning models, as well as their adjustable thresholds, parameters, and hyperparameters, can be incorporated into an automatic tagging engine for tagging and indexing new incoming data for future access and querying.

[0262] IX. The two preceding steps are described as including a data feedback loop for improving data analysis, statistical and machine learning performance.

[0263] X. The graphical user interface allows exporting a selected data set, along with selected features, components and parameters, for analysis.

[0264] XI. Contributing data to an open source data storehouse upon publication or completion of a project.

[0265]

[0311] Exemplary Embodiment

[0312] A professor of sociology wishes to conduct a study of disease-related words used over time in online discourse since 2002.

[0266]

[0313] The previous method would primarily allow the results of top words, phrases, or topics to be calculated, marked in the data, and reported. Figure 28 is a system diagram of user interface features for sociolinguistic data. Figure 29 is a flowchart of the sociolinguistic engine pipeline. This sociolinguistic pipeline is novel and non-trivial due to its ability to receive free-form spontaneous expressions and convert them into quantifiable and computable data.

[0267]

[0314] Previously, qualitative researchers were primarily able to focus on subjective and aggregated impressions. Datasets could be parsed solely by metadata, such as age, cancer stage, location, or marital status.

[0268]

[0315] The sociolinguistic pipeline is also novel due to its ability to facilitate correlation, computation, machine learning statistical, data analysis, class-based, or predictive classification from partially or purely qualitative data.

[0269]

[0316] The professor inputs the URLs into the engine's corpus builder, which then aggregates, normalizes, and merges the scraped data into a single text dataset.

[0270]

[0317] Next, the user imports audio files collected by the sociology department over the past five years using a tape recorder.

[0271]

[0318] Audio files are transcribed into text and a metacorpus is added. Each subcorpus includes a field that distinguishes its source.

[0272]

[0319] The professor enters a query with the aim of analyzing the historical trends of disease-related words.

[0273]

[0320] Audio files are automatically tagged with responses to open-ended questions that were previously difficult to quantify, such as the following:

[0274]

[0321] "Could you tell me a bit more about what happened then?"

[0322] "How do you cope with that?"

[0323] "Did you receive support from anyone at that time?"

[0324] "What do you think you learned from this experience?"

[0325] "How have you applied what you've learned to other aspects of your life?"

[0326] For each of these questions, a sociolinguistic pipeline can then quantify, but is not limited to, the following dimensions (and their respective subcomponents):

[0275]

[0327] emotions

[0328] Stance on the issue

[0329] Personality

[0330] belief

[0331] perspective

[0332] Sentiment

[0333] perception

[0334] opinion

[0335] The professor and her team conducted eight 90-minute audio-recorded interviews with the patients and their primary caregivers over a two-year period, including four interviews throughout the entire chemotherapy process and one follow-up chemotherapy course.

[0276]

[0336] The audio from the remaining interviews is transcribed and incorporated into a sociolinguistic pipeline, where the PI can simply upload the audio interviews using data preprocessing tools.

[0277]

[0337] The professor receives an email notification once the file has been processed and transcribed into text.

[0278]

[0338] The pipeline's data import wizard allows researchers to declare the names of relevant fields, enter metadata, or merge datasets.

[0279]

[0339] The professor can then work with the data through linguistic sentiment NLP, statistical or semantic tools, including, but not limited to, tokenization (receiving top words and phrases used by cancer patients during interviews), collocations to discover key terms surrounding queried terms from interviews, correlations, and sentiment scores to track feelings, opinions, or beliefs or stances at each interview checkpoint and / or follow-up.

[0280]

[0340] The professor may choose to perform comparative metrics to compare the language and / or sentiment used by patients with respect to time, geographical location, or other metadata, as well as to receive centrality measures, correlation measures, and points for datasets and further statistical interventions (which would normally be unavailable for qualitative interviews).

[0281]

[0341] Furthermore, the professor can perform inferential comparisons with similar studies of cancer patients receiving different or similar treatments, integrate qualitative interview datasets with written post-interview follow-up surveys, export the datasets for further analysis in statistical programs, and perform machine learning (user grouping, clustering, health prediction, and assessment of which features are important for health).

[0282]

[0342] If the research generates new data, the professor can train machine learning algorithms on that new data and / or contribute it back to the data store for use by others.

[0283]

[0343] If the professor wishes, they can study or discover patterns in the data for further conclusions.

[0284]

[0344] Furthermore, patients can be clustered, and cases can undergo T-tests or other measures for anomaly detection.

[0285]

[0345] The professor may choose to use additional tools to highlight significant features, such as, but not limited to, random forests.

[0286]

[0346] Using machine learning, professors can detect cancer patients with the lowest health levels, their associated characteristics, and / or those at the highest risk of suicide.

[0287]

[0347] Once a research paper is published, the professor can contribute her dataset to the general knowledge pool via a data store, approve its use under selected or all conditions, and refer to her research paper for categorization and use by other researchers.

[0288]

[0348] While several embodiments have been described, these embodiments are presented merely as examples and do not limit the scope of this disclosure. In fact, the novel methods, apparatuses and systems described herein may be embodied in various other forms, and furthermore, various omissions, substitutions, and modifications of the forms of methods, apparatuses and systems described herein may be made without departing from the spirit of this disclosure. The appended claims and their equivalents are intended to encompass such forms or modifications so as to fall within the scope and spirit of this disclosure. For example, this technology may be structured for cloud computing, so that a single function is shared and processed collaboratively among multiple devices over a network.

[0289]

[0349] Semiotic Linguistic Political Analysis

[0350] assignment

[0351] There is no sentiment analysis system that can reliably and accurately represent political stances, including party alliances and their psycholinguistic subcomponents. Furthermore, there is no semantic system that can capture the nuanced signals of subtextual political beliefs in transcribed, translated, or authored text or (audio) speech with immanent semantic framing.

[0290]

[0352] Machine learning solutions fail to capture the nuanced and implicit meanings of political speech in a finely tuned, valence-based quantifiable format. These features are increasingly important as speech and text communications, which are difficult to repeat, highlight their impact on the masses and their deterrent interests (such as the riots at the U.S. Capitol on January 6, 2021).

[0291]

[0353] Machine learning has never been applied to these cognitive linguistic elements, along with precise measurements of belief systems, self-and-other valence, or subcomponents of intra-messaging argumentative turns, in a way that produces robust personal profiles, clusters, and relationships between political stances and their subcomponents within communities, political parties, and groups of people.

[0292]

[0354] Emotional detection, both in the distribution of political communications and in the evoked and stimulated emotional responses within collective and individual audiences, is essential for protecting society, understanding and maintaining political discourse and free speech, and preventing potential future political violence.

[0293]

[0355] Furthermore, a normalized set of ratings and classifications of the system is needed to further classify and detect other aspects of sentiment implied and contained within text and spoken political speech, including, but not limited to, opinions, beliefs, and stances.

[0294]

[0356] Attempts to measure and improve outcomes of shifts in social belief systems, reductions in hate crimes, mitigation of political violence between parties, and other prosocial peacekeeping efforts require a set of systems for measuring online, cellular, digital, and analog communications, their implications, nuances, and impacts.

[0295]

[0357] Finally, measuring positions along the political spectrum is necessary to detect political extremism or domestic and international terrorism.

[0296]

[0358] Traditional approach

[0359] Automated classification of political stances lacked the granular, enriched micro-information about beliefs, opinions, and feelings (sentiments). No single system could reliably predict, all at once, shifts in political stance, types and tendencies of swing voters, and the likelihood of changing opinions and party affiliations.

[0297]

[0360] Furthermore, the theoretical cognitive linguistic elements capable of capturing the nuances of political beliefs have never been reliably quantified in a normalized way that allows for comparison points between sectors of society.

[0298]

[0361] Such systems are needed after political interference by foreign entities. Furthermore, the use of political language, video, and speech in a manner that highlights the emotional and cognitive impact on citizens has attracted attention in recent months and years. Systems for detecting subtle differences in messages and their impact on audiences during and after political communications are urgently needed.

[0299]

[0362] Furthermore, the rise of subcultures such as white supremacists and other extremist groups on each side of the political spectrum, as well as their social impact and increasing violence, necessitates a set of systems for quantifying, identifying, and predicting emotional escalation and the rise toward political violence.

[0300]

[0363] Previous solutions to this problem were able to predict violence itself, but they were unable to link it to the implications and implicit meanings in specific elements of communication.

[0301]

[0364] Previous solutions to this problem have failed to address the sub-elements of cognition, linguistics, emotion, intellectual processes, and assumptions, as well as the micro-locations across the political spectrum beyond major parties.

[0302]

[0365] The classification of opinions, beliefs, sentiments, and emotions in this domain is crucial for peaceful societies, threat detection, public discourse, demagoguery, and insights into political audiences applicable to both the human and computing domains.

[0303]

[0366] method

[0367] Ingestion of website or otherwise digitized text, transcribed audio, and their linguistic preprocessing and subsequent linguistic analysis measurements for aggregating multidimensional scores, valence, polarity, and deixis.

[0304]

[0368] Conversion to a single-score Political Lean Score (PLS) with sub-dimensions of attitude readings regarding governmental, constitutional, ideological, and social issue tracking of multidimensional authors, speakers, political parties, and news media or content sources.

[0305]

[0369] Continuous updates of content and media scores will be incorporated as a new data line.

[0306]

[0370] Detection and classification of data lines related to Swing Voter Likelihood Scores (SVLS), party loyalty thresholds, rates of change in continuous stances or political event communications over time, the degree to which they change with respect to time or events, and measurement of the severity and intensity of events.

[0307]

[0371] A class of swing voters that can be detected in terms of political ambivalence, openness, flexibility, susceptibility, adaptability, and movement along the political spectrum and related sub-axes, according to central causes and major factors.

[0308]

[0372] Capabilities that should be utilized in national security operations, human-robot interaction, educational applications and experiments, emotion modeling for social robots, and other emotion-detecting security interactions.

[0309]

[0373] The APS3 score measures the position of force, control, refinement, direction, nature, action, and status relationships as represented in linguistically calculated syntax and strings.

[0310]

[0374] The ability to incorporate and implement the translation of social psychology into robot-readable measures, including group dynamics, intergroup relations, bias or stereotyping, and organizational psychology.

[0311]

[0375] While not limited to these, the calculation and transmission of cognitive emotional linguistic elements such as force valence, Enacted Virtue Score (EVS), Comparative Value Score (CVS), Auto-Positioning Social Subcomponent Score (APS3), and their respective sub-dimensions are embedded.

[0312]

[0376] Measurement of feature components in multidimensional space, resistance, religion, locality score, ironic construction, rhetorical device type, speech act, and dynamic psycholinguistic state changes.

[0313]

[0377] User attribute scoring and valence derivation for sub-components such as bravery, curiosity, trustworthiness, vulnerability, emotional intelligence score (EQS), empathy, judgment, and learnability, when these are related to political position and status.

[0314]

[0378] Model Features

[0379] Profiles of archetypal users of a given cluster to identify micro-fractions of political stances and positions along relevant sub-axes, both collectively and individually.

[0315]

[0380] The inclusion of microlabels and sub-axes for political stances, breaking down political and party affiliations into granular sentiment-based sub-sub

[0316]

[0381] While not limited to them, the theoretical components of cognitive linguistics, including framing, conceptual metaphor, literary techniques, metaphor, conceptual blending, mental space, lexical semantics, frame semantics, force dynamics, and other cognitive linguistic subcomponents, are measured and incorporated, where they were previously not part of an integrated set of systems for measuring these subcomponents for the quantification and computation of these affective influences through language.

[0317]

[0382] Incorporation of semantic-centered theoretical cognitive linguistic elements applied to input data in a quantifiable form through statistical and machine learning elements, labels, matrices, and conversion to input, in order to capture rich semantic political information regarding stance, political leanings, and beliefs.

[0318]

[0383] A heuristic for flagging social media comments in response to audio (such as podcasts), visuals (such as videos and images), and text or speech (including, but not limited to, speeches, commentaries, interviews, news articles, comments, posts, emails, and other inputs) that do not match the observed or witnessed posted content.

[0319]

[0384] Our model possesses a unique ability to combine machine learning (supervised and unsupervised learning) with the system's rule-based set, providing more empirical classification of sentiment, feelings, beliefs, and opinions, stronger profiling of political parties using language, more effective classification, and greater flexibility around datasets.

[0320]

[0385] The use of metadata, synchronized streams, model deployment, partitioning, baseline comparisons to perceptual outcomes, and valence calculations to detect political sectors and sentiment inferences.

[0321]

[0386] Technical details

[0387] The ability to collect labeled data is achieved by using heuristics, filtering, location-based and rule-based proxies for political affiliations and stances on a predefined list of sub-axis, themes, topics, and lexical frames.

[0322]

[0388] Collection and integration of signature n-grams associated with people, websites, locations, or time periods, by statistical and machine learning approaches, including, but not limited to, n-gram counts and measures of specificity for such entities compared to other people, websites, locations, or time periods.

[0323]

[0389] This system begins by identifying characteristic sub-axes, themes, topics, and vocabulary fields and frames specific to particular politicians and public figures through text mining of the top n grams of user-generated data in various forms, including but not limited to tweets and transcribed speeches, and through topic modeling.

[0324]

[0390] These systems can also identify characteristic sub-axes, themes, topics, and vocabulary fields and frames unique to specific social media users through text mining of top n grams of user-generated data in various forms, including, but not limited to, tweets, Reddit posts, transcribed speech from social media conversations, photos, liked content, and audio where available, as well as topic modeling.

[0325]

[0391] These systems can also identify characteristic sub-axes, themes, topics, and vocabulary fields and frames specific to particular social media outlets through text mining of top n grams of user-generated data in various forms, including, but not limited to, items, posts, transcribed speech-format videos, social media interactions from official outlet accounts, photos, likes, and audio where available, as well as topic modeling.

[0326]

[0392] These systems can then define a final list of issues on which to measure the user's stance, based on sub-axes that are predefined and discovered through statistics and machine learning.

[0327]

[0393] These systems can then identify user stances on each issue and sub-axis with certainty scores using machine learning approaches, including, but not limited to, rule-based indicators, such as virus signature n-gram adoption, dominant theme identification in words, n-gram, skip-gram and part-of-speech (POS)-gram co-occurrence and special collocations, Bayesian classifiers, collaborative filtering, matrix decomposition, clustering algorithms, LSTM neural networks, and fine-tuned transformer neural networks. These systems then associate this data with mined user metadata that can indicate age, racial group, and socioeconomic status.

[0328]

[0394] These systems can then predict user similarity based on the stance similarity of the manifested list of sub-axis, using machine learning approaches, including but not limited to neural networks, Bayesian classifiers, k-nearest neighbor algorithms, clustering algorithms, logistic regression, SVM, and random forests.

[0329]

[0395] These systems can then predict political alliances using user stances as input features, employing machine learning approaches that include, but are not limited to, rule-based indicators, Bayesian classifiers, LSTM neural networks, and finely tuned transformer neural networks trained on political data and public responses.

[0330]

[0396] The collection of user information may be integrated with respect to discovered and predefined dimensions, including, but is not limited to, sympathy, stubbornness, likelihood of political stance change, and tendency toward violence.

[0331]

[0397] These steps can then be repeated for any specific point in time, and by analyzing changes in the data, it becomes possible to track political movements and identify trends in political stances over time, for a single user, a specific group of users, or on a dataset of all users as a whole.

[0332]

[0398] By cross-referencing user-generated data exposed to propaganda, advertising, and other identified sources of influence, these systems can quantify the degree of change in stance on any identified sub-axis.

[0333]

[0399] The results of the analysis for specific users and time periods may be visualized in GUIs, including but not limited to websites, browser plugins, and smartphone applications.

[0334]

[0400] These systems can also be applied to assess a user's stance on a single or limited data point, such as a single post, video, or any form of incoming text, audio, or video.

[0335]

[0401] Prejudice, discrimination, and bias detection

[0402] These methods are unique in their ability to recognize hidden and overt biases, and their fine-grained linguistic features are extracted particularly from natural language and applied to bias and discrimination-based expressions, which can be applied and refined to cases of racism, sexism, ageism, homophobia, transphobia, xenophobia, sexual harassment, classism and meritocracy in spontaneous or crafted speech or text by private and public individuals and entities, non-profit organizations, publications, websites, and other media sources and their communication media.

[0336]

[0403] assignment

[0404] The detection of discrimination against a given group of people, a protected class, a minority, or a vulnerable or disadvantaged group, the prejudice held against them, and the otherwise negative or biased sentiment held about them, has not been readily standardized and quantified in computational linguistics. Because detection from these writings is overly dataset-specific, no formalized system has emerged that can be reliably applied across time, text, and bias.

[0337]

[0405] Furthermore, when sociolinguists, sociologists, psychologists, and ethics researchers, women's studies, and queer studies (as well as their equivalent humanities and science studies) examine language up close, there is little reliable set of rules and cues developed as standards for detecting racial bias.

[0338]

[0406] As a result, there is no unified measure for assessing natural language or spontaneous speech. Therefore, a consistent relative degree of bias and prejudice has not been established, making it difficult to exercise accountability and promote more rapid social change.

[0339]

[0407] This creates a society where the suffering of minorities is "invisible," and specific groups are effectively deceived and confused due to the lack of definitive and unified evidence in language. This also makes it difficult for organizations and companies to measure their performance and responsiveness as collective entities with respect to implicit bias.

[0340]

[0408] Finally, real-world outcomes require a set of systems of measures to be correlated from a lens of implicit bias and discrimination, so that public and private individuals, entities, and systems can be kept accountable, assessed, and improved (and interventions can be designed to improve performance in the communications of such systems, individuals, and entities in fair treatment). Societies need solutions for bias prediction and degree, which will be addressed by these systems to identify, prevent, and detect the probability of personal or collective violence from or against groups of people, both in general and in any given time or location.

[0341]

[0409] Subtle forms of bias, such as "outsider treatment" (treating minority and oppressed classes as if they were outside the norms of whiteness, heterosexuality, cisgenderity, or other majority status), "white knight" (placing majority status members at the center as saviors and treating minority and oppressed classes as if they were in need of salvation), or "microaggressions" (small, hidden, and easily overlooked instances, often unconscious, but collectively causing psychological harm), are difficult to enumerate, anticipate, and quantify.

[0342]

[0410] Therefore, to date, there has been no standardized method for quantifying bias and discrimination.

[0343]

[0411] Features

[0412] Computational efficiency of discrimination detection through the implementation of indexes in all integrated algorithms specified herein.

[0344]

[0413] High clarity and precision through time and text slicing techniques to pinpoint emotional and biased responses during interviews, observations, and communications input into the system.

[0345]

[0414] Further computational efficiency (query speed) of accessing results through the implementation of indexes in the included algorithms.

[0346]

[0415] Integrating multisensory inputs into a single bias detection system that outputs a set of normalized scores and bias master scores indicating the presence and degree of bias, prejudice, discrimination, or favoritism in the perception and behavior of a given input.

[0347]

[0416] The ability to link discrimination and bias ratings and inputs to real-world outcomes, including, but not limited to, judicial decisions, fair housing assessments, police brutality allegations, employment discrimination, media bias, persuasion and marketing outcomes, commercial ROI (results in impressions, literally, on web, tablet and smartphone entities, and other conversions from expectation to sale on other media), and social norms assessments.

[0348]

[0417] The ability to assess the implicit bias of the observer when witnessing and internally processing multisensory data, as well as the implicit bias present in the media itself and the actors within it, allows for correlation and statistical joining between the two.

[0349]

[0418] A tweakable application that can be adapted to parameters for discriminatory speech and linguistic expressions, and can be applied beyond conventional classifications of bias to finely tuned features of intersexism bias, country of origin discrimination, pregnancy discrimination, religious discrimination, parental or partner status, and socioeconomic class.

[0350]

[0419] The use of metadata and computational linguistic assessments, along with other sensory inputs, to create profiles of geographical, socioeconomic, political and social stances, age-related, and other characteristics for predicting entities and individuals that may transmit bias, and to implement effective interventions.

[0351]

[0420] The ability to monitor interventions on timelines and location tracks for adjustment, improvement, and approval purposes.

[0352]

[0421] Base State Detection (BSD) markers enable automated data line identifiers that are copied and sorted into a data store according to markers instantiated for inclusive dimensions such as gender, race, socioeconomic class, sexual preference, degree of physical ability, and poverty or socioeconomic index transformation.

[0353]

[0422] Calculation of comparison signals between data with BSD marker attachments and data in the incoming data flow, enabling AI to improve the efficiency of subtext channel processing time and achieve reductions in both training rounds and system resources.

[0354]

[0423] method

[0424] These bias and prejudice detection systems quantify discriminatory stances and negative attitudes toward protected, minority, disadvantaged, or particular classes of people. These systems clarify the detection of discriminatory language through the detection and measurement of subtextual linguistic cues by the presence and placement of features within a sentence, as well as by measuring instances (in range [0,n]), intrasentence relative position (IRP), multiple syntax sequences, fuzzy or exact string matching or combinations thereof, linguistic syntactic and prominent sociolinguistic features of any type in the following categories, and their associated measurement points in microdimensional comparison (MDC).

[0355] 1. Minimize 2. Shifting blame 3. Reframing the argument as a form of non-bias 4. Ignore 5. Treated like an outsider 6. Fear-based descriptive language 7. Threatening-based language 8. Unfair accusations 9. Discriminatory rejection 10. Animal Languages 11. Biased language 12. The Strawman Technique and Exaggerated Mockery 13. Justifying Bias 14. Nullification of pain ( 15. Extremist framing 16. Rebellious language 17. Blaming the Victims 18. Keep quiet. 19. Clear threat 20.Domination techniques 21. Claim of honorific titles 22. The argument of what should be done. 23. Defensive Techniques 24. Favoritism in explanations (mansplaining, whitesplaining, etc.) 25. Non-recognition of victims 26. Stereotyping 27. Exclusive language 28. Class-based language 29. Doubts about the strengths 30. Personality assessment based on rank 31. Defamation campaign 32. Double Standards 33. Linguistic mimicry 34. Deletion and pretentious attitude 35. Exoticization 36. Play Quality Nostalgia 37. Microinspection 38. Perception and praise of majority characteristics 39. Leading Questions 40. Models praise minorities 41. Paternalization 42. Appropriate language 43. Language that treats people like criminals 44. Sermon 45. Inhuman language 46. ​​Genuine speech 47. Intrusion / Invasion / Takeover Language 48. Allegations of fraud / abuse of social welfare services 49. Allegations of social spoiling 50. Complaints about assimilation 51. Intelligent Language 52. Asserting prejudice as honesty / realism 53. Conservation Terminator 54. Misconceptions about special treatment 55. Prejudiced preamble / modifier before a racist comment 56. Prejudice disguised as humor 57. Use of targets for social validity confirmation 58. Invalidating the trauma caused 59. (False / disappointing) generalization ("one of them," "someone like that") 60. Demonstrative pronouns and indefinite pronouns 61. To deceive and confuse 62. Dehumanization 63. The normalization of bias 64. The Bandwagon Fallacy 65. An order to ignore narrow-mindedness. 66. Claims of minority status as leverage / privilege 67. The Bootstrap Myth 68. Justifying Freudian Gaffes 69. Trivialization 70. Racialization 71. Restorative Justice as a Double Standard Assertion 72. Ironic flattery 73. Intimidating minority languages 74. The Single Data Point Fallacy 75. Dominant Language / Power Assertion 76. Reminders of low social status 77. Sacrifice as an agitator (worthwhile damage / assertion of power) 78. Uncivilized claims 79. Infantile language 80. Nickname 81. Objectification 82. Sexualization 83. Fetishization 84. Majority-Oriented Language 85. Condemnation of Dishonesty 86. Condemnation of the lack of strengths 87. Justification by token existence or success 88. Assumptions of superiority, such as majority race, majority ethnicity, dominant sex, majority sexual orientation, being able-bodied / non-disabled, or otherwise privileged by society. 89. Contempt or condemnation implied within a semantic framework 90. Implicit demands for persecutor education

[0425] Embodiment

[0426] These elements are distinguished in language by an ensemble of one or more of the following linguistic features:

[0356] 1. From the feature list, identify 0 to n discriminatory and biased linguistic markers, where each linguistic marker represents a second linguistic embedding of a cognitive component measurement (CCM) that contributes to the overall bias score and discriminatory label.

[0357] 2. Skipgrams of phrases in a corpus that can highlight differences in the treatment of groups of people, as well as available enumerations of computed cognitive linguistic features associated with prejudice, bias, or discriminatory views or practices directed towards a given group of people, minority, or protected class, depending on their presence or absence, intensity and degree.

[0358] 3. Any assembly of corpora of sound, images and videos, as well as social media data and metadata, that are observed by humans and analyzed and rated by machine learning algorithms, in order to assess implicit bias, discrimination and prejudice through microexpression analysis, sonar analysis and computation in speech transcriptions from permitted access, uploads, live video, audio, or camera captures, in all aspects.

[0359] 4. Additional optional capture of input from the observer themselves, through images, videos, audio, written or transcribed or translated text, survey data, interviews or comments, and any corresponding metadata, from live, website or social media interactions.

[0360] 5. Perform natural language package expansion on the text corpus, generate part-of-speech (POS) tags for each word in the corpus context, and create a mapping of each word to its corresponding part of speech.

[0361] 6. Where appropriate, but not limited to, develop further linguistic enumerations, notations, and tagging, including structural dependencies and collocations.

[0362] 7. Identification and preprocessing of any face image, including but not limited to semantic segmentation, reframing, and normalization.

[0363] 8. Identification and preprocessing of relevant, timed, user-generated audio fragments that can express user emotions, intentions, and opinions.

[0364] 9. Train selected models on controlled corpora or prototexts that, where applicable to input, demonstrate bias, discrimination, prejudice, or favoritism towards a given group of people, protected classes, or minorities, as well as on transcribed, translated, inputted, or scraped natural language text, gesture input, facial expressions, or spontaneous speech.

[0365] 10. If applicable to the input type, train a computer vision emotion detection model on the collected and pre-processed facial data.

[0366] 11. If applicable to the input, train an audio emotion recognition model on collected, identified, and pre-processed sounds and user-generated speech.

[0367] 12. In some cases, facial sentiment prediction is used to predict the sentiment label of an image and associate that label with the user's existing text. This allows additional manifested data to be tagged with levels of credibility, trust, guilt assumption, truth assumption, and favoritism in their responses.

[0368] 13. In some cases, use sound sentiment prediction to predict the sentiment label of an image and associate that label with the user's existing text. This allows additional manifested data to tag the sound and text data at levels of credibility, trustworthiness, guilt assumption, truth assumption, and favoritism in their responses.

[0369] 14. List the part-of-speech grams (POS grams) that potentially highlight differences in how people are treated by different groups.

[0370] 15. List and examine the frequencies of POS grams and skip grams present in the data.

[0371] 16. Use one or more statistical approaches to discover skipgrams and part-of-speechgrams that potentially highlight or emphasize differences between how groups of people are treated.

[0372] 17. Augment the data, where necessary, using human-generated or machine-generated word lists that include arrays of sociolinguistic indicators of bias in multiple contexts directed at a given group of people, a control group, or the general population.

[0373] 18. Apply appropriate data augmentation methods to generate similar synograms (semantically similar n-grams) to be used as features.

[0374] 19. Frequent n-gram augmentation by replacing each word, as needed, with an array of the respective word or sequence of words that have similar semantic meanings, valencies, sentiment labels, sentiment clusters, semantic associations, or polarities to the identified word.

[0375] 20. Augment one of the lists of frequent n-grams using k-nearest neighbors in the Word2Vec embedding space, and additionally sort the antonyms and synonyms from the list, respectively.

[0376] 21. For the cases shown, compare skipgrams distinguished by significant differences, distance and polarity, sentiment, valence, clustering, or two or more terms or words that each have semantic association or meaning.

[0377] 22. Identify features that are indicators of bias, and detect the presence of enumerated and augmented features in input, corpus-based, scraped, translated, or transcribed text or speech.

[0378] 23. Calculate a score for each applicable label based on the detected features, including but not limited to text-based facial image features.

[0379] 24. To quantify bias, train machine learning methods, including but not limited to LSTM, transformers, CNNs, KNNs, clustering, random forests, linear regression, and Bayesian probabilities, on any augmented data and selected features from the input.

[0380] 25. Use image and text data to output implicit bias, discrimination score, prejudice score, credibility score, truth score, guilt assumption score, and favoritism score, and to grade them.

[0381] 26. Use the resulting trained model to predict bias, discrimination, or prejudice in any video, image, or sound wave file associated with a given speech or text input.

[0382] 27. Assessment of audio, text, and computer vision responses to similar or the same communications by different separate groups of people by analyzing the ratio, time or passage boundary, or word count of favorable, unfavorable, and neutral facial text or auditory expressions and responses (as well as their subdimensions and specific emotions) to group communications, gestures, images, voice, or profile data of other people.

[0383]

[0427] The above disclosure also includes the embodiments described below.

[0384]

[0428] (1) A method for automatically augmenting natural language content with emotion recognition using a processing circuit. The method includes receiving natural language content as text input via an input device; searching for a match between a given set of language rules for emotion recognition and components of the text input using a processing circuit; activating a matched language rule, wherein an instance of the language rule has at least one human dimension, and evaluating at least one human dimension of the activated matched language rule using a processing circuit; scoring each human dimension using a processing circuit to obtain an archetypal profile of the dimension score of a given emotion recognition; aggregating the dimensions in the obtained profile of the dimension score using a processing circuit to obtain an intensity indication of a given emotion recognition; and relating the matched language rules to a given emotion recognition and displaying the augmented natural language content on a display in a manner that signals the respective intensity indications of the given emotion recognition.

[0385]

[0429] (2) Human dimensions include sentiment, emotion, emotional cognition attitude, values, social customs, way of thinking, viewpoint, manner, response, characteristics, beliefs, opinions, perspective, motivation, bias, state, manner, approach, dynamics, personality traits, emotional approach, emotional selection, reaction, tendency, temporary state, change of state, cognitive manner, behavioral manner, internal conditions, external conditions, feeling, emotion, assertion, attitude, assertive attitude, oriented attitude, unoriented attitude, spontaneous attitude, conscious emotional cognition, unconscious emotional cognition, anger, expectation, disgust, fear, joy, sadness, surprise, trust, ego, blame, suitability, sanctity, kindness, respect, time, favor, approval, sincerity, vulnerability, judgment, isolation, purpose, stiffness, and most A method of feature (1) comprising one or more of the emotional emotions of minimization, specificity, force, action, agency, curiosity, clarity, intention, emphasis, energy, certainty, interest, engagement, shock or surprise, tension, speed, nuance, logic, paranoia, distance, discrimination, respect, objectification, attachment, empathy, and patience, wherein each dimension has a value of one of the following: +1 for positive force, -1 for negative force, 0 for neutral force, and φ for non-existence or inapplicability, wherein scoring each human dimension by a processing circuit comprises scoring the human dimensions for all matching rules.

[0386]

[0430] (3) The method of feature (1) or (2), wherein the step of searching using multiple language rules further includes detecting syntax based on the language rules and evaluating the human dimensions of each detected syntax.

[0387]

[0431] (4) Any method of feature (1) to (3), wherein the scoring step includes comparing an intensity indication to a given threshold of emotion cognition in order to obtain an emotion intensity level.

[0388]

[0432] (5) The method of feature (3), wherein the step of detecting syntax further includes detecting the presence or absence of syntax having components related to a given sentiment cognition in natural language content.

[0389]

[0433] (6) Any method of features (1) to (5), further comprising: determining a pattern of emotion cognition including a given emotion cognition by linking it with other emotion cognitions detected by other language rules using a processing circuit, identifying the pattern of emotion cognition as dynamic emotion cognition, and tracking the given emotion cognition and other emotion cognitions together with related components in a time sequence on natural language content.

[0390]

[0434] (7) The method of feature (3), further comprising the step of detecting syntax determining a numerical value among one or more of the following: part-of-speech tagging or syntax rules, string matching rules which are exact, inaccurate, masked, or wildcard, token proximity rules, punctuation rules, lemmatization rules, stemming rules, lexical rules, and word lookup or dictionary-based rules.

[0391]

[0435] (8) The method of feature (7), wherein the step of determining the numerical value of the token proximity rule comprises accessing all tokens in the natural language content that are less than n tokens away from a given point, where n is a positive integer.

[0392]

[0436] (9) Any method of feature (1) to (8), further comprising generating new language rules by a machine learning engine that performs at least one of supervised learning and unsupervised learning.

[0393]

[0437] (10) The method of feature (9), further comprising: receiving multiple natural language data items from a repository; normalizing and tokenizing the received multiple natural language data items using preprocessing to generate multiple preprocessed natural language data items; labeling the multiple preprocessed natural language data items by expressed emotion cognition and the intensity of the expressed emotion cognition; providing the multiple preprocessed natural language data items in parallel to an unsupervised learning engine and a supervised learning engine; training the unsupervised learning engine and the supervised learning engine in parallel over multiple training epochs to identify specific emotion cognitions in natural language data and to determine the intensity of specific emotion cognitions, wherein each training epoch of the unsupervised learning engine provides rule suggestions to subsequent training epochs of the rule-based engine, and each training epoch, the rule-based engine provides an output representing at least one of the trained unsupervised learning engine and the trained supervised learning engine, which provides aggregated and scoring data to subsequent epochs of the unsupervised learning engine and the supervised learning engine.

[0394]

[0438] (11) Any method of feature (1) to (10), further comprising generating new language rules by a processing circuit that performs human dimension matching by matching human dimensions present in natural language content to an existing dimension array having wildcards or pattern skips, in order to identify new rules for a rule-based engine.

[0395]

[0439] (12) Any method of feature (1) to (11), wherein the receiving step further comprises sequential reading of a streaming live video or animated video source along with coordinated text transcription, and the method further comprises determining contextual cues based on word co-occurrence, argumentative elements and topical elements, marking individual strings or n-grams with ternary dimensional scores, detecting further information evident in the visual data or tone elements evident in the auditory data and inputting them into separate, but time-coordinated sources of video, and performing juxtaposition from contextual cues and further information to create a contextual score for each scene in the video.

[0396]

[0440] (13) The method of feature (12), the display step of displaying a text transcription in such a manner that a given emotion cognition and its respective intensity indication are enclosed in parentheses and inserted inline adjacent to the constituent elements.

[0397]

[0441] (14) A method of any of features (1) to (13), further comprising generating new language rules by a rule discovery engine, the method further comprising: detecting hook words or pairs of words in natural language content by a processing circuit; evaluating one or more human dimensions associated with the detected hook words or pairs of words to determine whether the hook words or pairs of words indicate possible sentiment cognition; extracting a predetermined window of words surrounding the hook words or pairs of words when possible sentiment cognition exists; scoring one or more human dimensions by a processing circuit to obtain a dimension score profile of the hook words or pairs of words; and constructing a new rule for possible sentiment cognition based on the hook words or pairs of words and the extracted surrounding words when the dimension score profile exceeds the majority.

[0398]

[0442] (15) Any method of feature (1) to (14), further comprising identifying the index position in text input at the position where the language rule is matched.

[0399]

[0443] (16) The method of feature (15), further comprising annotating text input at index locations with emotion recognition and respective intensity indications.

[0400]

[0444] (17) Any method of feature (1) to (16), wherein the receiving step further comprises receiving natural language content as audio input via an input device and transcribing the audio input into text input, and the method further comprises matching fragments of the audio input to stored rules for similar sound fragments and assigning the audio fragments sentiment cognition labels of the stored rules.

[0401]

[0445] (18) An e-reader comprising a touchscreen display, a processing circuit, and a memory, wherein the touchscreen display is configured to display the text of an e-book, the processing circuit is configured to scan and tag the text using rules that, when triggered, detect an emotional recognition state and determine the intensity in which the emotional recognition state has occurred, the processing circuit is configured to generate and display one or more sidebars for listing dynamic and emotional recognition intensity information based on detected components of the displayed text, the touchscreen is configured to select a dynamic or emotional recognition intensity when touched at a position in the display, and the processing circuit is further configured to generate and display a color-coded highlight indicating the occurrence of the selected dynamic or emotional recognition intensity.

[0402]

[0446] (19) A system for alleviating psychological disorders, comprising a mobile device having a processing circuit and memory, and a peripheral device having a communication device and one or more actuators, wherein the memory of the mobile device stores program instructions, and when the program instructions are executed by the processing circuit of the mobile device, the mobile device performs a method comprising: receiving natural language content as text input via an input device; the processing circuit searching for a match between a plurality of language rules for a given emotion cognition and components of the text input; the processing circuit detecting a matched language rule in order to obtain an intensity indication of a given emotion cognition having at least one human dimension; and transmitting a first activation signal identifying the emotion cognition intensity when the intensity indication of a given emotion cognition reaches an emotion cognition intensity exceeding a first threshold, and the peripheral device being configured to receive the transmitted first activation signal via a communication device and to activate one or more actuators to produce a sensory distraction for alleviating psychological disorders.

[0403]

[0447] (20) A system of feature (19) wherein a program instruction, when executed by a processing circuit of a mobile device, causes the mobile device to further implement a method including continuing to receive natural language content as further text input via an input device, and transmitting a second activation signal that identifies a negative emotion intensity when a given emotion cognition intensity indication reaches a negative emotion intensity that exceeds a second threshold, and a peripheral device is further configured to receive the transmitted second activation signal and activate one or more actuators to produce a variety of randomized sensory distractions to alleviate a personality disorder.

[0404]

[0448] (21) The method of feature (1), further comprising highlighting words in natural language content based on intensity indication, transmitting the natural language content together with the highlighted words to a display, and displaying the natural language content together with the highlighted words as an augmented reality display on the display during a video streaming session.

[0405]

[0449] (22) An electronic reader of feature (18), further comprising detecting the presence or absence of syntax having components relating to emotional cognitive states in text, and displaying emotional and cognitive representations of text when the user touches text displayed on a touchscreen, wherein the representation is a color heatmap.

[0406]

[0450] (23) The electronic reader of feature (18), further comprising annotating text with emotional cognition states and their respective intensities at index positions indicated when the electronic reader is touched while reading.

[0407]

[0451] (24) A system of feature (19) further comprising: in response to the detection of a matched language rule, the mobile device being configured to transmit an electrical signal or short radio wave to trigger color-coded lighting of a peripheral device based on the language rule.

[0408]

[0452] (25) The system of feature (19), further comprising a peripheral device which further comprises a color geometric display configured to activate LEDs according to an indication of the intensity of a given emotion perception.

[0409]

[0453] (26) A system of feature (19) wherein a method implemented by a mobile device further comprises comparing an intensity indication with a given threshold of emotional cognition in order to obtain an emotional intensity level of a cognitive emotional state, and the peripheral device comprises a color light-emitting diode and a vibrator, and is configured to broadcast the cognitive emotional state via the color light-emitting diode and vibrate via the vibrator when the intensity indication exceeds the threshold.

[0410]

[0454] (27) A system of feature (19) wherein a method implemented by a mobile device further comprises determining a pattern of emotion cognitions including a given emotion cognition by linking it with other emotion cognitions detected by other language rules, and identifying the pattern of emotion cognitions as dynamic emotion cognitions, the peripheral device comprising an LED light and a vibrating device that vibrates in coordination with the pulsing of the LED light to shift as a shift of emotion cognitions.

[0411]

[0455] (28) A system of feature (19) provided by a mobile device, further comprising identifying an index position in text input at a position where a language rule is matched during an audio conversation that is received and transcribed when spoken by the wearer of the peripheral device.

Claims

1. A method for automatically augmenting natural language content with emotion recognition using a processing circuit, wherein the method is The natural language content is received as text input via an input device, The processing circuit searches for a match between a plurality of linguistic rules for a given emotion recognition and the components of the text input, wherein an instance of the linguistic rule has at least one human dimension. The processing circuit activates the matched language rule and evaluates at least one human dimension of the activated matched language rule. In order to obtain the archetypal profile of the given emotion cognition dimension score, the processing circuit scores each human dimension, In order to obtain the intensity indication of the given emotion cognition, the processing circuit aggregates the dimensions in the obtained profile of the dimension score, The matching language rules are related to the given emotion cognition, and the augmented natural language content is displayed on the display in a manner that signals the respective intensity indications of the given emotion cognition. A method that includes [something].

2. The aforementioned human dimensions include sentiment, emotion, emotional cognition, attitude, values, social customs, way of thinking, viewpoint, manner, response, characteristics, beliefs, opinions, perspective, motivation, bias, state, manner, approach, dynamics, personality traits, emotional approach, emotional selection, reaction, tendency, temporary state, change of state, cognitive manner, behavioral manner, internal conditions, external conditions, feeling, emotion, assertion, attitude, assertive attitude, oriented attitude, unoriented attitude, spontaneous attitude, conscious emotional cognition, and unconscious. Including one or more of the following emotional affections: cognition, anger, anticipation, disgust, fear, joy, sadness, surprise, trust, ego, blame, suitability, sanctity, kindness, respect, time, favor, approval, sincerity, vulnerability, judgment, isolation, purpose, stiffness, minimization, specificity, force, action, agency, curiosity, clarity, intention, emphasis, energy, certainty, interest, involvement, shock or surprise, tension, speed, nuance, logic, paranoia, distance, discernment, respect, objectification, attachment, empathy, and patience. Here, each dimension has a value of +1 for a positive force, -1 for a negative force, 0 for a neutral force, and one of φ for non-existence or inapplicability. Here, the scoring of each human dimension by the processing circuit includes scoring the human dimension for all matching rules. The method according to claim 1.

3. Searching for the matching means Detecting syntax based on the aforementioned language rules, Evaluating the human dimensions of each detected syntax and The method according to claim 1, further comprising:

4. The scoring is To obtain the emotion intensity level, the intensity indication is compared with the given emotion cognition threshold. The method according to claim 1, comprising:

5. The detection of the syntax is To detect the presence or absence of a syntax having components related to the given emotion cognition in the natural language content. The method according to claim 3, further comprising:

6. The processing circuit determines a pattern of emotion cognition, including the given emotion cognition, by linking it with other emotion cognitions detected by other language rules, and identifies the pattern of emotion cognition as dynamic emotion cognition. Tracking the given emotion cognition and other emotion cognitions along with related components in a time sequence on the natural language content. The method according to claim 1, further comprising:

7. The aforementioned syntax is detected. Part-of-speech tagging or syntax rules, String matching rules that are accurate, inaccurate, masked, or wildcards. Token proximity rule, Punctuation rules, Lemma rules, Stemming rules, Vocabulary rules, and Word lookup or dictionary-based rules The method according to claim 3, further comprising determining one or more numerical values ​​from among them.

8. The method according to claim 7, wherein determining the numerical value of the token proximity rule involves accessing all tokens in the natural language content that are less than n tokens away from a specified point, where n is a positive integer.

9. The method according to claim 1, further comprising generating new language rules by a machine learning engine that performs at least one of supervised learning and unsupervised learning.

10. Receiving multiple natural language data items from a repository, The process involves normalizing and tokenizing the received multiple natural language data items using preprocessing to generate multiple preprocessed natural language data items, The plurality of preprocessed natural language data items are labeled by the expressed emotion cognition and the intensity of the expressed emotion cognition. The aforementioned multiple preprocessed natural language data items are provided in parallel to the unsupervised learning engine and the supervised learning engine, In order to identify specific emotion cognitions in the natural language data items and to determine the intensity of those specific emotion cognitions, the unsupervised learning engine and the supervised learning engine are trained in parallel over multiple training epochs, wherein each training epoch of the unsupervised learning engine provides rule suggestions to subsequent training epochs of the rule-based engine, and each training epoch, the rule-based engine provides aggregated and scored data to subsequent epochs of the unsupervised learning engine and the supervised learning engine. To provide an output representing at least one of the trained unsupervised learning engine and the trained supervised learning engine. The method according to claim 9, further comprising:

11. The method according to claim 1, further comprising generating new language rules by a processing circuit that performs matching of human dimensions by matching human dimensions present in the natural language content to an existing dimension array having wildcards or pattern skips, in order to identify new rules for a rule-based engine.

12. The receiving further comprises continuous reading of a streaming live video or animated video source along with coordinated text transcription, The method described above is Determining contextual clues based on word co-occurrence, argumentative elements, and topical elements, Marking individual strings or n-grams with a three-dimensional score, Detecting additional information evident in visual data or tonal elements evident in auditory data and inputting them into separate, but time-coordinated, sources of video, In order to create a context score for each scene in the aforementioned video, the context cues and further information are juxtaposed. Furthermore, The method according to claim 1.

13. The method according to claim 12, wherein the display includes displaying the text transcription in such a manner that the given emotion recognition and the respective intensity indications are enclosed in parentheses and inserted inline adjacent to the components.

14. Furthermore, it includes the ability to generate new language rules using a rule discovery engine. The aforementioned method, The processing circuit detects hook words or pairs of words in the natural language content, In order to determine whether the hook word or pair of words indicates a possible emotional cognition, one or more human dimensions associated with the detected hook word or pair of words are evaluated. When a possible emotional cognition exists, a predetermined window of words surrounding the hook word or pair of words is extracted, To obtain a profile of the dimension score of the hook word or word pair, the processing circuit scores one or more of the human dimensions. When the aforementioned profile of the dimension score exceeds the majority, a new rule for the possible sentiment cognition is constructed based on the hook word or word pair and the extracted surrounding word. Furthermore, The method according to claim 1.

15. The method according to claim 1, further comprising identifying the index position in the text input at the position where the language rule is matched.

16. The method according to claim 15, further comprising annotating the text input at the index position with emotion recognition and respective intensity indications.

17. The receiving further comprises receiving the natural language content as audio input via the input device and transcribing the audio input into text input, wherein the method Matching the audio input fragments to stored rules for similar sound fragments, and assigning the emotional recognition labels of the stored rules to the audio fragments. The method according to claim 1, further comprising:

18. Highlighting words in the natural language content based on the aforementioned intensity indication, Transmitting the natural language content along with the highlighted words to the display, During a video streaming session, the natural language content is displayed on the display as an augmented reality display along with the highlighted words. The method according to claim 1, further comprising:

Images