System and method for analyzing physical properties

Abstract

The method can identify a feature of the video, select the feature in the video, and convert the feature to feature data at a plurality of time points using a video transformation algorithm. The method can store the feature data, compare the feature data to previous feature data using a comparison algorithm, score the comparison using a scoring algorithm, and determine whether the score exceeds a threshold.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 602,206, filed November 22, 2023, entitled “System and Method for Analyzing Physical Properties,” the entire contents of which are incorporated herein by reference. Background Technology

[0003] Many industries will benefit from analyzing physical attributes to gain clues about the emotions a person is feeling and the thoughts they are having. Past attempts to find emotions or thoughts in images or videos have been somewhat crude, using cameras to look for smiles or frowns and noting the emotions or thoughts often associated with these facial movements. These analytical attempts are not individualized and are oversimplified because they fail to account for other indications of emotions or thoughts. Summary of the Invention

[0004] A method for analyzing physical properties, such as those from video, can be disclosed. This method identifies features in the video, selects features from the video, and uses a video transformation algorithm to convert the features into feature data at multiple time points. The method can store the feature data, use a comparison algorithm to compare the feature data with previous feature data, use a scoring algorithm to score the comparison, and determine whether the score exceeds a threshold. Attached Figure Description

[0005] Figure 1 It could be a diagram of a computer-based learning system;

[0006] Figure 2 It may be a diagram of the method according to the claims;

[0007] Figure 3 It can be a diagram of a convolutional neural network; and

[0008] Figure 4 It can be a diagram of a computer that can be physically transformed to perform the method.

[0009] Those skilled in the art will understand that the elements in the accompanying drawings are shown for simplicity and clarity, and therefore not all connections and options are shown to avoid obscuring aspects of the invention. For example, common but easily understood elements that are useful or necessary in commercially viable embodiments are generally not depicted to facilitate a clearer view of the various embodiments of this disclosure. It will be further understood that certain actions and / or steps may be described or depicted in a particular order of occurrence, and those skilled in the art will understand that such specificity regarding the order is not actually necessary. It will also be understood that the terms and expressions used herein will be defined relative to their respective fields of inquiry and research, unless otherwise set forth herein. All dimensions specified in this disclosure are intended to be illustrative only and not restrictive. Furthermore, the scales shown in these figures may not necessarily be drawn to scale. As will be understood, the actual dimensions and scales of any system, device, or part of a system or device disclosed herein may be determined by its intended use. Detailed Implementation

[0010] Many industries will benefit from analyzing human attributes to gain clues about a person's emotions or thoughts. Past attempts to find emotions or thoughts in images, videos, or other sensors have been somewhat crude; cameras would look for smiles or frowns and note the emotions often associated with these facial movements. These analytical attempts were not individualized and were oversimplified because they did not consider other indicators of emotion.

[0011] A method for analyzing physical sensor information into data can be disclosed. This method can identify features from sensor data, select features within the data, and transform the features at multiple time points. The method can store the feature data, use a comparison algorithm to compare the feature data with previous feature data, use a scoring algorithm to score the comparison, and determine whether the score exceeds a threshold. In the examples used throughout this specification, the sensor can be an image sensor, and features can be identified in the output video. The method can use a video transformation algorithm to transform the features into feature data at multiple time points. The method can store the feature data, use a comparison algorithm to compare the feature data with previous feature data, use a scoring algorithm to score the comparison, and determine whether the score exceeds a threshold.

[0012] Methods and apparatuses that can implement various features of the invention will now be described with reference to the accompanying drawings. The drawings and related descriptions are provided to illustrate embodiments of the invention, but not to limit the scope of the invention. References to “one embodiment” or “embodiment” in the specification may be intended to indicate that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of the invention. The phrases “in one embodiment” or “embodiment” appearing throughout the specification may not necessarily refer to the same embodiment.

[0013] Throughout the accompanying drawings, reference numerals may be used repeatedly to indicate the correspondence between referenced elements. As used in this disclosure, unless the context otherwise requires, the term "comprising" and variations thereof, such as "including," "comprises," and "included in," may not be intended to exclude other additives, components, integers, or steps.

[0014] In the following description, specific details are set forth to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments can be practiced without these specific details. Well-known circuits, structures, and techniques may not be shown in detail to avoid obscuring the embodiments. For example, circuits may be shown as block diagrams to avoid obscuring the embodiments with unnecessary detail.

[0015] It should also be noted that these embodiments may be described as processes depicted as flowcharts, flow diagrams, structural diagrams, or block diagrams. The flowcharts and block diagrams in the accompanying drawings can illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer programs according to the various embodiments disclosed. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which may include one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions mentioned in the blocks may occur in a different order than shown in the figures.

[0016] Although flowcharts can describe operations as a sequential process, many operations can be executed in parallel or simultaneously. Furthermore, the order of operations can be rearranged. A process can terminate when its operations are completed. A process can correspond to a method, function, procedure, subroutine, subroutine, etc. When a process corresponds to a function, its termination can correspond to the function returning to the calling function or the main function. Additionally, each box in a block diagram and / or flowchart illustration, and combinations of boxes in block diagrams and / or flowchart illustrations, can be implemented by a system based on dedicated hardware or a combination of dedicated hardware and computer instructions that performs the specified function or action.

[0017] Furthermore, a storage device can refer to one or more devices for storing data, including read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, and / or other non-transitory machine-readable media for storing information. The term "machine-readable media" can include, but is not limited to, portable or fixed storage devices, optical storage devices, wireless channels, and various other non-transitory media capable of storing, including, containing, executing, or carrying (one or more) instructions and / or data.

[0018] Furthermore, embodiments can be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. When implemented as software, firmware, middleware, or microcode, program code or code segments used to perform the necessary tasks can be stored in a machine-readable medium such as a storage medium or other memory. One or more processors can execute the necessary tasks serially, distributedly, concurrently, or in parallel. Code segments can represent procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or combinations of instructions, data structures, or program statements. Code segments can be coupled to another code segment or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., can be passed, forwarded, or transmitted through suitable means, including memory sharing, messaging, token passing, network transmission, etc., and are also referred to as interfaces, where an interface is a point of interaction with software or computer hardware or peripheral devices.

[0019] Figure 2 A method for analyzing the physical properties of thought data that can be analyzed can be made public. For example, this method can analyze movement in a video to obtain thought data to be analyzed, such as emotional cues and patterns. At a very high level, bodily reactions, whether intentional or unintentional, can provide insights into what a person or animal is thinking, planning, or feeling. Bodily reactions can take many forms, such as increased body temperature, the smell of sweat, sounds of pain or distress, or changes in taste. In a very simple example, if a human is confused, they might scratch their head. By studying physical movement using machine learning or artificial intelligence, behavioral patterns may emerge. If a human is exhibiting a behavioral pattern, insights into their thoughts can be determined. Similarly, if a human deviates from a behavioral pattern, additional insights into the human's thoughts can be determined. Finally, the analysis can be individual-specific, as each individual may have different habits, which can provide insights into the individual's thoughts.

[0020] Past technologies have faced numerous challenges. Identifying which features are crucial for indicating whether a thought process has been hit or missed is critical. Similarly, feature-based pattern recognition has always been challenging and prone to false alarms because systems don't improve over time or take feedback into account. The ability to utilize features has been enhanced by employing specially designed artificial intelligence systems and identifying them. Furthermore, artificial intelligence has significantly improved the ability of systems and methods to analyze large amounts of data to determine patterns.

[0021] The system and method have many applications. As an example, and not a limitation, analyzing human characteristics can greatly improve the determination of whether a person understands a concept in an educational session. Identification and verification can be performed by studying a wide range of characteristics. Criminal intent can be determined by studying characteristics. The ability to sell goods or services can be improved by studying characteristics. There are many embodiments and they are varied, and should not be limited to these embodiments.

[0022] In some embodiments, sensor data, such as video, can be converted into multiple images for analysis. In some embodiments, the video may have a frame rate, and the frame rate can be used to determine the number of images to be used per second. Obviously, more or fewer frames per second may also be used. In some embodiments, the images can be normalized so that comparisons across videos can be consistent. For example, the size, brightness, or contrast of the images constituting the video can be adjusted to be consistent between the images used for comparison.

[0023] Physical sensors can take many forms. They can be image sensors, voice sensors, odor sensors, touch sensors, taste sensors, magnetic sensors, or temperature sensors. Output data can be analyzed to obtain characteristics associated with the physical sensor. Furthermore, there can be multiple physical sensors. Data from multiple physical sensors can be analyzed together to create a more detailed set of data for analysis, in an attempt to determine the thoughts of a person or animal. Throughout this manual, for simplicity, video sensors and video data will be used, and this is not intended to limit the scope in any way.

[0024] At box 200, features from the sensor output (such as video) can be identified. Some sample features may include edges, colors, shapes, or keypoints. Multiple features may exist that can be analyzed individually or together. For example, a face may have 50 or 100 features that can be analyzed. Logically, combinations of features can be used. As an example, the edges of a face and points on the nose can be used together as features to be analyzed. Features can also be points related to parts of the human body.

[0025] Features can be extracted in various ways. In one embodiment, computer vision techniques are used to extract features, and multiple computer vision techniques are possible. In other embodiments, a pre-trained machine learning model is used to extract features. For example, the pre-trained machine learning model could be a convolutional neural network (CNN). A CNN can be trained on images of millions of people and may have learned to understand ideas from photographs. The CNN may be novel because it has only been created and trained on known images. Logically, other types of learning algorithms can be used in the estimator. For example, in one embodiment, the learning algorithm could be a fully connected neural network (FCN).

[0026] Converting an image into data may require measuring different points on the object. These points can be compared to a measurement baseline of the object, and changes can be recorded. The system can then analyze these changes to determine the extent of movement.

[0027] More specifically, see reference Figure 3 The learning algorithm may include a convolutional neural network 510 (CNN) and a transformer 320. In one embodiment, the CNN 310 may determine one or more features 351 to 354 in each image 341 to 344. In one example, the CNN may determine features 351-354, which may be a set of numbers, but the magnitude of features 351-354 may vary upward or downward depending on many factors.

[0028] A CNN can be trained on images of millions of people and may have already learned to understand human thoughts from photographs. This CNN may be novel because it has already been created and trained on known images. Logically, other types of learning algorithms can be used. For example, in one embodiment, the learning algorithm could be a fully connected neural network (FCN). The analysis of features can indicate changes in physical appearance.

[0029] During training, transformer 320 can use features 351-354 (CNN output) from multiple images 341-344 of the same person, along with additional data (e.g., the thought processes of the individual in photo 360), to create a model. Once the model is trained, the transformer can generate predictions of the thought processes of individual 370. In some embodiments, the estimation of thought processes 370 can be real-time. The transformer 320 used in this invention can be trained on a dataset specifically created for predicting thoughts 370.

[0030] The trained model in transformer 320 can use features from multiple images 341-344 of the same object, as well as external information, to predict the object's thought processes. The learning algorithm can also analyze other relevant information about the object.

[0031] At box 205, you can select features from the video you want to analyze. Different features can be observed for different purposes. For example, trying to determine if a person is preparing to commit a crime might involve looking at one set of features, while determining if a person is watching a comedy might involve looking at another set of features.

[0032] Machine learning can be used to help determine one or more features that should be studied for different scenarios. As mentioned earlier, films of people preparing for crimes in the past can be used to identify features that might need to be observed. Similarly, past films of people who enjoy comedy films can be studied to identify features that might need to be observed. As with any machine learning algorithm, additional data can be fed into the model over time to improve it.

[0033] In yet another embodiment, the features studied can be individual-specific. Over time, various features can be captured and analyzed to determine what the features represent. For example, students can be monitored and features can be stored to determine the amount of understanding a student has. For some students, scratching their heads might be a sign of confusion, but for others it might be a sign of complete mastery. Personalized analysis makes better and more personalized results possible.

[0034] At box 210, a transformation algorithm, such as a video transformation algorithm, can be used to convert the features into feature data at multiple time points. Various video transformation algorithms can be used. In one embodiment, the video transformation algorithm is a Fast Fourier Transform (FFT) algorithm. Logically, other video transformation algorithms are possible and anticipated. Video transformation algorithms can make the analysis of feature data easier because the feature data can be transformed into numbers that are easier to compare than photos or videos.

[0035] Transformation algorithms can be used to transform features into a representative format. This format can be consistent in analysis and time, allowing comparisons or videos to be based on the same format. In some embodiments, the transformation algorithm creates a data structure to store data representing movement in the video. For example, the data structure may include feature vectors or multiple feature vectors.

[0036] At box 215, feature data can be stored in memory. The memory can be selected based on the type and amount of data. For example, large amounts of data may require a different type of memory than small amounts. Furthermore, depending on the data volume, speed requirements, etc., the memory can be local, remote, or in the cloud.

[0037] At box 220, a comparison algorithm can be used to compare the feature data with previous feature data. Various comparison algorithms are possible and anticipated. At a high level, the comparison algorithm can compare the video being analyzed with videos in a video database. More specifically, the images constituting the various videos can be compared with images in an image database. Even more specifically, data representing images can be compared with data representing previously analyzed images. In one embodiment, the comparison algorithm may include a similarity measure selected from a group of Euclidean distance, cosine similarity, or dynamic time warping of a time pattern. Of course, other similarity measures are possible and anticipated.

[0038] At box 225, a scoring algorithm can be used to score the comparison. The scoring algorithm can be an indicator of the similarity between images or videos (or features within images or videos) at a given point in time. For example, if a video image is transformed using a Fast Fourier Transform (FFT), a similarity measure can compare the FFT results, which is easier than comparing the images themselves. At a higher level, the FFT converts the signal into individual spectral components, thus providing frequency information about the signal. As an example, a similarity measure can determine the Euclidean distance between the FFT result of the image in question and a past scored image. In another embodiment, a feature can be in the form of a grayscale or color intensity value representing a pixel that may be a feature. Grayscale or color intensity values ​​can be readily compared mathematically.

[0039] In a more detailed approach, the pixel intensity values ​​of points on the face at each time point can be analyzed as a time series, and an FFT can be performed on two time series to convert them to the frequency domain. This yields amplitude and phase information of the frequencies present in the intensity variations of the points on the face. A comparison metric can be defined to evaluate the difference or similarity between the FFT results at two time points, where the comparison metric can be based on the amplitude and phase of the dominant frequency components. Logically, a scoring mechanism can be used to determine when a point changes significantly or remains relatively stable between two time points.

[0040] A pattern database can include a large number of videos used to create pattern or feature representations from these videos. The database can be organized for efficient retrieval and comparison. The data and database can be in various formats, depending on the data volume, required retrieval speed, and data type.

[0041] Machine Learning

[0042] Machine learning can be used to identify patterns. A machine learning model can be trained on an existing dataset and used to predict whether movement in a new video matches a known pattern. Machine learning models can also be used to predict future actions based on past pattern recognition. Furthermore, machine learning models can be used to determine pattern bias. Logically, pattern bias can be used to determine future actions.

[0043] Frameworks for machine learning algorithms (such as large language models) may involve a combination of one or more components, sometimes involving three: (1) representation, (2) evaluation, and (3) optimization components. A representation component refers to a computational unit that performs steps to represent knowledge in different ways, including but not limited to one or more decision trees, rule sets, instances, graphical models, neural networks, support vector machines, model ensembles, and / or others. An evaluation component refers to a computational unit that performs steps to represent evaluation hypotheses (e.g., candidate programs) in a manner that includes, but is not limited to, accuracy, prediction and recall, squared error, probability, posterior probability, cost, margin, entropy K-1 divergence, and / or others. An optimization component refers to a computational unit that performs steps to generate candidate programs in different ways, including but not limited to combinatorial optimization, convex optimization, constrained optimization, and / or others. In some embodiments, other components and / or sub-components of the aforementioned components may exist in the system to further enhance and complement the aforementioned machine learning capabilities.

[0044] Machine learning algorithms sometimes rely on unique computational system architectures. One such architecture is the neural network, a system that approximates a biological neural network (e.g., the human brain). While far more complex than traditional computer systems, this architecture is advantageous for enabling machine learning. For example, an artificial neural network can comprise a large set of nodes that, like neurons in the brain, can be dynamically configured to perform learning and decision-making.

[0045] Machine learning tasks are sometimes broadly categorized into unsupervised learning and supervised learning. In unsupervised learning, the machine learning algorithm is left to generate any output (e.g., labeling as needed) without feedback. The machine learning algorithm can learn on its own (e.g., observe past outputs) but operates without (or with minimal) feedback from, for example, a human administrator. Meanwhile, in supervised learning, the machine learning algorithm is provided with feedback on its output. Feedback can be provided in various ways, including via active learning, semi-supervised learning, and / or reinforcement learning. In active learning, the machine learning algorithm is allowed to query answers from an administrator. For example, a machine learning algorithm might make a guess in a face detection algorithm, asking an administrator to identify photos in an image and comparing the guess with the administrator's response. In semi-supervised learning, the machine learning algorithm is provided with a set of example labels along with unlabeled data. For example, a dataset of 100 photos with labeled faces and 10,000 random unlabeled photos might be provided to the machine learning algorithm. In reinforcement learning, the machine learning algorithm is rewarded for correct labels, allowing it to iteratively observe conditions until a reward is consistently obtained. For example, for each correctly identified face, the machine learning algorithm might be given points and / or scores (e.g., "75% correct"). This article describes examples involving supervised machine learning.

[0046] As detailed in this paper, in practice, machine learning systems and their underlying components are tuned by data scientists to perform numerous steps to refine the system. This process is sometimes iterative and may require iterating through a series of steps: (1) understanding the domain, prior knowledge, and objectives; (2) data integration, selection, cleaning, and preprocessing; (3) learning the model; (4) interpreting the results; and / or (5) merging and deploying the discovered knowledge. Given the virtually infinite number of variables that may be optimized in a machine learning system, this can further include consulting with domain experts to refine and make the objectives more specific. Meanwhile, one or more of the data integration, selection, cleaning, and / or preprocessing steps can sometimes be the most time-consuming, as the old adage "garbage in, garbage out" also applies to machine learning systems.

[0047] As an example, Figure 1 A simplified example of an artificial neural network 100 on which machine learning algorithms can be executed is explained. Figure 1 This is merely an example of using nonlinear processing with artificial neural networks; based on the features described in this article, other forms of nonlinear processing can be used to implement machine learning algorithms.

[0048] exist Figure 1In the diagram, each of the input nodes 110 an is connected to the first group of processing nodes 120 an. Each of the first group of processing nodes 120 an is connected to each of the second group of processing nodes 130 an. Each of the second group of processing nodes 130 an is connected to each of the output nodes 140 an. Although only two groups of processing nodes are shown, any number of processing nodes can be implemented. Similarly, although in Figure 1 The diagram only shows four input nodes, five processing nodes, and two output nodes per group, but any number of nodes can be implemented per group. Figure 1 The data flow is depicted from left to right: data can be input into input nodes, flow through one or more processing nodes, and be output by output nodes. Input to input node 110 an can originate from external source 160. Output can be sent to feedback system 150 and / or memory 170. Feedback system 150 can send output to input node 110 an for continuous processing iterations with the same or different input data.

[0049] In one illustrative approach using a feedback system 150, the system can use machine learning to determine the output. The output may include anomaly scores, heat scores / values, confidence values, and / or classification outputs. The system can use any machine learning model, including XG-boosted decision trees, autoencoders, perceptrons, decision trees, support vector machines, regression, and / or neural networks. The neural network can be any type of neural network, including feedforward networks, radial basis function networks, recurrent neural networks, long / short-term memory, gated recurrent units, autoencoders, variational autoencoders, convolutional networks, residual networks, Kohonen networks, and / or other types. In one example, the output data in the machine learning system can be represented as a multidimensional array, a two-dimensional table (such as a matrix), or an extension of data with higher dimensions.

[0050] A neural network may include an input layer, multiple intermediate layers, and an output layer. Each layer may have its own weights. The input layer may be configured to receive one or more feature vectors as described herein as input. Intermediate layers may be convolutional layers, pooling layers, dense (fully connected) layers, and / or other types. The input layer passes input to the intermediate layers. In one example, each intermediate layer may process the output from the previous layer and then pass the output to the next intermediate layer. The output layer may be configured to output a classification or a real value. In one example, layers in a neural network may use activation functions such as the sigmoid function, the Tannh function, the ReLU function, and / or other functions. Furthermore, a neural network may include a loss function. In some examples, the loss function may measure the number of missed positives; alternatively, it may also measure the number of false positives. The loss function can be used to determine error when comparing output values ​​with target values. For example, when training a neural network, the output of the output layer may be used as a prediction and compared with the target value of the training instance to determine the error. This error can be used to update the weights in each layer of the neural network.

[0051] In one example, a neural network may include techniques for updating weights in one or more layers based on error. The neural network may use gradient descent to update the weights. Alternatively, the neural network may use an optimizer to update the weights in each layer. For example, the optimizer may use various techniques or combinations of techniques to update the weights in each layer. Where appropriate, the neural network may include mechanisms to prevent overfitting—regularization (such as L1 or L2), exit mechanisms, and / or other techniques. The neural network may also increase the amount of training data used to prevent overfitting.

[0052] Once the data for machine learning is created, an optimization process can be used to transform the machine learning model. The optimization process may include (1) training the data to predict outcomes, (2) defining a loss function that serves as a precise measure of the machine learning model's performance, (3) minimizing the loss function, for example, through gradient descent or other algorithms, and / or (4) optimizing the sampling method, for example, using stochastic gradient descent (SGD), where instead of feeding the entire dataset to the machine learning algorithm to compute each step, subsets of the data are sampled sequentially. In one example, optimization involves minimizing the number of false positives to maximize the user experience. Alternatively, the optimization function could minimize the number of missed positives to optimize the minimization of the loss from exploitation.

[0053] In one example Figure 1Nodes are depicted that can perform various types of processing, such as discrete computations implemented by computing devices, computer programs, and / or mathematical functions. For example, input node 110 an may include logical inputs from different data sources, such as one or more data servers. Processing node 120 an may include parallel processes executed on multiple servers in a data center. Furthermore, output node 140 an may be the logical output ultimately stored in a result data repository, such as on the same or different data servers as input node 110 an. It is noteworthy that nodes do not need to be different. For example, two nodes in any two sets can perform the exact same processing. The same nodes can be repeated for the same or different sets.

[0054] Each node can be connected to one or more other nodes. Connections can link the output of one node to the input of another node. Connections can be associated with weights. For example, one connection can be weighted to be more important or significant than another connection, thus influencing the degree of further processing as the input traverses the artificial neural network. Such connections can be modified so that the artificial neural network 100 can be learned and / or dynamically reconfigured. Although in Figure 1 A node is described as having only connections to consecutive nodes, but connections can be formed between any nodes. For example, a processing node can be configured to send output to a previous processing node.

[0055] The input received in input node 110 an can be processed by processing nodes (such as the first set of processing nodes 120a-n and the second set of processing nodes 130 an). This processing can result in the output in output node 140 an. As depicted by the connection from the first set of processing nodes 120 an and the second set of processing nodes 130 an, the processing can include multiple steps or sequences. For example, the first set of processing nodes 120 an can be a coarse data filter, while the second set of processing nodes 130 an can be a more detailed data filter.

[0056] Artificial neural network 100 can be configured to make decisions. As a simplified example for interpretive purposes, artificial neural network 100 can be configured to detect faces in a photograph. A digital copy of the photograph can be provided to input node 110 an. A first set of processing nodes 120 an can each be configured to perform specific steps to remove non-facial content, such as large, continuous red areas. A second set of processing nodes 130 an can each be configured to find a coarse approximation of the face, such as facial shape and skin tone. Multiple subsequent sets can further refine this processing, each set seeking further, more specific tasks, where each node performs some form of processing that does not necessarily need to operate in the advancement of that task. Artificial neural network 100 can then predict the location on the face. The prediction may be correct or incorrect.

[0057] Feedback system 150 can be configured to determine whether artificial neural network 100 has made a correct decision. Feedback may include indications of correct answers and / or incorrect answers and / or the degree of correctness (e.g., percentage). For example, in the face recognition example provided above, feedback system 150 may be configured to determine whether a face was correctly recognized, and if so, what percentage of the face was correctly recognized. Feedback system 150 may already know the correct answer, allowing it to train artificial neural network 100 by indicating whether it made the correct decision. Feedback system 150 may include human input, such as an administrator telling artificial neural network 100 whether it made the correct decision. Feedback system 150 may provide feedback to artificial neural network 100 via input node 110 (e.g., an indication of whether a previous output was correct or incorrect), or it may send such information to one or more nodes. Feedback system 150 may be additionally or alternatively coupled to storage device 170 such that the output is stored. Feedback system may not have a correct answer at all, but rather base the feedback on further processing: for example, feedback system may include a system programmed to recognize faces such that feedback allows artificial neural network 100 to compare its results with those of a manually programmed system.

[0058] Artificial neural network 100 can be dynamically modified to learn and provide better input. Based on, for example, previous inputs and outputs and feedback from feedback system 150, artificial neural network 100 can modify itself. For example, processing in nodes can be changed and / or connections can be weighted differently. Continuing with the previously provided example, face prediction may be incorrect because the photograph provided to the algorithm is colored in a way that makes all faces appear red. Therefore, nodes that exclude large, continuous portions of the photograph containing red can be considered unreliable, and connections to those nodes can be weighted significantly less. Additionally or alternatively, nodes can be reconfigured to process the photograph differently. These modifications can be predictions and / or guesses of artificial neural network 100, allowing artificial neural network 100 to change its nodes and connections to test hypotheses.

[0059] Artificial neural network 100 does not need to have a predetermined number of processing nodes or a predetermined set of processing nodes, but its complexity can be increased or decreased. For example, artificial neural network 100 can determine that one or more processing nodes are unnecessary or should be repurposed, and discard or reconfigure these processing nodes accordingly. As another example, artificial neural network 100 can determine that all or part of the input needs further processing, and add additional processing nodes and / or sets of processing nodes accordingly.

[0060] The feedback provided by the feedback system 150 can be simply reinforcing (e.g., providing an indication that the output is correct or incorrect, rewarding the machine learning algorithm with multiple points, etc.) or it can be specific (e.g., providing the correct output). For example, the machine learning algorithm 100 can be asked to detect faces in a photograph. Based on the output, the feedback system 150 can indicate a score (e.g., 75% accuracy, an indication that the guess was correct, etc.) or a specific response (e.g., specifically identifying where the face is located).

[0061] The artificial neural network 100 can be supported or replaced by other forms of machine learning. For example, one or more nodes of the artificial neural network 100 can implement decision trees, association rule sets, logic programming, regression models, clustering analysis mechanisms, Bayesian networks, propositional formulas, generative models, and / or other algorithms or decision-making forms. The artificial neural network 100 can implement deep learning.

[0062] Large language models are characterized by their large size. Their size is achieved by AI accelerators, which are capable of processing vast amounts of text data scraped from the internet. The artificial neural networks constructed can contain tens or even billions of weights and are (pre)trained using self-supervised and semi-supervised learning. Changing the architecture helps accelerate the training process.

[0063] As language models, they work by taking input text and repeatedly predicting the next label or word. Until 2020, fine-tuning was the only way a model could be adapted to perform a specific task. However, larger models, such as GPT-3, can be rapidly designed to achieve similar results. They are thought to acquire specific knowledge about the inherent syntax, semantics, and "ontology" within human language corpora. Large language models are trained using self-supervised or semi-supervised learning. This means they are trained on large amounts of unlabeled text. Large language models can tune their internal parameters and learn from new input from users over time.

[0064] Training a large language model to predict the next word in a sentence based on previous input sentences. This is a self-supervised learning task because you don't define separate output labels. Repeat this process until the model reaches an acceptable level of accuracy. Some large language models (such as InstructGPT and ChatGPT) use both supervised learning and reinforcement learning. A combination of both is crucial for optimal performance.

[0065] Refer again Figure 2 At box 230, this method determines whether the score exceeds a threshold. The similarity score can be compared to the threshold. The threshold can be initially set and adjusted over time to reflect the model's accuracy.

[0066] At box 235, in response to a score exceeding a threshold, physical movement can be categorized, and the categorization can be related to what the subject is thinking about. For example, head scratching can be identified as confusion.

[0067] Validation and Refinement

[0068] Additional validation steps can be used to reduce false positives in comparisons. More specifically, comparative analysis can be refined based on feedback and new information to improve the accuracy of pattern recognition algorithms. For example, for many people, head scratching indicates confusion. In a smaller proportion of people, head scratching might indicate confidence in the answer. Individual-specific feedback can be used to improve individual-specific systems and methods. Similarly, some movements may be influenced by media or current events, and the meaning of a movement or other physical manifestations of an idea may change over time.

[0069] This method and system utilize computing devices. For example... Figure 4 As shown, the computing device 401 performing the method may include a processor 402 coupled to an interconnect bus. The processor 402 may include a register set or register space 404, which... Figure 4The processor is depicted as being entirely on-chip, but it may alternatively be wholly or partially off-chip and directly coupled to processor 402 via dedicated electrical connections and / or via an interconnect bus. Processor 402 may be any suitable processor, processing unit, or microprocessor. Although not described in detail... Figure 4 As shown, however, computing device 401 may be a multiprocessor device and may therefore include one or more additional processors that are the same as or similar to processor 402 and are communicatively coupled to an interconnect bus.

[0070] Figure 4 The processor 402 may be coupled to a chipset 406, which includes a memory controller 408 and a peripheral input / output (I / O) controller 410. Chipsets are known to typically provide I / O and memory management functions, as well as multiple general-purpose and / or special-purpose registers, timers, etc., that can be accessed or used by one or more processors coupled to the chipset 406. The memory controller 408 may perform functions that enable the processor 402 (or multiple processors, if there are multiple processors) to access system memory 412 and mass storage 414, which may include any one or both of an in-memory cache (e.g., a cache within memory 412) or an on-disk cache (e.g., a cache within mass storage 414).

[0071] System memory 412 may include any desired type of volatile and / or non-volatile memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, read-only memory (ROM), etc. Mass storage memory 414 may include any desired type of mass storage device. For example, computing device 401 may be used to implement module 416 (e.g., various modules as described herein). Mass storage memory 414 may include hard disk drives, optical disk drives, magnetic tape storage devices, solid-state memories (e.g., flash memory, RAM memory, etc.), magnetic memories (e.g., hard disk drives), or any other memory suitable for mass storage. As used herein, the terms module, block, function, operation, process, routine, step, and method refer to tangible computer program logic or tangible computer-executable instructions that provide the specified function to the computing device 401, system, and method described herein. Therefore, modules, blocks, functions, operations, processes, routines, steps, and methods may be implemented in hardware, firmware, and / or software.

[0072] In one embodiment, program modules and routines may be stored in mass storage 414, loaded into system memory 412, and executed by processor 402, or may be provided from a computer program product stored in a tangible computer-readable storage medium (e.g., RAM, hard disk, optical / magnetic media, etc.).

[0073] The peripheral I / O controller 410 can perform functions that enable the processor 402 to communicate with peripheral input / output (I / O) devices 424, network interface 426, and local network transceiver 428 (via network interface 426) via the peripheral I / O bus. I / O device 424 can be any desired type of I / O device, such as a keyboard, display (e.g., liquid crystal display (LCD), cathode ray tube (CRT) display, etc.), navigation device (e.g., mouse, trackball, capacitive touchpad, joystick, etc.), etc. I / O device 424 can be used with modules such as 416 to receive data from transceiver 428, transmit data to components of system 100, and perform any operations related to the methods described herein. Local network transceiver 428 may include support for Wi-Fi networks, Bluetooth, infrared, cellular, or other wireless data transmission protocols. In other embodiments, an element may simultaneously support each of the various wireless protocols employed by computing device 401. For example, software-defined radio may be able to support multiple protocols via downloadable instructions. In operation, computing device 401 may be able to periodically poll visible wireless network transmitters (both cellular and local networks). This polling is possible even when normal wireless traffic is supported on computing device 401. Network interface 426 may be, for example, an Ethernet device, an asynchronous transfer mode (ATM) device, an 802.11 wireless interface device, a DSL modem, a cable modem, a cellular modem, etc., which enables system 100 to communicate with another computer system having at least the elements described with respect to system 100.

[0074] Although the memory controller 408 and the I / O controller 410 are in Figure 4The modules 416 are depicted as individual functional blocks within chipset 406, but the functions performed by these blocks can be integrated within a single integrated circuit, or implemented using two or more separate integrated circuits. Computing environment 400 may also implement module 416 on remote computing device 430. Remote computing device 430 can communicate with computing device 401 via Ethernet link 432. In some embodiments, module 416 may be retrieved by computing device 401 from cloud computing server 434 via Internet 436. When using cloud computing server 434, retrieval module 416 may be programmatically linked to computing device 401. Module 416 may be a collection of various software playgrounds including artificial intelligence software and document creation software, or it may be Java® software. In some embodiments, computing device 401 may include a virtual machine (JVM) environment residing in computing device 401 or remote computing device 430. Module 416 may also be a "plugin" suitable for execution in a web browser located on computing devices 401 and 430. In some embodiments, module 416 may communicate with backend component 438 via Internet 436.

[0075] System 400 can be any combination of, but not limited to, LAN, MAN, WAN, mobile, wired or wireless networks, private networks or virtual private networks. Furthermore, although only one remote computing device 430 is shown in Figure 6 for simplification and illustrative purposes, it should be understood that any number of client computers can be supported and can communicate within system 400.

[0076] Additionally, some embodiments herein may be described as including logic or multiple components, modules, blocks, or mechanisms. Modules and method blocks may constitute software modules (e.g., code or instructions embodied on a machine-readable medium or in transmitted signals, wherein the code is executed by a processor) or hardware modules. Hardware modules may be tangible units capable of performing certain operations and may be configured or arranged in a certain manner. In example embodiments, one or more computer systems (e.g., standalone client or server computer systems) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or an application portion) to operate as hardware modules to perform certain operations as described herein.

[0077] In various embodiments, the hardware module may be implemented mechanically or electronically. For example, the hardware module may include dedicated circuitry or logic permanently configured to perform certain operations (e.g., as a dedicated processor, such as a field-programmable gate array (FPGA) or application-specific integrated circuit (ASIC)). The hardware module may also include programmable logic or circuitry temporarily configured by software to perform certain operations (e.g., contained within a processor or other programmable processor). It should be understood that the decision to implement the hardware module mechanically in dedicated and permanently configured circuitry or in temporarily configured circuitry (e.g., software-configured) may be driven by cost and time considerations.

[0078] Therefore, the term "hardware module" can be understood to encompass tangible entities, i.e., entities physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate or perform certain operations described herein. As used herein, "hardware-implemented module" can refer to a hardware module. Consider embodiments where hardware modules are temporarily configured (e.g., programmed), and each hardware module does not need to be configured or instantiated at any given time. For example, in cases where hardware modules include processors configured using software, the processor can be configured as corresponding different hardware modules at different times. The software can accordingly configure the processor, for example, to constitute a specific hardware module at one time and different hardware modules at different times.

[0079] Hardware modules can provide and receive information from other hardware modules. Therefore, the described hardware modules can be considered communication-coupled. In the presence of multiple such hardware modules, communication can be achieved through signal transmission connecting the hardware modules (e.g., via appropriate circuitry and buses). In embodiments where multiple hardware modules are configured or instantiated at different times, such communication between hardware modules can be achieved, for example, by storing and retrieving information in a memory structure accessible to the multiple hardware modules. For example, one hardware module can perform an operation and store the output of that operation in a memory device communicationally coupled to it. Another hardware module can then access the memory device at a later time to retrieve and process the stored output. Hardware modules can also initiate communication with input or output devices and can operate on resources (e.g., collections of information).

[0080] The various operations of the example methods described herein can be performed, at least in part, by one or more processors configured, either temporarily (e.g., by software) or permanently, to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute modules of processor implementations that operate to perform one or more operations or functions. In some example embodiments, the modules referred to herein may include processor-implemented modules.

[0081] The methods or routines described herein may be implemented, at least in part, by a processor. For example, at least some operations of the methods may be performed by one or more processors or hardware modules implemented by processors. The execution of certain operations may be distributed across one or more processors, residing not only within a single machine but also deployed across multiple machines. In some example embodiments, one or more processors may reside in a single location (e.g., in a home environment, an office environment, or as a server cluster), while in other embodiments, the processors may be distributed across multiple locations.

[0082] One or more processors may also operate to support the execution of related operations in a “cloud computing” environment or as “Software as a Service” (SaaS). For example, at least some operations may be performed by a group of computers (as an example of a machine that includes processors), and these operations may be accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., application programming interfaces).

[0083] The execution of certain operations can be distributed across one or more processors, residing not only within a single machine but also deployed across multiple machines. In some example embodiments, one or more processors or processor-implemented modules may reside in a single geographic location (e.g., within a home environment, office environment, or server cluster). In other example embodiments, one or more processors or processor-implemented modules may be distributed across multiple geographic locations.

[0084] Some portions of this specification may be presented as algorithms or symbolic representations of operations on data stored as bit or binary digital signals in machine memory (e.g., computer memory). These algorithms or symbolic representations may be examples of techniques used by those skilled in the art of data processing to convey the substance of their work to others skilled in the art. As used herein, an "algorithm" may be a self-consistent sequence of operations or similar processing that leads to a desired result. In this context, algorithms and operations may involve the physical manipulation of physical quantities. Typically, but not necessarily, such quantities may take the form of electrical, magnetic, or optical signals that can be stored, accessed, transmitted, combined, compared, or otherwise manipulated by a machine. Sometimes, primarily for general reasons, it is convenient to use terms such as "data," "content," "bit," "value," "element," "symbol," "character," "term," "number," "numerical value," etc., to refer to such signals. However, these terms may simply be convenient labels and will be associated with appropriate physical quantities.

[0085] Unless otherwise specified, discussions using terms such as “processing,” “computing,” “operation,” “determining,” “presenting,” or “displaying” in this document may refer to the actions or processes of a machine (e.g., a computer) that manipulate or transform data representing physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.

[0086] As used herein, any reference to “multiple embodiments,” “some embodiments,” or “embodiment” or “teaching” may mean that a particular element, feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment. The phrases “in some embodiments” or “teaching” appearing throughout the specification may not necessarily refer to the same embodiment.

[0087] Some embodiments can be described using the expressions “coupled” and “connected”, as well as their derivatives. For example, the term “coupled” can be used to describe some embodiments to indicate that two or more elements are in direct physical or electrical contact. However, the term “coupled” can also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other. Embodiments may not be limited to this context.

[0088] Furthermore, the accompanying drawings depict preferred embodiments for illustrative purposes only. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein can be employed without departing from the principles described herein.

[0089] Upon reading this disclosure, those skilled in the art will understand alternative structural and functional designs for the systems and methods described herein based on the principles disclosed herein. Therefore, while specific embodiments and applications have been shown and described, it should be understood that the disclosed embodiments are not limited to the precise constructions and components disclosed herein. Various modifications, alterations, and variations that are obvious to those skilled in the art may be made to the arrangement, operation, and details of the systems and methods disclosed herein without departing from the spirit and scope defined in any of the appended claims.

Claims

1. A method for analyzing the physical properties of the ideas and patterns to be analyzed, comprising: Identify features from physical sensors; Select features from the physical sensors; The features are transformed into feature data at multiple time points using a transformation algorithm. Store the feature data; The feature data is compared with the previous feature data using a comparison algorithm; The comparisons were scored using a scoring algorithm; Determine whether the score exceeds the threshold; and In response to the score exceeding a threshold, the result is classified as an idea.

2. The method according to claim 1, wherein, The physical sensor includes at least one from the group consisting of: Image sensor; Voice sensor; Odor sensor; Touch sensor; Taste sensor; Magnetic sensor; Temperature sensor.

3. The method according to claim 2, further comprising: If the physical sensor is an image sensor, the scoring agency determines when the point changes significantly or remains relatively stable between two points in time.

4. The method according to claim 2, wherein, If the physical sensor is an image sensor, then the video transformation algorithm is the Fast Fourier Transform (FFT) algorithm.

5. The method according to claim 2, wherein, If the physical sensor is an image sensor, the feature includes at least one of the following: edge, color, shape, or key point.

6. The method according to claim 2, further comprising: If the physical sensor is an image sensor, a large amount of video is analyzed to create a database of patterns or feature representations from these videos.

7. The method according to claim 2, wherein, If the physical sensor is an image sensor, the comparison algorithm compares the video with videos in the video database.

8. The method according to claim 7, wherein, The comparison algorithm includes a similarity metric, which includes at least one of Euclidean distance, cosine similarity, or dynamic time warping of a time pattern.

9. The method according to claim 2, wherein, If the physical sensor is an image sensor, a machine learning model is trained on a model on an existing dataset, and the model is used to predict whether movement in the new video matches a known pattern.

10. The method according to claim 9, wherein, The machine learning model is used to predict future actions based on past pattern recognition.

11. The method according to claim 9, wherein, The machine learning model is used to determine pattern bias, and the pattern bias is used to determine future actions.

12. The method of claim 4, further comprising determining a similarity metric to compare the FFT results.

13. The method of claim 12, further comprising using the Euclidean distance between the FFT results.

14. The method of claim 13, further comprising: The similarity is scored; and Determine whether the similarity exceeds a threshold.

15. The method according to claim 2, further comprising: If the physical sensor is an image sensor, then: Analyze the pixel intensity values ​​at each time point in the time series; Performing an FFT on the two time series to transform them into the frequency domain will produce amplitude and phase information of the frequencies present in the intensity changes at that point on the face.

Images