Techniques for measuring or modifying control in digital interactions

Abstract

Techniques for determining control in digital interactions include recording a time series of digital sub-second sensor data for each of multiple subjects. A mean and a variance are determined for a feature of micropeaks in the time series of digital sub-second sensor data. An autonomy and control score are determined for each subject of the multiple subjects based at least in part on the mean and variance of the feature. Sometimes a micropeak occurrence feature of the micropeaks is transformed to a binary time series, and the autonomy and control score is also based at least in part on entropy rate of the binary time series or transfer entropy between two binary time series, or some combination.

Description

Docket Number 15290-036PC0 (RU 2024-006) Patent TECHNIQUES FOR MEASURING OR MODIFYING CONTROL IN DIGITAL INTERACTIONS CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims benefit of Provisional Appln. 63 / 584,027, filed September 20, 2023, the entire contents of which are hereby incorporated by reference as if fully set forth herein, under 35 U.S.C. §119(e). BACKGROUND

[0002] Communication among subjects more than ever involve digital streaming over communications networks of images, video, audio, or biosensor signals, or some combination, called hereinafter digital interactions. These interactions are subject to various filters that mask or otherwise modify the digital streaming presented by a subject. Variations in such masks or other filters can affect the quality and effectiveness of communication. SUMMARY

[0003] Techniques are provided for using micropeaks formerly associated with mental biorhythms and neurodevelopment, to measure (and adapt using filters) relative control in digital interactions.

[0004] In a first set of embodiments, a method for determining control in digital interactions includes recording a time series of digital sub-second sensor data for each of a plurality of subjects. The method also includes determining a mean and variance of a feature of micropeaks in the time series of digital sub-second sensor data. The method further includes determining an autonomy and control score for each subject of the plurality of subjects based at least in part on the mean and variance of the feature.

[0005] In some embodiments of the first set, the method still further includes transforming a micropeak occurrence feature of the micropeaks to a binary time series. In these embodiments, determining autonomy and control score for each subject includes determining autonomy and control score for each subject of the plurality of subjects also based at least inDocket Number 15290-036PC0 (RU 2024-006) Patent part on entropy rate of the binary time series. In some of these embodiments, the method still further includes determining transfer entropy between a pair of the binary time series. In these embodiments, determining autonomy and control score for each subject further includes determining autonomy and control score for each subject of the plurality of subjects also based at least in part on the transfer entropy.

[0006] In some embodiments of the first set, the method even further includes filtering a subsequent time series of digital sub-second sensor data from a first subject of the plurality of subjects to increase an entropy rate for the subsequent time series. In these embodiments, the method still further includes transmitting the filtered subsequent time series to a different subject of the plurality of subjects.

[0007] In some embodiments of the first set, the time series of digital sub-second sensor data includes one or more of video data, audio data, movement data of one or more appendages, movement data for one or more facial poses, electrical voltage or current data at one or more electrodes, or brain wave data, or some combination.

[0008] In other sets of embodiments, a non-transitory computer readable medium, or an apparatus, or a system is configured to perform one or more of the above methods.

[0009] Still other aspects, features, and advantages are readily apparent from the following detailed description, simply by illustrating a number of particular embodiments and implementations, including the best mode contemplated for carrying out the invention. Other embodiments are also capable of other and different features and advantages, and its several details can be modified in various obvious respects, all without departing from the spirit and scope of the invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Embodiments are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings in which like reference numerals refer to similar elements and in which:

[0011] FIG. 1 is a block diagram that depicts a digital interactions setup, according to an embodiment;Docket Number 15290-036PC0 (RU 2024-006) Patent

[0012] FIG. 2A and FIG. 2B are graphs that illustrate example properties of micropeaks in digital interactions, according to an embodiment;

[0013] FIG. 2C is a graph that illustrates example normalized peaks in digital interactions, according to an embodiment;

[0014] FIG. 2D is a graph that illustrate example differences in parameters of distributions of digital interactions, according to an embodiment;

[0015] FIG. 3 is a graph that illustrate example binary peak time series, according to an embodiment;

[0016] FIG. 4 is a flowchart that illustrates an example method to detect or effect control and autonomy during digital interactions, according to an embodiment;

[0017] FIG. 5 is a flowchart that illustrates an example method to detect or effect control and autonomy during digital interactions, according to an experimental embodiment;

[0018] FIG. 6A and FIG. 6B are plots that illustrate examples of stochastic characterization of micro-movement spikes, MMS, derived from interactive socio-motor dyads during ADOS evaluation, according to an embodiment;

[0019] FIG. 7A and FIG. 7B are plots that illustrate examples of socio-motor agency according to index of autonomy, according to an embodiment;

[0020] FIG. 8A and FIG. 8B are plots that illustrate examples of autonomy index, according to an embodiment;

[0021] FIG. 9A through FIG. 9D are plots that illustrate example of theoretical considerations for relationships between NSR and Entropy metrics, according to an embodiment;

[0022] FIG.10 and FIG. 10B are plots that illustrate examples of child or clinician autonomy with respect to the log-NSR, VMR and the Spike Rate, according to an embodiment.

[0023] FIG. 11A through FIG. 11C are plots that illustrate examples of a linear relationships between transfer entropy and the entropy rates (autonomies) for child and clinician differentiating between NT and ASD participants, according to an embodiment;Docket Number 15290-036PC0 (RU 2024-006) Patent

[0024] FIG. 12A and FIG. 12 B are plots that illustrate effectiveness of autonomy-to- control ratio as a clinical diagnosis tool for detecting ASD in both male and female children, according to an embodiment;

[0025] FIG. 13A through FIG. 13C are diagrams that illustrate an example of a method that filters imagery before, during or after transmission to mitigate control in a video interaction, according to an embodiment;

[0026] FIG. 14 is a block diagram that illustrates a computer system upon which an embodiment of the invention may be implemented;

[0027] FIG. 15 is a block diagram that illustrates a chip set upon which an embodiment of the invention may be implemented; and

[0028] FIG. 16 is a block diagram of exemplary components of a mobile terminal (e.g., cell phone handset) for communications, which is capable of operating in the system of FIG. 1A, according to one embodiment. DETAILED DESCRIPTION

[0029] Techniques are described to detect or quantify or effect differences in control and autonomy in digital interactions. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.

[0030] Some embodiments of the invention are described below in the context of detecting Autism Spectrum Disorder (ASD) during interactions with a therapist. However, the invention is not limited to this context. In other embodiments, autonomy and control for subjects with the same or different normal developing or neurodevelopmental disorders are detected or quantified or effected, or some combination.Docket Number 15290-036PC0 (RU 2024-006) Patent 1. Overview

[0031] FIG. 1 is a block diagram that depicts a digital interactions system 100, according to an embodiment. Although faces of two subjects 190a and 190b, among other subjects indicated by ellipses, collectively referenced hereinafter as subjects 190, are depicted, the subjects 190 are not part of the system 100. The system 100 includes sensors sets 110a and 110b, among other indicated by ellipses, collectively referenced hereinafter as sensor sets 110, which sets are each configured to detect activities by each subject 190, such activities including one or more facial or bodily movements, sounds, electrical activity, neural activity, or other activities. Each sensor set therefore includes one or more cameras, microphones, electrodes, orientation sensors, velocimeters, accelerometers, or other sensors, or some combination. Each sensor set is configured to collect sensor data at sub-second time resolutions, i.e., at resolutions greater than 1 Hertz (Hz), e.g., greater than 10 Hz, or greater than 100 Hz, or greater than 1 kiloHertz (kHz, 1kHz = 103Hz).

[0032] The system 100 includes computer system 140 in communication with the sensor set 110 by wired or wireless connections, collectively called digital communication links 120, depicted as dashed lines. The computer system includes hardware and software configured to perform as a digital interaction detection module 150. The function of module 150 is to record and process on a computer-readable medium or display, or both, the signals detected at sensor set 110, and to determine relative autonomy and control of each of subjects 190. In some embodiments, the system 100 includes display devices 112a and 112b, among other indicated by ellipses, collectively referenced hereinafter as display devices 112, such as display device 1114 depicted and described below with reference to FIG. 8. Each display device 112 is configured to present sensory stimuli, such as image, video, audio, or robot activity, or some combination, to each subject 190.

[0033] Module 150 is configured to implement one or more of the methods described herein. Module 150 includes one or more data structures 154 to hold data, such as data that indicates a control and autonomy score or video or audio showing a different subject, or some combination.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0034] Although processes, equipment, and data structures are depicted in FIG. 1 as integral blocks in a particular arrangement for purposes of illustration, in other embodiments one or more processes or data structures, or portions thereof, are arranged in a different manner, on the same or different hosts, in one or more databases, or are omitted, or one or more different processes or data structures are included on the same or different hosts.

[0035] Sensor set measurements are collected from sensor sets 110 before, during or after each digital interaction, or some combination. Statistics of fluctuations in these measurements, called micropeaks, have been found to indicate neurodevelopmental conditions. The improvement proved herein is that, according to some embodiments, these micropeaks are used to indicate control and autonomy in digital interactions. Any time series in the space and time domain from sensor sets 110 can be used. For example, one can construct time series of other parameters derived from the kinematics of positional data. In an experimental embodiment, described in more detail in a later section, the time series are from IMUs (accelerometers) and gyroscopes (orientation data giving angular speed) which are very common in wearables these days. IMUs are becoming popular to provide positional data derived from kinematic-chained maps and recent IMU sets give a full body avatar tractable in real time. In body and facial positional data (very rich data!) one can use various properties, such as curvature, torsion, or amount of curve bending and twisting, speed, acceleration of the curve, among others.

[0036] FIG. 2A and FIG. 2B are graphs that illustrate example properties of micropeaks in sub-second sensor measurements, according to an embodiment. For each graph, the horizontal axis indicates a number of frames at 25kHz, and the vertical axis indicates the measurement amplitude (electrical current or voltage, acoustic pressure, video intensity or intensity temporal change or color intensity or color intensity change, position of one or more landmarks on subject 190) for data collected around the digital interaction. Trace 214 illustrates a portion of example output from one sensor. Trace 214 include multiple fluctuations. A micropeak is considered to be the maximum value between successive minimum values. FIG. 2A shows multiple micropeaks along a portion of trace 214. The opportunities to obtain micropeak data are great. For example, in the domain of video atDocket Number 15290-036PC0 (RU 2024-006) Patent 30Hz with face data, only 5 seconds of video are sufficient because of the density of the micropeaks. So, the brevity of the data collection that these new methods afford is phenomenal, as such sensors make it possible to sample very frequently when screening and doing so at scale with commercially available methods. In the FIG. 2A example (based on auditory brainstem response, ABR, waveforms), the micro-peaks were available right after activity, under highly stressful circumstances.

[0037] FIG. 2B shows a single micropeak from FIG. 2A. The single micropeak in FIG. 2B at frame 12 along the horizontal axis has properties, also called features, that include a prominence 245 indicated by a dashed line segment from the larger of two minimum values on each side of the peak to the maximum value, an amplitude of half the prominence value, a width 246a at half height, and a width 246b at the base which is at the value of the larger of the two minima bracketing the micropeak.

[0038] In some embodiments the micropeaks in the signal are normalized by the value of the largest prominence of all the micropeaks in the signal being processed. In some embodiments, the normalization of any peak, including the primary peak, is computed using Equation 1. Pn* = Pn / [Pn+Avg(Pmin1, Pmin2) ] (1) Where Pn is the prominence of the nth peak, Pn* is the normalized prominence of the nth peak, and Avg(Pmin1, Pmin2) is the average of all the points between the two local minima Pmin1 and Pmin2, including the local maximum (micropeak) Pn When the Avg(Pmin1, Pmin2) is small compared to the prominence of the nth peak, such as for the primary peak, the value of Pn* approaches 1. Such normalization tends to scale out allometric differences in length, weight, anatomical size, and other characteristics for individuals in the same group (e.g., same age or stage of development or neurodevelopmental condition or interaction control, or some combination)

[0039] The normalized prominences of the micropeaks and primary peak are plotted in FIG. 2C. FIG. 2C is a graph that illustrates example normalized peaks in movement measurements during digital interactions, according to an embodiment. The horizontal axis indicates frame at 25 kHz, and the vertical axis indicates normalized prominences ofDocket Number 15290-036PC0 (RU 2024-006) Patent micropeaks. Each micropeak is plotted at the frame where it occurs with a normalized prominence value between 0 and 1.

[0040] Similar normalization can be performed for other peak properties, such as width.

[0041] According to improvements presented herein, different families of empirically estimated probability distribution functions (PDFs) of the micropeaks characterize the control in digital interactions. A PDF can be characterized by two parameters, shape and scale, of a Gamma parametrization. FIG. 2D is a graph that illustrates example differences in parameters of distributions of digital interactions, according to an experimental embodiment. FIG. 2D is a log-log Gamma parameter plane spanned by the Gamma shape and scale parameters for each vertex representing a different temporal portion sub-second sensor measurements for various subjects. From these parameter values one can calculate statistical properties such as mean, variance, skewness and kurtosis. The variance to mean ratio (VMR) is especially useful and, in a sui generis terminology is sometimes called the noise to signal ratio (NSR) in the example embodiment.

[0042] Here, instead of averaging the peaks and smoothing out as noise the fluctuations across the dataset, the full waveform was obtained, inclusive of activity prior and after the digital interaction. The pdfs characterizing the variations in the fluctuations were examined, including micropeak widths, prominences, and full amplitudes.

[0043] FIG. 3 is a graph that illustrate example binary peak time series, according to an embodiment. This plot is produced from normalized peaks, such as illustrated in FIG. 2C. The binary signal can be based on any of the normalized or non-normalized peak features by selecting peak features greater than the window mean values for those features. In the amplitude (spatial) domain one can use full amplitude from 0, or prominence from the base (valley) of the previous peak to the maximum. In the temporal domain one can use inter peak interval times, or peak's width. In the experimental embodiment described in a later section, the amplitude of the peak from 0 to the maximum is used to construct the VMR metrics; and then the inter peak intervals were converted to binary to do a Poisson analyses. In general, the binary series is constructed using a value of zero at times of average to low micropeak activity and a 1 at the times of above average micropeak activity, using anyDocket Number 15290-036PC0 (RU 2024-006) Patent threshold for high activity, such as 75thor 90thpercentile. Micropeaks are (standardized) fluctuations away from the empirical (Gamma) mean best fitting the empirical distribution of all peaks (in an MLE sense) in the raw time series or a portion thereof (called a window). So, 0s are places where the activity is at mean value or lower (we call these quiet times) and 1s are times when there is a fluctuation away from the mean activity (pockets of these are bursts of activity). In other words, 1s are times when movement significantly deviates from baseline activity and 0s times when the system stays at baseline. Baseline is determined by the empirical means. That way, the standardized data is advantageously transformed into a binary code that describes when the system bounces back and forth between “active” and “baseline” states.

[0044] FIG. 4 is a flowchart that illustrates an example method to detect or effect control and autonomy during digital interactions, according to an embodiment, using micropeak features. Although steps are depicted in FIG. 4, and in subsequent flowchart FIG. 5, as integral steps in a particular order for purposes of illustration, in other embodiments, one or more steps, or portions thereof, are performed in a different order, or overlapping in time, in series or in parallel, or are omitted, or one or more additional steps are added, or the method is changed in some combination of ways.

[0045] In step 401, sub-second sensor data are collected as one or more digital streams from each subject during an interaction of multiple subjects; thus, recording a time series of digital sub-second sensor data for each of a plurality of subjects. In various embodiments, the time series of digital sub-second sensor data includes one or more of video data, audio data, movement data of one or more appendages, movement data for one or more facial poses, electrical voltage or current data at one or more electrodes, or brain wave data, or some combination.

[0046] In step 403, micropeak features are determined in one or more of the time series of sensor data. In some embodiments, the micropeaks are normalized as shown in FIG. 2C. Variance (σ2) and mean (µ) are determined for each of one or more time periods, also called time windows, e.g., by fitting the Gamma Function (Γ) to the values of the micropeak feature during each window, as depicted for multiple windows in FIG. 2D, and using the shape andDocket Number 15290-036PC0 (RU 2024-006) Patent scale parameter values of the Gamma Function to determine mean and variance. In some embodiments, the variance-to-mean ratio (VMR) is called the noise-to-signal ratio (NSR) of the peaks and the mean to variance ratio (MVR) is called the signal-to-noise ratio (SNR) of the peaks.

[0047] In step 405, the temporal variation of micropeak activity, e.g., the number of peaks as depicted in FIG. 2C, for normalized peaks is transformed to a binary time series, e.g., as depicted in FIG. 3; thus, transforming a micropeak occurrence feature of the micropeaks to a binary time series indicating baseline low activity (one value, e.g., 0) and above baseline activity (other value, e.g., 1).

[0048] In step 411 an autonomy metric is determined for each subject based on the VMR for the subject or based on the entropy rate for the binary time series, or some combination. The computation of entropy rate for a window of the binary time series is explained in further detail in the example embodiments. Here autonomy is linked to randomness level, which can be obtained from the Gamma shape parameter (which also relates to the GAMMA scale or VMR / MVR by the scaling power law) and / or by the approximate entropy measure. The essence of autonomy is randomness vs. deterministic (at the high and low extremes, respectively).

[0049] In step 413, a control metric is determined for each subject based on the MVR for the subject or based on the transfer entropy for a window of the binary time series between two different subjects, or some combination. The control metric indicates how likely a subject’s activities are being influenced by the activities of a different subject. The computation of transfer entropy between two subjects is explained in further detail in the example embodiments. Transfer entropy (TE) provides the directionality of the flow of information (who causally leads / lags the interaction). The binary case affords the Poisson treatment and further invites other methods. In contrast to autonomy, control is about predictability (to compensate for internal delays in the system's transduction, transmission, integration, and transformation of sensory to motor signals).

[0050] In step 415, an autonomy and control score for each subject is determined based on the autonomy metric and the control metric. In various embodiments, the autonomy andDocket Number 15290-036PC0 (RU 2024-006) Patent control score constitutes a pair of values for the control metric and the autonomy metric or a ratio of one of these metrics to the other, or some combination, such as a ratio of the SNR (MVR) to the entropy ratio or the inverse thereof. Socio-motor agency is defined as the ratio of index of autonomy / index of control. In some embodiments, the score is presented to one or more subjects. If an evaluator who is not a subject is available to review the scores, then the score is presented to the evaluator. Thus, this step includes determining an autonomy and control score for each subject of the plurality of subjects based at least in part on the mean and variance of the feature, or based at least in part on entropy rate of the binary time series, or based at least in part on the transfer entropy, or some combination.

[0051] In some embodiments, the autonomy metric or control metric or autonomy and control score is used to reduce or enhance control of one subject by a different subject, as performed by steps 421 through 441. In step 421 the processor sifts through all the subjects. For each subject, it is determined in step 423 whether the subject is appropriately controlled by another subject. If so, then, in step 425, a digital stream from the subject is transmitted to a different subject without any entropy filtering e.g., as a frame in a video conference. Control then passes to step 441. In various contexts, “appropriate controlled” can mean being controlled by another or controlling another or effectively neither controlling nor being controlled.

[0052] If it is determined in step 423 that the subject is not appropriately controlled by another subject, then control passes to step 431. In step 431, a digital stream from the inappropriately controlled subject to remaining subjects is filtered to increase the entropy rate in the filtered stream to reduce such control; or is filtered to decrease the entropy rate in the filtered stream to increase such control. In step 433, the filtered digital stream from the subject is transmitted to a different subject, e.g., as a frame in a video conference. Control then passes to step 441.

[0053] In step 441 it is determined whether there is another subject in the interaction. If so, control passes back to step 423. If not, control passes to step 451. In step 451, it is determined whether end conditions are satisfied. If not, control passes back to step 401, described above. If so, the process ends.Docket Number 15290-036PC0 (RU 2024-006) Patent 2. Example embodiments 2.1 Example method

[0054] This section describes more details on entropy rate and transfer entropy among other details of various methods for various steps of the flowchart of FIG. 4. Here, the variance of the peak features is also called the “noise” in the peak features signal and the mean is called the “signal” so that signal-to-noise ratio (SNR) and noise-to-signal ratio (NSR) in the peak features signal is defined in those terms.

[0055] FIG. 5 is a flowchart that illustrates an example method to detect or effect control and autonomy during digital interactions, according to an experimental embodiment using facial pose sub-second movements in a digital video stream. A related embodiment is described in more detail below. The use of SNR and NSR in this section corresponds to MVR and VMR, respectively.

[0056] 1. START: A social interaction between two or more people is taking place in a remote setting, where each participant can see each other’s face in a videocall platform.

[0057] 2. Start Recording (step 501): The system chooses whether to start tracking face kinematics using the face tracking software at time ^^from the start of the session. If NO, go to step 20. If YES, go to step 3.

[0058] 3. Init Clock (step 503): Set an internal clock to 0 and initialize the face kinematic tracking software.

[0059] 4. Buffer Data (step 505): Fetch data samples and store them into short term memory, e.g. data structure 154.

[0060] 5. T seconds Elapsed (step 507): Check if T seconds have passed from clock initialization. If NO, go back to step 2 and keep collecting data. If YES, go to step 6. In some embodiments, T is selected in a range from 1 to 100 seconds. In various embodiments, the number of seconds / milliseconds of the sliding window depends on the sampling resolution of the sensors and the known physiologically-relevant temporal ranges of the signal of interest. An advantage of 100 sub-second peaks at a minimum ensures statistical power across all bio- signals and determines the sampling unit of time for the sliding window to estimate one set ofDocket Number 15290-036PC0 (RU 2024-006) Patent parameters along the stochastic trajectory being computed across the movie. For example, 5 seconds encompasses at 60Hz 300 measurements for each of the 69 points of the grid across the face. We find sufficient number of peaks, e.g., 20 points across the eyes and eye brows area, describes that area. Other embodiments can use 1 second if the sampling resolution were higher. For example, using a sensor that samples 1 kHz we need much less time, as with the brainstem response data, which was merely a pulse. An advantage of 50% overlap is to provide smoothness in the stochastic trajectory of the empirically estimated Gamma parameters or entropy rate, etc.

[0061] 6. Measure Processes (step 509): Measure kinematic data from the raw positional data.

[0062] 7. Standardize Processes (step 511): Transform the kinematic data to standardized Micro-Movement spike trains. The mean is estimated by fitting the continuous Gamma family of distributions to the raw peak data. The Gamma family of distributions has been consistently found to be the best candidate to fit human peak activity motion data, according to Maximum Likelihood Estimation (MLE). After the data is shifted and centered around the Gamma mean, peak motion activity is scaled and mapped to the (0,1) interval according to the local minima average. The normalized peak series, called Micro Movements Spikes (MMS) conserve the temporal structure of the original speed / acceleration time series. They represent “quiet” times interspersed with bouts of activity away from mean activity. ^^^^^^^^ =^^^^^^^^^^^^^^^ ^^ ^^^Equation (2)

[0063] 8. Split Windows into N Windows (step 513): Synchronously split data from each participant into N non-overlapping consecutive windows.

[0064] 9. Init Windows Index i=1 (step 515): Begin the analysis for the first window.

[0065] 10. If ^ ≤ ^ (step 521): Check whether there are still data windows to be processed or the last window has been reached. If NO, go to step 14. If YES, go to step 11.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0066] 11. Compute the Autonomy index (step 523), which is the entropy rate of the standardized kinematic processes: &(')=())li→m.&('(, '0, … , ')) (3) This quantity measures how much the process changes over time, i.e., the information that is carried in a new value. It measures the degree of randomness (unpredictability) of the underlying dynamical system. It is obtained from the data, using the Approximate Entropy estimator: 2345(^, ^, ^) ≅(789∑MEN8(9;^<∑ IJ^ [| ( ) ( ) F^KLD [# ?,@ B^CD ? 8B^CD E | GHF∑IKLJD^[# ?,@[|B^(?)8B^(E)|FGHF (4)

[0067] can safely bepresumed that it also measures the ability of an observer to predict the motor behavior of another, when the two of them engage in a dyadic social interaction.

[0068] For details on definitions and choices of parameters see Methods section below.

[0069] Compute the Control Index, which is the Signal-to-Noise ratio of the standardized kinematic processes. The continuous Gamma family of probability distributions has the first (mean) and second (variance) moments expressed in terms of the shape and scale: O = ^P, Q0= ^P0(5) Then, unfolding Equation 1, the Noise-to-Signal Ratio (NSR) of the MMS reduces to: UU^RS =TV =^W^W= P (6) The parameters of the Gamma distribution, shape k and scale θ have been found across multiple studies from our laboratory, including the present one (see FIG. 2D), to follow a Power Law of the form: ^ ≅ ^PX→ ;^<(^) = ^ ;^<(P) + Z + [ (7) Where ε is a small errorMMS time series reveals a maturation process of the motor code for voluntary and involuntary motions. This law is very advantageous because it provides us with a quantitative framework to interpret fluctuations in biorhythmic data that range from random to predictive.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0070] Noise-to-Signal Ratio measures the degree of motion variability away from mean activity. Small NSR characterizes steady and smooth motion, akin to goal-oriented behavior. On the other hand, a high NSR indicates unpredictable and random motion. In that sense, the NSR is a proxy for motor control and quantifies the existence of predictable motor patterns. Because the NSR is calculated on the standardized MMS, motor noise does not depend on the anatomy of the individual as it is scaled by the mean amplitude of motion.

[0071] Compute Transfer Entropy: Using transfer entropy, we can quantify the amount the causal influence between participants, without the need to use any model or make any other assumptions.

[0072] 12. Append Indexes (step 525): Store the indexes that were calculated for the current window into a multidimensional array, e.g., in data structure 154.

[0073] 13. i=i+1 (step 527): Move to the next window.

[0074] 14. If i>N (step 529): Check if window count index is out of bounds. If NO, go to step 10. If YES, go to 15.

[0075] 15. Diagnose interactions (step 531): Based on the indexes stored in the array, perform diagnostics on the state of the social interaction and print the results.

[0076] 16. Reached Target (step 533): Check whether indexes for one or more participants are within a desired range. If YES, go to step 2. If NO, go to step 17.

[0077] 17. Update Policy (step 535): Determine policy required to bring the social interaction to a desired state. Note: We can determine such a policy because using our techniques, we can establish relationships between the indexes. Such a relationship can be found in FIG. 11, described in more detail below, where autonomy and Transfer Entropy are approximately linearly correlated. More advantageously, statistical analysis can show us the relationship between autonomy, NSR and rate of MM peak activity. See FIG. 10, described in more detail below.

[0078] 18. Update Filter (step 537): Generate a filter to be applied on the video interfaces of one or more participants, according to the updated policy. Note: The filter will generate patterns of pixels that, when superimposed on each frame, over multiple frames, will changeDocket Number 15290-036PC0 (RU 2024-006) Patent the spike rate or NSR in such a way, that, following the established relationships between the indexes, will allow us to tune them towards the desired range.

[0079] 19. End Session (step 541): Check whether the videocall session has come to an end. If YES, the process terminates. If NO, go to step 2.

[0080] 20. Implement Policy (step 543): The system or the user decides whether to intervene and implement the policy to the social interaction. If NO, go to step 19. If YES, go to step 21.

[0081] 21. Init Clock (step 545): Initialize an internal clock.

[0082] 22. Apply Filter (step 547): Start applying the image filter to each and every frame of the video interfaces of one or more participants.

[0083] 23. T seconds Elapsed (step 549): Check the clock to see if T seconds have passed by. If NO, go to step 22 and keep filtering. If YES, go to step 2. 2.2 Example embodiment.

[0084] An example embodiment was implemented in the context of Autism Diagnosis Observation Schedule (ADOS, currently in version 2 [1; 2; 3; 4].) to measure control in dyadic interactions between clinician and child-patient, or modify such interactions in videoconference sessions, or some combination. Although clinically validated, the ADOS- based diagnosis misses females [5; 6; 7]. Moreover, there are not enough raters to absorb the large number of toddlers, children, and adults that according to various screening tools, are suspected as autistic today. The test is long and taxing on both the child and the clinician administering it because it has an average of 27 tasks aimed at engaging the child through social presses and expecting overtures from the child. many of the tasks artificially rob the child of a chance to be naturally social, as the interaction is also taxing on the clinician and at times, awkward and seemingly forced. In this sense, several of the tasks might be biased, interfering with the child’s agency, and robbing the clinician of the spontaneity characteristic of a natural social exchange. In this sense, it is advantageous to develop objective ways to quantify this potential bias that such a taxing effect may produce on both social agents. Furthermore, there currently are no means to define naturalistic social agency in the dyad andDocket Number 15290-036PC0 (RU 2024-006) Patent to identify tasks that enhance it. Still further, currently there are no means to implement these tasks using artificial intelligence (AI) and machine learning (ML) methods. Such approaches are advantageous to speed up, automate, and scale the assessment process, particularly doing so with respect to currently underdiagnosed females [5].

[0085] Herein, digital ADOS

[0011] , i.e., the ADOS that is digitally recorded while the child and clinician interact, is leveraged to enhance the validity of this test as the gold standard for clinical and research use, while providing a streamlined version of it that could help (1) identify objective biometrics of social agency and (2) automate the process of identifying socially compliant tasks using methods from Artificial Intelligence (AI) and Machine Learning (ML).

[0086] Herein, socially compliant tasks are those which provide social agency to the child that is being diagnosed. More precisely, here social agency is defined as a balance between autonomy and control during a social exchange. Autonomy is defined as the ability of the child to lead the conversation as much as the clinician does, rather than always following the lead of the clinician. Control is defined as the ability of the child to effectively predict the consequences of impending social actions and overall behaviors, based on intact motor control.

[0087] In an experimental embodiment, snapshots of the interaction between a child and clinician, each wearing 3 biosensors, provide synchronously recorded motion at 128 Hz. The clinician leads interactions and takes notes while rating the interaction. Sample standardized micromovement spikes (MMS) are derived from angular speed capturing during 1500 frames of social exchange (approximately 11.7 seconds). Frequency histograms of the MMS peaks, are derived (one frame) from each sensor on the child and clinician. Pairwise comparison of the histograms evolution were evaluated using the Earth Movers’ distance similarity metric. A 6x6 matrix (child and clinician, 3 sensors at the torso, right and left wrist) of coordinated MMS features Is formed. Off diagonal entries of the matrix are the shared dyadic space, while entries along or near the diagonal are the child’s or clinician’s activities in standalone mode. An EMD scale ranks from most to least similar spike patterns in the matrix for each frame.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0088] In a non-limiting interpretation, socio-motor agency can be impeded if neurodevelopment undergoes a different maturation path [3; 14; 15]. If the child, for example, has excessive motor noise (variance) and motor randomness in its performance, the predictive ability required for self-motor control will be compromised [3] and with it, the overall control ability will be altered. This alteration will also, in turn, affect the clinician’s perception of the child’s nuanced micro-motions underlying social behaviors, thus biasing the assessment [3; 11; 14]. Under such circumstances, socio-motor agency can be impeded, as can be the rating of the child by the clinician. Therefore, the tasks that manifest excess random noise of the joint dyadic motor patterns (lower control of the dyad) and / or excess lead of the clinician within the dyad (lower autonomy of the child), are inevitably bound to bias the clinician’s scoring towards a deficit model of autism. In contrast, the tasks that manifest high dyadic control and autonomy of the child are bound to boast social agency, according to our biometric definition. These tasks can provide a more appropriate model of readiness potential for social exchange. While detecting a problem relative to normative data, this new model can also do so in a fair, unbiased manner. In this sense, the child has a chance to succeed. In turn, the clinician can presume competence and identify areas of strength to recommend treatments more appropriately. Such treatments will rather be grounded on the non-obvious, nuanced aspects of behaviors occurring at a micro-level that escapes the naked eye, yet they will be quantifiable with biosensors that read out biorhythmic activities from the nervous systems with sub-second resolution.

[0089] In the experimental embodiments, a total of 29 children including 19 males and 10 females spanning 4 to 15 years of age and two adult clinicians participated in the study. Children participated in multiple sessions over the span of 2 ½ years with one clinician per session and were administered a specific module per session, i.e., a specific subset of ADOS tasks, in accordance with their age, level of development and spoken language.

[0090] Digital data were acquired during each session using light wearable sensors (APDM Opals, Portland, OR, USA). Six sensors were used, one each on the left and right wrist and one on the torso, for each of the child and clinician. The sensors continuously and synchronously recorded triaxial accelerometry and gyroscopic data at a sampling frequencyDocket Number 15290-036PC0 (RU 2024-006) Patent of 128 Hz. The recording environment followed the standardized ADOS requirements using similar table and sitting arrangements for the clinician-child dyad. The children and the two clinicians were unaware of the goals of the study.

[0091] Scalar values of angular speed from orientation data that the gyroscopes recorded (or acceleration from the inertial measurement units) were acquired using the Euclidean norm of the coordinate components of motion as measured by the sensors using Equation 8. \ =]\B0+ \^0+ \_0(8) From here onward, all analyses refer to the scalar value the

[0092] Human motion data suchbiased by allometric effects and anatomical differences across subjects. To scale out such artifacts, normalized motion data fluctuations (peaks and valleys) are used as relative deviations from the empirically estimated Gamma mean, as given by Equation 2, described above. The parameters of the Gamma distribution, shape k and scale θ have been found across multiple studies from our laboratory, including the present one, to follow a Power Law of the form given above by Equation 7. The distribution mean and variance can be determined from the Gamma parameters as given above in Equation 5. The VMR (also called NSR) is then given by Equation 6 above. The Gamma scale parameters fully characterizes the variance (noise) of the motor patterns of the interactive dyad (or of the participant), e.g., in relation to the level of fluctuations of angular speed during the ADOS activities .

[0093] Empirical estimation of these parameters in thousands of participants over a decade of work with humans along the lifespan, and across disorders of the nervous system, has revealed an interpretation for the Gamma log-log parameter plane. Distributions that fall along high NSR regimes are also close to the memoryless random regime of the special exponential distribution case (when k = 1). Points in mid NSR correspond to heavy-tailed Gamma distributions. Then, low NSR (or high signal = 1 / NSR) are congruent with symmetric shapes (Gaussian-like) distributions. High-signal Gaussian regimens are highly predictable in contrast to High-noise memoryless random Exponential regimes. As such, this parameter plane is empirically interpretable.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0094] When quantifying motor control from the perspective of an agent, Noise-to-Signal Ratio (NSR,VMR) measures the degree of motion variability away from mean activity. Small NSR,VMR characterizes steady and smooth motion, akin of goal-oriented behavior, as experienced from the perspective of the agent / child. On the other hand, a high NSR,VMR indicates unpredictable and random motion. In that sense, the NSR,VMR is a proxy for motor control and quantifies the existence of predictable motor patterns. Because the NSR,VMR is calculated on the standardized MMS, motor noise does not depend on the anatomy of the individual as it is scaled by the mean amplitude of motion.

[0095] The presence of MMS peaks indicates an outburst of activity away from baseline. This is informative of a motor activity. When the temporal distribution of MM peaks is also taken into account, a MMS train can be viewed as a representation of information regarding human motion variability through time. When multiple sensors sampling in synchrony are considered, the MMS spikes carry spatiotemporal information about the bursts of activity in the motor system.

[0096] If the MMS data are transformed so that the presence of a peak corresponds to the binary “1” and an absence of a peak corresponds to the binary “0”, we can represent normalized speed (or acceleration) as a stochastic binary sequence. An underlying mechanism stochastically generates bursts of activity, and this is equivalent to randomly generating 0s and 1s from an underlying binary alphabet. The transform is given by Equation 9. `a= 1, if ^^^^^^^^a> ^ℎ^^dℎ^;e (9) Assuming that `ais a random sample drawn from an underlying probability distribution at time ^, as expressed in Equation 10. `a= 1 with probability 3a, 0 with probability 1 − 3a(10) Entropy is an information theoretic measure that quantifies the amount of information in a random variable that follows a probability distribution ^B

[0019] and is given by Equation 11: & = −∑B3B;^<^3B(11) In the case of the binary process, the amount of information of the random variable of an activity outburst (MM spike) is given by Equation 12.Docket Number 15290-036PC0 (RU 2024-006) Patent &a= −3a;^<^(3a) − (1 − 3a) ;^<^(1 − 3a) (12) Which takes the maximum value of 1 when 3a= 0.5 and the minimum value of 0, when 3a= 0 or 1.

[0097] Intuitively, entropy measures either the uncertainty regarding the outcome of a random realization of the random variable before that variable is measured or equivalently the amount of information one gets when one observes the variable. If for examples, that with a 100 % chance `a= 1, the entropy is zero and there is no uncertainty about the outcome of the measurement, and no valuable information is provided to us. However, if with a 50 % chance `a= 1, the entropy is maximum because it is totally uncertain whether the outcome will be 0 or 1 and observing the outcome gives maximal information, specifically, 1 bit of information in the case of a base 2 logarithm (^ = 2).

[0098] Randomness vs. predictability is measuring using Entropy Rate (H). The definition of entropy can be generalized for the case of multiple random variables '(, '0, … , 'M,, by considering the joint probability distribution ^iD,iU,…,iIusing Equation 13. &('(, '0, … , 'M) = − ∑iD,iU,…iI,^iD,iU,…,iIlogl(^iD,iU,…,iI) (13) In theover time) which takes values from a discrete alphabet m (in the case of the binary MMS), the entropy rate of the process is given by Equation 3 defined above and rewritten here for convenience. &(')=())li→m.&('(, '0, … , ')) (3) This quantity measures how much the process changes over time, i.e., the information that is carried in a new value. It measures the degree of randomness (unpredictability) of the underlying dynamical system [19; 20; 21; 22].

[0099] The concept of entropy rate is not limited to random processes, but it can also be defined in the case of deterministic dynamical systems. Let nabe a continuous univariate times series. Then one can construct a state-space representation of the process if one chooses an appropriate dimension e of the presumed underlying dynamical system and an embedding delay o [23; 24].as given by Equation 14.Docket Number 15290-036PC0 (RU 2024-006) Patent n( n(^)æn(^ + o)öThe existence and are ensured by Taken’s embedding theorem

[0025] . For more information on dynamical systems theory see e.g., [26; 27]. Essentially, any univariate time series can be viewed as being sampled from a high dimensional dynamical system

[0028] . The dynamical system follows a trajectory in the d- dimensional space defined by the d degrees of freedom. All possible states of the dynamical system define the phase space of the system.

[0100] If one partitions the phase space across v dimensions, with v ≤ e, one has an F- dimensional grid of cells of volume ^w. Then, one can measure the state of the system at constant time intervals equal to the embdedding delay o. Then one can define the joint probability 3(x(, x0, … , x@) that 'yis in cell x(, '0yis in cell x0,…., '@yis in cell x@. The degree of “randomness” of the determistic system can then be calculated using the Kolmogorov-Sinai (KS) entropy expressed in Equation 15. mR = − ;Hx→^^ @;→x^(.@y∑ED,EU,…,Ez3(x(, x0, … , x@);^<^3(x(, x0, … , x@)(15)naand characterizes the degree of randomness of the system (and subsequently the sampled one- dimensional signal). For completely deterministic systems it is equal to zero and infinite for random systems.

[0101] In practice, the entropy rate is approximated using what is known as the correlation integral given in Equation 16. D (@−0~U^Docket Number 15290-036PC0 (RU 2024-006) Patent i.e., the number of pairs of trajectory points that are close to each within a tolerance threshold ^ . The correlation integral measures the regularity (frequency) of patterns like a given template of specific length.

[0102] It can be shown that the relationship in Equation 17 holds.: ( @→;.x^^z(H),H→^ y;^<^ ^zCD(H)~ m0(17) entropy is a generalization of the usualShannon entropy and is defined by Equation 18.: m^=((8^;^<^(∑B3B^) (18) the entropy rate in the case of adiscrete time series ^(5). Consider two different blocks of length ^ sampled from the time series as expressed in Equations 19a and 19b. n(x) =^^(x), ^(x + 1)… , ^(x + ^ − 1)^(19a)e[n(x), n(^)] =^N^(,^0,n…9(|^(x + ^ − 1)− ^(^ + ^ − 1)|) (19c) i.e., the maximum distance between the two vectors (Chebysev distance). Then, one can define a quantity like the correlation integral, for a template of length ^ at n(x) using Equation 20. {# ?^M89^(,@[B(E),B(?)]E9=^HM89^(20) Then, the entropy4S = ;9 98(Hx→^^ 9;x→^. M;x→^.[^ (^) − ^ (^)] (21) where ^9^9(^) =(∑MEN8(9^(;^<^{E9(^) (22)

[0104] ^ is very close to the template sequence at time x, and {E98((^) the probability that the same holds true ^ for sequences of length ^ − 1, then^^ (H)(H)is the conditional probability that anyDocket Number 15290-036PC0 (RU 2024-006) Patent of length ^ is very close to the template of length ^ at time x given that the same holds true for ^ − 1. Then the logarithm of this conditional probability is given by Equation 23a. ;^<^ ^^ ^^ (H) 9^^^JD(H)^ = ;^<^|{E(^)~ − ;^<^({E98((^)) (23a) It is of thisto in time series analysis, the entropy rate can be estimated by what is known as Approximate Entropy (ApEn)

[0026] and is given by Equation 23b. 2345(^, ^, ^)(^) = ^9− ^98((23b) Where ^ is the template and ^ isthe threshold tolerance choice. Approximate entropy (ApEn) measures the logarithmic frequency with which segments of length ^ that are very close together (according to the threshold), stay together through time. An approximate formula for ApEn, which was implemented in this example embodiment is given by Equation 4, rewritten here for convenience. 345∑IJ^[# ?,@[|B ? 8B E GH]2(^, ^, ^)≅(∑ MEN8(9;^<KLD ^CD( )^CD( )|]^ (4)MMS speed sequence. Generally, in the case of discrete alphabet sequences with k symbols, Equation 24 holds. 0 ≤ 2345 ≤ ;^<^^ (24) Where 2345 = 0 for deterministic time series and 2345 = ;^<^^ for random series. In the example embodiment (binary MM series), ^ = 2 and so Equation 25 holds. 0 ≤ 2345^^≤ ;^<^2 (25) For ^ = ^ (natural logarithm choice), the maximum value is ;5(2)= 0.69, which is the base used in the example embodiment.

[0106] A good choice of ^ is equal to the embedding dimension, which can be estimated using the False Nearest Neighbor (FNN) algorithm. Usually, ^ is of low dimension, in theDocket Number 15290-036PC0 (RU 2024-006) Patent example embodiment, the dimension of the data was estimated to be 2. The threshold ^ is usually set between 0.1 to 0.25 standard deviations of the time series.

[0107] Herein, control by a prompting party over a responding party is detected objectively by a MMS response that is more predicted and less random compared to a MMS prompt, i.e., with an decrease in entropy. Similarly, independence of the responding party is indicated objectively by an increase in entropy of the MMS response over the MMS prompt The self- control index is purely based on the SNR (MVR) which in the Gamma sense also provides information about the shape of the distribution and tells us if the self sub-second peaks are random noise or predictive, Gaussian signal (high SNR,MVR).

[0108] Quantifying Information Flow Between Binarized MMS with Local Transfer Entropy is thus used as a proxy for objectively determining control in dyadic interactions. Local Shannon Entropy is defined as the negative logarithm of the probability of an outcome x of a random variable as expressed in Equation 26. ℎ(n)= − ;^<03(n) (26) where low probability outcomes carry more information than high probability outcomes. Entropy as we defined it can then be expressed as the average value of all such outcomes, as given by Equation 27. &(') = 4[ℎ(n)] = ∑B3(n) ;^<03(n) (27) where 4[. ] is the expectation (average) operator. An estimator based on samples n}is given by Equation 28. &(n) ≅( MM∑}N(ℎ(n}) (28)

[0109] are respectively defined as: x(n; ^) = ;^<^|0n^^~= ℎ(n) − ℎIt quantifies the information that is gained when observing ' (e.g., a response MMS) after having already observed another variable ^ (e.g., a prompt MMS)Docket Number 15290-036PC0 (RU 2024-006) Patent

[0110] Similarly, the Local Conditional Mutual Information and Conditional Mutual Information are given by Equations 30a and 30b, respectively. x(n; ^|^) = ;^<^|0n^^, ^~^(n|^)= ℎ(n|^) − ℎ(n|^, ^) (30a)These quantify the information that is gained when observing ' after considering both ^ and ^ versus considering only ^.

[0111] Local transfer entropy quantifies the flow of information from Y to X and is defined by Equation 31a. ^^→i(5 + 1, ^, ;, ^)= x(^(}^)^(8^; n}^(|^(}^)) (31a) Where ; and process Y witha memory of ; samples up to point 5 + 1 − ^ given by Equation 31b ^(}^)^(8^= ^^}^(8^8^^(, … , ^}^(8^8(, ^}^(8^^ (31b) up to point5 + 1 − ^ given by Equation 31c. ^(}^)= ^n}8^^(, … , n}8(, n}^. (31c) The^ and ' that is of interest when calculating the transfer entropy from ^ to '. For ^ = 1, a typical choice of source-destination lag, the transfer is given by Equation 32. ^^→i(5 + 1, ^, ;) = x(^(}^); n}^(|^(}^)) (32) The localstate of X, ^ samples ahead, conditioned on the history of X. In other words, it measures the information gained about the future state of X when considering both its own past and the past states of Y versus considering only its past state. Transfer entropy is the expected information gain, averaging over all states as given by Equation 33. ^^→i(^, ;) = 4[^^→i(5 + 1, ^, ;)] (33)

[0112] Entropy transfer rate is used herein to quantify autonomy of an agent from the perspective of an observer. For a child / clinician dyad, the normalized MMS derived from theDocket Number 15290-036PC0 (RU 2024-006) Patent fluctuations in angular speed from the right- and left-wrist sensors throughout the course of the dyadic interaction were obtained. These are then converted into binary MMS trains and the entropy rate estimates (using approximate entropy) were calculated. Since ER is a proper way to quantify predictability vs. randomness, one can safely presume that it also measures the ability of an observer to predict the motor behavior of an agent, when the two of them engage in a dyadic social interaction.

[0113] For example, when the clinician observes the behavior of a child that engages in repetitive and predictable motions, they can easily learn their behavioral and motor patterns. This also implies that they can more easily lead and control a child that behaves predictably.

[0114] Following this argument, in this embodiment, socio-motor agency is redefined as the balance between control and autonomy. Signal-to-Noise ratio (SNR,MVR) and entropy rate objectively characterize the ability of an agent to control their own behavior and the ability to act autonomously in a social interaction. Transfer entropy can objectively quantify the amount the causal influence from the clinician to the child and vice versa, without the need to use any model or make any other assumptions.

[0115] In the example embodiment, angular speed samples (128 Hz) from wearable sensors on the wrists and torso of the child and clinician are normalized and binarized to obtain discrete sequences of 0’s and 1’s.entropy rate estimates measure the unpredictability of the underlying binary processes to characterize the agents’ autonomy in the dyadic social interaction. The analysis is performed on data from time windows of about 7.8 secs which proved advantageous to attain high confidence intervals.

[0116] In the context of ADOS-2, there are task modules that each consist of tasks that the clinician performs with the child / patient to observe behavior related to the diagnosis of ASD and reach a conclusion. There are different modules. Each child is administered a single module based on their expressive language level, developmental age and their unique interests and abilities. However, the modules are designed in such a way that ensures that judgements about social and communicative abilities are as independent as possible from level of language ability and chronological age. Both Module T (toddler) and Module 1 are administrated to non-speaking children, Module T for ages 12-30 months and Module 1 forDocket Number 15290-036PC0 (RU 2024-006) Patent children over 31 months. Module 2 is administrated to children of all ages who are using phrase speech but are not yet verbally fluent. Module 3 and Module 4 are administrated to individuals that are verbally fluent with Module 3 specifically designed for Children / Younger adolescents that can still play with action figure-type toys and Module 4 for adults. All Modules are administrated under the assumption that the individual can walk independently and is free of visual or hearing impairments.

[0117] The experimental embodiment focuses on Modules 1,3 and 4. We suggest the following categorization of tasks to relate our digital biomarkers to the clinical criteria.

[0118] Socio-Motor Tasks. These are tasks that engage interactive movements within the child and between the child and clinician. Construction Task, Joint Interactive Play, Demonstration Task, Cartoons, Conversation and Reporting and Break Tasks all have in common the Child’s Socio-Motor behavior involvement. Construction Task consists of an interaction between the Clinician and Child that involves reaching over the Clinician’s arm to ask for block pieces. Joint Interactive Play consists of a Play Sequence between the Child and the Clinician that involves body movements. During Demonstration Task the Child uses their body to represent objects and mime the use of each object. During Cartoon Task, the Clinician observes the Child’s gestures and coordination with speech. Similarly, during Conversation and Reporting body language and facial expressions / gestures are observed alongside general communicative skills. During Break the Child is expected to move around the room.

[0119] Emotional Tasks. These are tasks that probe the child’s emotional states. Emotions, Social Difficulties, Friends, Relationships, and Marriage and Loneliness all evoke a strong emotional response from the child. During the Emotions Task, the child is asked questions about social relationships, different emotions such as happiness, fear and anxiety and details about the manifestation of these emotions under different circumstances. Social Difficulties and Annoyance consist of questions related to social interactions at school or work, such as bullying or teasing. Friends, Relationships, and Marriage are designed to evaluate the child’s concepts on topics such as friendship and social relationships and the questions asked can cause strong emotions in the child. Similarly, during Loneliness task, questions are askedDocket Number 15290-036PC0 (RU 2024-006) Patent about the concept of loneliness, which is a heavy topic, especially for children in the Autism Spectrum, that struggle with social rejection and bullying from a young age.

[0120] Abstract Tasks. These are tasks that require higher, abstract-level cognition. Make- Believe Play, Description of a Picture, Telling Story from a Book, and Creating a Story all help observe higher cognitive skills. Make-Believe Play involves the use of dolls an action figures and the child is tested for their ability to perceive them as animate beings and produce imaginative sequences of actions that involve these objects. Perception, or the lack of it, for objects as animate beings is a concept frequently encountered within the context of the Theory of Mind. During Description of a Picture Task the clinician observes the child’s use of language / communication and the level of interest in the picture presented. Telling a Story from a Book is similar but involves a story from a book instead of a picture, so humor and presumption of the feelings of the characters from the book are evaluated as well.

[0121] After the administration of Module 3 a scoring system is used to evaluate the levels of Social Affect (SA) and Restricted and Repetitive Behavior (RRB). The scores are added up to determine a final score, from 0 to 10. A score of 0 or 1 indicates Minimal to No Evidence of ASD related symptoms, scores between 2 and 4 indicate a Low Level of ASD related symptoms, 5 to 7 Moderate and 8 to 10 High. A score of 9 or more determines that the child is Autistic whereas a score of 7 of more that the child is in the Autism Spectrum. Furthermore, Social Affect consist of Communication scoring (Reporting of Events, Conversation and Descriptive, Conventional, Instrumental, or Informational Gestures) and Reciprocal Social Interaction scoring (Unusual Eye Contact, Facial Expression Directed to Examiner, Shared Enjoyment in Interaction, Quality of Social Overtures, Amount of Reciprocal Social Communication, Overall Quality of Rapport). RRB consists of scoring Stereotyped / Idiosyncratic Use of Words or Phrases, Unusual Sensory Interest in Play Material / Person, Hand and Finger and Other Complex Mannerisms and Excessive Interest or Highly Specific Topics / Objects or Repetitive Behaviors. 2.3 Example results.

[0122] FIG. 6A and FIG. 6B are plots that illustrate examples of stochastic characterization of micro-movement spikes, MMS, derived from interactive socio-motorDocket Number 15290-036PC0 (RU 2024-006) Patent dyads during ADOS evaluation, according to an embodiment. The MMS standardized fluctuations in angular speed are obtained from the right (dominant) wrist during ADOS social interactions. Filled markers represent first visits to the clinician, unfilled markers are subsequent visits. FIG. 6A plots empirically estimated Gamma Shape and Scale (NSR) for each participant using Modules 1, 3 and 4 of the ADOS test as backdrop of behavioral assay. The values encompass all activity during the social exchange with the clinician, across the entire ADOS session. The size of the marker is proportional to the age of the participants. The empirically estimated Gamma parameters localize each child-clinician’s dyadic interaction for each task on the Gamma parameter plane with 95% confidence intervals. These points represent the empirical probability density function (PDF) of their joint dyadic interaction. When we plot the full scatter estimated from each task in the ADOS, for all children, a tight linear relation emerges whereby the log-log plot follows a scaling power law, of the form k≅aθb. As the Gamma scale value decreases, the Gamma shape value increases. Because knowing one, we can predict the other with high certainty, we can then reduce these two parameters of interest to one parameter summarizing these motor signatures of the interacting dyad. We can also do so for each individual signature, i.e., those of the child and those of the clinician in standalone mode. Using this relationship in Equations 5 and 6, above, the Gamma scale parameter θ fully characterizes the noise, variance (NSR,VMR) of the biorhythmic motor patterns of the interactive dyad, e.g., in relation to the level of fluctuations of angular speed during the ADOS activities.

[0123] From this plot a child’s negative Gamma scale parameter (-log NSR), which indicates control as a function of age can be plotted (not shown). Such a plot shows increasing NSR with age for ASD in contrast to neurotypical (NT) children. the log-SNR (denoted as the index of control) of the interacting socio-motor dyad has an age-dependent pattern. In TD children, as the age increases, control tends to slightly increase i.e., a slight positive trend is reflected in the slope of the line fitting the (blue) TD scatter (TD: intercept = 3.0271 p = 0.0067, slope = 0.0786 p = 0.362.) In contrast, as ASD children age, control tends to decrease, i.e., a strong negative trend is quantified in the slope of the line best fitting the (red) scatter, (ASD: intercept = 5.5317 p = 5.65 x 10-12, slope = -0.1074 p = 0.0463s.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0124] As indicated above, one can derived from the Gamma parameters plotted in FIG. 6A, mean, standard deviation and skewness. These PDF properties show a tendency of the joint dyad moments to separate NTs from ASD participants, as they interact with an adult clinician, with marked differences between males and females.

[0125] FIG. 6B plots Transfer Entropy for social dyads involving clinician and child obtained for males (M) and females (F) in the NT and the ASD groups, using 6 sensors, 3 on the clinician and 3 on the child, outputting time series of angular speed motion on the left and right wrists and the trunk of each social agent in the dyad. Each matrix of cells shows pairwise Transfer Entropy (TE) from child to clinician and from clinician to child. Recall that TE is the reduction in uncertainty of predicting the future of X when we consider the process Y. Dark cells indicate little transfer entropy between parties and thus low independence, while light cells indicate more transfer entropy and thus more independence.

[0126] The cross terms in the off-diagonal entries of the matrix (top right-hand entries 1,4 to 1,6; 2,4 to 2,6; 3,4 to 3,6; and bottom left-hand entries 4,1 to 6,1; 4,2 to 6,2 and 4,3 to 6,3) represent the dyadic cases of child to clinician and clinician to child, respectively. There, in the shared entries of the matrix we see that in ASD, males show a decrease in TE values while females show an increase. In the context of the ADOS, females evoke a reduction in the clinician’s uncertainty predicting the impending females’ motions. , This might indicate a bias that partly accounts for the disparate ratio of 4-5 males per each female diagnosed with ASD. These differences demonstrate the interplay between the NSR, VMR as an index of controllability (predictability) and the overall sense of socio-motor agency in each of the ADOS tasks, for males and for females.

[0127] In the diagonal sub-matrices (top left-hand entries 1,1 to 1,3; 2,1 to 2,3; 3,1 to 3,3; and bottom right-hand entries 4,4 to 4,6; 5,4 to 5,6 and 6,4 to 6,6) we represent the patterns within the individual’s body parts. There we appreciate higher values of TE from child to child in both ASD males and females, with ASD females having higher TE than ASD males. As with the shared dyadic activity, here in the individual patterns, the highest differences for clinician to clinician can be appreciated in the ASD females.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0128] Quantifying dyadic social agency reveals differences between NT and ASD. In a dynamic dyadic social interaction, such as that taking place during the ADOS, it is difficult to exert control over the interaction because prompts by the clinician and responses by the child are not occurring at the expected timely rates. This temporal mismatch in autism alone can bias the rating by the clinician in ways that differ between NT and ASD, but also differ between males and females. Here we equate high NSR,VMR with low predictive control and propose the type of socio-motor agency required in a naturalistic social interaction will be impacted by poor controllability levels on one side of the dyad. We then explore whether dyad-based control (e.g., shared by the child and clinician) is observably different in ASD participants.

[0129] Another aspect of dyadic social agency, besides control, is autonomy. As mentioned earlier, autonomy is the ability of the child to lead the conversation as much as the clinician does, rather than always following the lead of the clinician. An obvious way to quantify the degree to which the clinician is leading would be by using some form of causal analysis between data recorded on the clinician and data recorded on the child. As our main approach however, we choose to quantify autonomy in ways that depend on data recorded from wearable sensors on a single agent which, as will be shown, is intuitive and can be applied in a clinical setting, to help digitize the ADOS.

[0130] In this embodiment, entropy metrics are used to examine the degree to which the MMS behave randomly or deterministically (contain periodic, systematic patterns). To that end, entropy rate [26; 27], a metric well suited to interrogate the stochastic regimes of spike trains, is used. To derive entropy rate from MMS data, a time widow is used which is small enough that the process can be viewed as stationary, yet large enough to contain a satisfactory number of samples lending statistical power to our empirical estimation per window. As such, 7.8 seconds was observed to be an advantageous unit of time for the spike trains. Using this window, MMS per unit of time was determined. The degree of randomness of the child’s motions is measured by estimating the entropy rate. Furthermore, entropy rate is compared to transfer entropy obtained from the child and clinician, a causal metric that can inform who leads the interaction for any given task.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0131] In this embodiment, a suitable scale of autonomy is one in which, at one extreme, a high degree of randomness is a measure of a system at its highest degree of autonomy. This is the type of state where the system is uncontrollably “hidden” from the initiator. There is no opportunity to control the respondent. At the other end, the lowest degree or randomness leads to a systematic, deterministic pattern, highly controllable. While in the former, the child’s excessive autonomy prevents social exchange with the clinician in that the clinician cannot control the child, in the latter, the clinician can absolutely control the child. Either extreme is detrimental to the development of rapport and turn-taking in a social exchange. A happy medium is one in which, while the child preserves a degree of autonomy that enables a balanced social exchange, the clinician also partakes in a give-and-take interaction, rather than leading the child most of the time.

[0132] This embodiment is tested by comparing transfer entropy (TE, a measure of causality) from the child to the clinician, with entropy rate, a measure spanning a scale from totally random to totally deterministic states of the MMS. FIG. 7A and FIG. 7B are plots that illustrate examples of socio-motor agency according to index of autonomy, according to an embodiment. FIG. 7A depicts average transfer entropies (TE) between child to clinician taken over windows of ~7.8 secs duration, per participant (filled in markers are first visit to the clinician, unfilled markers are subsequent visits) plotted against horizontal axis indicating child. autonomy (Ap Ent). This plot reveals higher autonomy index in NT children, a trend that is also quantified in FIG. 7B, a histogram of occurrences of different child autonomy scores. As the child autonomy increases, the CL to CH TE (left hand) decreases.

[0133] FIG.7A shows an age-dependent trend spanning two scatters. In older neurotypical (NT) children, the scatter aligns such that as the child’s entropy rate (denoting a scale of autonomy) increases, the TE decreases, denoting a causal lead of the child over the clinician. In contrast a second scatter emerges for younger children whereby the trend is less visible, indicating that these children’s index of autonomy is not as evident during the exchange and the causal lead (TE) denoting the child’s lead over the clinician’s lead, is less evident. We can appreciate the shift in this metric of autonomy in FIG. 7B where the histogram of the ASD children is shifted to the left, indicating lower density values than NT children.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0134] Adding the clinician (CL) past activity does not contribute more information about the child (CH) state than looking at the CH past activity alone. This is also the case for the right hand. Since the left hand is not the dominant hand in these children, we plotted the histograms pertaining to the left hand as well, to see if these effects consistently emerged. it was found that across multiple time windows, the pdf for the neurotypical group is shifted to the right, meaning that on average, NTs have higher values of autonomy than ASDs.

[0135] Since autonomy also varies throughout sessions, plotting the autonomy variability (variance of this index over the mean of this index) for different participants shows that for ASDs, child and clinician variability is higher than most NTs. This variability index tends to separate ASD from NT participants, particularly for later visits (as the child aged, over 2 years and a half that the study spanned). Thus, it is observed that autonomy variability (variance over the mean) throughout a session, is higher for the ASD group, both for the child and the clinician involved.

[0136] As described above, NT children show increasing index of control with age and ASD children show a decreasing trend. Likewise, here we ask if the index of autonomy also changes with age. We examine this index as a function of age across the children. We also examine it for the clinician across the children’s ages.

[0137] FIG. 8A and FIG. 8B are plots that illustrate examples of autonomy index, according to an embodiment. Plots reflect the average child and clinician index of autonomy for left hand motions vs. age as well as right vs. left hand index of autonomy, index of autonomy variability and index of control. These plots indicate non-equivalence of the index of autonomy and the index of control.

[0138] FIG. 8A plots child (CH) index of autonomy as a function of age along the horizontal axis. This shows CH index of autonomy is positively and linearly correlated with age. We find that the child’s index of autonomy for both TD and ASD increases with age in all cases. This result reveals that the ability of the ASD child to actively participate in a dyadic interaction is a human socio-motor developmental trait that improves with age. FIG. 8B plots clinician (CL) index of autonomy as a function of child age along the horizontalDocket Number 15290-036PC0 (RU 2024-006) Patent axis. There is no discernable trend between the clinician index of autonomy across children’s ages.

[0139] The child’s index of autonomy and the variability of this index extracted from the sensors in both hands, are linearly correlated. In the case of the index of control however, there is higher variability of the mean autonomy index across subjects when we use data from the left-hand sensor, where separation of the NT from ASD is evident. For this reason, it is advantageous to focus analysis on the left-hand motions. Furthermore, the index of autonomy derived from the left-hand motions as well as its index of control, are positively correlated for small values of control index, but negatively correlated for higher values. In other words, autonomy and control are not equivalent metrics.

[0140] The experimental embodiment illustrates that males and females respond differently to ADOS tasks. The average indexes of autonomy and control were calculated across all participants, derived from samples corresponding to the different ADOS tasks. Then, potential differences between ASD and NT participants we assessed, focusing on the comparison of males vs. females. It was found that ASD males respond with lower index of autonomy than do NT males. In contrast, ASD females vs. NT females, manifest very modest differences, including three tasks with no significant differences (Social Difficulties and Annoyance, Loneliness (both Emotional type tasks) and Construction Task (Socio-motor type task)).

[0141] It is therefore clear, that ADOS tasks inherently bear a lack of differentiation between NT and ASD females, unlike their male counterparts for which the differences are large. Data were separated according to the name of the ADOS tasks. Modest differences were observed between females. Tasks with nonsignificant differences in females were Social Difficulties and Annoyance, Loneliness (both Emotional type tasks) and Construction Task (Socio-motor type task). Despite the non-significance, emotional tasks have broader spread in ASD females along the index of control than do NT females. In contrast, the index of autonomy has comparable spread for both.

[0142] As socio-motor agency is defined as the ratio of index of autonomy / index of control, this implies that across these ADOS emotional tasks, NT females have more socialDocket Number 15290-036PC0 (RU 2024-006) Patent agency than ASD females. In contrast to females, a statistically significant difference between the two male groups was observed for all tasks. ASD males shift significantly to lower values of the index of autonomy across all tasks, but socio-motor tasks are more deeply affected.

[0143] In another example embodiment, a relationship between entropy-Spike rate and NSR is determined in child-clinician dyadic interactions. Recall that entropy rate H(X) is defined by Equation 3, above. In the case that all 'Eare independently identically distributed (i.i.d.), entropy rate is given by Equation 34. 1 1 &(')=}li→m.5&('(, '0, … , '}) =}li→m.5|&('()+ &('0)+ ⋯ &('})~ = ( } li→m. }|5&('() + 5&('() + ⋯ 5&('()~ = &('() (34) In any other case the following inequality of Equation 35 holds: &('(, '0, … , '})≤ &('() (35)

[0144] The method described above outputs non-zero values (“peaks” that are gamma distributed) and zero values (“quiet moments” at the person’s average level of activity) and can be treated as the product between two processes. A gamma distributed process '^aand a binary process '^a, where t denotes discrete time. If for small time windows the process is roughly stationary, then at each point in time we have the processes '^and '^, respectively. Then, from Equation 35 we have the upper bound for the entropy rate given by Equation 36. &('a)≤ &('^'^) (36) If &('^, '^) is the joint entropy, because ^(n, ^)= n^ is a measurable function, we have Equation 37. &('^'^)≤ &('^, '^) (37) The upper bound for the joint entropy is given by Equation 38. &('^, '^) ≤ &('^) + &('^) (38) Equality holds true if and only if '^and '^are independent. Ultimately, from Equations (36 to 38), Equation 39 holds. &('a) ≤ &('^) + &('^) (39)Docket Number 15290-036PC0 (RU 2024-006) Patent

[0145] In practice, '^is a discrete approximation of a continuous gamma variable '^. If ∆ is the size of the bin used in the approximation and ℎ('^) the differential (continuous) entropy of '^, it can be shown that Equation 40 holds. ℎ|'^~ =∆li→m^(&('^, ∆) + ;^<∆) (40)ℎ|'^~ ≅ &('^, ∆) + ;^<∆ (41) Equivalently, this can be written as Equation 42. &('^) ≅ ℎ|'^~ − ;^<∆ (42) The is known to begiven by Equation 43. ℎ|'^~ = ^ + ;5P + ln (^)+ (1 − ^)¡(^) (43) where

[0146] Because of the power law between shape and scale that we discovered from analyzing human motion data and given in Equation 7, above, repeated here for convenience: ;^<^ ≅ ^;^<P + Z (7) Where ^ ≤ 0 and b are parameters that are determined by fitting a regression model on the population of interest. Because of this, differential entropy ends up being a univariate function of P, for a specific set of parameters.

[0147] For the ranges of values typically found in MMS, we see that Differential Entropy has almost a positive linear relationship with the natural logarithm of the scale, which is equivalent to the NSR, VMR of the gamma process. For greater values, that are not usually encountered in human data, the Differential entropy diverges. (Here the calculation of differential entropy for smaller values failed due to numerical instability). FIG. 9A through FIG. 9D show these results. FIG. 9A through FIG. 9D are plots that illustrate example of theoretical considerations for relationships between NSR and Entropy metrics, according to an embodiment.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0148] FIG. 9A illustrates near positive linear relationship between Differential Entropy h and the natural logarithm of the Gamma scale, equal to the NSR, VMR of the gamma process.

[0149] FIG. 9B illustrates that the empirically estimated entropy is the same as the equivalent Poisson process. Within the range of physiological data from the ADOS dyadic interaction, the entropy of the gamma component increases with the NSR,VMR and the entropy of the binary component increases with the Spike Rate.

[0150] FIG. 9C illustrates that sampling the empirical ranges across the entropy of the gamma component and the entropy of the binary component of the MMS yields the theoretical maximal entropy defining the upper bound (maximum). Their ratio indicates that the maximum entropy is greater than the empirical entropy by a quantity that is increasing as the binary entropy drops (e.g., as the spike rate decreases) and it increases as the gamma entropy increases.

[0151] FIG. 9D illustrates that the binary spike process is independent from the gamma process, suggesting that in human motion timing and amplitude (spatial) aspects of these motions are independent.

[0152] Because of Equation 42, we expect the empirical discrete entropy to also behave linearly with respect to the log-NSR, VMR which turned out to be the case. For the range of physiological values, we found a slope of 0.4694 nats (p<0.01, a nat is the fundamental unit of information, using the natural logarithm for the computation of entropy) for the differential entropy and a slope of 0.2606 nats (p<0.01) for the empirical entropy.

[0153] Moving on to the process &('^), the theoretical entropy is given by Equation 44 (if probabilities do not changei.e., spike process is Bernoulli). &('^) = −3 log 3 − (1 − 3)log(1 − 3) (44) Where 3 is the probability of a spike occurring. In practice: &('^) = −S log S − (1 − S)log(1 − S) (45) Where S is the spike rate measured as “number of spikes” per “number of samples”.

[0154] As can be seen, the empirically estimated entropy is the same as the equivalent Poisson process. As a conclusion, for the range of physiological data from the ADOS dyadicDocket Number 15290-036PC0 (RU 2024-006) Patent interaction, the entropy of the gamma component is increasing with the NSR, VMR and the entropy of the binary component is increasing with the Spike Rate.

[0155] The underlying process '^is observable only at the peak activity values. We make the hypothesis that &('^)≅ &('^|'^∈ '(), i.e. the entropy of the gamma process is approximately equal to the entropy measured by the observable continuous peaks of the micro-movement process. Note that this does not imply that &('^)= &('^|'^), i.e. that the two processes are independent.

[0156] Peak activity is separate from standardized speed. If we plot the entropy of the process &('^'^) and the maximum theoretical entropy defining the upper bound, &('^) + &('^), we see that the maximum entropy is greater than the empirical entropy by a quantity that is increasing as the binary entropy drops, or equivalently (by the previous finding) as the spike rate decreases. It increases as the gamma entropy increases. Fitting a surface function, we find Equation 46. &9^B('^, '^) − &('^'^) = −2.368&('^) + 0.8141&('^) + 1.114, S¦R4 = 0.04857 (46) Then, we get an approximate relation given by Equation 47. &('^'^) = 3.416&('^) − 0.1859&('^) − 1.114 (47)

[0157] Recall that the following inequality of Equation 48 holds from Equations 37 and 38. &('^'^) ≤ &('^, '^) ≤ &('^) + &('^) (48) Which yields Equation 49 3.368&('^) − 0.1859&('^) − 1.114 ≤ &('^, '^) ≤ &('^) + &('^) (49) Or Equation 50. 3.368[&('^)+ &('^)F− 3.1821&('^) − 1.114 ≤ &('^, '^)≤ &('^)+ &('^) Finally, yielding Equation 51.3.368&9^B('^, '^) − 3.1821&('^) − 1.114 ≤ &('^, '^) ≤ &9^B('^, '^) (51) If '^and '^were to be independent, we can get Equation 52a.Docket Number 15290-036PC0 (RU 2024-006) Patent 3.368&9^B('^, '^) − 3.1821&('^) − 1.114 = &9^B('^, '^) (52) Or Equation 52b. &9^B('^, '^) = 1.34&('^) + 0.47 (52b)

[0158] This independence criterion, is obviously data dependent. In the general case where we can fit a surface of the form of Equation 53a: &9^B('^, '^)− &('^'^) = ^&('^)+ Z&('^)+ ¨ (53a) The&9^B('^, '^) = ©1 −X^ª &('^) −«^ (53b)

[0159] smalldifferences between the intercepts of the two linear models, an easy way to quantify departure from independence is by computing the angle P between the two lines given by Equation 53c: P = tan8( ^8­J®­(^­J®(53c) ­^Then, we define the degree of departure from independence as the ratio between the P and¯0. If the data cannot be well fit by a linear model, it’s advantageous to perform a standard goodness-of-fit test and / or measure the mean error between the model and the data. If P > 0, the rate of maximum joint entropy increase over the gamma entropy is bigger than in the case of independence. If P < 0 then the rate of maximum joint entropy increase over the gamma entropy is smaller than in the case of independence.

[0160] From our data, we see that this condition holds well, with 9.93 % departure from independence for the ASD group and 11.55 % for the NT group.

[0161] This implies Equation 54. &('^, '^) ≅ &('^) + &('^) (54) This shows that the binary spike process is independent from the gamma process. This points towards the separability of time and space in human motion.

[0162] Next is shown that controllability of an agent in a dyadic social interaction is inversely proportional to autonomy. This allows leveraging socio-motor agency to protect the agent.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0163] Transfer Entropy (TE) ^^→i(^, ;) between two processes X and Y was defined above in Equation 33 and repeated here for convenience. ^^→i(^, ;) = 4[^^→i(5 + 1, ^, ;)F (33) This leads to Equation 55. ^^→i(5 + 1, ^, ;)= x(^(}^); n}^(|^(}^)) (55)

[0164] rate(which is equal to entropy rate for stationary processes) of process X and the generalized entropy rate ℎi,^of X conditioned on the source Y as given in Equations 56a through 56c. ^^→i(^, ;) = ℎi− ℎi,^(56a)

[0165] values of X, given its history and the past values of Y. Transfer Entropy is the reduction in uncertainty of predicting the future of X when we consider the process Y. If we call ℎ^,iuncertainty, then ℎ^is what we already defined as autonomy and ^i→^is the transfer entropy. We chose embedded history of length 20 for TE and for the entropy rate of our processes we used a template (embedding) length equal to the average distance between two spikes, to ensure that in the reconstructed space, the coordinates of a point in time include both zeros (“quiet moments”) and spikes and that the system doesn’t bounce back and forth from a single coordinate of zeros components. The embedding delay was chosen using Average Mutual Information.

[0166] FIG.10 and FIG. 10B are plots that illustrate examples of child or clinician autonomy with respect to the log-NSR, VMR and the Spike Rate, according to an embodiment. Non i.i.d. process is revealed by the relationship between autonomy, NSR, VMR and spike rate for clinician in FIG. 10A and child in FIG. 10B for the gamma and binary components of the MMS, relative to the process entropy. This establishes that the speed is independent of the peak activity and the positive correlation between entropy rate and NSR, VMR or Spike Rate.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0167] Next, we study how TE behaves in the shared space of the child-clinician dyad. FIG. 11A through FIG. 11C are plots that illustrate examples of a linear relationships between transfer entropy and the entropy rates (autonomies) for child and clinician differentiating between NT and ASD participants, according to an embodiment. FIG. 11A shows all clinician (CL) child (CH) interactions. FIG. 11B shows interactions with neurotypical (NT) children; and FIG. 11C shows interactions with autism disorder spectrum (ASD) children. We find that ^4^±→^²decreases when the child exhibits high autonomy and increases when the clinician has higher autonomy and vice versa for ^4^²→^±. In fact, this relationship is well characterized by linear relationships between transfer entropy and the entropy rates (autonomies), as the fitted linear surfaces indicate in FIG. 11A through FIG. 11C. In this sense, we can safely conclude that by manipulating standardized human biorhythmic time series either by increasing the NSR, VMR or by increasing peak activity, we can increase autonomy and reduce the controllability of human agents by other human or by artificial agents, including those potentially created by artificial intelligence (AI).

[0168] Next is demonstrated the Validation of the Digitization of the ADOS as an automated, streamlined and scalable screener of socio-motor agency. The digital data was validated with the clinical criteria, a paradigm that we have coined clinically interpretable digital biomarkers. In this model, the objective digital indexes that we used to define socio- motor agency as the autonomy-to-control ratio, are examined in relation to the ADOS clinical scores that a trained human rated during the session. We employ a machine learning technique, Support Vector Machine (SVM) to classify the digital data as a function of the clinical score. Then we apply tools from signal detection theory, specifically the receiving operating characteristic curve, ROC, to assess the validity of our classifier.

[0169] Each of 26 participants with the full ADOS session (digital and clinical) produces on average between 50 – 60 minutes of time series digital data from biosensors registering motion at 128Hz. We used the left-hand wrist sensor in these analyses, as we showed that it is highly correlated with the right wrist, yet more variable, thus expanding our sampling space. Upon exploration of several time windows to segment the data, sweeping across theDocket Number 15290-036PC0 (RU 2024-006) Patent time series and tasks, while maximizing statistical power in each locally stationary segment, we arrived at 7.8 second windows as advantageous.

[0170] The data were validated using the Leave-one-person-out cross-validation (LOOCV) method. As features for our classifier, we used autonomy (entropy rate), NSR, VMR and the embedding delay of the data, which is the time scale at which deterministic properties arise and characterize the dynamical behavior of motion. Two classifiers were used, one trained on female subjects and the second one trained exclusively on male participants. When trying to digitally diagnose autism in one participant, we train our classifier on the data from the remaining male or female participants and then test how accurately the trained model predicts the participant class (NT vs. ASD). This method avoids overfitting and trains models that can diagnose autism in novel participants, thus automating the screening process. Digitizing the ADOS in this way makes the diagnosis of autism more inclusive of females, historically underdiagnosed by a test that we objectively showed has biases towards males across all tasks.

[0171] FIG. 12A and FIG. 12B are plots that illustrate effectiveness of autonomy-to-control ratio as a clinical diagnosis tool for detecting ASD in both male and female children, according to an embodiment. FIG. 12A plots the positive diagnosis rates as a function of ADOS scores. Support Vector Machines (SVM) classifiers were trained on all subjects except one and tested on the remaining subjects of the same sex (Leave-one-person-out cross-validation (LOOCV)). Therefore, each of the 26 subjects was digitally diagnosed with a classifier trained on a different dataset, which ensured zero overfitting and bias. Training and testing features were the entropy rate, the signal-to-noise ratio, and the embedding delay (the time scale at which a dynamical system behaves in the most deterministic way) calculated on normalized speed samples of ~7.8 secs duration windows. Here, we report the percentage of time windows per subject that gave a positive diagnostic label and plot them versus the ADOS scores, as determined by the clinicians. As shown in Figure 12A, there is no confusion of our biometrics about the clinician ADOS scores, which classify ASD males with 100% accuracy and performs remarkably well for ASD vs. TD females.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0172] FIG. 12B plots the ROC curve with true positive rates on vertical axis against false positive rate along horizontal axis. We use the positive rate scores as a metric used to diagnose ASD and report the Receiver Operating Characteristic curve (ROC curve), which shows the true positive and false positive rates of the digital diagnostic tool we developed for different thresholds. The Area Under the Curve (AUC) is 0.9576, which indicates great performance. FIG. 12B confirms the validity of these biometrics for clinical use.

[0173] To reiterate, in these example embodiments, autonomy was defined as the non-parametric entropy rate spanning from totally random to totally deterministic behavior of standardized micro-movements spike trains derived from nuanced fluctuations in motion data. Control was defined in terms of the NSR, VMR empirically estimated from such spike trains as well, such that high regimes of NSR, VMR correspond to the memoryless regimes denoting high uncertainty (poor predictability and randomness) in the motor code. We took a step further to examine the parameterization of the MMS as a binary-spike and a Gamma process and demonstrated the independence between them. The binarized sequences of spikes bear a motor code whereby the observer may or may not be able to predict and therefore control the observed agent. At high randomness, the observed agent affords more autonomy than at deterministic ranges. At deterministic ranges, the observer can predict and control the actions of the observed agent. At higher NSR, VMR the agent has lower self-control, as predicting his / her / their motor actions consequences can then be compromised by noise. In this case then, the observer can also exert higher control over the observed agent and the person’s socio-motor agency become compromised.

[0174] Underlying both indexes and the ratio of autonomy to control are then discrete pockets of information making up a continuous stream of dyadic motor code, contributed by both social agents. Thus, we can infer the existence of an underlying shared alphabet in the motor code that manifests during dyadic social interactions of the type studied here. Agents with discrete motor signatures that appear more random are thus harder to control and behave more autonomously and independently than agents with systematically predictable motions sharing their codes.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0175] Besides describing new biometrics of shared socio-motor agency in dyadic social interactions, our analyses showed ways to streamline the ADOS test, thus making it less taxing on the child and the clinician. A handful of tasks affording more socio-motor agency to the child can indeed uncover the social readiness potential of the child rather than biasing the diagnosis by the clinician towards a deficit model. Along those lines, using these newly defined indexes of dyadic autonomy and control, we demonstrated fundamental differences across the tasks for males and females, thus confirming that despite previously quantified differences in motor control separating males and females at the voluntary [4] and involuntary [12; 13] levels, the ADOS remains biased towards males. These digital indexes of shared socio-motor agency, nevertheless, used within the context of an unbiased ML classifier, could detect the differences between males and females for both the NT and ASD randomly chosen participants. This digitized automated version of the test resembles the type of scenario that a clinician faces at the clinic, any given day. Namely, a random arrival of a case that the clinician may see for the first time. In that sense, the leave-one-person-out classifier provides robust digital screening of autism and may be a way to scale our pilot study to encompass larger numbers of NT, ASD participants across ages, sexes, and do so longitudinally as well.

[0176] The theoretical considerations at the intersection of stochastic analyses and information theoretic approaches with non-linear dynamics offers the MMS and analyses as a viable way to obtain the personalized signatures of autonomy and control and tweak the NSR to mask the spike trains derived from the person’s physiological biorhythmic activity. This ability to separate the binary spike rate code from the gamma process denoting levels of randomness vs. predictability, enables the person in a dyadic social exchange to attain socio- motor agency by enhancing autonomy and avoiding external control by the other agent, be that agent another human or an AI-driven one. 2.4 Example filtering.

[0177] This methodology can be used to protect privacy from surveillance systems, as ultimately these systems rely on biometric data, which can now, using the presentDocket Number 15290-036PC0 (RU 2024-006) Patent personalized methods, be manipulated to hide our fingerprint-like MMS signatures from an external agent trying to control us.

[0178] FIG. 16A through FIG. 16C, are diagrams that illustrate an example of a method that filters imagery before, during or after transmission to mitigate control in a video interaction, according to an embodiment.

[0179] FIG. 16A depicts a training phase of the method. Videos from multiple subjects (at 25 Hz frame rate) are used to calculate Approximate Entropy and log-SNR 1301 throughout the course of each video for sliding time windows of approximately 3.6 seconds, by use of open source face video processing software. The computation takes place for each of the face landmarks that the software generates, specifically the Micro-Movement transformation of speed vectors of each landmark. The socio-motor index pairs are used to construct a parameter space. Quadrants are constructed based on the median values of Approximate Entropy and log-SNR.

[0180] Each quadrant corresponds to a categorical class the parameter divisions for which are stored in a data structure 1311. Approximate Entropy measures Autonomy and log-SNR measures Control. Thus, we have the four categorical classes: 1. High Autonomy Low Control 2. High Autonomy High Control 3. Low Autonomy Low Control 4 Low Autonomy High Control. (A.3),

[0181] A Conditional Generative Adversarial Network (cGAN) 1317 is trained on 128x128 resolution face frames from the videos 1313. In 1315, each frame is labeled according to the categorical classes defined in data structure 1311. If the frame is sampled from a time window which has received one of the four classifications, the frame belongs to the same class. The network, with class as input and video frame as output of the network, is trained on all frames from all videos and the class labels of each frame. Once the network 1317 is trained within the desired accuracy, it will receive as input noise and a class label, and it will generate random novel images that resemble the original frames that belong to that class. The AI generated image 1324 output by 1317 is, if similar enough as determined in 1325, blended with the subject’s actual image 1321 in a separate process 1327 as described below.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0182] FIG. 16B depicts a filtering phase of the method. Input is a few seconds video 1321 of a participants face, from which we extract a frame at each point in time. [The goal is to generate a new video (filtered video 1330) that resembles as much as possible the original video 1321 but for which the Approximate Entropy and Control indexes belong in a input desired class 1322 of the four classes. For example, focusing on the first frame of the video 1321, at time t=0 . We choose the desired class 1322 and feed it along with random noise 1323 into the pre-trained cGAN 1317b. Call O the original frame of video 1321, and call P the frame 1327 output from the trained network 1317b.

[0183] In process 1325 the Structural Similarity Index Measure (SSIM) is computed between the two images, a measure developed in the field of Computer Vision that measures how closely the frame / image O resembles P. Values closer to 1 indicate higher similarity compared to values further away. For a preselected tolerance level, SSIM_tol, we check if the output SSIM≥ SSIM_tol. If the condition is satisfied, then we keep P. Otherwise, repeat the process of generating an image 1324 from trained network 1317b until we get a frame P that satisfies that condition.

[0184] In process 1327, a weighted average of P and O, as aP+(1-a)O, is produced. The constant a is initially chosen to be close to 1. In process 1329, the Peak Signal-to-Noise Ratio (PSNR) between the original frame O and the weighted average aP+(1-a)O is computed. This will measure how much noise is added to the original frame O during our approximation aP+(1-a)O. We want PSNR[aP+(1-a)O,O]≥ PSNR_tol , where PSNR_tol is a preselected tolerance level. Also in process 1329, when it is determined that PSNR[aP+(1-a)O,O]≥ PSNR_tol, the filtered frame aP+(1-a)O is kept and added to the output video 1330. Otherwise, α is reduced by a small amount Δa to obtain a new value a. Repeat 1327 and 1329 until the PSNR_tol is exceeded.

[0185] The process continues by choosing the next frame of video 1321 at time t←t+1 and repeating the operations of 1317b, 1325, 1327 and 1329, as described above for the first frame. Once there are no more frames left, concatenate all filtered frames a_t P_t+(1-a_t ) O_t to produce the filtered video 1330.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0186] FIG. 16C depicts results from an evaluation phase. Original and Filtered face video are used to compute the average Approximate Entropy (Autonomy) over all face landmarks, for sliding time windows of approximately 3.6 seconds in plot 1340. Approximate Entropy is averaged over time, for each landmark for the original video depicted in 1341 and for the filtered video in 1342. Original and Filtered face video are used to compute the average log- SNR (Control) over all face landmarks, for sliding time windows of approximately 3.6 seconds in plot 1350. The log-SNR (Control) is averaged over time, for each landmark for the original video depicted in 1351 and for the filtered video in 1352.

[0187] This solution to the controllability issue can then be extended from individuals to dyads, from dyads to social groups and from social groups to society. In this sense, socio- motor agency can serve as a foundation for societal agency, now quantifiable using the methods that we offer in this work.

[0188] We have uncovered new indexes of shared, dyadic autonomy and control, objectively defined socio-motor agency and provided new means to automate its digital screening with already routinely used clinical tools. This works offers novel ways to scale our clinical science and make it actionable, diverse, and inclusive at more than one level. 3. Computational Hardware Overview

[0189] FIG. 14 is a block diagram that illustrates a computer system 1400 upon which an embodiment of the invention may be implemented. Computer system 1400 includes a communication mechanism such as a bus 1410 for passing information between other internal and external components of the computer system 1400. Information is represented as physical signals of a measurable phenomenon, typically electric voltages, but including, in other embodiments, such phenomena as magnetic, electromagnetic, pressure, chemical, molecular atomic and quantum interactions. For example, north and south magnetic fields, or a zero and non-zero electric voltage, represent two states (0, 1) of a binary digit (bit). Other phenomena can represent digits of a higher base. A superposition of multiple simultaneous quantum states before measurement represents a quantum bit (qubit). A sequence of one or more digits constitutes digital data that is used to represent a number or code for a character. In some embodiments, information called analog data is represented byDocket Number 15290-036PC0 (RU 2024-006) Patent a near continuum of measurable values within a particular range. Computer system 1400, or a portion thereof, constitutes a means for performing one or more steps of one or more methods described herein.

[0190] A sequence of binary digits constitutes digital data that is used to represent a number or code for a character. A bus 1410 includes many parallel conductors of information so that information is transferred quickly among devices coupled to the bus 1410. One or more processors 1402 for processing information are coupled with the bus 1410. A processor 1402 performs a set of operations on information. The set of operations include bringing information in from the bus 1410 and placing information on the bus 1410. The set of operations also typically include comparing two or more units of information, shifting positions of units of information, and combining two or more units of information, such as by addition or multiplication. A sequence of operations to be executed by the processor 1402 constitutes computer instructions.

[0191] Computer system 1400 also includes a memory 1404 coupled to bus 1410. The memory 1404, such as a random access memory (RAM) or other dynamic storage device, stores information including computer instructions. Dynamic memory allows information stored therein to be changed by the computer system 1400. RAM allows a unit of information stored at a location called a memory address to be stored and retrieved independently of information at neighboring addresses. The memory 1404 is also used by the processor 1402 to store temporary values during execution of computer instructions. The computer system 1400 also includes a read only memory (ROM) 1406 or other static storage device coupled to the bus 1410 for storing static information, including instructions, that is not changed by the computer system 1400. Also coupled to bus 1410 is a non-volatile (persistent) storage device 1408, such as a magnetic disk or optical disk, for storing information, including instructions, that persists even when the computer system 1400 is turned off or otherwise loses power.

[0192] Information, including instructions, is provided to the bus 1410 for use by the processor from an external input device 1412, such as a keyboard containing alphanumeric keys operated by a human user, or a sensor. A sensor detects conditions in its vicinity andDocket Number 15290-036PC0 (RU 2024-006) Patent transforms those detections into signals compatible with the signals used to represent information in computer system 1400. Other external devices coupled to bus 1410, used primarily for interacting with humans, include a display device 1414, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), for presenting images, and a pointing device 1416, such as a mouse or a trackball or cursor direction keys, for controlling a position of a small cursor image presented on the display 1414 and issuing commands associated with graphical elements presented on the display 1414.

[0193] In the illustrated embodiment, special purpose hardware, such as an application specific integrated circuit (IC) 1420, is coupled to bus 1410. The special purpose hardware is configured to perform operations not performed by processor 1402 quickly enough for special purposes. Examples of application specific ICs include graphics accelerator cards for generating images for display 1414, cryptographic boards for encrypting and decrypting messages sent over a network, speech recognition, and interfaces to special external devices, such as robotic arms and medical scanning equipment that repeatedly perform some complex sequence of operations that are more efficiently implemented in hardware.

[0194] Computer system 1400 also includes one or more instances of a communications interface 1470 coupled to bus 1410. Communication interface 1470 provides a two-way communication coupling to a variety of external devices that operate with their own processors, such as printers, scanners and external disks. In general, the coupling is with a network link 1478 that is connected to a local network 1480 to which a variety of external devices with their own processors are connected. For example, communication interface 1470 may be a parallel port or a serial port or a universal serial bus (USB) port on a personal computer. In some embodiments, communications interface 1470 is an integrated services digital network (ISDN) card or a digital subscriber line (DSL) card or a telephone modem that provides an information communication connection to a corresponding type of telephone line. In some embodiments, a communication interface 1470 is a cable modem that converts signals on bus 1410 into signals for a communication connection over a coaxial cable or into optical signals for a communication connection over a fiber optic cable. As another example, communications interface 1470 may be a local area network (LAN) card to provide a dataDocket Number 15290-036PC0 (RU 2024-006) Patent communication connection to a compatible LAN, such as Ethernet. Wireless links may also be implemented. Carrier waves, such as acoustic waves and electromagnetic waves, including radio, optical and infrared waves travel through space without wires or cables. Signals include man-made variations in amplitude, frequency, phase, polarization or other physical properties of carrier waves. For wireless links, the communications interface 1470 sends and receives electrical, acoustic or electromagnetic signals, including infrared and optical signals, that carry information streams, such as digital data.

[0195] The term computer-readable medium is used herein to refer to any medium that participates in providing information to processor 1402, including instructions for execution. Such a medium may take many forms, including, but not limited to, non-volatile media, volatile media and transmission media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 1408. Volatile media include, for example, dynamic memory 1404. Transmission media include, for example, coaxial cables, copper wire, fiber optic cables, and waves that travel through space without wires or cables, such as acoustic waves and electromagnetic waves, including radio, optical and infrared waves. The term computer-readable storage medium is used herein to refer to any medium that participates in providing information to processor 1402, except for transmission media.

[0196] Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic medium, a compact disk ROM (CD-ROM), a digital video disk (DVD) or any other optical medium, punch cards, paper tape, or any other physical medium with patterns of holes, a RAM, a programmable ROM (PROM), an erasable PROM (EPROM), a FLASH-EPROM, or any other memory chip or cartridge, a carrier wave, or any other medium from which a computer can read. The term non-transitory computer-readable storage medium is used herein to refer to any medium that participates in providing information to processor 1402, except for carrier waves and other signals.

[0197] Logic encoded in one or more tangible media includes one or both of processor instructions on a computer-readable storage media and special purpose hardware, such as ASIC 1420.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0198] Network link 1478 typically provides information communication through one or more networks to other devices that use or process the information. For example, network link 1478 may provide a connection through local network 1480 to a host computer 1482 or to equipment 1484 operated by an Internet Service Provider (ISP). ISP equipment 1484 in turn provides data communication services through the public, world-wide packet-switching communication network of networks now commonly referred to as the Internet 1490. A computer called a server 1492 connected to the Internet provides a service in response to information received over the Internet. For example, server 1492 provides information representing video data for presentation at display 1414.

[0199] The invention is related to the use of computer system 1400 for implementing the techniques described herein. According to one embodiment of the invention, those techniques are performed by computer system 1400 in response to processor 1402 executing one or more sequences of one or more instructions contained in memory 1404. Such instructions, also called software and program code, may be read into memory 1404 from another computer-readable medium such as storage device 1408. Execution of the sequences of instructions contained in memory 1404 causes processor 1402 to perform the method steps described herein. In alternative embodiments, hardware, such as application specific integrated circuit 1420, may be used in place of or in combination with software to implement the invention. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.

[0200] The signals transmitted over network link 1478 and other networks through communications interface 1470, carry information to and from computer system 1400. Computer system 1400 can send and receive information, including program code, through the networks 1480, 1490 among others, through network link 1478 and communications interface 1470. In an example using the Internet 1490, a server 1492 transmits program code for a particular application, requested by a message sent from computer 1400, through Internet 1490, ISP equipment 1484, local network 1480 and communications interface 1470. The received code may be executed by processor 1402 as it is received or may be stored in storage device 1408 or other non-volatile storage for later execution, or both. In this manner,Docket Number 15290-036PC0 (RU 2024-006) Patent computer system 1400 may obtain application program code in the form of a signal on a carrier wave.

[0201] Various forms of computer readable media may be involved in carrying one or more sequence of instructions or data or both to processor 1402 for execution. For example, instructions and data may initially be carried on a magnetic disk of a remote computer such as host 1482. The remote computer loads the instructions and data into its dynamic memory and sends the instructions and data over a telephone line using a modem. A modem local to the computer system 1400 receives the instructions and data on a telephone line and uses an infra-red transmitter to convert the instructions and data to a signal on an infra-red a carrier wave serving as the network link 1478. An infrared detector serving as communications interface 1470 receives the instructions and data carried in the infrared signal and places information representing the instructions and data onto bus 1410. Bus 1410 carries the information to memory 1404 from which processor 1402 retrieves and executes the instructions using some of the data sent with the instructions. The instructions and data received in memory 1404 may optionally be stored on storage device 1408, either before or after execution by the processor 1402.

[0202] FIG. 15 illustrates a chip set 1500 upon which an embodiment of the invention may be implemented. Chip set 1500 is programmed to perform one or more steps of a method described herein and includes, for instance, the processor and memory components described with respect to FIG. 14 incorporated in one or more physical packages (e.g., chips). By way of example, a physical package includes an arrangement of one or more materials, components, and / or wires on a structural assembly (e.g., a baseboard) to provide one or more characteristics such as physical strength, conservation of size, and / or limitation of electrical interaction. It is contemplated that in certain embodiments the chip set can be implemented in a single chip. Chip set 1500, or a portion thereof, constitutes a means for performing one or more steps of a method described herein.

[0203] In one embodiment, the chip set 1500 includes a communication mechanism such as a bus 1501 for passing information among the components of the chip set 1500. A processor 1503 has connectivity to the bus 1501 to execute instructions and process information storedDocket Number 15290-036PC0 (RU 2024-006) Patent in, for example, a memory 1505. The processor 1503 may include one or more processing cores with each core configured to perform independently. A multi-core processor enables multiprocessing within a single physical package. Examples of a multi-core processor include two, four, eight, or greater numbers of processing cores. Alternatively, or in addition, the processor 1503 may include one or more microprocessors configured in tandem via the bus 1501 to enable independent execution of instructions, pipelining, and multithreading. The processor 1503 may also be accompanied with one or more specialized components to perform certain processing functions and tasks such as one or more digital signal processors (DSP) 1507, or one or more application-specific integrated circuits (ASIC) 1509. A DSP 1507 typically is configured to process real-world signals (e.g., sound) in real time independently of the processor 1503. Similarly, an ASIC 1509 can be configured to performed specialized functions not easily performed by a general purposed processor. Other specialized components to aid in performing the inventive functions described herein include one or more field programmable gate arrays (FPGA) (not shown), one or more controllers (not shown), or one or more other special-purpose computer chips.

[0204] The processor 1503 and accompanying components have connectivity to the memory 1505 via the bus 1501. The memory 1505 includes both dynamic memory (e.g., RAM, magnetic disk, writable optical disk, etc.) and static memory (e.g., ROM, CD-ROM, etc.) for storing executable instructions that when executed perform one or more steps of a method described herein. The memory 1505 also stores the data associated with or generated by the execution of one or more steps of the methods described herein.

[0205] FIG. 16 is a diagram of exemplary components of a mobile terminal 1600 (e.g., cell phone handset) for communications, which is capable of operating in the system of FIG. 1A, according to one embodiment. In some embodiments, mobile terminal 1601, or a portion thereof, constitutes a means for performing one or more steps described herein. Generally, a radio receiver is often defined in terms of front-end and back-end characteristics. The front- end of the receiver encompasses all of the Radio Frequency (RF) circuitry whereas the back- end encompasses all of the base-band processing circuitry. As used in this application, the term “circuitry” refers to both: (1) hardware-only implementations (such as implementationsDocket Number 15290-036PC0 (RU 2024-006) Patent in only analog and / or digital circuitry), and (2) to combinations of circuitry and software (and / or firmware) (such as, if applicable to the particular context, to a combination of processor(s), including digital signal processor(s), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions). This definition of “circuitry” applies to all uses of this term in this application, including in any claims. As a further example, as used in this application and if applicable to the particular context, the term “circuitry” would also cover an implementation of merely a processor (or multiple processors) and its (or their) accompanying software / or firmware. The term “circuitry” would also cover if applicable to the particular context, for example, a baseband integrated circuit or applications processor integrated circuit in a mobile phone or a similar integrated circuit in a cellular network device or other network devices.

[0206] Pertinent internal components of the telephone include a Main Control Unit (MCU) 1603, a Digital Signal Processor (DSP) 1605, and a receiver / transmitter unit including a microphone gain control unit and a speaker gain control unit. A main display unit 1607 provides a display to the user in support of various applications and mobile terminal functions that perform or support the steps as described herein. The display 1607 includes display circuitry configured to display at least a portion of a user interface of the mobile terminal (e.g., mobile telephone). Additionally, the display 1607 and display circuitry are configured to facilitate user control of at least some functions of the mobile terminal. An audio function circuitry 1609 includes a microphone 1611 and microphone amplifier that amplifies the speech signal output from the microphone 1611. The amplified speech signal output from the microphone 1611 is fed to a coder / decoder (CODEC) 1613.

[0207] A radio section 1615 amplifies power and converts frequency in order to communicate with a base station, which is included in a mobile communication system, via antenna 1617. The power amplifier (PA) 1619 and the transmitter / modulation circuitry are operationally responsive to the MCU 1603, with an output from the PA 1619 coupled to the duplexer 1621 or circulator or antenna switch, as known in the art. The PA 1619 also couples to a battery interface and power control unit 1620.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0208] In use, a user of mobile terminal 1601 speaks into the microphone 1611 and his or her voice along with any detected background noise is converted into an analog voltage. The analog voltage is then converted into a digital signal through the Analog to Digital Converter (ADC) 1623. The control unit 1603 routes the digital signal into the DSP 1605 for processing therein, such as speech encoding, channel encoding, encrypting, and interleaving. In one embodiment, the processed voice signals are encoded, by units not separately shown, using a cellular transmission protocol such as enhanced data rates for global evolution (EDGE), general packet radio service (GPRS), global system for mobile communications (GSM), Internet protocol multimedia subsystem (IMS), universal mobile telecommunications system (UMTS), etc., as well as any other suitable wireless medium, e.g., microwave access (WiMAX), Long Term Evolution (LTE) networks, code division multiple access (CDMA), wideband code division multiple access (WCDMA), wireless fidelity (WiFi), satellite, and the like, or any combination thereof.

[0209] The encoded signals are then routed to an equalizer 1625 for compensation of any frequency-dependent impairments that occur during transmission though the air such as phase and amplitude distortion. After equalizing the bit stream, the modulator 1627 combines the signal with a RF signal generated in the RF interface 1629. The modulator 1627 generates a sine wave by way of frequency or phase modulation. In order to prepare the signal for transmission, an up-converter 1631 combines the sine wave output from the modulator 1627 with another sine wave generated by a synthesizer 1633 to achieve the desired frequency of transmission. The signal is then sent through a PA 1619 to increase the signal to an appropriate power level. In practical systems, the PA 1619 acts as a variable gain amplifier whose gain is controlled by the DSP 1605 from information received from a network base station. The signal is then filtered within the duplexer 1621 and optionally sent to an antenna coupler 1635 to match impedances to provide maximum power transfer. Finally, the signal is transmitted via antenna 1617 to a local base station. An automatic gain control (AGC) can be supplied to control the gain of the final stages of the receiver. The signals may be forwarded from there to a remote telephone which may be another cellular telephone, any other mobile phone or aDocket Number 15290-036PC0 (RU 2024-006) Patent land-line connected to a Public Switched Telephone Network (PSTN), or other telephony networks.

[0210] Voice signals transmitted to the mobile terminal 1601 are received via antenna 1617 and immediately amplified by a low noise amplifier (LNA) 1637. A down-converter 1639 lowers the carrier frequency while the demodulator 1641 strips away the RF leaving only a digital bit stream. The signal then goes through the equalizer 1625 and is processed by the DSP 1605. A Digital to Analog Converter (DAC) 1643 converts the signal and the resulting output is transmitted to the user through the speaker 1645, all under control of a Main Control Unit (MCU) 1603 which can be implemented as a Central Processing Unit (CPU) (not shown).

[0211] The MCU 1603 receives various signals including input signals from the keyboard 1647. The keyboard 1647 and / or the MCU 1603 in combination with other user input components (e.g., the microphone 1611) comprise a user interface circuitry for managing user input. The MCU 1603 runs a user interface software to facilitate user control of at least some functions of the mobile terminal 1601 as described herein. The MCU 1603 also delivers a display command and a switch command to the display 1607 and to the speech output switching controller, respectively. Further, the MCU 1603 exchanges information with the DSP 1605 and can access an optionally incorporated SIM card 1649 and a memory 1651. In addition, the MCU 1603 executes various control functions required of the terminal. The DSP 1605 may, depending upon the implementation, perform any of a variety of conventional digital processing functions on the voice signals. Additionally, DSP 1605 determines the background noise level of the local environment from the signals detected by microphone 1611 and sets the gain of microphone 1611 to a level selected to compensate for the natural tendency of the user of the mobile terminal 1601.

[0212] The CODEC 1613 includes the ADC 1623 and DAC 1643. The memory 1651 stores various data including call incoming tone data and is capable of storing other data including music data received via, e.g., the global Internet. The software module could reside in RAM memory, flash memory, registers, or any other form of writable storage medium known in the art. The memory device 1651 may be, but not limited to, a single memory, CD, DVD, ROM,Docket Number 15290-036PC0 (RU 2024-006) Patent RAM, EEPROM, optical storage, magnetic disk storage, flash memory storage, or any other non-volatile storage medium capable of storing digital data.

[0213] An optionally incorporated SIM card 1649 carries, for instance, advantageous information, such as the cellular phone number, the carrier supplying service, subscription details, and security information. The SIM card 1649 serves primarily to identify the mobile terminal 1601 on a radio network. The card 1649 also contains a memory for storing a personal telephone number registry, text messages, and user specific mobile terminal settings.

[0214] In some embodiments, the mobile terminal 1601 includes a digital camera comprising an array of optical detectors, such as charge coupled device (CCD) array 1665. The output of the array is image data that is transferred to the MCU for further processing or storage in the memory 1651 or both. In the illustrated embodiment, the light impinges on the optical array through a lens 1663, such as a pin-hole lens or a material lens made of an optical grade glass or plastic material. In the illustrated embodiment, the mobile terminal 1601 includes a light source 1661, such as a LED to illuminate a subject for capture by the optical array, e.g., CCD 1665. The light source is powered by the battery interface and power control module 1620 and controlled by the MCU 1603 based on instructions stored or loaded into the MCU 1603. 4. Alternatives, Deviations and modifications

[0215] In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. Throughout this specification and the claims, unless the context requires otherwise, the word “comprise” and its variations, such as “comprises” and “comprising,” will be understood to imply the inclusion of a stated item, element or step or group of items, elements or steps but not the exclusion of any other item, element or step or group of items, elements or steps. Furthermore, the indefinite article “a” or “an” is meant to indicate one or more of the item, element or step modified by the article.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0216] Notwithstanding that the numerical ranges and parameters setting forth the broad scope are approximations, the numerical values set forth in specific non-limiting examples are reported as precisely as possible. Any numerical value, however, inherently contains certain errors necessarily resulting from the standard deviation found in their respective testing measurements at the time of this writing. Furthermore, unless otherwise clear from the context, a numerical value presented herein has an implied precision given by the least significant digit. Thus, a value 1.1 implies a value from 1.05 to 1.15. The term ”about” is used to indicate a broader range centered on the given value, and unless otherwise clear from the context implies a broader range around the least significant digit, such as “about 1.1” implies a range from 1.0 to 1.2. If the least significant digit is unclear, then the term “about” implies a factor of two, e.g., “about X” implies a value in the range from 0.5X to 2X, for example, about 100 implies a value in a range from 50 to 200. Moreover, all ranges disclosed herein are to be understood to encompass any and all sub-ranges subsumed therein. For example, a range of "less than 10" for a positive only parameter can include any and all sub- ranges between (and including) the minimum value of zero and the maximum value of 10, that is, any and all sub-ranges having a minimum value of equal to or greater than zero and a maximum value of equal to or less than 10, e.g., 1 to 4. 5. References

[0217] The following references are hereby incorporated by refernce as if fully set forther herein, except for terminology inconsistent with that used herein. [1] C. Lord, S. Risi, L. Lambrecht, E.H. Cook, Jr., B.L. Leventhal, P.C. DiLavore, A. Pickles, and M. Rutter, The autism diagnostic observation schedule-generic: a standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord 30 (2000) 205-23. [2] K. Gotham, S. Risi, A. Pickles, and C. Lord, The Autism Diagnostic Observation Schedule: revised algorithms for improved diagnostic validity. J Autism Dev Disord 37 (2007) 613-27.Docket Number 15290-036PC0 (RU 2024-006) Patent [3] E.B. Torres, M. Brincker, R.W. Isenhower, P. Yanovich, K.A. Stigler, J.I. Nurnberger, D.N. Metaxas, and J.V. Jose, Autism: the micro-movement perspective. Front Integr Neurosci 7 (2013) 32. [4] E.B. Torres, R.W. Isenhower, P. Yanovich, G. Rehrig, K. Stigler, J. Nurnberger, and J.V. Jose, Strategies to develop putative biomarkers to characterize the female phenotype with autism spectrum disorders. J Neurophysiol 110 (2013) 1646-62. [5] A.M. D'Mello, I.R. Frosch, C.E. Li, A.L. Cardinaux, and J.D.E. Gabrieli, Exclusion of females in autism research: Empirical evidence for a "leaky" recruitment-to-research pipeline. Autism Res (2022). [6] S. Lundstrom, C. Marland, R. Kuja-Halkola, H. Anckarsater, P. Lichtenstein, C. Gillberg, and T. Nilsson, Assessing autism in females: The importance of a sex-specific comparison. Psychiatry Res 282 (2019) 112566. [7] R. Loomes, L. Hull, and W.P.L. Mandy, What Is the Male-to-Female Ratio in Autism Spectrum Disorder? A Systematic Review and Meta-Analysis. J Am Acad Child Adolesc Psychiatry 56 (2017) 466-474. [8] E. Somoza, and D. Mossman, ROC curves and the binormal assumption. J Neuropsychiatry Clin Neurosci 3 (1991) 436-9. [9] M. Hollander, and E.A. Pena, Nonparametric Methods in Reliability. Stat Sci 19 (2004) 644- 651.

[0010] E.B. Torres, R. Rai, S. Mistry, and B. Gupta, Hidden Aspects of the Research ADOS Are Bound to Affect Autism Science. Neural Comput 32 (2020) 515-561.

[0011] H. Bokadia, R. Rai, and E.B. Torres, Digitized Autism Observation Diagnostic Schedule: Social Interactions beyond the Limits of the Naked Eye. J Pers Med 10 (2020).

[0012] E.B. Torres, and K. Denisova, Motor noise is rich signal in autism research and pharmacological treatments. Sci Rep 6 (2016) 37422.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0013] E.B. Torres, S. Mistry, C. Caballero, and C.P. Whyatt, Stochastic Signatures of Involuntary Head Micro-movements Can Be Used to Classify Females of ABIDE into Different Subtypes of Neurodevelopmental Disorders. Front Integr Neurosci 11 (2017) 10.

[0014] M. Brincker, and E.B. Torres, Noise from the periphery in autism. Front Integr Neurosci 7 (2013) 34.

[0015] E.B. Torres, H. Varkey, J. Vero, E. London, H. Phan, P. Kittler, A. Gordon, R.E. Delgado, C.F. Delgado, and E.A. Simpson, Sensing echoes: temporal misalignment in auditory brainstem responses as the earliest marker of neurodevelopmental derailment. PNAS nexus 2 (2023) pgac315.

[0016] E.B. Torres, Two classes of movements in motor control. Exp Brain Res 215 (2011) 269- 83.

[0017] E.B. Torres, R.W. Isenhower, J. Nguyen, C. Whyatt, J.I. Nurnberger, J.V. Jose, S.M. Silverstein, T.V. Papathomas, J. Sage, and J. Cole, Toward Precision Psychiatry: Statistical Platform for the Personalized Characterization of Natural Behaviors. Front Neurol 7 (2016) 8.

[0018] E.B. Torres, C. Caballero, and S. Mistry, Aging with Autism Departs Greatly from Typical Aging. Sensors (Basel) 20 (2020).

[0019] C.E. Shannon, and W. Weaver, The mathematical theory of communication, University of Illinois Press, Urbana, 1998.

[0020] C.E. Shannon, Claude Elwood Shannon papers, 1932-1995 (bulk 1938-1989), pp. 7,000.

[0021] C.E. Shannon, N.J.A. Sloane, and A.D. Wyner, Claude Elwood Shannon : miscellaneous writings, Mathematical Sciences Research Center, AT&T Bell Laboratories, Murray Hill, N.J., 1993.

[0022] C.E. Shannon, N.J.A. Sloane, A.D. Wyner, and IEEE Information Theory Society., Claude Elwood Shannon : collected papers, IEEE Press, New York, 1993.Docket Number 15290-036PC0 (RU 2024-006) Patent

[0023] J. Shi, L. Chen, and K. Aihara, Embedding entropy: a nonlinear measure of dynamical causality. J R Soc Interface 19 (2022) 20210766.

[0024] E. Tan, S. Algar, D. Correa, M. Small, T. Stemler, and D. Walker, Selecting embedding delays: An overview of embedding techniques and a new method using persistent homology. Chaos 33 (2023) 032101.

[0025] F. Takens, Detecting strange attractors in turbulence. in: D.A.R.a.L.-S. Young, (Ed.), Dynamical Systems and Turbulence, Lecture Notes in Mathematics, Springer-Verlag, 1981, pp. 366–381.

[0026] A. Delgado-Bonal, and A. Marshak, Approximate Entropy and Sample Entropy: A Comprehensive Tutorial. Entropy (Basel) 21 (2019).

[0027] J.T. Lizier, JIDT: an information-theoretic toolkit for studying the dynamics of complex systems. Frontiers in Robotics and AI 1 (2014) 1-20.

[0028] N.H. Packard, J.P. Crutchfield, J.D. Farmer, and R.S. Shaw, Geometry from a time series. Physical Review Letters 45 (1980) 712-716.

[0029] E.B. Torres, Atypical signatures of motor variability found in an individual with ASD. Neurocase 19 (2013) 150-65.

[0030] E.B. Torres, Atypical signatures of motor variability found in an individual with ASD. Neurocase 19 (2011) 150-65.

[0031] M. Brincker, and E.B. Torres, Chapter 1- Why Study Movement Variability in Autism_. in: E.B. Torres, and C. Whyatt, (Eds.), Autism : the movement sensing perspective, CRC Press / Taylor & Francis Group, Boca Raton, 2018, pp. xviii, 386 pages.

[0032] S.N. Mohamed Thangal, and J.M. Donelan, Scaling of inertial delays in terrestrial mammals. PLoS One 15 (2020) e0217188. ________________________________________________

Claims

Docket Number 15290-036PC0 (RU 2024-006) Patent CLAIMS What is claimed is:

1. A method for determining control in digital interactions, the method comprising: recording a time series of digital sub-second sensor data for each of a plurality of subjects; determining a mean and variance of a feature of micropeaks in the time series of digital sub-second sensor data; and determining an autonomy and control score for each subject of the plurality of subjects based at least in part on the mean and variance of the feature.

2. The method as recited in claim 1, further comprising transforming a micropeak occurrence feature of the micropeaks to a binary time series, wherein said determining autonomy and control score for each subject further comprises determining autonomy and control score for each subject of the plurality of subjects also based at least in part on entropy rate of the binary time series.

3. The method as recited in claim 2, further comprising determining transfer entropy between a pair of the binary time series, wherein said determining autonomy and control score for each subject further comprises determining autonomy and control score for each subject of the plurality of subjects also based at least in part on the transfer entropy.

4. The method as recited in claim 1, further comprising: filtering a subsequent time series of digital sub-second sensor data from a first subject of the plurality of subjects to increase an entropy rate for the subsequent time series; and transmitting the filtered subsequent time series to a different subject of the plurality of subjects.Docket Number 15290-036PC0 (RU 2024-006) Patent 5. The method as recited in claim 1, wherein the time series of digital sub-second sensor data includes one or more of video data, audio data, movement data of one or more appendages, movement data for one or more facial poses, electrical voltage or current data at one or more electrodes, or brain wave data, or some combination.

6. A non-transitory computer-readable medium carrying one or more sequences of instructions for determining control in digital interactions, wherein execution of the one or more sequences of instructions by one or more processors causes the one or more processors to perform the steps of the method of at least one of claims 1 to 5.

7. An apparatus for determining control in digital interactions, the apparatus comprising: at least one processor; and at least one memory including one or more sequences of instructions, the at least one memory and the one or more sequences of instructions configured to, with the at least one processor, cause the apparatus to perform the steps of the method of at least one of claims 1 to 5.

8. A system for determining control in digital interactions, the system comprising: a sensor; at least one processor; and at least one memory including one or more sequences of instructions, the at least one memory and the one or more sequences of instructions configured to, with the at least one processor, cause the apparatus to perform the steps of the method of at least one of claims 1 to 5.

Images