Computer control by distortion of facial geometry

Abstract

To provide a system that can freely focus a user's visual attention on something other than the system.SOLUTION: A system detects, with data provided by one or more sensors, a user's alteration of the geometries of parts of the face, head, neck, and / or shoulders. The system determines the extent of each alteration and normalizes it with respect to the maximum possible range of each alteration so as to assign a numeric score indicative of its extent to each part-specific alteration. The normalized part-specific scores are combined so as to produce a composite numeric code representative of the complete set of simultaneously-executed geometric alterations. Each composite code is translated or interpreted relative to an appropriate context defined by an embodiment, an application executing on an embodiment, or the user. For example, each composite code may be interpreted as, or assigned to, a specific alphanumeric letter, a color, a musical note and the like.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] (No corresponding description) [Background technology]

[0002] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of the filing date of co-pending Provisional Application No. 62 / 125,758, filed January 31, 2015, the entire contents of which are incorporated herein by reference.

[0003] Computerized devices are common today. Such devices include editing documents, exchanging verbal information (e.g., "telephone"), creating and displaying digital graphics and photographs, financial transactions (e.g., "automated teller machines" (ATMs) and "point of sale" (POS) terminals), playing interactive and multiplayer games, and the like. Users typically interact with, control, and send data to these computerized devices through the controls of a physical device. Examples include a user pressing keys on a physical keyboard, touching a screen when a virtual keyboard is displayed, moving a computer mouse with their hands, and the like. Users also interact with computerized devices by speaking verbal commands and data. Summary of the Invention [Problem to be solved by the invention]

[0004] In the system of the present invention, a user communicates data and / or commands to a computerized device while fully utilizing the user's hands and voice for other tasks and freeing the user to focus their visual attention on things other than the system. [Means for solving the problem]

[0005] A system disclosed herein includes at least one sensor configured to measure a condition of a portion of a user's face, head, neck, and / or shoulders, and a conversion circuit configured to convert at least one of the portion-specific measurements of the condition generated by the at least one sensor into at least one symbolic value and input at least one of the at least one symbolic value to at least one processor that modifies at least one component and / or behavior of at least one computing device.

[0006] As used herein, a method and / or device for detecting and / or determining whether or not a face, head, neck, and / or shoulder geometry of at least one user is provided, comprising: (a) at least one sensor configured to capture at least one block of data relating to at least one factor that changes contemporaneously with at least one change in geometry of at least one user's face, head, neck, and / or shoulders, the at least one geometric change being manifested via innervation of at least one nerve of at least one user by the at least one user, the innervation of the at least one nerve resulting in contraction of at least one muscle, the contraction of the at least one muscle being contemporaneous with the manifestation of the at least one change in geometry of the at least one user's face, head, neck, and / or shoulders; A system is disclosed that includes, at least in part, a sensor that contributes to the at least one of the at least one blocks of data; (b) means for determining at least one factor-specific measurement from at least one portion of at least one of the at least one blocks of data; (c) means for quantizing each of the at least one factor-specific measurement of at least one of the at least one users to generate at least one "quantized factor measure"; (d) means for converting the at least one quantized factor measure into at least one user-specific "change signature"; (e) means for mapping the at least one change signature to at least one "symbolic value"; and (f) means for processing the at least one symbolic value. This system The system may further include a first processor including a determining means, a quantizing means, a transforming means, and a mapping means, and a second processor including a processing means.

[0007] Further disclosed herein is a method including: using at least one sensor to measure the extent to which certain muscles located at least partially within a user's face, head, neck, and / or shoulders contract, thereby deriving a plurality of unique muscle measurements; quantizing each of the unique muscle measurements; combining the quantized unique muscle measurements into a unique composite number for each unique combination of quantized muscle contractions; converting the composite number into a context-specific symbolic value; and using the context-specific value to modify at least one bit in a file or buffer of digital information and / or modify the behavior of at least one computing device.

[0008] Further disclosed herein is a system for detecting user-initiated changes in the geometry of face, head, neck, and / or shoulder parts using data provided by one or more sensors. The extent of each change is determined and normalized with respect to the maximum possible range of each change, and each part-specific change is assigned a numerical score indicative of its extent. The normalized part-specific scores are combined to generate a composite numerical code representing the complete set of concurrently performed geometric changes. Each composite code is translated or interpreted for an appropriate context defined by the embodiment, by an application executed by the embodiment, or by the user. For example, each composite code may be interpreted as or assigned to a specific alphanumeric character, color, musical note, etc. Using this system, a user can communicate data and / or commands to a computerized device while fully utilizing the user's hands and voice for other tasks and freeing the user to focus visual attention on things other than the system.

[0009] For a fuller understanding of the nature and objects of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0010] [Figure 1]FIG. 1 illustrates an exemplary embodiment of the present disclosure, in which a user uses only face, head, neck, and shoulder curvatures to generate commands for an operating system of the embodiment or for applications running within the embodiment. [Diagram 2] Sagittal, frontal, and transverse views used to define and explain the relative positions of muscles and other features of the human body and the direction of displacement of those features. [Diagram 3] A diagram of some of the muscles that cause expressions, movements, and changes to the geometry of the human face. This diagram can be found in "Sobotta's Atlas and Text-book of Human Anatomy" published in 1909. The diagram was created by K. Hajek and A. Schmitson. [Figure 4] FIG. 1 is a diagram of seven user-controlled body signals used by an exemplary embodiment of the present disclosure. This diagram is intended only to characterize these signals, but one may envision the diagram as representing the types of images that may be captured by a camera incorporated into or connected to an embodiment. [Diagram 5] 5 is a diagram of the same user shown in FIG. 4, but now communicating a signal to an exemplary embodiment and generating a signal value. [Figure 6] FIG. 1 illustrates an exemplary embodiment of the present disclosure, in which a user creates music using only four-signal movements. Muscle contractions that the user specifies as the four signals are detected and measured with sensors attached to the skin. [Figure 7] FIG. 1 illustrates an exemplary embodiment of the present disclosure, in which a user generates text (e.g., a text message) using only displacements of selected parts of the geometry of his or her face, head, neck, and shoulders. [Figure 8] FIG. 1 is a diagram of a user using an exemplary embodiment of the present disclosure, the device being similar in size to a smartphone. [Figure 9]1 is a flow chart illustrating some of the steps performed by an exemplary embodiment of the present disclosure, which allows a user to input commands and / or data to the embodiment, its operating system, and / or applications running therein (i.e., to the "consumers") through the generation of "signals," which then process each composite signal value. [Figure 10] 1 is a flow chart illustrating some of the steps performed by an exemplary embodiment of the present disclosure to allow a user to input commands and / or data to the embodiment, its operating system, and / or applications running therein (i.e., to a "consumer") through the generation of a "signal." In this embodiment, a user can customize the encoder by initializing it with a file that specifies the user's preferences, e.g., specifying the signal characteristics the user prefers. [Figure 11-1] FIG. 2 illustrates the differences between “face tracking”, “eye tracking” and / or “retina tracking” systems and the present disclosure. [Figure 11-2] FIG. 2 illustrates the differences between “face tracking”, “eye tracking” and / or “retina tracking” systems and the present disclosure. [Figure 11-3] FIG. 2 illustrates the differences between “face tracking”, “eye tracking” and / or “retina tracking” systems and the present disclosure. [Figure 11-4]11_1 and 11_2, a change in the orientation of the user's face, body, and / or "direction of gaze" is sufficient to change the value input to the system (e.g., the horizontal (y) screen coordinate of the cursor) in response to a change in the user's appearance from the system's viewpoint. In contrast, as shown in FIG. 11_3 and 11_4, a change in the orientation of the user's face, body, and / or "direction of gaze" is not sufficient to change the value generated within an exemplary embodiment of the present disclosure relative to the spatial configuration of the user's face and / or body; only a change in the spatial configuration of the user's face and / or body relative to the user itself can change the value encoded. [Figure 12-1] FIG. 2 illustrates the differences between “face tracking”, “eye tracking” and / or “retina tracking” systems and the present disclosure. [Figure 12-2] FIG. 2 illustrates the differences between “face tracking”, “eye tracking” and / or “retina tracking” systems and the present disclosure. [Figure 12-3] FIG. 2 illustrates the differences between “face tracking”, “eye tracking” and / or “retina tracking” systems and the present disclosure. [Figure 12-4] 12_1 and 12_2, a change in the orientation of the user's face and / or body relative to the retina tracking system and / or its camera can result in the assignment of the same value (e.g., horizontal (y) screen coordinate of the cursor) if it coincides with a "complementary" change in "gaze direction". In contrast, a change in the spatial configuration of the user's face and / or body parts relative to the inherent reference frame of the user's body, as shown in 12_3 and 12_4, will always result in the assignment of a different value according to an exemplary embodiment of the present disclosure (which is equally impractical in either case unless a valid code is specified in either configuration). [Figure 13-1]13A-13C illustrate the constancy in values ​​interpreted by an exemplary embodiment of the present disclosure in response to a user moving relative to the embodiment and / or its camera. [Figure 13-2] 13A-13C illustrate the constancy in values ​​interpreted by an exemplary embodiment of the present disclosure in response to a user moving relative to the embodiment and / or its camera. [Figure 13-3] 13A-13C illustrate the constancy in values ​​interpreted by an exemplary embodiment of the present disclosure in response to a user moving relative to the embodiment and / or its camera. [Figure 14-1] FIG. 1 illustrates the constancy of values ​​interpreted by an exemplary embodiment of the present disclosure in response to a user's acceleration, either absolutely (i.e., relative to the Earth) or relative to the embodiment, device, and / or its camera. [Figure 14-2] 14_1 illustrates the constancy of values ​​interpreted by an exemplary embodiment of the present disclosure in response to the acceleration of a user, either absolutely (i.e., relative to the Earth) or relative (i.e., relative to the embodiment, device, and / or its camera). A "head orientation tracking" system, such as that shown in FIG. 14_1, changes the values ​​input to the system, and / or the system's interpretation of those values, in response to the acceleration (absolute and / or relative) of the user, the user's head, and / or the system, device, and / or its camera. In contrast, an exemplary embodiment of the present disclosure generates inputs only in response to the spatial configuration (of face, head, body, etc.) generated by the user relative to the user's original body frame(s) of reference. Thus, for this exemplary embodiment, the acceleration (absolute and / or relative) of the user, the user's head, and / or the embodiment and / or its camera does not change the values ​​generated or the subsequent interpretation by the embodiment if the spatial configuration of the user's face, head, body, etc. remains constant. [Figure 15-1]1 illustrates the constancy of values ​​generated and / or interpreted by an exemplary embodiment of the present disclosure in response to movement of the user, the user's face, or any other body part of the user relative to the embodiment and / or its sensors (also in response to "back and forth" motions). [Figure 15-2] 1 illustrates the constancy of values ​​generated and / or interpreted by an exemplary embodiment of the present disclosure in response to movement of the user, the user's face, or any other body part of the user relative to the embodiment and / or its sensors (also in response to "back and forth" motions). [Figure 15-3] 1 illustrates the constancy of values ​​generated and / or interpreted by an exemplary embodiment of the present disclosure in response to movement of the user, the user's face, or any other body part of the user relative to the embodiment and / or its sensors (also in response to "back and forth" motions). [Figure 15-4] 1 illustrates the constancy of values ​​generated and / or interpreted by an exemplary embodiment of the present disclosure in response to movement of the user, the user's face, or any other body part of the user relative to the embodiment and / or its sensors (also in response to "back and forth" motions). [Figure 15-5] 1 illustrates the constancy of values ​​generated and / or interpreted by an exemplary embodiment of the present disclosure in response to movement of the user, the user's face, or any other body part of the user relative to the embodiment and / or its sensors (also in response to "back and forth" motions). [Figure 16] FIG. 1 illustrates the generation of a series of values ​​by a user in accordance with the present disclosure. [Figure 17] FIG. 1 illustrates the utility of an exemplary embodiment of the present disclosure. [Figure 18] 1 illustrates the communication of the American Sign Language spelling of the word "friend." The signs shown are produced and communicated by the presentation of "static" (i.e., unmoving, unchanging) hand configurations. [Figure 19-1] FIG. 1 illustrates the communication of eight different words through the use of American Sign Language. [Figure 19-2]FIG. 1 illustrates the communication of eight different words through the use of American Sign Language. [Figure 19-3] FIG. 1 illustrates the communication of eight different words through the use of American Sign Language. [Figure 19-4] FIG. 1 illustrates the communication of eight different words through the use of American Sign Language. [Figure 19-5] FIG. 1 illustrates the communication of eight different words through the use of American Sign Language. [Figure 19-6] FIG. 1 illustrates the communication of eight different words through the use of American Sign Language. [Figure 19-7] FIG. 1 illustrates the communication of eight different words through the use of American Sign Language. [Figure 19-8] Figure 1 illustrates the communication of eight different words through the use of American Sign Language. The signs illustrated are produced and communicated through the presentation of "dynamic" (i.e., moving, changing) hand configurations, where the presentation of any static parts is expected to violate the protocol and prevent its comprehension by a third party. [Figure 20] FIG. 1 illustrates an ATM embodiment of the present disclosure in which a customer communicates their password, or a portion thereof, to the embodiment via changes in the geometry of their face, head, neck, and / or shoulders. [Figure 21] FIG. 1 illustrates an embodiment of the present disclosure in which a customer communicates with a third party (e.g., another person) by symbolic values ​​encoded by the customer himself through changes in the geometry of his face, head, neck, and / or shoulders, and thus can be communicated to the third party even from within an environment that is so noisy that voice communications may not be understood by the third party. [Figure 22] FIG. 1 illustrates an embodiment in which an artist composes a digital image by modifying the geometry of the face, head, neck, and / or shoulders, issuing color values ​​encoded by the artist to an embodiment in which the artist is also using gestures to simultaneously control the movement of a virtual paintbrush. [Diagram 23]FIG. 1 illustrates an embodiment in which a gamer adjusts parameters suitable for controlling the gamer's own virtual player in a game by modifying the geometry of the face, head, neck, and / or shoulders. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] A. Introduction The present disclosure relates to computer system architectures and designs that allow for modifications to the geometric shapes associated with the head, neck, and / or upper body of a user and / or one or more other humans, to be used as a means of communicating and / or inputting values ​​into a computerized device and / or software executed thereon. The present disclosure includes detection and quantification of specific parts or aspects of such altered shapes, the innervations that cause them, and / or their consequences (e.g., local changes in blood flow). Once detected, each such specific part of the overall geometric configuration is measured and associated with a "score." While each of the individual scores can be used directly as input to a computer processor, it is generally preferred to convert the complete set of scores into a single digital value that the processor can interpret as a command or data value, e.g., as an ASCII code.

[0012] The systems disclosed herein facilitate "direct" input of "individual" values ​​by a user into the system. Unlike systems described in the prior art, some embodiments of the present disclosure provide users with the ability to input individual "user-understood" and "user-intended" values ​​into such embodiments without the use or involvement of the user's hands or voice, and regardless of the user's position and / or movement relative to the embodiment. Furthermore, some embodiments of the present disclosure provide users with the ability to input individual values ​​into such embodiments without the user's access to, awareness of, and operation of a user interface (e.g., a "virtual" keyboard displayed by an output device) dedicated to supporting the specification, configuration, and / or submission of such values.

[0013] With respect to an exemplary embodiment of the system herein, the exemplary system detects coded signals intentionally manifested by the user through contraction and / or relaxation of specific muscles and / or muscle groups in the user's face, head, neck, and / or shoulders. The extent to which the exemplary system determines that each targeted muscle and / or muscle group is contracted is quantized to one of two or more numerical values, exponents, and / or their equivalents. The individual quantized muscle-specific numbers are then scaled to a predetermined and / or equivalent in order to generate a generic "composite" value that the exemplary system converts to a "value" specific to a particular embodiment and / or application. or are combined in a pre-specified manner. The resulting "values" may represent particular alphanumeric characters, particular words, emojis, particular colors, textures, particular musical notes, chords, and / or tones, etc.

[0014] Some embodiments of the present disclosure provide a user with an alternative and / or additional channel that allows the user to control and / or interact with a computerized device through and / or with a computerized device, which is a new user-controlled channel for input of user-specified values ​​into a computerized system and / or one or more applications executing thereon.

[0015] 1. Potential Embodiments Examples of systems herein include, but are not limited to, the following: A user of the exemplary embodiments of the present disclosure herein: using both hands for different purposes, or having paralysis and / or missing hands and / or fingers, You use your vision to observe something other than the system's output display (if any), or you are visually impaired and / or blind; and You use your voice to communicate with someone or something outside the system, or you lack the ability to speak or want to avoid using your voice, In case, The user may also, by selective contraction of appropriate muscles in their face, head, neck, and / or shoulders, Specifying and / or controlling parameters, attributes, and / or settings (e.g., colors, dimensions, rates, textures, etc.) of exemplary embodiments; Specifying and / or structuring text (e.g., for use in documents, text messages, chat utilities, etc.); Specifying colors (e.g., for use in graphic arts and / or image creation and / or editing applications); Specifying musical notes, tones, chords, tempos, etc. (e.g., for use in music creation, editing, sharing and / or streaming applications);

[0016] Specifying a password and / or passphrase to be used to gain and / or maintain access to the exemplary embodiment, such as where the embodiment also provides functionality such as an "Automated Teller Machine" (ATM), a "Point of Sale" (POS) terminal, an Automated Airport Ticket and / or Boarding Pass (ATB) dispenser, etc.;

[0017] Specifically, but not exclusively, specifying messages, such as by a hearing-impaired user (who may have difficulty articulating words due to hearing loss), to a portable embodiment that articulates certain words to facilitate the user's oral communication with the hearing person or people with whom the hearing-impaired user is communicating;

[0018] For example, where an exemplary embodiment is a modified telephone, speaking specific words to generate a maximally understandable verbal representation of a particular message to another person (e.g., on the other end of the telephone call), designating a message by a person who has suffered damage to a portion of the glottal apparatus (e.g., pharynx, tongue, etc.) and / or is in the process of recovering from such damage, etc.

[0019] Message transmission, such as by a person in a noisy environment operating a tool requiring the use of both hands, but where it is desirable or necessary to communicate text and / or simulated verbal messages and / or a stream of such messages to another person or system; Specify the message,

[0020] Designating a telephone number to be dialed, such as by a visually impaired and / or blind person, thereby initiating a telephone call according to an exemplary embodiment including telephony capabilities and functions; Navigating the directory structure of the exemplary embodiments by a quadriplegic or the like who is unable to consciously control any muscles of the body other than the muscles of the face, and who perhaps cannot speak clearly enough to verbally control such computerized devices due to distortions caused by reliance on a ventilator;

[0021] For example, when the user is a user, driver, and / or passenger of a vehicle, issuing commands (e.g., "slow down"), requests (e.g., "use air conditioning to reduce cabin temperature"), instructions (e.g., "prioritize fuel conservation"), and / or parameters (e.g., destination) to example embodiments that control all or a portion of the operation of the vehicle (e.g., steering, speed, braking);

[0022] navigating within and among the virtual and / or symbolic directory structures of the exemplary embodiments (such as when the embodiments also operate as a personal computer, computerized television, gaming system, database, server, etc.), for example, enabling a user to gain virtual access rights to particular directories and / or "folders" and / or open and / or initiate editing of files and / or execute or start applications or programs and / or control playback and / or display of media;

[0023] issuing commands and / or requests where the user is a building occupant, e.g., where the user's facial code is visible by one or more cameras in communication with the exemplary embodiments or with parts of the exemplary embodiments, and where, e.g., the exemplary embodiments constitute a centralized environmental control system and the cameras are mounted on the walls and / or ceilings of the building, e.g., one occupant may request an increase in the ambient temperature by communicating a facial code specifying the message "it's too hot" and another occupant may request a decrease in the ambient light levels by communicating a facial code specifying the message "it's too bright";

[0024] setting up and / or configuring local and / or private settings in example embodiments that provide functionality of the gaming environment while the user simultaneously controls aspects of game play with their own hands, monitors game-related events on an output device, and verbally communicates strategic comments to other game players;

[0025] Controlling games and / or other entertainment programs in which the issuance of a facial code by a user, and the resulting generation of a coded value, is an integral part of the game play and / or experience, e.g., in an exemplary embodiment, a game in which music is generated by users and scored for quality and originality;

[0026] sending to the exemplary embodiments data regarding a patient's symptoms, treatment, measurements (e.g., temperature), diagnosis, and / or prognosis that is generated, discovered, and / or communicated by a medical professional during the examination and / or treatment of the patient, such as when the user is a medical professional, particularly when the medical professional is wearing protective clothing that restricts the movement of their hands and mouth, such as while treating a patient in an area contaminated with a virulent pathogen (e.g., the Ebola virus);

[0027] sending to the exemplary embodiment data regarding a patient's symptoms, measurements, diagnoses, and / or prognoses generated, discovered, and / or communicated during an examination of that patient, such as when the user is a medical professional, to avoid alerting the patient with early disclosure of preliminary but potentially significant observations;

[0028] Controlling the dynamic configuration of media and / or textual information in an exemplary environment where the user is, for example, a teacher in a classroom scenario, where manual control and / or issuance of voice commands by the teacher would distract or interfere with the student's focus on the lesson being taught; Controlling the exemplary embodiment as a robot, particularly as a robot designed to facilitate a user's ability to simulate fantasies, functioning without voice or manual control when voice or manual control would be a hindrance to simulating the fantasies;

[0029] Controlling the exemplary embodiment functioning as a robot, in particular a robot that must be controlled from a distance in a noisy environment (e.g., a factory) by an operator whose hands and attention must be dedicated to controlling and manipulating other (possibly dangerous) objects; can be done.

[0030] Exemplary embodiments of the present disclosure utilize one or more sensors to detect and provide a basis for measuring the extent to which a user moves, twists, and / or otherwise alters certain aspects of the geometry of their face, head, neck, and / or shoulders. Through appropriate nerve innervation and appropriate muscle contraction, the user can generate distinct signals that the embodiments can recognize, measure, and uniquely identify. The embodiments ignore any geometry that they do not recognize. The embodiments then convert each recognized user-specific signal into a symbolic value appropriate to the design of the embodiments, the operation of a body of computer code that runs in conjunction with the embodiments' operating system (e.g., a text editor), and / or the task that the user of the embodiments is attempting to accomplish and / or complete.

[0031] The determining means, quantizing means, converting means, mapping means, and processing means each involve specialized manipulations of digital data and require appropriate specialized computer instructions / code capable of accomplishing those operations. Those computer codes may be executed on a single processor, e.g., in a serial manner, or may be executed on two or more processors, providing the possibility that some or all of the operations may be performed in parallel (i.e., streams of data may be moved from processor to processor and multiple streams may be processed simultaneously).

[0032] 2. Innervation A nerve is "innervated" when the exchange of ions across the cell membrane transmits an electric current along at least one axon of the nerve.

[0033] 3. Muscle Contraction When a muscle contracts, it typically shortens in length and at least increases in cross-sectional area within the muscle, causing the muscle to tense. Muscles contract in response to appropriate nerve innervation. Contraction is accompanied by the release of calcium ions and is associated with an electrical voltage that can be detected by electromyography.

[0034] 4. Conversion of "relative displacement" to "symbolic value" Exemplary embodiments of the present disclosure detect and measure changes in the structure and / or shape of a user's face, head, neck, and / or shoulders. The user's face, head, neck, and / or shoulder portions include portions (i.e., "source portions") that the user can displace relative to at least one other portion (i.e., "reference portion"). The displacement of each source portion is measured relative to at least one corresponding reference portion (i.e., "relative displacement"). The portion monitored and measured by one embodiment may be different from the portion monitored by one or more other embodiments.

[0035] Exemplary embodiments of the present disclosure determine the extent to which a user displaces one or more particular portions of the user's face, head, neck, and / or shoulders. These embodiments respond to determining a collection of measurements that are characteristic of a particular, characteristic set of portion-specific dislocations that characterize the user's face, head, neck, and / or shoulders at a particular moment in time by generating a "value."

[0036] Each value generated by one of these embodiments in response to the determination and / or generation by this embodiment of a particular collection of part-specific relative dislocation measurements represents a symbolic entity that is processed, executed, and / or responded to in a manner characteristic of, and appropriate to, the purpose, functionality, design, and / or utility of the respective embodiment and / or of the application or program executed by the respective embodiment.

[0037] The set of embodiment-specific and / or application-specific "values" associated with each unique collection of portion-specific relative displacement measurements may include, but are not limited to, alphanumeric characters, numbers, colors (e.g., RGB color value specifications), iconic graphics, embodiment-specific and / or application-specific commands (e.g., "save file"), words (e.g., "apple"), phrases (e.g., "Best Regards, Joe Smith"), or any other data values ​​that can be processed by an embodiment.

[0038] However, the areas of a user's face, head, neck, and / or shoulders that the exemplary embodiments may monitor and measure, and the relative displacements that form the basis of user-specified embodiment-specific and / or application-specific values ​​that the embodiments subsequently process, are not uniform.

[0039] Different users have different faces, heads, necks, and shoulders. Thus, when different users perform the same displacement, for example when the user fully raises his eyebrows, the resulting measurements of the relative displacement will be different. In other words, source and reference parts of different sizes, geometric shapes, and relative orientations are characterized by different measures (magnitudes) of their relative displacement.

[0040] Therefore, in order to standardize the partial-specific relative displacement measurements that form the basis for generating a "value," exemplary embodiments of the present disclosure "quantize" these measurements, so that the same type of relative displacement, e.g., "moving eyebrows all the way up," "opening mouth slightly," etc., results in partial-specific relative displacement measurements that lead to the generation of the same value.

[0041] 5. Measurable parts Exemplary embodiments of the present disclosure use external features of the user's face, head, neck, and / or shoulders as source and reference features. These source features and corresponding reference features include, but are not limited to: [Table 1]

[0042] Exemplary embodiments of the present disclosure use the muscles of the user's face, head, neck, and / or shoulders as source parts. There is no reference part, since displacement is measured relative to the degree of muscle contraction.

[0043] Exemplary embodiments of the present disclosure use nerves in the user's face, head, neck, and / or shoulders as source portions, and there is no reference portion since translocation is measured relative to the degree of intrinsic nerve innervation.

[0044] 6. Differences from retinal tracking Some embodiments of the present disclosure differ from "retinal tracking" systems in which a user controls the position of a cursor on a display device by looking at a desired cursor position, and perhaps controlling a simulated "click" of a mouse button at any particular cursor position. Such retinal tracking systems, by definition, involve user control of elements of a user interface. And, without a user interface that displays a cursor to be controlled, the utility and use of a retinal tracking system would be impossible.

[0045] In contrast, some embodiments of the present disclosure do not require a user interface, and if the embodiments of the present disclosure were used to control a user interface, they would only be able to do so to the extent that the control exerted was mediated through the user's specification of discrete values ​​(e.g., the letter "a", the number "81", the color "blue", etc.) that were well-defined, fully understood, and consciously generated by the user. This is contrary to the experience of a user who controls cursor position through retinal tracking (or use of a mouse), who typically, if not entirely, does not know the coordinates (in terms of the display device's coordinate system) of the current cursor position, much less the coordinates of the location on the display device to which the cursor is pointed.

[0046] 7. Differences from detecting and interpreting body movements Some embodiments of the present disclosure are distinct from systems in which a user controls a user interface with respect to the user's detected movements and / or accelerations, particularly when those movements and / or accelerations of the user are relative to the system and / or its sensors (i.e., not necessarily related to the user himself / herself). For example, a system that modifies a display to adjust the user's viewpoint within a "virtual world" in response to the user's head and / or eye movements and / or accelerations operates by matching the user's gaze changes in the real world with the user's gaze in the virtual world (e.g., to facilitate the user's exploration of the virtual world through images displayed to the user's eyes from inside specially equipped goggles or glasses and / or to improve the user's satisfaction). However, in this case, the user has not generated a specific muscle-mediated code to achieve synchrony of the viewpoints in the real and virtual worlds, and in fact, the user of such a system may not even be aware of the mechanisms by which that synchronization is effected, much less have knowledge of, or the means to consciously change, the signals sent from the system sensors to the part of the system responsible for adjusting the viewpoint of the output device.

[0047] 8. Differences from "gesture" detection and interpretation Some embodiments of the present disclosure differ from systems that respond to "gestures" performed by a user. Such gestural actions are typically interpreted by such systems in terms of orientation relative to the system and / or its sensors. In contrast, some embodiments of the present disclosure detect and respond only to static configurations of specific muscles and / or muscle groups, regardless of any movement of the user and / or the user's body parts. In these exemplary embodiments, the muscle configurations that encode the numbers, and their associated context-specific values, are defined at "moments" in time. And by definition, movement must be present and detected in relation to an "interval" of time, not an "instant."

[0048] 9. Differences from "sign language" detection and interpretation Some embodiments of the present disclosure differ from systems that respond to "signs" performed by a user through hand-mediated signals (i.e., by "signs" characteristic of "sign languages" such as "American Sign Language" (ASL)). In contrast, some embodiments of the present disclosure detect and respond to configurations of specific muscles and / or muscle groups located only on the face, head, neck, and / or shoulders. These exemplary embodiments keep the user's hands completely free to operate, including but not limited to operating a keyboard, mouse, joystick, etc., so that the user is free to use his or her hands for some other purpose.

[0049] Also, some of the signs used in sign languages ​​such as ASL involve repetitive movements. For example, in ASL, the following signs rely on defining and characteristic movements and cannot be conveyed in the absence of such movements: "fear", "sadness", "please", "sorry", "more", "hello", "goodbye", "thank you", "yes", "no", "can't hear", "listen", "people", "friends", "he / they", etc. The letters "j" and "z", and the names of the days of the week (e.g., "Monday", "Tuesday", etc.) cannot be performed correctly without certain characteristic hand movements. In contrast, some embodiments of the present disclosure detect and respond only to static configurations of certain muscles and / or muscle groups, regardless of any movement of the user and / or the user's body parts.

[0050] 10. Differences from physically operated input devices Some embodiments of the present disclosure differ from systems that require a user to specify values ​​(e.g., alphanumeric characters) through the interaction and / or manipulation of physical devices and / or “peripherals” such as a keyboard, touch screen, trackball, mouse, etc.

[0051] Pressing and releasing keys on a keyboard allows a user to input certain discrete values ​​(e.g., the letters "g", "r", and "w"). A user's use of a keyboard to input discrete values ​​may occur in the absence of a user interface that provides feedback. However, the user interface requires that the user input discrete values ​​whether through the use and control of a cursor control device such as a trackball, mouse, or through the use of an alternative to a cursor such as a touch screen, because these devices require the user to select and / or "click" or "touch" discrete values ​​displayed on an interface screen.

[0052] In contrast, some embodiments of the present disclosure do not require a user to manipulate a physical device or peripheral to specify the discrete values. These example embodiments do not require the availability or provision of a user interface, or a display or output device of any kind.

[0053] 11. Differences from voice input Some embodiments of the present disclosure are distinct from systems that detect, interpret, and / or respond to words spoken by a user. Some embodiments of the present disclosure are similar to such verbal control systems in that discrete values ​​may be entered into the system in the absence of a user interface and in the absence of physical contact between the user and the system, e.g., the user touching or pressing a key on the system. However, unlike these "verbal" systems, these exemplary embodiments do not require the user to speak or make any kind of sound, which is advantageous for many situations and many potential uses.

[0054] 12. Additional input channels, not necessarily exclusive In addition to allowing a user to communicate encoded values ​​through facial expressions (i.e., through muscle contractions and muscle-mediated changes to the shape and / or relative position of the user's facial, head, neck, and / or shoulder features), some embodiments of the present disclosure may incorporate additional traditional input channels, such as a keyboard, mouse, touch screen, detectors of user speech, etc. Embodiments of the present disclosure need not operate in the absence of the "traditional" input channels described in the prior art. Conversely, they may incorporate detection and conversion of facially encoded values ​​into embodiments as an additional (but not the only) channel through which a user may interact with those embodiments.

[0055] 13. Detecting Contractile Coding Signals Some embodiments of the present disclosure can detect contractions of target muscles and / or muscle groups in a variety of ways and through the use of a variety of sensors and / or sensor types, including but not limited to: Visually, by one or more cameras, through detection of changes in the relative distances of various surface features of the head, face, neck, and / or shoulders from each other and / or from one or more "landmark" features (such as the tip of the nose); Visually, by one or more cameras, through detection of changes in "shading" of surface features and / or contours associated with the user's head, face, neck, and / or shoulders, indicative of changes in height and / or shape of those features and / or contours; Visually, through detection of changes in the three-dimensional shape of the user's head, face, neck, and / or shoulders, which is built on image data collected from one or more cameras; Electrically, through the detection of muscle contractions and / or the detection of the innervation that causes those contractions; Through detection of changes in shape, stretch, and / or twist by stretch-responsive sensors; Magnetically, through the use of sensors that detect and / or measure changes in magnetic fields associated with the electric fields that arise in association with muscle contractions and / or innervation of the nerves involved in those contractions; and Thermally, via visual detection of changes in thermal irradiance from the user's skin using a camera sufficiently sensitive to infrared wavelengths; It can be detected.

[0056] 14. One or more values A user of the exemplary system herein may communicate many reduced encoded values ​​to the exemplary system, for example, when composing text or music. Alternatively, the user may communicate a single value. For example, when used in conjunction with a facial recognition based security system, the exemplary system may display a letter, number, or color to the user, who may then "straightforward" that code to the system by encoding it via appropriate contractions of the necessary muscles in his or her face, head, neck, and / or shoulders. This allows the exemplary secure system to have greater confidence that the image of the user's face used to at least partially identify the user is in fact authentic and not a stolen image of the true user sent by a hacker to defeat the security of the exemplary system and steal the true user's money and / or identity.

[0057] 15. Differentiating adjacent values Some embodiments of the present disclosure detect and interpret (i.e., transform, modify, map, etc.) the instantaneous configurations of specific muscles and / or muscle groups that are differentially (i.e., stepwise, variably, separately, etc.) contracted. Such exemplary embodiments can distinguish each value-specific configuration of muscle contraction from its temporally adjacent (i.e., before and after) configurations in a series of "sequential" (i.e., temporally ordered and temporally distinct) value-specific configurations in a number of different ways and / or in a number of different manners, including, but not limited to, the following:

[0058] Each unique value Respond to and / or process (i.e., convert into an embodiment, and / or application context specific value) each recognizable and valid configuration of contraction detected, measured, and / or recognized within a particular set of face, head, neck, and / or shoulder muscles occurring within any particular image within a series, or stream, of images generated by at least one of the cameras of the present embodiment, possibly avoiding duplicate processing of contraction encoded values ​​until after a new value or no valid value has been detected.

[0059] Each value that exists at least for a given period Respond to and / or process (i.e., convert into an embodiment, and / or application context specific value) each recognizable and valid configuration of contraction detected, measured, and / or recognized within a particular set of facial, head, neck, and / or shoulder muscles maintained by the user for at least a certain minimum period of time (e.g., during a certain number of consecutive images in the stream of images), and do not repeat responding to the same contraction encoding value and / or processing such value until a different, or invalid, contraction configuration is detected, measured, and / or recognized.

[0060] Each value occurs after at least a certain latency. Respond to and / or process (i.e., convert into an embodiment- and / or application-situation-specific value) each recognizable and valid configuration of contraction detected, measured, and / or recognized within a particular set of facial, head, neck, and / or shoulder muscles that occurs after a certain minimum interval (but not before) since detection of the previous value, but signal an error after a certain maximum interval since detection of the previous value.

[0061] 16. Feedback from the system Exemplary embodiments of the present disclosure herein may provide feedback to a user issuing muscle contraction encoded values ​​in a variety of ways, including but not limited to: Displaying the value detected by the present embodiment (e.g., the letter "e", the color "green", etc.) on an output device screen and / or in one or more locations of a graphical display. By By performing a change or behavior of the embodiment that is detectable to the user, such as by issuing an audible note; by changing the color of one or more pixels on a graphical display and / or on a projected image; by changing the position of the embodiment (relative to the user, the Earth, and / or parts of the embodiment itself) and / or a portion thereof; by flashing a non-obtrusive light of one color (e.g., green) one or more times when a valid value is detected on the user's face, head, neck, and / or shoulders and when a valid value is received by the embodiment, and, optionally, by flashing a light of a different color (e.g., red) in response to detection of an invalid and / or improper value; and For example, by parroting back to the user with an audible pronunciation of each detected value such as "a", "b", "o", "u", "t", "about"... or "8", "0", "5", "4" (if dialing a phone number on the user's behalf). can be provided.

[0062] Exemplary embodiments of the present disclosure may indicate to a user that they are ready to detect and process and / or execute values ​​and / or instructions issued by the user via a user expression of muscle contraction encoded values ​​in a variety of ways, including but not limited to: by displaying, at one or more locations on the screen and / or graphical display of the output device, a message, status indicator, icon, color, a cursor (of a particular shape and / or color), or other graphical element that indicates that the embodiment is ready; By outputting an audible signal, such as a beep or buzz, indicating that it is ready; and By saying the word "ready" (or any other word) It can be shown.

[0063] Certain muscle contraction encoding values ​​may be useful for controlling exemplary embodiments of the present disclosure. For example, certain values ​​may be defined for the exemplary embodiments that allow a user to "control" various embodiment functions, including, but not limited to: terminating the detection and processing of the previous set of values; To start composing new documents, music, graphics, etc. Deleting one or more of the previous values; Audibly uttering (e.g., parroting) the draft message; Save recently edited documents, music, graphics, etc. Sending recently created text messages; Hanging up (i.e. ending a call), etc. Includes.

[0064] 17. User or Users Exemplary embodiments of the present disclosure can search for, detect, and process values ​​encoded via muscle contractions of a single user's face, head, neck, and / or shoulders, a common limitation appropriate for many embodiments (e.g., those that convert contraction-encoded values ​​into passwords and / or passphrases to verify user identity before granting access to the embodiment).

[0065] However, another exemplary embodiment of the present disclosure may be to simultaneously search for and detect values ​​encoded via muscle contractions in the face, head, neck, and / or shoulders of as many users as possible, Such an embodiment can consolidate the individual values ​​encoded by each individual user into a group of contemporaneous values. An embodiment can associate the values ​​encoded by each particular user with a note associated with a particular instrument. In this type of embodiment, a group of people could potentially generate music associated with the simultaneous playing of various instruments. Such an embodiment can allow the pitch of the music output by the system for each user's associated virtual instrument to be specified by the user via intrinsic muscle contraction. It can allow the loudness of the music output by the system for each user's associated virtual instrument to be controlled by the degree to which each user's right hand extends from their respective chest. And it can allow each user to adjust the timbre of the synthesized music contributing to the concert by adjusting the degree to which that user's left hand extends from their chest.

[0066] 18. Steps performed by the system Exemplary embodiments of the present disclosure detect contractions or absence of specific muscles and / or muscle groups on the user's face, head, neck, and / or shoulders. These contractions are then measured and quantized, thereby converting the raw analog measurements for a substantially continuous range of distances into one of muscle-specific gray scale numbers. For example, the "lift" of a user's eyebrows may have a raw measurement expressed as a real number (e.g., 18.3 mm, 17.9 pixels, 4 minutes, etc.), but may be quantized to an integer within an embodiment-specific and / or muscle-specific range (e.g., 2 within a range of 0 to 3). The resulting degrees of contraction of each muscle-specific are then combined to generate a composite number within a specific range of such numbers (e.g., resulting from a combination of four quantized contraction measurements having ranges of 0 to 1, 0 to 1, 0 to 2, 0 to 2, and 0 to 2, respectively, from 0 to 107).

[0067] For example, an exemplary embodiment of the present disclosure may perform the following steps to provide the disclosed functionality. 1. Receiving a stream of images from a camera whose field of view can include the user of the system. 2. Attempt to place the person's (i.e., the user's) face, head, neck, and / or shoulders in each image. 3. Identifying the location of the user's face, head, neck, and / or shoulders found within the image. 4. Identifying the location of each point and / or feature on the user's face, head, neck, and / or shoulders (i.e., "target muscles") that correlates to the user's coded value designation. 5. Measure the degree to which each target muscle contracts. Because the actual muscles cannot be directly viewed, an embodiment may measure, for each target muscle or muscle group, corresponding and / or associated changes in visible feature characteristics of the user's face, head, neck, and / or shoulders. As an alternative to the degree to which the associated muscles contract, the types of features and feature-specific attributes that an embodiment may measure include, but are not limited to, the following: a) the degree to which the distance between the target features of those muscles (e.g., eyebrows) and their corresponding reference features (e.g., outer corners of each eye) has changed; b) the degree to which the angular orientation of the target features of those muscles (e.g., sagittal plane of the head) and their corresponding reference features (e.g., transverse plane across the shoulder) has changed; and c) the degree to which the cross-sectional area, volume, and / or shape of the target features of those muscles (e.g., nostrils of the nostrils) has changed.

[0068] The measurements of the alternative features and feature-specific attributes may correspond to measurements expressed in a variety of units, including, but not limited to, millimeters and / or pixels (for measuring separation), degrees (for measuring angular orientation), square millimeters (for measuring area), and cubic millimeters (for measuring volume).

[0069] 6. Quantize each muscle's unique measurement (or the average of measurements taken for pairs of correlated and / or equivalent target muscles or features, such as eyebrows). Thus, each muscle-specific raw measurement is converted (i.e., quantized) to an integer between muscle- and / or feature-specific minimum and maximum values. For example, the degree to which a user's eyebrows are raised up and / or lowered, corresponding to millimeters of deviation above or below their resting position for each outer corner of the user's eye, perhaps, can be quantized to a value (averaged over the eyebrow pair) ranging from 0 to 4. In this example, 0 can correspond to fully lowered eyebrows, 1 can correspond to partially or slightly lowered eyebrows, 2 can correspond to the resting or relaxed position of the eyebrows, 3 can correspond to partially or slightly raised eyebrows, and 4 can correspond to fully raised eyebrows.

[0070] 7. Combine the individual quantized muscle specific numbers into composite numbers. Each composite number corresponds to a unique order specific set of muscle specific quantized numbers. For example, assume we have two features (e.g. eyebrows and head orientation). Assume eyebrow position is converted / quantized to a number between 0 and 4. Assume head can be tilted left, right or not tilted and is converted / quantized to a number between 0 and 2 (0=left tilt, 1=no tilt and 2=right tilt). With these two features, the user can specify 15 unique values. The definitions of these 15 unique composite values ​​are given in the table below. [Table 2]

[0071] 8. Converting composite numbers to "values" appropriate for the design and / or intended purpose of the embodiment and / or for the design and / or intended purpose of the application executed by the embodiment. For example, if the embodiment is a "graphics drawing" system (i.e., compositing of graphic images), then each composite value is converted to a corresponding color as shown in the table below. [Table 3]

[0072] 9. Respond in a manner appropriate to the circumstances. Such responses may include: The exemplary embodiment may, but is not limited to, a) update an image by changing the value of at least one pixel on a display of the exemplary embodiment (e.g., if the exemplary embodiment is a graphics rendering embodiment), b) update a document by changing or adding the value of at least one character in a memory containing the document's contents (e.g., in a random access memory of a text editing embodiment), or c) synthesize and output musical notes, chords, tones, etc. via at least one speaker associated with the embodiment.

[0073] 19. User-specific Calibration of Contraction In accordance with the present disclosure, different users may have different ranges of motion for the movements of the muscles and associated features of the face, head, neck, and / or shoulders. Exemplary embodiments of the present disclosure may measure and translate the contractions and movements of targeted facial muscles and features using uniform and / or fixed standards of measurement and / or quantization. Other exemplary embodiments may provide the user with the opportunity to indicate their range of contraction and / or movement for embodiment-specific muscles and / or features. These exemplary embodiments may then use these user-specific ranges as limits for measuring the "relative" degree of future contractions and / or movements.

[0074] 20. Contraction, distance, area, volume, angle, etc. This disclosure refers to the detection and transformation of the extent to which particular muscles and / or muscle groups are contracted. However, it should be understood that each such reference, unless expressly stated to the contrary, should be considered equally applicable in whole or in part to the detection and transformation of the extent to which external surface features of a user's face, head, neck, and / or shoulders (e.g., eyebrows) are positioned, dislocated, repositioned, bent, expanded, and / or volumetrically and / or geometrically altered from a "resting" position, shape, and / or volume by the contraction of one or more involved muscles and / or muscle groups.

[0075] It should be understood that such references to the degree of muscle contraction should be considered equally applicable in whole or in part to the detection and transformation of the extent to which one or more related features are altered with respect to the relative alignment, orientation and / or axis of alignment of other parts of the body (e.g., the degree to which the sagittal plane of the head is rotated from normal to the transverse plane of the shoulder - see Figure 2), unless expressly stated to the contrary.

[0076] It should also be understood that each such reference should be considered equally applicable in whole or in part to the detection and transduction of the degree to which an innervated target muscle and / or muscle group is activated, unless expressly stated to the contrary. Detection of the degree to which a particular nerve is innervated can be accomplished via measurements including, but not limited to, a) electrical properties (e.g., voltage, current, capacitance, etc.); b) magnetic properties.

[0077] It should also be understood that each such reference, unless expressly stated to the contrary, should be considered equally applicable in whole or in part to the detection and translation of alternative types of changes indicative of muscle contraction, including, but not limited to, the degree to which the skin of the user's face, head, neck, and / or shoulders is distorted (as measured by strain gauges), as well as optical properties (e.g., degree of redness, degree of shading, 3D models constructed from stereoscopic images and / or shading, etc.).

[0078] 21. Measurable features of the head, neck, and upper body Measurable facial features Expressive facial features often result from two or more muscles working together, so control of larger facial features is likely easier for an individual to learn and execute than control of the individual muscles that cause movement of those features.

[0079] Facial features that can be consciously controlled by a person and are readily identifiable by automated detection and measurement systems (e.g., based on analysis of image streaming from a video camera), and therefore may be useful for generating user-controlled (i.e., voluntary and deliberate facial expressions) input to a computer, include, but are not limited to, the following: Lip shape (e.g., pursed or relaxed lips) Oral cavity shape (i.e., the shape of the mouth opening) Relative percentage of "open" mouths Relative exposure of upper and / or lower teeth Relative vertical extension of the chin and / or mandible Relative lateral extension of the chin and / or mandible Tongue position (e.g. exposed or not) The angle of the mouth relative to the vertical axis of the face Relative eyelid position (e.g., open or closed) Relative percentage of eyes that are "open" The relative direction of "gaze" (to the user, not to the computer) Relative width of the nostrils Relative position of eyebrows (e.g. raised, at rest, or lowered) Relative depth of forehead wrinkles Emotional facial expressions (happy, surprised, scared, angry, sad) b. Measurable head and neck features Relative orientation and / or position of head

[0080] If both the user's face and neck are available for monitoring and assessment (e.g., visible to a computer camera), the relative orientation of the head with respect to the frontal, sagittal, and transverse planes of the body (as established by the user's neck and / or shoulders) can be used to generate further information.

[0081] Absolute head orientation and / or position Additionally, the absolute orientation of the user's head (with respect to the vertical defined by gravity) can be used to generate even more information by comparing the position of the user's head to vertical and / or horizontal edges (e.g., doorways, walls, and / or windows) visible behind the user. Absolute orientation of the user's head with respect to "true vertical" can also be achieved using "gravity" sensors, magnetometers, and / or accelerometers in a smartphone or other computing device that is measuring the user's characteristics (e.g., via video).

[0082] Relative and absolute head positions can be consciously controlled by a person and easily identified by automated systems, and thus can be used to allow a user to directly control the computer processor of the devices disclosed herein. These relative and absolute head positions include, but are not limited to: The left / right tilt of the head (i.e., relative to the front of the body) Forward or backward tilt of the head (i.e., relative to the sagittal plane of the body) The vertical axis of the head relative to the vertical axis of the body (i.e., relative to the transverse plane of the body) Rotation of the head relative to the torso (i.e., looking left and right)

[0083] C. Measurable features of the upper body Lifting of the left and / or right shoulders, even if slight, can be easily identified by an automated system and thus used to allow the user to directly control the computer processor of the devices disclosed herein. Other features of the upper body (e.g., pushing out the chest) can also be used as signals for the devices disclosed herein.

[0084] d. Measurable angle scale By adjusting the distance of the user's head from the camera that detects the measurable facial features and head orientation values, the absolute size (e.g., in pixels) of the user's head in the captured image and / or video stream changes. For example, the size of the user's head, relative to a baseline size, can be used by the user to identify additional bits of information for the invention disclosed herein. For example, if the user was using a small handheld version of the invention, where the device was using a "selfie" camera to detect and measure the movement of features generated by the user, the user could move the smartphone closer to or further away from his or her body to change the "signal" associated with the absolute pixel size of his or her head in the measured image.

[0085] e. Measurable facial muscles While it may require more practice on the part of the person to achieve proficiency in conscious control, and furthermore may require more sophisticated detection systems to determine their state and measure their "values," monitoring of a user's controlled contractions of individual facial muscles is nevertheless provided, enabling the user to generate high-definition input values ​​(i.e., multi-bit values), thereby exerting high-resolution control of the inventive processor disclosed herein.

[0086] Facial muscles that are consciously controlled by a person and easily identified by automated "signal" detection and measurement systems, and thus enable a user to directly control a computer through voluntary facial expressions, include, but are not limited to, the following: i. "levator palpebrae superioris" muscle, “Upper tarsal plate” muscle, Those controlled by the third (III) cranial nerve (i.e., the oculomotor nerve), including, but not limited to, ii. Those controlled by the fifth (V) cranial nerve (i.e., the "trigeminal"), including but not limited to:

[0087] A. "masseter" (i.e., one of the muscles of mastication), "Temporalis muscle" (i.e., one of the muscles of mastication), "Medial pterygoid muscle" (i.e., one of the muscles of mastication), "lateral pterygoid muscle" (i.e., one of the muscles of mastication), "tensor tympani", "Tensor veli palatini" "Mylohyoid muscle" "Upper belly of the digastric muscle", those controlled by the "mandibular" nerve (i.e., the "V3" nerve), including, but not limited to:

[0088] B. Those controlled by the seventh ("VII") cranial nerve (i.e., the "facial nerve"), including but not limited to: 1. Without limitation, "Occipitofrontalis" (i.e. "occipital muscle"); "posterior auricular muscle", those controlled by the "posterior auricular" nerve, including 2. Without being limited thereto, "Occipitofrontalis" (i.e., "frontalis"), "orbicularis oculi" (i.e., the orbital and eyelid areas); "Frown lines", "Nose root muscle" "Anterior auricular muscle", "Superior auricular muscle" Including the "superficial cranial muscles" Those controlled by the "head" nerves. 3. Without being limited thereto, "orbicularis oculi" (i.e., the lacrimal gland part), "levator labii superioris", "Zygomatic major muscle" those controlled by the "zygomatic" nerve, including 4. Without limitation, "Bridge of the nose" "Levator anguli oris" "Zygomatic major muscle" "Zygomaticus minor" "Orbicularis oris" "lol muscle" "Cheek muscle" "Nasal septal depressor muscle" "Levator labii superioris nasalis" those controlled by the "buccal" nerve, including 5. Without being limited thereto, "Depressor labii inferioris" "Mental muscle" "Depressor angularis oris" "Transverse mentalis muscle" those controlled by the "mandibular" nerve, including 6. Without limitation, "Platysma muscle" Those controlled by the "cervical" nerves, including:

[0089] C. Without limitation, "Genioglossus muscle" "hyoglossus" "Styloglossus muscle" "Geniohyoid muscle" "thyrohyoid" those controlled by the twelfth ("XII") cranial nerve (i.e., the "hypoglossal nerve"). f. Measurable head, neck, and upper body muscles Head, neck, and / or upper body muscles that may be consciously controlled by a person are readily identifiable by the automated video system and therefore may be used to measure spontaneous movements of the person's head. For example, muscles that may be beneficial for direct control of a computer by a user through objective tilt include, but are not limited to, the following: i. Without limitation, "Trapezius" "Sternocleidomastoid muscle" those controlled by the eleventh ("XI") cranial nerve (i.e., the "accessory nerve"), including ii. Without limitation, "Serratus anterior superior" "serratus anterior medial" "serratus anterior inferior" those controlled by the "long thoracic nerve", including iii. Without limitation, "Rhomboid major muscle" "Rhomboid minor" Those controlled by the "dorsal scapular nerve", including the iv. Without limitation, "Anterior scalene muscle" those controlled by the "anterior rami of the C5 and C6 cervical nerves," including v. Without limitation, "Middle scalene muscle" Those controlled by the anterior rami of the cervical nerves C3 to C8, including: vi. Without limitation, "Posterior scalene muscle" Those controlled by the "C6 to C8 cervical nerves", including: vii. Without limitation, "Latissimus dorsi" Those controlled by the "thoracic and dorsal nerves", including the viii. Without limitation, "Deltoid" "Teres minor" Those controlled by the "axillary nerve", including: ix. Without limitation, "Semispinalis capitis" Those controlled by the greater occipital nerve, including the x. Without limitation, "Splenius capitis muscle" those controlled by the "posterior rami of spinal nerves C3 and C4," including xi. Without limitation, "rectus capitis posterior major muscle" those controlled by the "suboccipital nerve" (i.e., the posterior rami of C1), including g. Detection and Measurement Methodology The movement, contraction, and / or other changes in each of the above characteristics and / or muscles can be detected and measured in a number of ways, including but not limited to the following.

[0090] i.Video Surveillance The features and / or muscle movements and / or contractions used as signals for the present invention can be detected and measured visually (eg, with a camera). The computing device may be equipped with one or more video cameras and / or other means for capturing images, thereby enabling a user of the device and / or other A person can interact with a computer while images of their face, head, neck, and upper body are captured. Where appropriate, the stream or sequence of images acquired by said camera is analysed with respect to one or more aspects of shape, orientation, distortion and / or appearance of relevant features. The instantaneous configuration of the imaged feature is measured and / or calculated. Changes in the imaged feature over time are also measured and / or calculated. The resulting instantaneous feature configurations are checked to determine whether they map to one or more types and / or values ​​of defined and / or enabled inputs to the computing device, such as, for example, computer commands and data, application commands and data, etc.

[0091] ii. Electrical monitoring The features and / or muscle movements and / or contractions used as signals for the present invention can be detected and measured electrically. Contractions of the muscles involved can in some cases at least be detected by changes in voltage and / or capacitance on the skin overlying the affected area. The computing device includes one or more voltage, current and / or capacitance sensors and / or other means for detecting electrical properties / characteristics of one or more "signature" muscles and / or the overlying skin, such that the computing device can detect and monitor electrical conditions and / or changes associated with innervation of muscles of the face, head, neck and / or upper body and / or changes in muscle innervation while a user of the device and / or other person interacts with the computer. The resulting stream of instantaneous muscle-related electrical characteristics of a user of a computing device are checked and monitored over time to determine whether they map to one or more types and / or values ​​of defined and / or enabled inputs to the computing device, such as, for example, computer commands and data, application commands and data, etc.

[0092] iii. Magnetic monitoring The features and / or muscle movements and / or contractions used as signals for the present invention can be detected and measured magnetically. As indicated above, the movements and / or contractions can be detected and measured electrically, where electrical currents as they appear in the nerves innervating the affected muscles generate magnetic fields that can be detected.

[0093] iv.Spectral monitoring The features and / or muscle movements and / or contractions used as signals for the present invention can be detected and measured spectroscopically due to their effect on blood flow in the affected areas of the associated muscle contraction. When a muscle contracts, its localized expansion increases the pressure exerted on the skin overlying the muscle. These contraction-induced pressure increases impede the flow of blood through the skin in the affected area. Such changes in the amount of blood flow through the skin can be measured spectroscopically (i.e., through analysis of the spectrum of colors detected within an image of the user's skin (or with individual "single pixel" photodetectors)). When a muscle contracts, a local "bump" may develop. That bump can squeeze out blood (i.e., resist blood inflow and promote blood outflow) not only in that muscle, but also in adjacent tissues, particularly the skin. This change in the local volume and / or rate of blood flow in the area of ​​the relevant muscle and / or feature can be detected by spectral analysis of an image of that area of ​​the user's skin. Such monitoring can be done with standard video cameras, as well as by other methods and using other sensor technologies. It can be achieved.

[0094] For example, a "pulse oximeter" is a sensor that can monitor changes in blood flow associated with the beating of a person's heart (to monitor the person's heart rate). These same types of sensors or similar technology can also be used to monitor the rate and / or volume of blood flow through parts of a user's face, head, neck, and / or upper body, thus enabling a computing device to indirectly monitor the degree of contraction in at least one muscle of the user.

[0095] Oxygenated blood absorbs light at 660 nm (red light), while deoxygenated blood preferentially absorbs light at 940 nm (infrared light). A pulse oximeter consists of two light-emitting diodes, one at 600 nm and one at 940 nm, and two light-collecting sensors that measure the amount of red and infrared light that emerges from the tissue the light beam traverses. The relative absorption of light by oxyhemoglobin (HbO) and deoxyhemoglobin is processed by the device to report the oxygen saturation level. The device focuses on the pulsating arterial blood and ignores local noise from the tissue. The result is a continuous, qualitative measurement of the patient's oxyhemoglobin status. The oximeter measures pulse rate, oxygen saturation (SpO 2 ), and further provides data on cardiac output. http: / / www.ccmtutorials.com / rs / oxygen / pagel7.htm

[0096] By using one or more blood flow and / or blood flow velocity sensors, the computing device can thereby detect and monitor the extent to which muscles in the face, head, neck, and / or upper body of a user of the device and / or other persons have contracted and / or the degree to which the degree of contraction has changed while interacting with the computer.

[0097] The resulting stream of instantaneous facial blood flow characteristics of a user of the computing device are checked and monitored over time to determine whether they map to one or more types and / or values ​​of defined and / or enabled inputs to the computing device, such as, for example, computer commands and data, application commands and data, etc.

[0098] v. Thermal monitoring The characteristics and / or muscle movements and / or contractions used as signals for the present invention are detected and measured by capturing and analyzing infrared images of the user's skin. As noted above, muscle contraction reduces the amount of blood flowing through the area of ​​skin overlying the contracted muscle, and it is possible to detect changes (e.g., reductions) in the skin temperature overlying the contracted muscle, and the magnitude of those temperature changes should correlate with the magnitude of muscle contraction.

[0099] vi. Audio monitoring Movement and / or contraction of the features and / or muscles used as signals for the present invention can be detected and measured through the changes that occur in the volume of the "pulse" associated sound that results from the pumping of blood through the skin overlying the associated muscle. As mentioned above, muscle contraction reduces the amount of blood flowing through the area of ​​skin overlying the contracted muscle. The reduction in blood flow changes the volume and characteristics of the sound of pulsating blood flow.

[0100] vii. Twist / Space Surveillance There are currently many techniques available to directly measure stretch and bending, some of which are specifically designed to measure stretch and bending along human surfaces. Others analyze shading and / or shading changes on a person's face, head, neck, and / or upper body to measure stretch and bending directly. It is also possible to measure and monitor changes in the contours of a person's face, head, neck, and upper body. Additionally, by using two or more video cameras, it is possible to build a 3D model of a person's face, head, neck, and / or upper body and monitor changes therein, thus enabling detection and / or monitoring of the relative degree of contraction of the associated muscles.

[0101] These and / or other techniques and / or methods can be used to monitor the shape of a user's face, head, neck, and / or upper body, thereby determining and / or monitoring changes in that shape and the relative degree of contraction of the muscles supporting it. The resulting stream of instantaneous facial muscle contraction readings of the computing device user are checked and monitored over time to determine whether they map to one or more types and / or values ​​of defined and / or enabled inputs to the computing device, such as, for example, computer commands and data, application commands and data, etc.

[0102] 22. Exemplary Feature Sets a. An exemplary embodiment of the present disclosure responds to changes in the position of four features of the user's face, head, neck, and shoulders: the degree to which the eyebrows are raised or curled, the degree to which the head is tilted in the coronal plane, the degree to which the left and right eyes are partially closed, and the degree to which the head is tilted in the sagittal plane. i. Find the average approximate vertical distance of the eyebrows relative to the inner corner of each eye (i.e., nose part). These vertical eyebrow states are quantized into five relative numbers, where 0 represents fully lowered eyebrows (shortest distance between the center of both eyebrows and the nose part), 1. represents partially lowered eyebrows, 2. represents eyebrows at rest and / or in a natural position, 3. represents partially raised eyebrows, and 4. represents fully raised eyebrows.

[0103] ii. Detect the angular tilt of the head in the "lateral" direction (i.e., in the coronal plane) relative to the transverse plane through the shoulders. These relative tilt degrees are quantized into seven relative numerical values, where 0 represents the head tilted approximately 15 degrees to the user's left, 1 represents the head tilted approximately 10 degrees to the user's left, 2 represents the head tilted approximately 5 degrees to the user's left, 3 represents the head is vertical (not tilted), 4 represents the head tilted approximately 5 degrees to the user's right, 5 represents the head tilted approximately 10 degrees to the user's right, and 6 represents the head tilted approximately 15 degrees to the user's right.

[0104] iii. Detect the degree to which the left and right eyelids are closed relative to the maximum separation of the upper and lower eyelids of each eye along an axis in a plane parallel to the sagittal plane of the head. These relative degrees of separation are quantized to four values, where 0 represents both eyes fully open, 1 represents the user's right eye is partially closed (squinting) and the user's left eye is fully open, 2 represents the user's left eye is partially closed and the user's right eye is fully open, and 4 represents both eyes are partially closed.

[0105] iv. Detect the angular tilt of the head in the "anteroposterior" direction (i.e., in the sagittal plane) relative to the coronal plane, as measured by the separation of the user's chin from the chest. These relative degrees of tilt are quantized into seven relative numerical values, where 0 represents the head tilted back approximately 15 degrees, 1 represents the head tilted back approximately 10 degrees, 2 represents the head tilted back approximately 5 degrees, 3 represents the head is in a vertical orientation (not tilted), 4 represents the head tilted forward approximately 5 degrees, 5 represents the head tilted forward approximately 10 degrees, and 6 represents the head tilted forward approximately 15 degrees.

[0106] The user-performed transposition of these four features provides four possible states for eyebrow position and four possible states for head tilt for each of the user's body landmarks. The feature-specific states provide seven states for available eye squints, four states for available eye squint degrees and combinations, and seven states for available head sagittal tilts. Permutations of these feature-specific states thus provide the basis for 784 composite states and, therefore, 784 embodiment-specific and / or application-specific values.

[0107] Providing the user with this feature set, and up to 784 implementation-specific and / or application-specific values, has the advantage that it provides the user with sufficient values ​​to specify 5 ASCII control characters (e.g., carriage return, delete, etc.), 27 non-alphanumeric characters (e.g., "!", "?", etc.), 10 numeric characters (e.g., "0" through "9"), 26 uppercase letters ("A" through "Z"), and 26 lowercase letters ("a" through "z"), for a total of 94 ASCII characters. This leaves an additional 690 values ​​that can be assigned to various colors, application-specific actions (e.g., "Create a New File", "Save File", etc.), and / or specific frequently used words, phrases, salutations (e.g., "Dear"), closings (e.g., "Sincerely"), and the like.

[0108] b. Another exemplary embodiment of the present disclosure responds to changes in the position of two features of the user's head: the degree to which the head is tilted in the coronal plane, and the degree to which the head is tilted in the sagittal plane. As described with respect to the first set of preferred features above, user-initiated transpositions of these two features relative to respective landmarks on the user's body provide seven available crown tilt states of the head and seven available sagittal tilt states of the head. Permutations of these feature-specific states thus provide the basis for 49 composite states and, therefore, 49 embodiment- and / or application-specific values.

[0109] Providing the user with this feature set, and up to 49 embodiment-specific and / or application-specific values, has the advantage of providing the user with sufficient values ​​to control a relatively wide variety of parameters via value-specific parameter specifications and at least 10 numbers. It has the added advantage of being relatively "light", allowing the values, and the facial and / or body curvatures required for their generation, to be learned relatively quickly.

[0110] 23. Exemplary muscles a. An exemplary embodiment of the present disclosure uses sensors responsive to changing voltages to detect and measure the degree to which specific muscles contract. This exemplary embodiment is identical to the embodiment described in the first preferred feature set above (i.e., responsive to changes in the relative position of the eyebrows and eyelids, and changes in the relative coronal and sagittal tilt of the head), with one exception. This exemplary embodiment detects, measures, and quantifies the degree to which the muscles involved in the transposition of each feature in the feature-specific embodiment are activated by detecting the differential voltage associated with their contraction.

[0111] i. This exemplary embodiment measures and quantizes eyebrow movement via measuring and quantizing voltages associated with muscles for raising and lowering the eyebrows, including the muscles that raise the eyebrows, i.e., the occipital frontalis (front abdominal) and temporoparietal muscles, and the muscles that lower the eyebrows, i.e., the corrugator supercilii and procerus muscles.

[0112] ii. This exemplary embodiment measures and quantifies left / right head tilt through measuring and quantizing voltages associated with muscles involved in positioning the head at such tilt, including the scalene, trapezius, and sternocleidomastoid muscles.

[0113] iii. This exemplary embodiment uses two sets of muscles: 1) the muscles involved in raising the upper eyelid, i.e., the levator palpebrae superioris, and 2) the muscles involved in closing the eye. That is, the degree to which the left and right eyelids are narrowed is measured and quantified by measuring and quantifying the voltage associated with the orbicularis oculi muscle.

[0114] iv. This exemplary embodiment measures and quantifies forward and backward head tilt by measuring and quantizing voltages associated with muscles to position the head at such tilt, including the sternocleidomastoid, semispinalis capitis, semispinalis cervix, semispinalis thoracic, splenius capitis, splenius cervix, spinalis capitis, and longissimus capitis.

[0115] 24. Exemplary nerve a. An exemplary embodiment of the present disclosure uses sensors responsive to changing voltages to detect and measure the degree to which specific nerves, and / or their corresponding muscles, are innervated. This embodiment is identical to the embodiment described in the first preferred feature set above (i.e., responsive to changes in the relative position of the eyebrows and eyelids, and changes in the relative coronal and sagittal tilt of the head), with one exception. This exemplary embodiment detects, measures, and quantifies the degree to which the nerves and muscles involved in the transposition of each feature in the feature-specific embodiment are innervated by detection of differential voltages associated with their innervation.

[0116] i. This exemplary embodiment measures and quantifies eyebrow movement by measuring and quantizing the voltage associated with the temporal branch of the facial nerve (cranial nerve VII), which innervates the muscles that lift the eyebrows, and the voltage associated with the buccal branch of the facial nerve, which innervates the muscles that frown the eyebrows.

[0117] ii. This exemplary embodiment measures and quantifies head tilt left and right through measurement and quantification of voltages associated with portions of cervical nerves 5 through 8 and associated nerves (cranial nerve XI) for innervation of the muscles that tilt the head left and right.

[0118] iii. This exemplary embodiment measures and quantifies the degree of narrowing of the left and right eyelids via measuring and quantifying voltages associated with two sets of nerves, including: 1) those innervating muscles that change the position of the upper eyelid, i.e., the infratrochlear, supratrochlear, supraorbital, and lacrimal nerves (from the ophthalmic branch of the trigeminal nerve, CN V, V1), and 2) those innervating muscles that change the position of the lower eyelid, i.e., the infratrochlear nerve (from the maxillary branch of the trigeminal nerve, V2), and a branch of the infraorbital nerve.

[0119] iv. This exemplary embodiment measures and quantifies forward and backward head tilt via measuring and quantizing voltages associated with the accessory nerve (cranial nerve XI), greater occipital nerve, posterior rami of cranial nerves C3 to T6, and posterior rami of cranial nerves C3 to C8, which are involved in the innervation of the muscles that tilt the head forward and backward.

[0120] 25. Commercialization Embodiments of the present disclosure can be integrated into a wide variety of functions and / or device types known in the prior art, such as personal computers, smart phones, ATMs, POS terminals, music synthesizers, self-driving cars, gaming consoles, smart televisions, etc. Converting these known devices into the novel systems disclosed herein involves the addition of one or more sensors (e.g., cameras) that monitor the user's face, head, neck, and / or shoulders, and / or muscles therein.

[0121] Some existing computerized devices have been developed to detect certain geometric changes to a user's face, head, neck, and / or shoulders either directly (e.g., via analysis of image data generated by a camera) or indirectly (e.g., electrical signals generated by a sensor in contact with the user's skin) by adding to each such device at least one sensor, e.g., a camera, and by downloading from the Internet at least one set of computer code that can operate within the device. The set of computer code may also be adapted to convert a particular geometric configuration of a user's face, head, neck, and / or shoulders into a digital value suitable for use and / or control by the user of the device.

[0122] 26. User customization of code An exemplary embodiment of the present disclosure allows a user to specify which features of the face, head, neck, and / or shoulders the user will use to communicate and / or assign values ​​to the embodiment. Another exemplary embodiment of the present disclosure allows a user to specify which muscles of the face, head, neck, and / or shoulders the user will use to communicate and / or assign values ​​to the embodiment. Another exemplary embodiment of the present disclosure allows a user to specify which nerves of the face, head, neck, and / or shoulders the user will use to communicate and / or assign values ​​to the embodiment.

[0123] 27. Implementation-specific code An exemplary embodiment of the present invention communicates to a user which features (i.e., which "site codes") of the user's face, head, neck, and / or shoulders are available to the user to communicate and / or assign values ​​to an embodiment. Another exemplary embodiment of the present invention communicates to a user which muscles of the user's face, head, neck, and / or shoulders are available to the user to communicate and / or assign values ​​to an embodiment. Another exemplary embodiment of the present invention communicates to a user which nerves of the user's face, head, neck, and / or shoulders are available to the user to communicate and / or assign values ​​to an embodiment.

[0124] Exemplary embodiments of the present disclosure communicate which features, muscles, and / or nerves are available to a user to communicate and / or assign values ​​to the embodiment via one or more graphic illustrations that are displayed to the user on a display device of the embodiment.

[0125] Another exemplary embodiment of the present disclosure communicates which features, muscles, and / or nerves are available for the user to communicate and / or specify values ​​to the embodiment by prompting the user to transmit feature encoding values, muscle encoding values, and / or nerve encoding values ​​to the embodiment. The embodiment responds to each user transmission by generating a valid or invalid signal, e.g., a light, beep, or some other binary indicator, indicating whether the user-provided composite code was composed of all the necessary partial codes. The user can then find a set of site codes that satisfied the embodiment, and then remove individual site codes to test whether each removed site code is actually necessary for communication with the embodiment.

[0126] Another exemplary embodiment of the present disclosure may indicate which standard set of site codes was used by the embodiment via an "input type" code (e.g., an industry standard code). Such an input type code may include multiple elements or sub-codes. One sub-code may indicate the set of features, muscles, and / or nerves that the embodiment monitored, detected changes in, and converted to a quantized numeric value for the site. Another sub-code may indicate all of the associated features, muscles, and / or nerves, or alternatively the "range" of quantized values ​​associated with each feature, muscle, and / or nerve.

[0127] In another exemplary embodiment of the present disclosure, a user may input codes, values, data files, etc. that specify the user's preferred communication features, muscles, and / or nerves, and the user's preferred "ranges" of quantized values ​​derived from each. Such an embodiment may map its internal composite values ​​to the user-specified features, muscles, and / or nerves in terms of the user-specified quantized ranges.

[0128] It will be apparent to one of ordinary skill in the art that there are many methods, techniques, protocols, etc. by which an embodiment can communicate to a user which features, muscles, and / or nerves are required for communication and / or control of the embodiment.

[0129] 28. Disease detection By storing statistics over time (e.g., months, years, decades) relating to a user's generation of contracted coding values ​​in accordance with one or more embodiments of the present disclosure, it is possible to detect changes (particularly decreases) in the pace and / or frequency of that user sending codes to one or more embodiments. It is also possible to detect changes (particularly increases) in the frequency with which a user sends erroneous codes that must then be corrected and / or repeats valid codes (particularly where such redundancies are not the same as clear errors that are ignored by one or more embodiments and thus sought to be corrected).

[0130] Some of the above changes, particularly decreases, in a user's efficiency and / or accuracy may be indicative of the onset of disease, particularly of a neurological nature. Some of the diseases that exemplary embodiments are capable of detecting include, but are not limited to, amyotrophic lateral sclerosis (ALS), Bell's palsy, brain tumors, brain atrophy (particularly age-related forms), Creutzfeldt-Jakob disease (CJD), dementia, diabetic neuropathy (potentially associated with undiagnosed diabetes), Huntington's disease, Parkinson's disease (PD), sleep apnea, and stroke.

[0131] Some embodiments of the present disclosure record metrics related to a user's efficiency and accuracy and alert the user and / or one or more third parties (e.g., a parent, account holder, medical professional, etc.) to the user's potential health problems as well as relevant preliminary diagnostic data.

[0132] 29. Exemplary Embodiments Embodiments may differ at least with respect to how many varied geometric configurations of the face, head, neck, and / or shoulders they support: some support only a few geometric configurations; others support a large number of geometric configurations.

[0133] The embodiments may differ at least with respect to how many variations they distinguish for any particular geometric change. This differs in practice in how many quantized measures are generated for any particular geometric change. For example, one embodiment only distinguishes between three subranges of changes associated with raising or lowering eyebrows (e.g., fully raised, natural, fully lowered). Another distinguishes between five subranges of those changes (e.g., fully raised, partially raised, natural, partially lowered, fully lowered).

[0134] Embodiments may differ at least with respect to how the change in geometry is detected, e.g., with cameras, voltage sensors, accelerometers, and which physical attributes of the changed geometry are measured (e.g., monoscopic (2D), stereoscopic (3D), electrical, etc.).

[0135] Embodiments may differ at least with respect to how the supported geometries are conveyed to the user. Embodiments may differ at least with respect to whether a user can specify the geometry and / or ranges used to specify each available symbolic value supported by an embodiment. Embodiments may differ at least with respect to whether an application developer can specify the geometry and / or ranges used to specify each available symbolic value supported by an embodiment. Embodiments may differ at least with respect to how accurately they can interpret (ie, "correct") malformed or poorly performed geometries. The embodiments may differ at least with respect to the maximum rate of geometrically encoded signals that they can support (i.e., "how fast a user can transmit a signal"). The embodiments may differ at least with respect to how the geometrically encoded signals that they convey are processed (i.e., what they are used for).

[0136] A. Smartphone The use of smartphones as a common computing platform is on the rise. Also, small handheld computing devices such as smartphones and tablet computers typically operate without the benefit of a physical keyboard. Thus, these devices are typically operated and controlled without high-resolution means of manual control. Instead, these devices typically rely on a "virtual" keyboard that is shown on the screen of these devices like any other graphic, requiring the user to touch that portion of the screen that covers the desired character or characters.

[0137] Many people have experienced the distraction and irritation that can result from sitting or standing very close to someone using a smartphone. The person may be talking (too loudly), what they are saying may be uninteresting, and there is little to no way to prevent the words they are saying from entering your ears and mind. The person may be engrossed in touching the screen of the device, composing some sort of text message (e.g., traditional text message, Twitter post, Facebook or Google chat, etc.). Visual disturbances are difficult to ignore, even if isolated in your peripheral vision.

[0138] What is needed is a method and / or technique that allows a person to interact with their friends and / or colleagues via a smartphone or other computer-mediated communication device without the noise caused by spoken words or the visual distractions caused by constantly tapping and sliding on a screen.

[0139] b. Security Issues This combination allows a user to submit a password "code" while the system monitors unique attributes of each code-specific geometric shape that the user attests to (e.g., detailed images of scars, blood vessels, warts, etc. visible on the user's facial surface as they execute each particular password-related symbolic value, user-specific tremors observable while the user executes the password-specific geometric shapes, user-specific characteristics related to electrical properties exhibited while the user performs code-related muscle contractions, etc.).

[0140] The additional user-specific information that the system can acquire while monitoring a user executing a geometrically encoded password code can provide a highly distinctive, though perhaps not unique, fingerprint that is difficult for a thief to simulate.

[0141] c. Deep Code Set Use of multiple site geometries and identification of more intermediate geometries between the defining extremes (i.e., a larger range of symbolic values ​​per site geometry) By implementing a symbolic language, a larger set of symbolic values ​​can be made available to users. Although a larger set of values ​​requires more time for users to learn, the benefits may outweigh the costs for many users, such as quadriplegics, who may choose to use embodiments of the present disclosure as their primary means of communication.

[0142] 30. Potential profit The invention disclosed herein can provide many advantages not available to people with respect to the current state of the art, including, but not limited to:

[0143] a. Silent / hands-free input The invention disclosed herein also allows a user to control their computer and compose text, documents, music, etc., without using their hands or fingers and without speaking verbal commands or information. This is particularly convenient when a user is attempting to use a computer in a crowded and / or public place where holding the computer in one hand and protecting the security of data entry into the computer is a concern. The present invention can also enable one to control and / or interact with one's computer in public situations without offending nearby people who may become frustrated and / or angered by chronic rapid hand movements and / or audible voiced commands and data.

[0144] b. Silent and private "word of mouth" conversations The invention disclosed herein can enable users to have private conversations in public places. By formulating messages through deliberate and coordinated contraction of facial muscles, users do not need to speak out loud, which may be accidentally or intentionally overheard by someone nearby. In fact, two users of the invention can each generate a message for the other by coordinated contraction of certain facial muscles, while listening to a synthesized translation of the message received from the other via headphones.

[0145] Such conversation would result in an equivalent, if not superior, solution in the complement of face-encoded commands and / or symbols available to increase the "richness" of communication, without the frustration and health risks associated with repetitive, high-frequency character selection via small manual or virtual keyboards.

[0146] c. Detecting damage to various nerves The facial muscles that produce facial expressions are innervated by pairs of facial nerves, one each for the right side of the face and one each for the left side (i.e., cranial nerves V and VII). The tongue muscles are innervated by the hypoglossal nerve. And other muscles that can serve as signals for embodiments of the present invention are innervated by other nerves.

[0147] By recording the type and extent of contractions, movements, and / or other changes detected in a particular user's signal muscles over time, particularly for laterally complementary signal muscles (i.e., one on each side of the user's body), the signal muscle control computer can alert the user that a decrease in the vigor of contractions and / or movements innervated by one or more nerves has been detected. Such a decrease may be indicative of damage and / or disease to the respective nerves, particularly if it is characteristic of only one side of the user's body.

[0148] d. Detection of depression and other psychological disorders The type and degree of facial expressions and / or body movements detected for a particular user over time By periodically recording the user's facial expressions, the signal muscle control computer can alert the user to an increase in the relative frequency of movements and / or expressions characteristic of various psychological illnesses, such as depression. In addition to helping detect an increase in the relative frequency of movements and / or facial expressions consistent with psychological illnesses, the signal muscle control computer can also detect a decrease in the magnitude and / or duration of movements and / or facial expressions that are antithetical to various psychological problems. For example, a decrease in the relative frequency of production and / or duration of smiling and other facial expressions consistent with happiness may indicate the onset of depression.

[0149] e. Monitoring various health-related characteristics Because the signal muscle control computer detects, analyzes, classifies, and quantifies the user's body language and facial expressions, and because embodiments may also record the relative frequency, magnitude, and / or other parameters associated with these expressions over time, it becomes possible for the computer, or its logs, to be used to provide feedback regarding the "quality" of the body language and facial expressions detected by the computer. If the user's body language and / or facial expressions deviate from what is "usual" or "normal" (perhaps defined by an aggregation of user-specific expression data collected over time), feedback regarding such "anomalies" may be provided to the user and / or therapist.

[0150] In this way, for example, the user's speech therapist can gain valuable insight into the health of the facial nerves, facial muscles, and neurological conditions (e.g., spasms, seizures, etc.) that can guide, improve, and / or accelerate the user's treatment. This type of facial and body language monitoring capability can also provide feedback to the user or the user's doctor or therapist regarding the effectiveness of treatment and / or therapy provided to the user with respect to any number of neurological, psychological, speech, and / or other health-related conditions.

[0151] B. Detailed Description of the Embodiments of the Present Disclosure 1 illustrates an exemplary embodiment of the present disclosure, in which a user 160 generates commands for the operating system of an embodiment or for an application running within an embodiment using only the seven signal feature movements shown in and described with reference to FIG.

[0152] The user 160 is running the same non-zero signal configuration as shown in and described with reference to FIG. 5. The embodiment has two cameras 170 and 171 that capture images 172 and 173 of the user 160 from different viewpoints, respectively. Such a two-camera embodiment may be implemented as a pair of cameras in the top left and top right corners of a smartphone. Each will view the user from a unique angular offset relative to the user's face. The cameras 170 and 171 are positioned such that the generated images 172 and 173 capture a stereoscopic view of the user. Such a stereoscopic view allows for the generation of a partial 3D reconstruction of the user's face, head, neck, and upper body.

[0153] A 3D reconstruction or model of the user's face, neck, head, and upper body allows for relatively easy and accurate determination of volumetric changes, such as those caused by muscle contraction or relaxation, and / or any changes in the relative position and / or orientation of other signal features (e.g., changes in the contours of the user's cheeks). Each camera sends each image it captures to an encoder module 174. The encoder module 174 is initialized with a "user configuration" file 175 that specifies which signal features this user 160 prefers and is most proficient with at run time. In this case, the user configuration file 175 indicates that this user 160 prefers to use the seven signal features already shown and described with reference to FIG.

[0154] Using preferences specified in a user configuration file 175, the encoder module 174 analyzes images 172 and 173 to identify the seven required signal features and measure their states. At the end of its analysis, the encoder module 174 outputs an array 176 (i.e., a "change signature") of regional signal values ​​(i.e., "quantized factor measurements"), which for this regional signal configuration (i.e., the same as shown in FIG. 5) is {2, 0, 1, 2, 1, 0, 0}.

[0155] In this embodiment, the transformer module 177 is initialized with an "application configuration" file 178. And in this example, the application configuration file indicates that only the four lowest quaternary signal values ​​are used to generate the composite signal value. The change signature 176 is sent from the encoder module 174 to the transformer module 177, and a (in this case) context-specific composite signal value 180 is generated from the truncated portion 179 specified by the application configuration file 178.

[0156] In this example, the lowest order signal values ​​are {2,1,0,0}. These are base 3 bits, so 2100 3 This composite signal value corresponds to a decimal value of 63, i.e. (2×27)+(1×9)+(0×3)+(0×1)=63.

[0157] The converter module 177 sends the number 63 to the "consumer" 181, which interprets it and executes the corresponding command. In this example, a "consumer" may be an application, a device operating system, a network interface module, etc. FIG. 2 is a set of sagittal, frontal, and transverse views used to define and explain the relative positions of muscles and other features of the human body and the directions of displacement of those features. Figure 3 is a diagram of some of the muscles that cause facial expressions, movements, and changes relative to the geometric shape of the human face. This diagram is in the public domain and can be found in "Sobotta's Atlas and Text-book Human Anatomy" published in 1909. The diagram was created by K. Hajek and A. Schmitson. 4 is a diagram of seven user-controlled body signals used by an exemplary embodiment of the present disclosure. This diagram is intended only to characterize these signals, but one may envision the diagram as representing the types of images that may be captured by a camera incorporated into or connected to the embodiment.

[0158] The user 100 is in a relaxed posture where each anatomy signal value (see Table 1 above) has a value of zero. The user 100 generates a first anatomy signal value by tilting 101 the user's head 100 to the left or right. The user 100 generates a second anatomy signal value by rotating 102 the head to the left or right (around the vertical longitudinal axis of the body) as if looking in the respective direction.

[0159] The user 100 generates a third region signal value by shrugging (i.e., raising) 103 his / her left or right shoulder. The user 100 generates a fourth region signal value by raising or lowering (i.e., grimacing) 104 his / her eyebrows. The user 100 generates a fifth region signal value by closing 105 either his / her left or right eye. (Ignoring simultaneous closing of both eyes allows the user to blink without generating spurious input to the embodiment.)

[0160] The user 100 generates a sixth region signal value by slightly or widely opening his / her mouth 106. The user 100 also generates a sixth region signal value by shifting his / her jaw sideways 107. A seventh region signal value is generated.

[0161] Figure 5 is an illustration of the same user 100 shown in Figure 4, but with user 110 generating non-zero composite signal values. The value of the first body part signal value 111 is zero because the head is not tilted left or right. The value of the second body part signal value 112 is zero because the head is not rotated either left or right. The value of the third body part signal values ​​113L and 113R is 1 because the user's right shoulder is raised (the user's left shoulder is at rest). (If the user's right shoulder was at rest and the user's left shoulder was raised, the signal value would be 2.)

[0162] The fourth region signal value 114 has a value of 1 because both eyebrows are raised. The fifth region signal value 115 has a value of 2 because the user's left eyelid is closed and the right eyelid is open. The sixth region signal value 116 has a value of zero because the user's mouth is closed. And the seventh region signal value 117 has a value of 2 because the user's chin has shifted laterally to the left of the user's nose.

[0163] Thus, the region signal configuration shown in FIG. 5 corresponds to region signal values ​​(ie, "quantized factor measurements") of {0,0,1,1,2,0,2}. The algorithm for converting such a set of raw site signal values ​​into a "composite" signal value is arbitrary. For example, the order of site signal values ​​is arbitrary, so long as it is consistent with respect to any particular interpretation and / or application of the composite signal values. Similarly, the determination of which of a number of potential site signals to collect from the user and on which user-generated composite signal values ​​to base is arbitrary. (However, it is useful for the user to freely specify which site signal values ​​are preferred, e.g., which site signal values ​​are easiest to implement and / or remember.)

[0164] The assignment of values ​​to the different states associated with each ankle signal value is also arbitrary. In the above example, a value of zero is associated with a state representing the user's resting or neutral posture. However, any state (including the neutral state) for each ankle signal value may be assigned any suitable value. In this case, there are three potential states for each ankle signal value, with suitable values ​​ranging from 0 to 2.

[0165] 6 illustrates an exemplary embodiment of the present disclosure, in which a user 120 creates music using only four-signal movements: muscle contractions that the user specifies as the four signals are detected and measured with sensors attached to the skin.

[0166] This exemplary embodiment detects and measures the state of the user's signal characteristics through the use of signals generated by "contact sensors" (121 through 126). Alternative embodiments use contact sensors that detect and measure electrical signals (voltage, capacitance, and / or current) associated with nerve innervation. Other alternative embodiments use contact sensors that detect and measure electrical signals (voltage, capacitance, and / or current) associated with muscle contraction. Yet other alternative embodiments use contact sensors that detect and measure changes in magnetic field strength, such as may be associated with nerve innervation and / or muscle contraction. Yet other alternative embodiments use contact sensors that detect and measure extension and / or flexion.

[0167] Regardless of their mode of operation, these sensors directly and / or indirectly detect and measure certain changes in the geometry of the user's face, head, neck, and / or shoulders that the user uses to specify codes that convey input values ​​to each respective embodiment.

[0168] In the exemplary embodiment of FIG. 6, the contact sensors transmit their state (e.g., the magnitude of strain and / or electrical activity they are detecting) wirelessly 127 (e.g., Wi-Fi) to a receiver 128 connected to the embodiment. The received signals at each instant in time (specified by time reference line 130) are stored in memory 129, where they are analyzed by a module 131 that converts (e.g., quantizes) the raw sensor signals into an array of "quantized factor measurements" 132 that act as a "change signature" that defines a unique combination of factor-specific geometric changes to the input signal. The change signature is then converted by a conversion module 133 into a corresponding embodiment- and / or application-specific "symbolic value" 134, which in this illustration is a specific musical chord. This exemplary embodiment then processes the user-specific symbolic value 134 in a synthesizer module 135 by synthesizing the indicated musical tone and outputting it as voice 137 on a speaker 136.

[0169] The user shown in Figure 6 specifies and communicates to the embodiment a symbolic value for "Guitar (chord) A#." The user does this through their own specification via the geometrical changes of their face, head, neck, and shoulders of the corresponding value-specific code "0,1,1,0."

[0170] The user 120 specified value-specific codes of 0,1,1,0 by self-performing four unique (i.e., four factor-specific) transformations to the geometry of their face, head, neck, and shoulders. The correspondence between the geometric transformations and the associated factor-specific codes corresponds to the codes shown in the table below. [Table 4]

[0171] The first signal feature indicates the first row of the above table, i.e. either the left or right shoulder is raised, or neither shoulder is raised. The two contact sensors 121 detect the lifting of the user's left shoulder 121L and right shoulder 121R. In this illustration, the user's right shoulder is raised, while the left shoulder is at rest. As shown in the above table, this corresponds to a signal value of 0 (according to an arbitrary definition).

[0172] The second signal feature is identified in the second row of the table above, namely, whether both eyebrows are raised or frowning, or both eyebrows are in a resting state. One contact sensor 124 detects the raising and lowering of the user's eyebrows. In the figure, the user's eyebrows are raised. As shown in the table above, this corresponds to a signal value of 1 (according to an arbitrary definition).

[0173] The third signal feature is identified in the third row of the table above, namely, one eye or the other eye is closed, or both eyes are open (or closed). The two contact sensors 125 detect the opening and closing of the user's left eyelid 124L and right eyelid 124R. In this illustration, the user's right eye is open and the left eye is closed. As shown in the table above, this corresponds to a signal value of 1 (according to an arbitrary definition).

[0174] The fourth and final signal feature is specified in the fourth row of the above table, namely, lateral displacement of the jaw to the left or right, or jaw at rest, centered. Two contact sensors 126 detect movement of the user's jaw to the user's left 126L and right 126R. In this illustration, the user's jaw is displaced to the user's left. As shown in Table 1, this corresponds to a signal value of 0 (according to an arbitrary definition). Thus, in the figure, within the context of arbitrarily selected anatomy signal values, the user 120 shown in the figure has communicated to the device a set of anatomy signal values ​​of {0,1,1,0}.

[0175] Each contact sensor (121 to 126) continuously transmits its state (127) to a receiving module 128 within the device. The receiving module 128 continuously transmits a current set of raw, possibly analog, sensor values ​​129, with the current values ​​indicated by a timestamp 130, to an encoder 131 which converts the raw data transmitted by the contact sensors to the receiver into a predetermined site signal value 132.

[0176] The encoder determines which of the potential regional signal values ​​associated with each regional signal best matches the raw (possibly analog) value transmitted by each respective sensor. For example, in this illustrated embodiment, each regional signal value has three possible "encoded" values: 0, 1, and 2, corresponding to three possible ranges of raw sensor values ​​(e.g., "resting state," "left," and "right"). If this embodiment were based on regional signals each having five values, e.g., ranging from 0 to 4, the encoder would attempt to convert the raw value of each sensor (or corresponding pair of sensors) into one of the five regional signal values.

[0177] In this embodiment, the predetermined region signal value 132 is transmitted to a converter 133 which generates a single composite signal value. In this case, four 3-state values ​​are transmitted to the converter. This means that the composite signal value can have a maximum of 3 x 3 x 3 x 3 = 81 unique values. This corresponds to a radix 3, 4-bit value. This corresponds to a radix 2, i.e. a binary number with 6.34 bits of information.

[0178] The symbolic value 134 generated by the transducer 133 in response to the composite signal value 132 (i.e., the "change signature") {0,1,1,0} and communicated to the synthesizer 135 happens to specify the guitar chord A#, and upon receiving this encoded part signal value from the transducer, the synthesizer 135 generates the specified guitar chord and sends it to the audio output module 136, which outputs the specified note 137.

[0179] If the audio output module 136 is a headphone, the user 120 can play and listen to the music he or she is creating silently, using only the movements of the four (set of) facial features to generate the music, and not disturb anyone nearby who may not want to hear the music at the time.

[0180] 7 illustrates an exemplary embodiment of the present disclosure, in which a user 140 generates text (e.g., a text message) using only displacements of seven selected regions of the geometry of his or her face, head, neck, and shoulders (i.e., the same seven regions shown in FIG. 4).

[0181] User 140 is running the same non-zero signal configuration shown and described with reference to Figure 5. This embodiment has a camera 141 that continuously captures images 142 of the user's face, head, neck, and upper body. Camera 141 transmits each captured image (e.g., each frame of video) to an encoder module 143.

[0182] The encoder module 143 determines which signal features this user 140 prefers and is most proficient at run time. The user 140 is initialized with a "user configuration" file 144 that specifies which of the seven signal features the user 140 prefers to use. In this case, the user configuration file 144 indicates that this user 140 prefers to use the seven signal features shown in and described with reference to FIG.

[0183] Using preferences specified in a user configuration file 144, the encoder module 143 analyzes the image 142, identifies the seven required signal features, and measures their status. At the end of its analysis, the encoder module 143 outputs an array 145 of quantized factor measurements, which are {2, 0, 1, 2, 1, 0, 0} for this regional signal configuration (i.e., the same as shown in FIG. 5).

[0184] Note that the sequence of anatomy signal values ​​for this user signal feature configuration is {2,0,1,2,1,0,0}, rather than {1,0,0,0,2,2,2} (which was the sequence associated with the set of anatomy signal values ​​associated with FIG. 5).

[0185] In this embodiment, the user configuration file 144 specifies not only WHICH signal features this user prefers to use, but also the ranked order of those signal features. For example, the user may choose to use those two or three signal features that they are most comfortable and proficient with as the least significant bits. The user may also choose to use the one or two signal features that they are least comfortable with as the most significant bits.

[0186] For the signal feature configuration shown in Figures 4 and 5, the (default) ranked order of the individual features is as follows: [Table 5]

[0187] In this case, the user has chosen to use the same seven signal features as shown in Figures 4 and 5. However, in contrast to the examples shown in Figures 4 and 5, this user 140 has chosen to assign a different ranked order (i.e., preferences) to these seven signal features. These preferences are also stored in and read from a user configuration file 144. They are then used by the encoder module to select which signal features to search for and measure in each image 142 sent to it by the camera 141. Thus, because of this user's preferences, the anatomy signal values ​​are output in the following order: [Table 6]

[0188] The array of ranked and quantized factor measurements 145 is sent from the encoder module 143 to a transformer module 146 which generates (in this case) application specific composite signal values ​​148. In this embodiment, the transformer module 146 is initialized with an "application configuration" file 147. Also, in this example, this application configuration file 147 indicates that only the four lowest order signal values ​​are used to generate the composite signal value 149.

[0189] In this example, the lowest order signal values ​​are {2,1,0,0}. These are base 3 bits, so 2100 3 This composite signal value corresponds to the decimal value of 63, i.e. (2 x 27) + (1 x 9) + (0 x 3) + (0 x 1) = 63, which is the numeric value of the ASCII character "?" (i.e., question mark).

[0190] Converter module 146 sends the numeric value 63 (ie, the ASCII value for a question mark) to text editing module 150 where it is appended to the text that user 140 is composing.

[0191] FIG. 8 is an illustration of a user 190 using an exemplary embodiment 191 of the present disclosure, which is similar in size to a smartphone.

[0192] A user 190 holds a device embodiment 191 in front of their face, allowing the selfie camera to capture an image of their face, head, neck, and upper body. The user 190 routes audio (if any) generated by the device 191 to their ears via headphones 192 to prevent anyone nearby from hearing the audio and / or to prevent the user 190 from accessing any of their possessed information that may be disclosed in an audio conversation (e.g., speech from someone they are having a "silent" (audible to participants) conversation with).

[0193] 9 is a flow chart illustrating some of the steps performed by an exemplary embodiment of the present disclosure to allow a user to input commands and / or data to the embodiment, its operating system, and / or applications running therein by generating "signals" (i.e., to "consumers"), which then process each composite signal value.

[0194] An embodiment generally indicated at 200 uses at least one sensor (i.e., "detector") to obtain data regarding the geometry of a user's face, head, neck, and / or shoulders, muscles therein, and / or nerves therein. In some embodiments, at least one sensor determines the "quantized" state of the user's anatomy features to use to generate at 202, and then transmits that data to a module within an embodiment that transmits a corresponding set (i.e., array) of "quantized factor measurements" for the set of anatomy features to a converter module at 203. In an embodiment generally shown at 204, the converter module converts the array of quantized factor measurements (i.e., a "change signature") into an embodiment-specific and / or application-specific symbolic value. In an embodiment generally shown at 205, the converter module transmits the symbolic value to an appropriate "consumer" within the embodiment and / or application for processing in an appropriate manner, for example, via adding a character specified by the symbolic value to a document being edited by the user.

[0195] 10 is a flow chart illustrating some of the steps performed by an exemplary embodiment of the present disclosure, allowing a user to input commands and / or data to the embodiment, its operating system, and / or applications running therein (i.e., to the "consumer") by generating "signals." In this embodiment, the user can customize the encoder module by initializing it with a file that specifies the user's preferences, e.g., the signal characteristics that the user prefers. The encoder module is also customized to rank the order in which the resulting anatomy signal values ​​are sent to the transducer module according to the user's preferences.

[0196] The steps illustrated herein differ from those of FIG. 9 in that initialization in an “application configuration” file customizes the behavior of the converter to synthesize composite signal values ​​appropriate for the “consumer” application or operating system from the composite signal values ​​passed by the encoder.

[0197] An embodiment generally indicated at 300 allows a user of an embodiment to specify preferences to customize the embodiment and make it easier for them to use. These preferences are stored in a "user configuration" file. An embodiment generally indicated at 301 initializes an encoder module with preferences specified in a user configuration file associated with a user of the embodiment. An embodiment generally indicated at 301 initializes a converter module with a file containing preferences and / or other customization parameters associated with an application the user is running on the embodiment. Steps 302 to 307 are identical to steps 200 to 205 described in FIG. 9.

[0198] 11 illustrates the differences between "face tracking", "eye tracking", and / or "retina tracking" systems and the present disclosure. As shown in FIG. 11_1 and FIG. 11_2, changes in the orientation of the user's face, body, and / or "direction of gaze" are sufficient to change the values ​​input to the system (e.g., the horizontal (y) screen coordinate of the cursor) in response to changes in the user's appearance from the system's viewpoint.

[0199] In contrast, as shown in Figures 11_3 and 11_4, changes in the orientation of a user's face, body, and / or "direction of gaze" are not sufficient to change the value generated within an exemplary embodiment of the present disclosure for the spatial configuration of the user's face and / or body; only changes in the spatial configuration of the user's face and / or body relative to themselves can change the value that is encoded.

[0200] FIG. 11 illustrates an exemplary embodiment 206 of the present disclosure based on the orientation of a user's face, body, and / or body parts relative to an exemplary embodiment, and / or an exemplary embodiment for detecting the spatial configuration and / or curvature of a user's body and / or body parts. This illustrates the independence of the symbolic values ​​encoded and communicated by user 200 to the cameras and / or other sensors used by the

[0201] 11_1 shows users 200a, 208a, and 212a controlling the position of a cursor 214a on displays 206a and 210a of a "retina tracking" computerized device. Cameras 207a and 211a attached to the displays 206a and 210a monitor the user's position relative to the camera's line of sight (perpendicular to the plane of the displays 206a and 210a) and the direction of the user's gaze 205a, 209a, 213a. The device interprets the direction of the user's gaze in conjunction with the position of the user's eyes relative to the cameras and displays to move the cursor 214a to the position on the display 210a where the user is looking.

[0202] FIG. 11_2 shows the result of a user 200b, 208b, and 212b reorienting his head relative to the displays 206b and 210b. The spatial configuration of the user's head (200b, 208b, and 212b) and the parts therein (201b, 202b, and 203b) is not changed relative to the user's body reference frame (with respect to the configuration of FIG. 11_1). However, the device of FIG. 11_2 interprets the user's changed gaze direction (205b, 209b, and 213b) as a user designation of a new cursor position 214b.

[0203] FIG. 11_3 shows users 200c, 208c, and 212c composing text 214c on displays 206c and 210c of an exemplary embodiment of a computerized facial and / or body decoding system of the present disclosure. Cameras 207c and 211c attached to displays 206c and 210c monitor the user's face 200c and / or body. The embodiment interprets the spatial configuration of the user's face and / or body parts (e.g., 201c, 202c, and 203c) that are associated with a particular binary value as a particular character (and / or other text element such as a word or phrase). In this case, the embodiment interprets the user's facial configurations 201c, 202c, and 203c as designating the letter "A" 214c.

[0204] FIG. 11_4 shows that there is no effect on the behavior of the exemplary embodiment of the present disclosure caused by the user 200d, 208d, and 212d reorienting his head relative to the displays 206d and 210d. The spatial configuration of the user's head (200b, 208b, and 212b) and the portions therein (201b, 202b, and 203b) is not changed relative to the user's body reference frame (with respect to the configuration of FIG. 11_3). Also, since the exemplary embodiment interprets the facial and / or bodily encoded values ​​regardless of the user's orientation relative to the embodiment and / or its sensors, the input values ​​to the embodiment and the text generated in response thereto are the same as those generated in FIG. 11_3. The alternative embodiment does not generate the second "A" 214d in the absence of an intervening indication that the user's transmission of the first "A" 214c character is complete (e.g., by a sufficiently long pause or delay, by the user's change and subsequent reworking of the "A" facial configuration, etc.).

[0205] 12 illustrates the differences between "face tracking", "eye tracking", and / or "retina tracking" systems and the present disclosure. As shown in FIG. 12_1 and FIG. 12_2, a change in the orientation of the user's face and / or body relative to the retina tracking system and / or its camera can result in the assignment of the same value (e.g., horizontal (y) screen coordinate of the cursor) if it coincides with a "complementary" change in "gaze direction".

[0206] In contrast, as shown in Figs. 12_3 and 12_4, the change in the spatial configuration of the user's face and / or body parts relative to the inherent reference frame of the user's body is an example of the present disclosure. The illustrative embodiment always results in the specification of different values ​​(which is equally impractical in either case unless a valid code is specified in either configuration).

[0207] FIG. 12_1 shows users 300a, 308a, and 312a controlling the position of a cursor 314a on displays 306a and 310a of a "retina tracking" computerized device. Cameras 307a and 311a attached to the displays 306a and 310a monitor the user's position relative to the camera's line of sight (perpendicular to the plane of the displays 306a and 310a) and the direction of the user's gaze 305a, 309a, 313a. The device interprets the direction of the user's gaze in conjunction with the position of the user's eyes relative to the cameras and displays to move the cursor 314a to the position on the display 310a where the user is looking.

[0208] Fig. 12_2 shows the result of users 300b, 308b, and 312b reorienting their heads relative to displays 306b and 310b. However, unlike the situation shown in Fig. 11_2, in this case users 300b, 308b, and 312b have modified their "gaze direction" such that the retina tracking system interprets the new combination of the users' camera-specific relative orientations as specifying the same input values, i.e., the same cursor positions, in combination with the users' modified facial structures as shown in Fig. 12_1.

[0209] FIG. 12_3 shows users 300c, 308c, and 312c composing text 314c on displays 306c and 310c of an exemplary embodiment of the present disclosure. Cameras 307c and 311c attached to displays 306c and 310c monitor the user's face 300c and / or body. The embodiment interprets the spatial configuration of the user's face and / or body parts (e.g., 301c, 302c, and 303c) that are associated with a particular binary value as a particular character (and / or other text elements such as words and phrases). In this case, the embodiment interprets the user's facial configurations 301c, 302c, and 303c as specifying the letter "A" 314c.

[0210] FIG. 12_4 illustrates the effect on the behavior of an exemplary embodiment of the present disclosure caused by users 300d, 308d, and 312d reorienting their heads relative to displays 306d and 310d. The spatial configuration of the user's head (300b, 308b, and 312b) and portions therein (301b, 302b, and 303b) is altered relative to the user's body reference frame (with respect to the configuration of FIG. 12_3). Also, because the exemplary embodiment interprets face- and / or body-encoded values ​​regardless of the user's orientation relative to the embodiment and / or its sensors, the input values ​​to the embodiment and the text generated in response thereto (i.e., "B") are different from the values ​​generated in FIG. 12_3 and the corresponding text (i.e., "A").

[0211] 13 illustrates the constancy in values ​​interpreted by an exemplary embodiment of the present disclosure in response to a user moving relative to the embodiment and / or its camera. The exemplary embodiment associates the same value with a consistent spatial configuration encoded on the user's face, regardless of the user's movement through the field of view of the embodiment and / or its camera.

[0212] FIG. 13_1 shows a user 400a, 408a located at the leftmost portion of a camera 407a of an exemplary embodiment 406a of the present disclosure. The user is moving from left to right across the field of view of the camera with a speed 417a. The user 400a is looking in a "front direction" 405a while conveying signals to the embodiment encoded by eyebrows 401a, eyelids 402a, and upper and lower lips 403a. This geometric configuration of the user's face results in the generation and processing of a particular symbolic value, for example, the embodiment of a "V".

[0213] FIG. 13_2 shows the same user 400a, 408a, here 408b, 401b, located at the center of the camera 407b of the exemplary embodiment 406b of the present disclosure. The user is still moving from left to right across the camera's field of view at a speed 417b. The user 400b is still looking in the "front direction" 405b, conveying the same signals to the embodiment encoded by the eyebrows 401a, eyelids 402a, and upper and lower lips 403a. This geometric configuration of the user's face further results in the generation and processing of the same symbolic value, e.g., the embodiment of "V", as generated in FIG. 13_1.

[0214] FIG. 13_3 shows the same user 400a, 408a, now 408c, 401c, located at the rightmost portion of the camera 407c of the exemplary embodiment 406c of the present disclosure. The user is still moving from left to right across the camera's field of view at a speed 417c. The user 400c is still looking in the "front direction" 405c, conveying the same signals to the embodiment encoded by the eyebrows 401a, eyelids 402a, and upper and lower lips 403a. This geometric configuration of the user's face further results in the generation and processing of the same symbolic values, e.g., "V" embodiments, as generated in FIG. 13_1 and FIG. 13_2.

[0215] 14 illustrates the constancy of values ​​interpreted by exemplary embodiments 506b, 507b of the present disclosure in response to the acceleration of a user 500b, either absolutely (i.e., relative to the Earth) or relative to the embodiment, device, and / or its camera. A "head orientation tracking" system such as that shown in FIG. 14_1 changes the values ​​input to the system, and / or the system's interpretation of those values, in response to the acceleration (absolute and / or relative) of the user 500a, the user's head, and / or the system, device, and / or its camera.

[0216] In contrast, the exemplary embodiment of the present disclosure generates inputs only in response to the spatial configuration (of face, head, body, etc.) generated by the user relative to the user's original body frame(s) of reference. Thus, with respect to this exemplary embodiment, acceleration (absolute and / or relative) of the user, the user's head, and / or the embodiment and / or its camera does not change the values ​​generated or subsequent interpretation by the embodiment if the spatial configuration of the user's face, head, body, etc. remains constant. 14_1 illustrates an embodiment of a "head orientation tracking" system 506. This embodiment controls the presentation of virtual landmarks 510a to the user and adjusts the presented perspective to match the rotation (i.e., rotational acceleration) undergone by the user's head 500a, 508a.

[0217] Initially, system 506a1 displays a view 510a1 of a virtual world, and user 512a1 can see tree 514a1 as he faces the display 521a1. The system has a camera 511a1, but is not responsive to the geometry of user 500a's eyebrows 501a or other facial features. User 500a is looking directly at camera 507a1 505a.

[0218] When the user rotates his / her head 508a from the original orientation 508a1 counterclockwise to a new orientation 508a2, the position of the "head orientation tracking" system 506a1 also rotates around the same axis of rotation (i.e., the user's vertical axis) to a new position 507a2. The system initially presented views 517a1, 510a1 to the user 512a1, but after the user and system rotation, the system presents views 517a2, 510a2 to the user 512a2. In this changed view 510a2, the user 512a2 can now see the house 520, and the tree that was originally on the right side 514a1 of the presented view 510a1 is now on the left side 514a2 of the view 510a2.

[0219] In contrast, FIG. 14_2 illustrates an exemplary embodiment 506b1 of the present disclosure, which facilitates the composition of text (i.e., a text editor) for users 500b, 508b.

[0220] Initially, the system 506b1 displays a view 510b1 of the document being edited, showing the addition of a single letter "A" 514b1. This embodiment has a camera 511b1 that detects and responds to at least the geometry of the eyebrows 501b of the user 500b. The user 500b is looking directly 505b with the camera 507a1.

[0221] As the user rotates his / her head 508b from the original orientation 508b1 counterclockwise to a new orientation 508b2, the position of the embodiment 506b1 also rotates about the same axis of rotation (i.e., the user's vertical axis) to a new position 507b2. The system initially presented views 517b1, 510b1 to the user 512b1, but after the user and system rotation, the system now presents the same views 517b2, 510b2 to the user 512b2. In this modified view 510b2, the geometry of the user's face 500b remained unchanged by the rotation, so that an additional letter "A" 514b2 has now been added to the document the user was editing.

[0222] FIG. 15 illustrates the constancy of values ​​generated and / or interpreted by an exemplary embodiment 606 of the present disclosure that responds to movement of a user 600, the user's face, or any other body part of the user relative to the embodiment and / or its sensors (also responds to "back and forth" motions).

[0223] FIG. 15_1 shows a camera 607a of an embodiment 606a for a user 608a whose head is at a distance "dA" from the camera. The normal 604a of the user's face 600a is parallel to the normal of the field of view of the camera 607a. The user 600a looks directly at the camera 605a. The user has a facial geometry consisting of raised eyebrows 601a, fully open eyelids 602a, and slightly parted upper and lower lips 603a. The user's facial geometry will evoke a particular symbolic value, e.g., "green", from the embodiment.

[0224] In Fig. 15_2 and Fig. 15_3, users 600b, 600c move directly towards cameras 607b, 607c. During processing, the distance of the user's head from the camera is decreased to "dB" and then to "dC". However, the geometric shape of the user's face remains unchanged. And the present embodiment will continue to generate the same symbolic value, e.g., "green", as was generated in Fig. 15_1.

[0225] In Fig. 15_4 and Fig. 15_5, the user 600d, 600e moves directly away from the camera 607d, 607e. During the process, the distance of the user's head from the camera is increased from "dC" to "dD" and then to "dE". However, again, the geometry of the user's face remains unchanged. And the present embodiment will continue to generate the same symbolic value, e.g., "green", as was generated in Fig. 15_1 to Fig. 15_3.

[0226] Despite the back and forth movement of the user 600 towards and then away from the camera of the embodiment, the symbolic values ​​generated remained unchanged because their definition depended only on the geometry of the user's face, which remained unchanged throughout the back and forth movement.

[0227] FIG. 16 is a diagram illustrating the generation of a series of values ​​by a user in accordance with the present disclosure. The user 700a generates a symbolic value of "F" as a result of the illustrated raised eyebrows 701a, slightly open mouth 703a, and tilted head 720a in accordance with an exemplary embodiment of the present disclosure. User 700a holds this geometric shape for a period of time "t0".

[0228] The user 700 then transitions 721ab from geometric shape 700a to geometric shape 700b over time period "t1." During time period "t2," the user 700b maintains the geometric shapes, namely, frowning eyebrows 701b, closed mouth 703b, and right tilted neck 720b, which are interpreted by the embodiment as symbolic value "A" 714b.

[0229] The user 700 then transitions 721bc from geometric shape 700b to geometric shape 700c over time period "t3." During time period "t4," the user 700c maintains the geometric shape, i.e., neutral eyebrows 701c, fully open mouth 703c, and tilted neck to the left 720c, which are interpreted by the embodiment as symbolic value "C" 714c.

[0230] The user 700 then transitions 721cd from geometric shape 700c to geometric shape 700d over time period "t5." During time period "t6," the user 700d maintains the geometric shape, i.e., raised eyebrows 701d, closed mouth 703d, and tilted neck 720d, which are interpreted by an embodiment as symbolic value "E" 714d.

[0231] The user 700 then transitions 721de from geometric shape 700d and back to the same geometric shape as 700d over time period "t7." During time period "t8," the user 7003 again maintains the same geometric shape 700d, namely, raised eyebrows 701e, closed mouth 703e, and untilted neck 720e, which are interpreted by the embodiment as symbolic value "E" 714e.

[0232] FIG. 17 illustrates the utility of an exemplary embodiment of the present disclosure. In this example, a user 800 cannot use his left hand 822 or its fingers to enter text into his smartphone 806 because the hand 822 is holding a briefcase 824. The user cannot use his right hand 823 or its fingers to enter text into his smartphone 806 because his field of view 805 is fixed on the object he is observing (left of the page). The user cannot issue a message regarding the object he is watching because he may direct his attention to the object. And the user cannot even see the screen 807 of the phone 806 because he is looking 805 at the object.

[0233] The present disclosure allows a user to input text and thereby compose a message to a contact using only muscle flexion patterns of the face, head and / or other body parts. Furthermore, by using a unique code (e.g., a mapping of physical spatial configurations to ASCII characters) known only to the user and the contact, it is possible to prevent someone else who can see the user's face and / or physically encoded message from understanding its meaning.

[0234] 18 is a diagram illustrating the delivery of the American Sign Language (ASL) spelling of the word "friend." The illustrated sign is produced and delivered by the presentation of "static" (i.e., unmoving, unchanging) hand configurations.

[0235] The letter "f" 906 is communicated via the ASL hand shape represented by hand 900. Similarly, the letters "r" 907, "i" 908, "e" 909, "n" 910, and "d" 911 are communicated via the ASL hand shapes represented by hand configurations 901 through 905, respectively. And even though ASL specifies no method or protocol for instructing a user how to properly transition from each letter to the next 912 through 916, someone interpreting such signs would likely understand them because the pace at which they are produced and displayed is fairly constant, i.e. ,They can be distinguished because the spacing between each pair of signatures is approximately constant.

[0236] Similarly, when a user generates symbolic values ​​encoded by value-specific geometric shapes of the face, head, neck, and / or shoulders and communicates them to an embodiment of the present disclosure, the user is expected to intuitively adopt a code generation cycle in which the codes are retained for a period of time and the latency between the generation of successive codes is consistent.

[0237] Figure 19 illustrates the communication of eight different words through the use of American Sign Language. The signs illustrated are produced and communicated through the presentation of "dynamic" (i.e., moving, changing) hand configurations, where the presentation of any static parts is expected to violate the protocol and prevent its comprehension by a third party.

[0238] Because embodiments of the present disclosure rely on static (and non-"moving") geometry of the face, head, neck, and / or shoulders, ASL signs (some of which are "moving" and utilize the user's hands) may not fit into the design and therefore may not be understood by the embodiments.

[0239] 20 illustrates an ATM embodiment 1006 of the present disclosure in which a customer 1000 communicates their password or a portion thereof to the embodiment via geometric changes in the face, head, neck, and / or shoulders. In this example, the user 1000 presents to the embodiment with raised eyebrows 1001, closed mouth 1003, and fully open eyes, thereby communicating to the embodiment 1006, via camera 1007, one symbolic value believed to represent a portion of the user's password and / or passphrase. This communication of the user's password or a portion thereof by a geometrically encoded signal may be done in conjunction with the entry of another password, or a portion of the same password, using the user's fingers 1028 to press keys on the embodiment's keyboard 1026.

[0240] While the user is communicating the password, the embodiment detects, analyzes, and transforms the four geometric configurations of the user's face, head, neck, and shoulders shown in representative images of users 1020a-d. The geometric shapes captured in the user's images are transformed into the symbolic values ​​"p" 1022a, "o" 1022b, "G" 1022c, and "0" (zero) 1022d.

[0241] If verified to be accurate by an embodiment, the user will be allowed to conduct the financial transaction.

[0242] FIG. 21 illustrates an embodiment 1107 of the present disclosure in which a worker 1100 is communicating with a third party (e.g., another person) by symbolic values ​​encoded by the worker himself through changes in the geometry of his face, head, neck, and / or shoulders, and thus can be conveyed to the third party even from within an environment 1141 that is so noisy that the voice communication would not be understood by the third party.

[0243] The worker 1100 is using both hands 1127L and 1127R to manipulate a jackhammer 1142, which bounces up and down 1140 and creates excessive noise 1141. To protect the worker's hearing, the worker 1100 wears sound-reducing ear covers 1130L and 1130R that include speakers that allow the worker to hear words spoken by the party speaking via the embodiment 1107 of the present disclosure.

[0244] The embodiment 1107 takes an image 1120 of the worker's face and shoulders, transforms certain parts of the face geometry, such as those related to eyebrows 1101 and lips 1103, and transmits the text to the other party. The other party's reply can be heard by the user 1100 via a speaker in the ear cover 1130.

[0245] Because the user must use both his hands 1127L and 1127R to control the noise-making implement 1142, he cannot use his fingers and / or the virtual keyboard on the smartphone to compose text messages, and because of the dangerous nature of the user's work, he must look closely at the implement and the ground below it and cannot see the smartphone display to compose such text messages.

[0246] The embodiment transmits 1131 the user's message (text and / or synthesized voice) to a cell tower or other compatible wireless receiver.

[0247] 22 illustrates an embodiment 1207 in which an artist 1200 composes a digital image on a display 1210 of an embodiment by issuing color values ​​encoded by the artist 1200 to an embodiment via modification of the geometry of his or her face, head, neck, and / or shoulders (as shown in representative image 1254, which is captured, analyzed, and interpreted by the embodiment). The artist is also using gestures to simultaneously control the movement of a virtual paintbrush. The user 1200 uses the geometry of at least the user's eyebrows 1201, eyelids 1255, and lips 1203 to identify the geometry that specifies a particular user-selected symbolic value, which for this embodiment is color (e.g., specified as an RGB value).

[0248] This embodiment also incorporates the ability to understand, detect and interpret various gestures performed by a user with left hand 1227L and right hand 1227R through appropriate movement of each of hands 1228L and 1228R towards and away from the embodiment's camera, for example to control movement of a cursor from position 1253 to position 1214.

[0249] This embodiment also incorporates the ability to change the perspective view displayed to the user in response to the user's head rotation 1221. This allows the user to shift attention to any portion of the digital image that exceeds the size of the display across 1210L-1200R (at the resolution of the display device), thereby limiting the view available on the display to only the selected portion 1210c of the full image.

[0250] FIG. 23 shows embodiments 1350, 1306, 1307 in which a gamer 1300 adjusts parameters suitable for controlling his or her virtual player in the game, possibly displayed on a screen 1310 of an embodiment of the game, by changes to the geometry of his or her face, head, neck, and / or shoulders, as detected, analyzed, and resolved based on images captured by a camera 1307 of the embodiment, e.g., by changes to at least the geometry of his or her eyebrows 1301 and eyelids 1302.

[0251] User 1300 cannot use his or her voice to adjust his or her private player-specific settings and / or preferences, as those commands are simultaneously broadcast to all other collaborative members of the team via his or her microphone 1360 (e.g., via the Internet 1331, 1362).

[0252] The user 1300 is unable to use his or her hands to manually adjust such player-specific settings and / or preferences because his or her hands 1327L, 1327R are dedicated to controlling the respective joysticks 1361L and 1361R.

[0253] While the present invention has been described in terms of the above preferred and alternative embodiments, it will be readily apparent to those skilled in the art that numerous modifications and / or additions can be made to the above-described embodiments. An embodiment can be defined as a system of one or more components in a particular structural and / or functional relationship, as a method of use or assembly performed by someone, any subset or all of the components, and / or a user, and / or as a subassembly or submethod. The present invention can include each of the individual components separately. However, it is intended that the scope of the present invention extends to all such modifications and / or additions, and that the scope of the present invention is limited only by the claims set forth herein.

[0254] Individual elements or features of a particular embodiment of the present teachings are generally not limited to that particular embodiment, but, where applicable, even if not specifically shown or described, can be interchanged and used in other embodiments. It can be varied in many ways. Such variations should not be considered as departures from the present teachings, and all such variations are intended to be included within the scope of the present teachings.

[0255] The terms used herein are for the purpose of describing certain exemplary embodiments only and are not intended to be limiting. As used herein, the singular forms "a," "an," and "the" may be intended to include the plural unless the context clearly indicates otherwise. The terms "comprises," "comprising," "including," and "having" are inclusive and thus identify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein should not be construed as necessarily requiring the execution in the particular order described or illustrated, unless specifically identified as the order of execution. It should also be understood that additional or alternative steps may be employed.

[0256] Terms such as first, second, third, etc. are used herein to describe various elements, components, regions, layers, and / or sections, but these elements, components, regions, layers, and / or sections should not be limited by these terms. These terms can be used to distinguish one element, component, region, layer, or section from another region, layer, or section. Terms such as "first," "second," and other numerical terms, when used herein, do not imply a sequence or order unless clearly indicated by the context. Thus, a first element, component, region, layer, or section described below can be referred to as a second element, component, region, layer, or section without departing from the aspects of the present teachings.

[0257] When an element or layer is referred to as being "on," "engaged," "connected," or "coupled" to another element or layer, it may be directly on, engaged, connected, or coupled to the other element or layer, or there may be intervening elements or layers. In contrast, when an element is referred to as being "directly on," "directly engaged," "directly connected," or "directly coupled" to another element or layer, there are no intervening elements or layers. Other words used to describe relationships between elements should be construed in a similar manner (e.g., "between" and "directly between," "adjacent" vs. "directly adjacent"). As used herein, the term "and / or" includes any and all combinations of one or more associated listed items.

[0258] Spatially relative terms such as "inside," "outside," "below," "below side," "upper," "top," "upper," "forward," "rear," "front" and "rear" are used herein for ease of description to describe the relationship of one element or feature to another, but the present disclosure is intended to encompass various orientations of the instrument in use or operation in addition to the orientation depicted in the figures. For example, if the instrument in the figures is turned upside down, an element described as "below" or "below" the other element or feature would be oriented "above" the other element or feature. Thus, the example term "below" can include both an upward and downward orientation. The device may be otherwise oriented (rotated 90 degrees or otherwise) and the spatially relative descriptors used herein would be interpreted accordingly.

Claims

1. using at least one sensor to measure the extent to which particular muscles located at least partially within the user's face, head, neck, and / or shoulders contract, thereby deriving specific measurements of the plurality of muscles; quantizing each of said intrinsic muscle measures; combining the unique quantized muscle measurements into a unique composite number for each unique combination of quantized muscle contractions; converting said composite number into a context-specific symbolic value; the symbolic values ​​are processed by the application such that the application can provide application-specific output based on the symbolic values; a transformation circuit inputting at least one of said at least one symbolic value to at least one processor that modifies at least one component and / or behavior of at least one computing device on which said application is implemented, and using said context-specific symbolic value to modify at least one bit in a file or buffer of digital information and / or modify the behavior of the at least one computing device; the conversion circuit uses an embedded processor and memory to generate an index of a predefined subrange within a range in which valid sensor-specific measurements are found, and uses the index as a key to look up the corresponding symbolic value in a table of key-value pairs; information is communicated to said embedded processor using at least three sets of body parts moved by said user to communicate alphanumeric symbols to said embedded processor; the conversion circuitry, using the embedded processor and the memory, quantifies the degree of distortion of the body part moved by the user and assigns a numerical value used to determine input to the application; The method, wherein the quantizing, combining, and converting are performed by a first processor embedded within the sensor, and modifying at least one bit in the digital information file or buffer is performed by a second processor within the computing device.

2. The method of claim 1 , wherein for the at least one sensor measurement, the generated symbolic value is the same whenever the same at least one sensor measurement is characterized by the same measurement value.

3. The method of claim 1 , wherein at least one of the at least one symbolic values ​​is an identifier of a command capable of modifying at least one aspect of at least one behavior.

4. At least one of the at least one symbolic value modifies at least one aspect of at least one behavior of at least one application. The method of claim 1 , wherein the command is an identifier of a command that can be executed.

5. at least one sensor configured to capture at least one block of data related to at least one factor that changes contemporaneously with at least one change in the geometry of at least one user's face, head, neck, and / or shoulders, wherein the at least one geometric change is manifested via innervation of at least one nerve of the at least one user by the at least one user, the innervation of the at least one nerve resulting in contraction of at least one muscle, the contraction of the at least one muscle contributing, at least in part, to the manifestation of the at least one change in the geometry of the at least one user's face, head, neck, and / or shoulders; means for determining at least one characteristic measurement from at least one portion of said at least one block of data; means for quantizing each of said at least one characteristic measurement of at least one of said at least one user to generate at least one "quantization factor measure"; means for converting said at least one quantization factor measure into at least one user-specific "variation signature"; means for mapping said at least one change signature to at least one "symbolic value"; means for processing said at least one symbolic value; Equipped with the symbolic values ​​are processed by the application such that the application can provide application-specific output based on the symbolic values; a transformation circuit inputting at least one of the at least one symbolic value to at least one processor that modifies at least one component and / or behavior of at least one computing device on which the application is implemented; the conversion circuit uses an embedded processor and memory to generate an index of a predefined subrange within a range in which valid sensor-specific measurements are found, and uses the index as a key to look up the corresponding symbolic value in a table of key-value pairs; information is communicated to the processor using at least three sets of body parts moved by the user to communicate alphanumeric symbols to the processor; the conversion circuitry, using the embedded processor and the memory, quantifies the degree of distortion of the body part moved by the user and assigns a numerical value used to determine input to the application; The system further comprises a first processor embedded within the sensor including the determining means, the quantizing means, the converting means, and the mapping means, and a second processor within the computing device including the processing means.

6. 6. The system of claim 5, wherein the mapping comprises: determining at least one index of a reference change signature in at least one list of at least one reference change signature that matches at least one of the at least one change signature; and searching for the at least one symbolic value associated with the same at least one index from at least one list of at least one symbolic value.

7. 6. The system of claim 5, wherein the determining means is configured to discontinue generation and / or processing by the system of at least one of the at least one symbolic value when the determining means determines that an ambiguity in at least one data block corresponds to at least one of the at least one changes in the geometric shape that changed while the at least one data block was captured.

8. 6. The system of claim 5, wherein the determining means is configured to distinguish between the characteristic measurements indicative of a stable geometric shape and those indicative of an early and / or incomplete change to the geometric shape, and to cease generation and / or processing by the system of at least one of the at least one symbolic value if the geometric shape is not stable.

Images