Methods and systems for object control and interaction based on non-sensory information rendering

Abstract

The processing subsystem communicates with brain regions of an animal subject that are responsible for forming concepts, without sensory input. The processing subsystem receives brain signals representing concepts formed by these brain regions, without sensory input. The processing subsystem processes the received brain signals to convert the concepts into computer-readable data that represents their tangible form. The processing subsystem identifies data records associated with one or more data elements of the computer-readable data, and also associated with real-world objects associated with the concepts.

Description

[Technical Field]

[0001] Cross-reference of related applications This application is a continuation-in-part application of U.S. Patent Application No. 17 / 863,480, filed on 13 July 2022, claiming priority to U.S. Provisional Patent Application No. 63 / 226,821, filed on 29 July 2021, all of which are incorporated herein by reference in their entirety.

[0002] This invention relates to the formation of concepts by the brain, and more particularly to methods and systems for rendering and / or injecting such concepts, and for controlling and / or interacting with real-world objects associated with those concepts. Technical Background

[0003] Imagination is the ability to form concepts in mind, including objects and sensations, without any direct input from senses. These concepts (objects and sensations) can take the form of, for example, mental images, phonological passages (i.e., non-acoustic sounds), analogies, and narratives. [Overview of the project]

[0004] A particular embodiment of the present invention provides a method and system for converting a concept formed in the target brain into computer-readable data, and for identifying data records associated with the computer-readable data in order to, for example, control and / or interact with real-world objects related to the concept using the computer-readable data.

[0005] According to the teaching of embodiments of the present invention, a method is provided for use on animal subjects having a brain including a region that plays a role in forming concepts without sensory input. The method includes the steps of: receiving brain signals representing a concept formed by the brain region without sensory input by a processing subsystem that communicates with the brain region; processing the received brain signals by the processing subsystem to convert the concept into computer-readable data representing a tangible form of the concept; and identifying data records associated with one or more data elements of the computer-readable data by the processing subsystem, the data records also associated with real-world objects associated with the concept.

[0006] Optionally, real-world objects are electronic devices.

[0007] Optionally, the method further includes the step of initiating at least one responsive action according to an identified data record.

[0008] Optionally, at least one response action may involve activating or controlling a real-world object.

[0009] Optionally, the method further includes the step of providing feedback to the sensory system in question in response to the success or failure of initiating at least one response action.

[0010] Optionally, the method further includes the steps of i) processing feedback data indicating the success or failure of the initiation of at least one response action by a processing subsystem, thereby converting the feedback data into one or more brain signals, and ii) providing feedback to a subject by providing one or more brain signals to a brain region such that the success or failure of the initiation of at least one response action is formed as a concept by the brain region without sensory input.

[0011] Optionally, at least one response action is selected from a group of response actions, each of which response action is associated with a corresponding data record among a group of data records, and each of which data records is associated with one or more corresponding elements of computer-readable data.

[0012] Optionally, a real-world object is one of several real-world objects, a data record is one of several data records, and several data records consist of several subsets of data records, each subset associated with a corresponding real-world object among the several real-world objects.

[0013] Optionally, the concept can be a mental image or a non-acoustic sound.

[0014] Optionally, computer-readable data can be image data or audio data.

[0015] According to a teaching embodiment of the present invention, a system is also provided for use on animal subjects having a brain including a region that plays a role in forming concepts without sensory input. The system includes a processing subsystem configured to communicate with a brain region, the processing subsystem is configured to receive brain signals representing concepts formed by the brain region without sensory input, process the received brain signals to convert the concepts into computer-readable data representing the tangible form of the concepts, and identify data records associated with one or more data elements of the computer-readable data, the data records also being identified as being associated with real-world objects related to the concepts.

[0016] Optionally, real-world objects include electronic devices.

[0017] Optionally, the processing subsystem may be further configured to initiate at least one response action according to the identified data record.

[0018] Optionally, at least one response action may involve activating or controlling a real-world object.

[0019] Optionally, at least one response action is selected from a group of response actions, each of which response actions is associated with a corresponding data record among a group of data records, and each of which data records is associated with one or more corresponding elements of computer-readable data.

[0020] Optionally, a real-world object is one of several real-world objects, a data record is one of several data records, the system further includes at least one storage medium that communicates with a processing subsystem to maintain the multiple data records, and the multiple data records consist of several subsets of data records, each subset associated with a corresponding real-world object of the multiple real-world objects.

[0021] Optionally, the concept can be a mental image or a non-acoustic sound.

[0022] Optionally, computer-readable data can be image data or audio data.

[0023] According to a teaching embodiment of the present invention, a method is also provided for use on animal subjects having a brain including a region that plays a role in forming concepts without sensory input. The method includes the steps of: receiving brain signals representing a concept formed by the brain region without sensory input by a processing subsystem that communicates with the brain region; processing the received brain signals by the processing subsystem to convert the concept into computer-readable data representing a tangible form of the concept; and activating or controlling a real-world object that is associated with the concept and associated with one or more data elements of the computer-readable data.

[0024] Unless otherwise defined herein, all technical and / or scientific terms used herein have the same meaning as those commonly understood by those skilled in the art in which the present invention pertains. Similar or equivalent methods and materials to those described herein may be used in carrying out or testing embodiments of the present invention, but exemplary methods and / or materials are described below. In the event of any conflict, the patent specification containing the definitions shall prevail. In addition, materials, methods, and examples are illustrative and not necessarily intended to be limiting. [Brief explanation of the drawing]

[0025] Some embodiments of the present invention are described herein merely as examples with reference to the accompanying drawings. Specifically, with reference to the drawings in detail, it is emphasized that the details shown are illustrative and for the purpose of illustrative consideration of embodiments of the present invention. In this regard, the description provided with the drawings will make it clear to those skilled in the art how embodiments of the present invention may be carried out.

[0026] Now, looking at the drawing, similar reference numbers or letters indicate corresponding or similar components. In the drawing, [Figure 1]This is a schematic diagram of a system according to one embodiment of the present invention, which has a processing device for interface with brain regions involved in forming concepts without direct input from perception, and converting brain signals representing the formed concepts or a plurality of formed concepts into data, and vice versa. [Figure 2] This is a block diagram of an exemplary processing device according to one embodiment of the present invention. [Figure 3] This is a schematic diagram of a system environment in which a processing device can operate, showing a memory for storing data received from the processing device according to an embodiment of the present invention, and a transceiver unit connected to the processing device for exchanging data with a remote server and a remote processing system via a communication network. [Figure 4] This is a schematic diagram of a non-limiting deployment configuration of a processing device in the human brain, showing a processing device coupled to the interface between the occipital lobe and the parieto-occipital sulcus, according to one embodiment of the present invention. [Figure 5] Figure 4 is a schematic diagram of an exemplary deployed configuration of a processing device according to one embodiment of the present invention, in which the processing device is embedded in a segment of the transmission route between the parietal lobe and the occipital lobe, providing input from the parietal-occipital sulcus to the occipital lobe. [Figure 6] This is a schematic diagram of an exemplary wired interface, according to one embodiment of the present invention, which includes an electrode array that can be used to interface between a processing device and a brain region. [Figure 7] This is a schematic diagram of an exemplary wireless interface that may be used to interface between a processing device and a brain region, showing a transmitter unit connected to a processing device and an electrode array connected to a receiver unit, according to one embodiment of the present invention. [Figure 8]This flowchart illustrates a process for identifying data records to enable the control of an object or interaction with an object, according to an embodiment of the present invention. [Figure 9] This is a schematic diagram of a system environment similar to Figure 3, showing a data record manager and one or more objects that can be interacted with or controlled, according to another embodiment of the present invention. [Figure 10] Figure 9 is a schematic diagram of a data record manager according to an embodiment of the present invention, showing exemplary data types stored in the data record manager and exemplary relational structures between stored data types. [Modes for carrying out the invention]

[0027] Description of Preferred Embodiments Embodiments of the present invention provide a method and system for receiving brain signals representing a concept or a group of concepts to be formed by a processing subdevice interfaced with a region of the brain of an animal subject that is responsible for forming concepts (i.e., the region responsible for imagination), without direct input from the subject's senses, and processing the received brain signals (by the processing device) to convert the concept or group of concepts into computer-readable data representing the tangible form of the concept or group of concepts, and for identifying data records (by the processing subsystem) that are associated with one or more data elements of the computer-readable data and also associated with real-world objects related to the concepts. The identification of data records can be used to invoke commands to activate or control the real-world objects. This process of receiving brain signals and converting those brain signals into data is interchangeably referred to herein as "imagination rendering," "non-sensory information rendering," or "concept rendering."

[0028] In certain embodiments, the processing subdevice is also configured to process computer-readable data representing a concept or set of concepts formed by a region of the brain, so as to convert the computer-readable data into brain signals, and to selectively provide these brain signals to a region of the brain so that the concept or set of concepts represented by the data is formed by that region of the brain without direct input from the senses of the subject. This process of converting data into brain signals and providing those brain signals to the brain is interchangeably referred to herein as “imagination injection,” “imagination inducement,” “concept injection,” or “concept inducement.”

[0029] Concepts formed by brain regions without direct sensory input, which are converted into data, are interchangeably referred to herein as "a concept that can be rendered," "a render-able concept," "imagination information that can be rendered," "render-able imagination information," "non-sensory information that can be rendered," or "non-sensory render-able information." Concepts converted into data are interchangeably referred herein as "rendered imagination information," "rendered non-sensory information," or "a rendered concept."

[0030] Concepts represented by data that are converted into brain signals and provided to a region to form a concept without direct sensory input are interchangeably referred to herein as "a concept that can be injected or induced," "an injectable or inducible concept," "imagination information that can be injected or induced," "injectable or inducible imagination information," "non-sensory information that can be injected or induced," or "injectable or inducible non-sensory information." Injectable (or inducible) imagination information may also refer to data that represents an injectable / inducible concept. Concepts that are converted into brain signals and represented by data provided to a brain region so that the concept is formed by that region are interchangeably referred to herein as "injected or induced imagination information," "an injected or induced concept," or "injected or induced non-sensory information."

[0031] As will be discussed in detail below, in certain embodiments, the imaginary information that can be rendered or injected using the methods and systems according to embodiments of the present invention may be visual information (e.g., images). In other embodiments, the concept may be visual imagination information, also called “imagined images,” “mental images,” or “mental images.” In other embodiments, the imaginary information (i.e., concepts) that can be rendered or injected using the methods and systems according to embodiments of the present invention may be auditory information (e.g., sounds). In other words, in certain embodiments, the concept may be auditory imagination information, also called “sound imagination information,” “imagination sounds,” “imagined sounds,” “non-acoustic sounds,” or “mental sound.” In further embodiments, the imaginary information may be information associated with other senses of the object, such as touch or smell.

[0032] In the context of this document, in certain cases, “imagination” also includes “dreams” and / or “thoughts.”

[0033] The principles and operation of the method and system according to the present invention can be better understood by referring to the drawings accompanying this specification.

[0034] Before describing in detail at least one embodiment of the present invention, it should be understood that the present invention is not necessarily limited in its application to the details of the construction and arrangement of components and / or methods described below and / or illustrated in the drawings and / or examples. Other embodiments of the present invention are possible and can be practiced or carried out in various ways.

[0035] Referring here to the drawings, Figure 1 is a schematic diagram of a system, generally shown as (10), according to one embodiment of the present invention, for rendering renderable imaginary information and preferably injectable imaginary information. Overall, the system (10) includes a computerized processing device (12) (hereinafter interchangeably referred to as the “processing device”) for interfaceing with a region (103) of the brain (102) of the object (100), which plays a role in forming concepts without direct input from the senses of the object (100) (specifically, at least one of visual or auditory). Note that in certain cases, one or more concepts may be formed by the region (103) based on inputs derived from direct input from the senses, e.g., one or more past or previous direct inputs from the senses of the object (100), or indirect inputs from the senses of the object (100). Therefore, the domain (103) may, in certain cases, form a concept using non-immediate input from the senses of the subject (100), and in other cases, form a concept using past direct and / or indirect input from the senses of the subject (100). Parenthetically, throughout this document, the expressions “without sensory input,” “without subject sensory input,” and “non-immediate sensory input” are used interchangeably to mean “without immediate input from the senses of the subject.” According to certain embodiments, the system (10) operates to convert renderable imaginary information (i.e., renderable concepts) from an intangible form to a tangible form, or to convert computer-readable data representing the tangible form of a renderable concept / renderable imaginary information.In certain embodiments, the system (10) operates to convert computer-readable data representing the tangible form of an injectable concept / injectable imagination information into brain signals provided to an object, such that the concept becomes an intangible form.

[0036] In certain non-limiting embodiments, the brain region (102) (103) interfaced with the processing device (12) is a region that forms visual concepts (i.e., “mental images”). In such embodiments, the imaginative information that can be rendered and injected is visual imaginative information. In other non-limiting embodiments, the brain region (102) (103) interfaced with the processing device (12) is a region that, without sensory input, is responsible for forming sound concepts (phonological passages) (i.e., forming non-acoustic sounds or imagined sounds). In such embodiments, the imaginative information that can be rendered and injected is auditory / sound imaginative information.

[0037] It should be noted that in the non-limiting and exemplary embodiments shown in Figure 1, the subject is a human subject. However, the principles of the present invention are equally applicable to non-human animal subjects, including, for example, canids, felids, non-human primates, rodents, reptiles, birds, and marine / aquatic mammals and non-mammalian species. Generally, for a given animal species, the corresponding brain regions responsible for forming visual or auditory concepts can be identified using brain scanning techniques such as magnetic resonance imaging.

[0038] In a preferred embodiment, the processing device (12) operates to receive brain signals representing at least one concept formed by a region of the brain (103) without sensory input (i.e., without direct input from the senses of the object (100)). In other words, the processing device (12) operates to receive brain signals carrying imagined information (e.g., visual imagined information or auditory imagined information), and thus the brain signals are imagined information-bearing signals. These brain signals can be any kind of information-bearing signals that can be read, picked up, or captured by a computer, or a machine, or a machine-object interface. In one particular non-limiting example, the brain signals are in the form of nerve impulses propagating along a neural pathway that connects to or provides input to a region (103) of the brain (102).

[0039] The process by which the processing device (12) receives brain signals is generally referred to herein as "collecting brain signals" or "collection of brain signals."

[0040] The processing device (12) further operates to process the received brain signals (collected brain signals) to generate (create) data (preferably digital data) from the collected brain signals. Specifically, the processing device (12) operates to process the collected brain signals to convert the renderable imaginary information carried by the brain signals into data representing the imaginary information in a tangible form, thereby rendering the renderable imaginary information into a tangible form. The data is machine-readable data (i.e., computer-readable data). In other words, the data is preferably in a form or format such that it can be provided to a suitable machine or computer (processor) and read by that machine or computer (for further processing or output).

[0041] In certain embodiments, the received signal processed by the processing device (12) may be a nerve impulse, or it may be a representative signal generated (i.e., produced) in response to the measurement or sampling of brain activity in region (103) by some kind of microdevice, such as a microdevice having microelectrodes or microtransducers associated with the processing device (12).

[0042] The processing device (12) processes the brain signals (collected brain signals) by applying a mapping function or a set of mapping functions (including mapping data) to the brain signals. The mapping function maps the imaginary information-carrying brain signals to the data so that the imaginary information (i.e., concepts) represented by the received brain signals is converted (transformed) into data, that is, it provides a conversion from brain signals to data and vice versa, thereby rendering the renderable imaginary information into a tangible form (as a result of the application of the mapping function by the processing device (12)). This brain signal-to-data mapping function will be interchangeably referred to below as "imagination-data mapping". The imagination-data mapping is preferably a one-to-one mapping. It is preferable that the mapping provides a faithful representation of the concepts formed by the region (103) in the data converted from the brain signals using the imagination-data mapping function(s). In other words, the mapping is preferably such that the data provides a faithful representation of the image or sound imagined by the subject (100).

[0043] As described above, according to a particular embodiment, the concept formed by the region (103) is a visual concept (i.e., a mental image, an imagined image), and the processing device (12) converts brain signals representing the mental image. In such an embodiment, for example, the data converted from the brain signals may be image data or pixel data, thereby the tangible form being a visible image corresponding to the image / pixel data, which is a faithful representation of the mental image that can be seen by an object (100) or another observer.

[0044] In other embodiments, the concept formed by the region (103) is the concept of sound (i.e., imagined sound, non-acoustic sound), and the processing device (12) translates brain signals representing the imagined sound. In such embodiments, for example, the data may be sound data in the form of an auditory signal or multiple auditory signals, which may be analog or digital (e.g., bits or bytes representing one or more sounds or tones), where the tangible form is an acoustic sound corresponding to an auditory signal which is a faithful representation of an imagined sound (non-acoustic sound) that can be heard (i.e., auditorily recognized) by the object (100) or another listener.

[0045] Refer to Figure 1, and also to Figure 2, which shows an exemplary block diagram of a processing device (12) according to a non-limiting embodiment of the present invention. The processing device (12) includes one or more processors (14) coupled to a computerized storage medium (16), such as computerized memory. The one or more processors (14) can be implemented as any number of computerized processors, including but not limited to microprocessors, microcontrollers, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), field-programmable logic arrays (FPLAs), and the like. The processors can be conventional processors, application-specific processors, or dedicated processors. Conventional processors, such as microprocessors, may be processors used in servers, computers, and other computerized devices, for example. For example, processors may include AMD and Intel x86 processors, Intel Xeon® and Pentium® processors, and any combination thereof. Implementations of one or more processors (14) as quantum computer processors are also contemplated herein. The aforementioned computerized processors include, or can electronically communicate with, a computer-readable medium that stores program code or instruction sets, and these program code or instruction sets, when executed by the computerized processor, cause the computerized processor to perform actions. Types of computer-readable media include, but are not limited to, electronic devices, optical devices, magnetic devices, or other storage or transmission devices that can provide computer-readable instructions to the computerized processor.It should be noted that the above-described implementations of one or more processors (14) represent a non-exhaustive list of exemplary implementations. It will be apparent to those skilled in the art that other forms of implementation of processing devices, including processors of processing technologies not described herein or not yet fully developed, such as biological processors or organic semiconductors in the field of biological computing technology, may be suitable for implementing any of the processing devices contemplated and considered herein.

[0046] The storage / memory (16) may be any conventional storage medium, or an application-specific or special-purpose storage medium for storing data or information, and may be shown as a single component for a representative purpose, but may be multiple components. The storage / memory (16) can be implemented in various ways, including, for example, one or more volatile or non-volatile memories, flash memory, read-only memory, random-access memory, or any combination thereof. In a particular embodiment, the storage / memory (16) may include one or more components for storing and maintaining imaginary-data mappings, and at least one component configured to store machine-executable instructions that can be executed by one or more processors (16).

[0047] In certain embodiments, the processing device (12) further operates to perform at least one operation on the generated data (generated by processing brain signals through the application of imaginary-data mapping) according to one or more rules or processing criteria. For example, the processing device (12) may be configured to perform an operation on the generated data according to a set of data storage rules or criteria such that the processing device (12) transmits some or all of the data representing a tangible form of at least one concept to one or more computerized storage / memory devices associated with the processing device (12). Such associated storage / memory devices may include, for example, the storage / memory (16) of the processing device (12) (Figure 3), or other storage / memory devices linked to or connected to the processing device (12), as will be considered below.

[0048] Referring further to Figure 3, examples of other storage / memory devices that can be linked to or connected to the processing device (12) include, for example, external storage / memory (32) and a server system (34) (having memory). In embodiments in which the processing device (12) transmits some or all of the data representing the tangible form of at least one concept to the server system (34), the server system may be a remote server system, thereby causing the processing device (12) to transmit the data representing the tangible form of at least one concept to the server system (34) via a communication network (36) (which may be one or more communication networks such as a cellular network, a local area network, or the Internet). In such embodiments, the processing device (12) may be linked to a transceiver (Tx / Rx) unit (30) that provides a communication / network interface for transmitting / receiving (i.e., exchanging data with the network (36)) to and from the network (36). Also shown in Figure 3 is a remote processing system (38) linked to the processing device (12) via the network (36).

[0049] In another non-limiting example, the processing device (12) may be configured to perform operations on the generated data according to a set of data modification rules or operation rules or criteria in order to create modified data. Generally, modified data can be created from the generated data by one or more of the following: adding data to the generated data (e.g., appending or inserting data into the generated data), deleting data from the generated data (e.g., deleting a subset of data from the generated data), or changing the data elements of the generated data (e.g., changing data values). In embodiments in which the generated data includes pixel data corresponding to an image representing visually imaginative information, the processing device (12) may modify the data by changing one or more pixels to change the image, and / or by combining the pixel data with other pixel data representing another image, which may be a computer-generated image or an image of a real-world scene captured by an image captured device (e.g., a camera). Other pixel data can be provided to the processing device (12) via, for example, an image capture device or memory linked, connected to, or otherwise associated with the processing device (12), such as a storage / memory (16), external storage / memory (32), or a server system (34).In embodiments where the generated data includes auditory signals or multiple auditory signals corresponding to a sound or multiple sounds representing auditory / imaginary information, the processing device (12) can modify the generated data by, for example, adding additional sounds (or multiple sounds) (from a sound capture device (e.g., a microphone) or memory associated with the processing device (12), e.g., storage / memory (16), external storage / memory (32), or server system (34)), and / or changing or deleting data elements (e.g., bits) of the digital version of the generated auditory signals (or multiple sounds), and / or adjusting auditory parameters of the auditory signals (or multiple sounds), including, for example, volume, pitch, and tone. For example, the processing device (12) can modify an auditory signal to increase or decrease the volume associated with the auditory signal. As another example, the processing device (12) can modify an auditory signal to change one or more frequencies (tones) of a sound. As a further example, the processing device (12) can modify the generated auditory signal by performing noise cancellation or interference reduction signal processing on the generated auditory signal.

[0050] In certain embodiments, the processing device (12) can also transform the modified data into a new set of one or more brain signals (by applying an imagination-data mapping to the modified data), and then provide these signals to a region (103) of the brain so that the concept represented by the new set of brain signals is formed by the region (103) without sensory input. In other words, the modified image or sound can then be “imagined” by the object (100).

[0051] Additionally, the modified data can be stored in memory (for example, storage / memory (16) and / or external storage / memory (32) and / or server system (34)).

[0052] In another non-limiting example, the processing device (12) may be configured to perform operations on the generated data according to a set of output rules or criteria. In embodiments where the generated data includes pixel data corresponding to images representing visually imaginative information, the output rules or criteria may include a set of display rules or criteria. For example, the processing device (12) may be configured to provide the generated data to a display device connected to or coupled to the processing device (12) so that the display device displays one or more images (or videos) represented by the pixel data. The processing device (12) may transmit or send the data to such a display device using any suitable image / video transmission format or transmission standard, or any standard commonly used for data transmission, including any of the formats and standards considered above. In another example, the processing device (12) may provide the generated data to a projection display or a holographic display.

[0053] In embodiments where the generated data includes auditory signals corresponding to one or more sounds representing auditory / imaginary auditory information, the output rules or criteria may include a set of playback rules or criteria. For example, the processing device (12) may be configured to provide the generated data (auditory signals or multiple auditory signals) in digital format to a digital audio playback device (e.g., MP3, digital stereo, etc.) connected to or coupled to the processing device (12) so that the sounds represented by the generated data are audibly reproduced. The processing device (12) may transmit or send data to such an audio playback device using any suitable audio transmission format or transmission standard, or any standard commonly used for data transmission, including any of the formats and standards considered above. Alternatively, the processing device (12) may be configured to provide the generated data in analog format to an analog audio playback device.

[0054] In certain preferred embodiments, the processing device (12) operates to process acquired / received data (preferably digital data) representing information corresponding to an injectable concept formed by a region of the brain (103) without sensory input, and to convert the data into one or more brain signals. The processing device (12) further operates to selectively provide or transmit one or more brain signals to the region (103) such that the injectable concept represented by the acquired / received data is formed as a concept by the region (103) without sensory input. In other words, the processing device (12) can acquire or receive data, for example in the form of an image represented by pixel data, or an analog or digital auditory signal carrying sound, convert that data into brain signals, and then provide those brain signals to the region (103) such that an image or sound concept is formed by the region (103), i.e., that an object (100) imagines the image or sound. Accordingly, a system (10) according to a particular aspect of the present invention operates to convert data representing the tangible form of an injectable concept / injectable imaginary information into brain signals provided to a target, as if the concept were in an intangible form. In a particular embodiment, a processing device (12) may manipulate (i.e., modify) the received / acquired data before converting it into brain signals. The manipulation / modification of the data may be carried out, for example, according to a set of data modification or manipulation rules or criteria (or a different set of modification or manipulation rules or criteria) considered above.

[0055] In certain embodiments, the processing device (12) is configured to transmit brain signals to region (103) using nerves or nerve fibers at input to region (103). For example, the processing device (12) may provide (transmit) brain signals by inducing the neurotransmission of nerve impulses. For example, the processing device (12) may provide brain signals by sending them to a microdevice, for example, one or more microelectrodes or microtransducers that induce the transmission of nerve impulses corresponding to the brain signals.

[0056] The data acquired or received by the processing device (12) and converted into brain signals by the processing device (12) may be of various types and / or from various sources. For example, the data to be converted may be image data or sound data and may be provided to the processing device (12) from memory associated with the processing device (12) (e.g., storage / memory (16), external storage / memory (32), server system (34)) or any other data source or storage medium. As another example, image data may be provided to the processing device (12) by an image capture device communicatively coupled to the processing device (12). As yet another example, sound data (e.g., auditory signals) may be provided to the processing device (12) by an sound capture device communicatively coupled to the processing device (12). In further examples, the data converted into brain signals by the processing device (12) may be data generated by the processing device (12) from collected brain signals, or modified data obtained from modifications of data generated by the processing device (12) from collected brain signals.

[0057] The conversion of received data into brain signals is achieved by applying the imaginary-data mapping discussed above. Since the brain of each object can form concepts in different ways, the mapping for each object can be object-specific (i.e., the mapping for one object may differ from the mapping for another object). However, regardless of whether a given imaginary-data mapping is specific, it is preferable that the mapping uses an imaginary-data mapping function (or more) such that the brain signals converted from the data form a faithful representation of the true image or sound (represented by the data) in the mind of object (100).

[0058] Various exemplary methods for generating imaginary-data mapping functions are described in detail in subsequent sections of this disclosure.

[0059] Mapping functions or a set of mapping functions can be stored in a memory device associated with the processing device (12), as will be further discussed below. In certain embodiments, mapping functions may be stored as data items or data structures, for example, in the form of a data table that stores mapping parameters and configurations. In other embodiments, mapping functions may be stored as equations or sets of equations that provide a functional relationship between data and brain signals. In yet another embodiment, mapping functions may be represented as computer-readable code that performs a mapping algorithm applicable to brain signals and / or received data. The formats described above are merely examples, and other formats of mapping functions are also contemplated herein.

[0060] Continuing to refer to Figure 1, the communication coupling (interfacing) between the processing device (12) and the region (103) can be realized by a machine-subject interfacing arrangement (18) (hereinafter interchangeably referred to as "interfacing arrangement," "machine-subject interface," "machine-brain interface," or simply "interface") which allows the processing device (12) to communicate with the region (103) of the brain (102). In certain embodiments, the interface (18) may include two interfacing parts, namely a first interfacing part (18a) and a second interfacing part (18b). The first interfacing part (18a), also called the electronic device interfacing part (18a), is connected to the processing device (12). A second interfacing portion (18b), also called the target interfacing portion (18b), can be connected to or coupled to a region (103) of the brain (102). The two portions (18a, 18b) are interconnected via a connecting portion (20), which in certain embodiments can provide a wired connection between the two portions (18a, 18b), and in other embodiments can provide a wireless connection between the two portions (18a, 18b).

[0061] Various deployment configurations for achieving communication coupling of the processing device (12) to the region (103) are contemplated herein, and some non-limiting and exemplary deployment configurations are described in further detail below. Some of the deployment configurations described herein require some kind of implantation within the subject (100), which can be achieved using invasive or semi-invasive techniques. For example, invasive techniques may include implantation by surgically accessing the region (103) through the skull of the subject (i.e., surgically opening the skull). Surgery performed on the brain has become common in recent years, and it is claimed that trained human surgeons and / or robotic surgeons (such as those used by Neuralink Corporation in San Francisco, USA) can perform the necessary implantations. Before describing some of the deployment configurations, it should be noted that the deployment configurations described herein are merely illustrative and represent only a non-exclusive subset of the possible deployment options for the processing device (12). As will be obvious to those skilled in the art, other development options may be possible.

[0062] In one example of a deployment configuration according to a specific, non-limiting embodiment, the processing device (12) communicates with region (103) by tapping the region (103), for example, by connecting the processing device (12) to a transmission route / path which is an input to region (103). A nerve, nerve bundle, or nerve pathway is an example of such a transmission route / path. In such a deployment configuration, the processing device (12) preferably remains outside the brain (102) of the object (100), and most preferably remains outside the skull so that it is at least partially visible when viewing the head of the object. When the processing device (12) is outside the object (100), the object's interfacing portion (18b) is embedded in or on a section of the transmission route (i.e., an input to region (103)) together with the entirety of the connecting portion (20) that connects to the object's interfacing portion (18b), or any section of the connecting portion (20). If only the section of the connecting portion (20) connected to the target interface portion (18b) is embedded, the remaining section of the connecting portion (20) connected to the interface portion (18a) of the electronic device is outside the target (100).

[0063] In another exemplary deployment configuration, the processing device (12) is deployed outside the object (100), and the object's interfacing portion (18b) is embedded in or on region (103) along with the entire or any portion of the connecting portion (20) that connects to the object's interfacing portion (18b). If only the portion of the connecting portion (20) that connects to the object's interfacing portion (18b) is embedded, the remaining portion of the connecting portion (20) that connects to the electronic device's interfacing portion (18a) is outside the object (100). Such an exemplary deployment configuration is schematically shown in Figure 1.

[0064] In yet another exemplary deployment configuration according to certain non-limiting embodiments, the processing device (12) itself, together with the entire interface (18), can be embedded in or on the region (103). In yet another exemplary deployment configuration according to a non-limiting embodiment, the processing device (12) is embedded in or on a section of the transmission route (i.e., the input to region (103)).

[0065] Furthermore, non-invasive deployment configurations are intended herein. For example, the interface (18) can be provided by an optical magnetic field sensor configuration or a non-contact modulation configuration employing, for example, optical, magnetic, or ultrasonic technology. In such a configuration, the interface (18) (and its associated components) and the processing device (12) are completely external to the brain (102). The external interface (18) captures brain signals in the region (103) via non-contact or non-invasive means and provides these captured brain signals to the processing device (12).

[0066] See also Figure 4, which schematically illustrates preferred but not limited examples of brain regions (103) that interface with the processing device (12) in embodiments where the imaginary information that can be rendered or injected is visual imaginary information, while continuing to refer to Figures 1-3. Figure 4 schematically illustrates various parts of the human brain, including the parieto-occipital lobe (104), the occipital lobe (106), and the parieto-occipital sulcus (Schematically represented as a vertical line (109)). As background, the occipital lobe is the center of visual processing in the mammalian brain and contains most of the anatomical region of the visual cortex. The occipital lobe is associated with visuospatial processing, distance and depth perception, color determination, object and face recognition, and memory formation. Visual information from actual events is received in the occipital lobe via the optic nerve and transmitted from the occipital lobe to the parieto-occipital lobe via the parieto-occipital sulcus. Therefore, visual real information and visual imaginative information (e.g., imagined images) flow in opposite directions between the occipital and parietal lobes. Some considerations regarding the human brain's perception of imagination and reality can be found in the Live Science Web article available at the following web address: https: / / www.livescience.com / 49244-imagination-reality-brain-flow-direction.html.

[0067] With the above in mind, in the non-limiting and exemplary embodiment shown in Figure 4, region (103) is the input from the parieto-occipital sulcus (109) to the occipital lobe (106) of the brain (102). In Figure 4, the transmission route schematically shown by a thick double-headed arrow (108) provides an information interface between the parieto lobe (104) and the occipital lobe (106) of the brain (102). The transmission route (also interchangeably referred to herein as a "transmission path") (108) provides the input from the parieto-occipital sulcus (109) to the occipital lobe (106) and may be, for example, nerve connections formed by one or more nerves or nerve fibers. In the illustrated example, the parieto-occipital sulcus (109) represents the boundary between the parietal lobe (104) and the occipital lobe (106). However, the corpus callosum can also provide an interface between the parieto-occipital lobe (104) and the occipital lobe (106) in place of or in addition to the parieto-occipital sulcus (109), and can therefore also be used as a region (103) that interfaces with the processing device (12).

[0068] In one example of a deployment configuration of the processing device (12) according to the embodiment shown in Figure 4, the processing device (12) communicates with the region (103) by tapping the interface between the occipital lobe (106) and the parietal lobe (104) and / or the parieto-occipital sulcus (109). For example, the processing device (12) can be connected to a transmission route (108) (e.g., a nerve) that connects the parietal lobe (104) to the occipital lobe (106). In such a deployment configuration, the target interfacing portion (18b) can be embedded in or on a section of the transmission route (108) between the occipital lobe (106) and the parieto-occipital sulcus (109), which in certain non-limiting implementation forms can be achieved by first cutting a nerve or nerve fiber to create a cut end of the nerve or nerve fiber, and then connecting the target interfacing portion (18b) to the cut end. In such a configuration, the processing device (12) preferably remains outside the brain (102) of the object (100), and most preferably remains outside the skull so that it is at least partially visible when viewing the head of the object. If the processing device (12) is outside the object (100), the object's interfacing portion (18b) is embedded in or on a section of the transmission route (108) along with either the entire or a section of the connecting portion (20) connected to the object's interfacing portion (18b). If only the section of the connecting portion (20) connected to the object's interfacing portion (18b) is embedded, the remaining section of the connecting portion (20) connected to the electronic device's interfacing portion (18a) is outside the object (100).

[0069] In another exemplary deployment configuration of the processing device (12) according to the embodiment shown in Figure 4, the processing device (12) is embedded in or on a section of the transmission route (108). Figure 5 schematically shows such a deployment configuration. Here, the embedding can be achieved, for example, by first cutting the transmission route (108) (e.g., a nerve) to create cut ends (50a, 50b) of the transmission route (108), then deploying the processing device (12) at the site of the cut (sight), and connecting the cut ends (50a, 50b) of the transmission route (108) to the processing device (12) via the interface (18).

[0070] In embodiments where region (103) is input from the parieto-occipital sulcus (109) to the occipital lobe (106), it should be noted that different types of information arriving in the occipital lobe (106) have a common format. Since the information input to the occipital lobe (106) has a common format, and because the occipital lobe (106) is located in the brain's visual processing center, in certain embodiments, the format of imagined images (i.e., visual imagined information) arriving from the parieto lobe (104) is the same as the format of real visual information. U.S. Patent No. 11,395,620, co-owned, which is incorporated herein by reference in its entirety, describes a technique for receiving signals (corresponding to nerve impulses) from the visual cortex (or equivalent visual processing area of ​​the brain) depending on an object viewing a scene, and processing these received signals (using a mapping function called “impulse-image mapping”) to generate image data representing the object’s visual perception of that scene. U.S. Patent No. 11,395,620 further describes techniques for using mapping functions to process image data corresponding to an image or a set of images, converting the image data into neural impulses, and providing these neural impulses to the visual cortex (or equivalent visual processing region) so that an object visually recognizes the image or a set of images corresponding to the image data. In certain embodiments, since the format of imagined images arriving from the parietal lobe (104) is visual real information, it is a special feature of certain aspects of embodiments of the present invention to utilize the mapping methodologies described in the co-owned U.S. Patent No. 11,395,620 (for converting neural impulses into image data and vice versa) to enable conversion between brain signals / imagined information and image data. Specifically, the mapping functions(s) described in U.S. Patent No. 11,395,620 can be used as imagination-to-data mappings in embodiments of the present invention where the imagined information is visual imagination information (i.e., the concepts are mental images).In other words, in certain embodiments, the mapping function(s) described in U.S. Patent No. 11,395,620 is suitable for converting brain signals representing visual concepts (i.e., mental images or imagined images) into image data, and for the reverse conversion.

[0071] The following paragraphs describe various methods and techniques for generating impulse-image mappings that can be used as imagination-data mappings in the context of visual imagination information. Further consideration of impulse-image mapping can be found in U.S. Patent No. 11,395,620.

[0072] In certain embodiments, the generation of impulse-image mappings (i.e., imagination-data mappings) may be assisted by machine learning (ML) algorithms or neural network (NN) algorithms. For example, a processing device (12) may use one or more ML or NN algorithms to learn the signal format of nerve impulses (in response to the eyes of an object viewing a scene or receiving a visual stimulus) and determine the imagination-data mapping by comparing the format of the nerve impulses with digital images stored in memory associated with the processing device (12). In certain embodiments, the stored digital images may be generated by an imaging device such as a camera associated with the processing device (12).

[0073] As part of a non-limiting, exemplary process for generating a mapping, a sample photograph (i.e., an image) can be positioned in front of the eye of an object (100) as a visual stimulus so that light from this sample is collected (captured) by the eye, and the processing device (12) collects nerve impulses sent from the eye to the brain (102) (along the optic nerve, which is a nerve or multiple nerves that transmit nerve impulses from the eye to the visual cortex or equivalent visual processing area of ​​the brain) in response to the object looking at the sample. A digital image having image data representing the same sample can also be stored in memory associated with the processing device (12) (e.g., storage / memory (16)). The digital image can be generated, for example, by an imaging device. The resolution of the digital image preferably follows a standard resolution such as, for example, 1920 pixels × 1080 pixels, 1280 pixels × 960 pixels, or 800 pixels × 600 pixels. A small change can then be made to the sample image to create a new sample image, for example, by changing one pixel of the sample image. Next, a new sample image is placed in front of the subject (100), and the processing device (12) collects nerve impulses sent from the eye to the brain (102) in response to viewing the new sample image. Preferably, a digital version of the new sample image, i.e., a digital image having digital image data representing the new sample, is stored in memory associated with the processing device (12) (e.g., storage / memory (16)). The digital version of the new sample image can be generated by the processing device (12) applying changes to the pixels of the original digital image. This process can be continued by applying progressively larger changes to the sample image (e.g., changing 2 pixels, then 5 pixels, then 10 pixels, etc.). For each changed pixel, the change in nerve impulses from the eye (compared to the previous sample) is compared with the change between the new digital image data and the previous digital image data.This process can be continued using several different sample images until each nerve impulse from the eye can be matched one-to-one with a corresponding image pixel. This matching between each nerve impulse and its corresponding image pixel constitutes a mapping between the nerve impulse and the image (i.e., an impulse-image mapping), which can then be used as an image-data mapping.

[0074] In certain embodiments, the mapping function is stored as, or in conjunction with, a configuration table that maintains the conversion parameters from nerve impulses to images and from images to nerve impulses. The configuration table contains all image attributes / features, such as color, intensity, location, and the encoding value of the nerve impulse. The size of this table may depend on the resolution of the image, such that for each pixel (or group of pixels), the image data of that pixel (or group of pixels) has corresponding values ​​for color, intensity, location, and the code of the nerve impulse. As previously stated, in certain embodiments, the mapping function can be stored as an equation or set of equations that provides a functional relationship between the data and the brain signal, while in other embodiments, the mapping function(s) can be expressed as a computer-readable code that performs a mapping algorithm applicable to the brain signal and / or received data.

[0075] In a preferred but non-limiting implementation of the process for generating the mapping, anchor points or regions of the digital image are processed first. Anchor points include pixels (or groups of pixels, typically consisting of at least four pixels) at each of the four corners of the digital image, as well as pixels (or groups of pixels) at the center of each edge of the digital image (i.e., top, bottom, left, and right), resulting in eight anchor points. The color and intensity of each of the eight pixels correlate to the corresponding nerve impulse when the corresponding anchor point in the sample photograph is viewed by the eye of subject (100) (based on the determined position of the anchor point). If groups of pixels are used, the average color and intensity of the pixels within each group are calculated and set as the color and intensity of the pixel group.

[0076] The color and intensity values ​​of the pixels are stored in a table along with the values ​​of the corresponding registered nerve impulses. Next, some or all of the pixel values ​​of the anchor point are changed, and the sample image displayed to the eye is also changed accordingly, so that the color and intensity of each of the eight pixels correlate with the corresponding nerve impulse when the corresponding anchor point in the sample photograph is viewed with the eye. This process can be repeated several times until the correlation between the anchor point pixels (either individual pixels or groups of pixels) and the corresponding nerve impulses is verified. The mapping function generation process can then proceed to change the color and intensity values ​​of selected pixels or groups of pixels that are not anchor pixels. This change can be performed according to a specific predefined sequence, which may include a sequence of color and intensity values ​​for selected pixels, and then a sequence for selected pixels. In this way, a pixel or group of pixels is selected (according to the pixel selection sequence), the color and intensity values ​​of the selected pixel(s) are changed according to the color / intensity sequence, then another pixel or group of pixels is selected (according to the pixel selection sequence), the color and intensity values ​​of the selected pixel(s) are changed according to the color / intensity sequence, and so on, until all combinations of color / intensity values ​​have been performed across all pixels and the corresponding nerve impulses have been recorded / stored (in the table).

[0077] Additionally, after selecting individual pixels or groups of pixels and gradually changing their color / intensity values ​​to create a correlation between nerve impulses and the color / intensity values ​​of these pixels, the accuracy of the correlation can be checked by optionally converting the nerve impulses into digital image data using a partial table containing the color / intensity values ​​of the selected pixels.

[0078] Next, using a complete table, nerve impulses (collected in response to the eye viewing the sample photograph) can be converted into digital images to create a generated digital image. The generated digital image is then compared with a digital image stored in memory associated with the processing device (12) (e.g., storage / memory (16)) (in certain embodiments, this may be generated by the imaging device camera in response to capturing an image of the sample photograph). This comparison can be performed pixel by pixel. If the comparison yields a pixel matching within a desirable level of accuracy (e.g., 90% of the pixels in the two images are the same), the mapping process is complete. If the comparison does not yield a pixel matching within a desirable level of accuracy, the correlation process can be repeated, i.e., anchor points can be selected and the color / intensity values ​​of the pixels can be changed incrementally.

[0079] The following paragraphs describe another non-restrictive, illustrative method for generating imagination-data mapping functions in the context of visual imagination information. To begin with, the method is particularly suitable for generating mapping functions used for human subjects. However, the method or similar methods may also be suitable for generating mapping functions used for other animal species, particularly intelligent animal species such as canids and primates.

[0080] With this in mind, a readily recallable (i.e., easily recognizable in the subject's mind) sample image is placed in front of the subject's eyes, where the subject's brain is interfaced with the processing device (12). The sample image preferably consists of readily perceptible and / or easily distinguishable colors, and is preferably initially in black and white. The sample image can be, for example, thick vertical lines with readily perceptible spaces between them.

[0081] In the next step, the subject may be asked to close their eyes and recall a sample image (e.g., black and white vertical lines) from memory or using their imagination. In response to the subject recalling the sample image, first data corresponding to brain signals collected by the processing device (12) is captured. Subsequently, a change can be made to the sample image to create a new sample image. This change is preferably a geometric change, such as a change in the length or thickness of the vertical lines, or a slope of the lines such that the lines are no longer vertical but slightly angled. As another example, the change may be a shape change, such as changing a thick vertical line into a circle. The geometric / shape change is preferably such that the new sample image is easily distinguishable from the original sample image, and the subject can easily recall / imagine the new sample image without confusing it with the original sample image.

[0082] Changes / differences between a sample image and a new sample image can be identified / marked / logged. The new sample image can be shown to the subject, and then the subject can be asked to recall the new sample image from memory / imagination. In response to the subject imagining the new sample image, second data corresponding to brain signals collected by the processing device (12) is captured and compared with the first data corresponding to the sample image. Changes between the first and second data can be correlated with changes between the sample image and the new sample image.

[0083] This process of altering a sample image and comparing the changes in brain signal data can continue until each component of the data (generated in response to the subject imagining the image) is mapped or associated with a pixel of the image.

[0084] The process described above can also be repeated by applying changes to the image's colors. For example, if the initial sample image is a black and white image, this process can be repeated by changing the black and white image to a blue and green image, then to a red and yellow image, and so on.

[0085] As described above, the exemplary methods described above for generating mapping functions are suitable in embodiments where region (103) is a part of the brain responsible for forming visual concepts (i.e., mental images) without sensory input, particularly in embodiments where region (103) is input from the parieto-occipital sulcus (109) to the occipital lobe (106). In embodiments where region (103) is a part of the brain responsible for forming auditory concepts (i.e., imagined information that can be rendered or injected is auditory / auditory imagined information), region (103) may include a segment along one or more neurons that transport sound-related neural impulses from the ear to the part of the brain that performs auditory processing (this is commonly called the auditory cortex in humans, as well as in other species including but not limited to canids, felines, non-human primates, and rodents). U.S. Patent Application No. 17 / 728,013, co-owned, is incorporated herein by reference in its entirety, and describes a technique for receiving signals (corresponding to nerve impulses) from the auditory cortex (or equivalent auditory processing area of ​​the brain) in response to an object hearing an acoustic sound (i.e., in response to an auditory stimulus being applied to the object's ear), and for processing these received signals (using a mapping function called “impulse-sound mapping”) to generate an auditory signal representing the auditory recognition of the sound heard by the object. U.S. Patent Application No. 17 / 728,013 further describes a technique for processing an auditory signal corresponding to a sound or a combination of sounds using a mapping function to convert the auditory signal into nerve impulses, and for providing these nerve impulses to the auditory cortex (or equivalent auditory processing area of ​​the brain) so that the object can auditorily recognize the sound or a combination of sounds corresponding to the auditory signal. Therefore, a special feature of certain embodiments of the present invention is the utilization of the mapping methodology described in co-owned U.S. Patent Application No. 17 / 728,013 (for converting nerve impulses to auditory signals and vice versa) to enable conversion between brain signals / imaginary information and auditory signals / data.Specifically, the mapping function(s) described in U.S. Patent Application No. 17 / 728,013 can be used as an imagination-to-data mapping according to embodiments of the present invention, where the imagined information is auditory / sound imagined information (i.e., the concept is a non-acoustic sound). In other words, in certain embodiments, the mapping function(s) described in U.S. Patent Application No. 17 / 728,013 is suitable for converting brain signals representing a sound concept (i.e., a non-acoustic sound or an imagined sound) into auditory signals or multiple auditory signals, and vice versa.

[0086] The following paragraphs describe various methods and techniques for generating impulse-sound mappings that can be used as imaginative-data mappings in the context of auditory / sound imaginative information. Further consideration of impulse-image mapping can be found in U.S. Patent Application No. 17 / 728,013.

[0087] According to a particular embodiment, the processing device (12) can generate an impulse-sound mapping (i.e., an imaginary-data mapping) by learning the signal format of nerve impulses (in response to auditory stimuli of the ear) using one or more ML algorithms or NN algorithms, and then determining an imaginary-data mapping by comparing the format of the nerve impulses with an auditory signal that includes, for example, digital data stored in memory associated with the processing device (12) and / or an analog auditory signal generated in response to capturing sound by a sound capture device (e.g., a microphone).

[0088] As part of a non-restrictive, exemplary process for generating a mapping, an audio sample signal can be generated, which is an amplitude-varying signal over some fixed time. The audio sample signal is an analog signal consisting of multiple frequency components corresponding to various sounds (frequency tones), and these frequency components can be separated using frequency analysis techniques, such as Fourier analysis including the Fast Fourier Transform (FFT). Sound vibrations from the audio sample signal are captured by the ear of the subject (100), and the processing device (12) collects nerve impulses sent from the ear to the auditory region of the brain (102) (along the auditory nerve) in response to hearing the sample audio.

[0089] To the extent of this specification, “acoustic nerve” refers to any nerve or segment of nerve that can transmit pulses (i.e., nerve impulses) converted from mechanical waves (e.g., vibrations) perceived by one or both ears to the brain (102) (specifically, the auditory region of the brain, e.g., the auditory cortex) so that the brain (and therefore the object) may interpret and recognize them as sound.

[0090] The same audio sample can then be played back so that the sample is captured by a sound capture device connected to the processing device (12). The processing device (12) collects the auditory signal transmitted from the sound capture device to the processing device (12) and analyzes / processes the audio sample signal. Analysis / processing may include, for example, digitization (sampling and quantization) and / or frequency analysis (e.g., FFT). Then, to create a new audio sample signal, small changes to one or more of the signal characteristics can be added to the audio sample signal, for example, by changing one or more of the frequency components or amplitude values ​​of the audio sample signal. The sound vibrations from the new audio sample signal are captured by the ear, and the processing device (12) collects the nerve impulses sent from the ear to the auditory region of the brain (102) (along the auditory nerve) in response to hearing the new audio sample signal. The same new audio sample signal can then be played back so that it is captured by the sound capture device, and the processing device (12) collects the auditory signal transmitted from the sound capture device to the processing device (12). The processing device (12) analyzes / processes the new audio sample signal (e.g., via digitization and / or FFT). This process can be continued by changing the characteristics of the audio sample signal one at a time individually (e.g., changing a single frequency component or an instantaneous amplitude value) or by changing a large group of signal characteristics stepwise (e.g., changing multiple frequency components and / or multiple instantaneous amplitude values). For each change in the audio sample signal, the change in nerve impulses from the ear (compared to the previous sample) is compared with the change in the auditory signal collected by the processing device (12) from the sound capture device. This process can be continued until each nerve impulse from the ear can be matched to the corresponding auditory signal component (e.g., sound) transmitted by the sound capture device.This matching between each nerve impulse and its corresponding auditory signal component constitutes a mapping between nerve impulses and sound (i.e., an impulse-sound mapping), which can be used as an imaginary-data mapping. It should be noted that the changes to the audio sample signal preferably cover multiple combinations of sounds (frequency tones), and more preferably cover sounds across any given range of amplitude and / or frequency.

[0091] It should be noted that, in this specification, the processing device (12) may employ a variety of techniques for acquiring brain signals (e.g., nerve impulses or bioelectrical signals) from a region (103) of the brain (102) and providing the brain signals (converted from the data) to that region (103). Such techniques may typically rely on the use of microdevices such as microelectrodes or microtransducers to measure (receive) brain signals and / or to guide the transmission of brain signals along a section of a transmission route (e.g., a neural pathway) that is an input to the region (103) (e.g., transmission route (108) in Figure 5).

[0092] In certain embodiments, the brain signal includes or is in the form of a nerve impulse, thereby the induction of transmission includes inducing the transmission of the nerve impulse along a nerve pathway or segment of nerve, for example, a nerve forming a transmission route (108).

[0093] Various organizations have conducted research, development, and experiments on connecting and interfacing computer processing devices to the brain, tissues, and nerves via implantation or other invasive or semi-invasive means. One example of such research can be found in the 2019 publication by the University of Luxembourg entitled "CONNECT - Developing nervous system-on-a-chip" (available at https: / / wwwfr.uni.lu / lcsb / research / developmental_and_cellular_biology / news / connect_developing_nervous_system_on_a_chip), which describes the culture of individual nervous system components and the connection of those components within a microfluidic chip (integrated circuit).

[0094] Examples of research and experimentation in the field of brain-machine interfacing are described in an article titled "Brain-Chip Interfaces: The Present and The Future" by Stefano Vassanelli of the Neurochip Laboratory at the University of Padua, Italy, published in Procedia Computer Science in 2011. In one example, a computerized processing device is interfaced with neurons using metal microelectrodes or oxide-insulated electric microtransducers (e.g., electrolyte-oxide-semiconductor field-effect transistors (EOSFETs) or electrolyte-oxide-semiconductor-capacitors (EOSCs)) to record (i.e. measure) or stimulate the electrical activity of the neurons. In another example, large-scale, high-resolution recordings (i.e., measurements) from individual neurons are acquired using microchips featuring large multi-transistor arrays (MTAs) or processing devices coupled to such microchips. In yet another example, a microchip featuring a large MTA is used to interface with cells in vitro by deploying the MTA in contact with brain tissue, and in one embodiment, the signal corresponding to a nerve impulse is in the form of a local-field-potential (LFP).

[0095] One example of a brain-machine interface device is the Neuralink device, developed by Neuralink Corporation in San Francisco, USA. The Neuralink device includes an ASIC that digitizes information acquired from neurons via microelectrodes.

[0096] With the above in mind, the following paragraphs provide a high-level description of various non-limiting implementations of the interface (18) that can be used to connect a processing device (12) to an object (100) / to interface between the processing device (12) and the object (100) so that a machine-object interface is provided according to non-limiting and exemplary embodiments of the present invention.

[0097] Refer to Figures 1-5, and also to Figure 6, which shows a schematic diagram of an interface (18) according to a non-limiting embodiment of the present invention. Here, the interface portion (18b) in question includes an electrode array (22) having a plurality of electrodes (23) deployed in or on a region (103) (which may, in a particular case, coincide with the optic nerve or auditory nerve). The electrodes (23) are preferably microelectrodes such as EOSFETs or EOSCs. In embodiments in which a processing device (12) operates to convert brain signals into data, the electrode array (22) operates to measure brain signals transmitted to the region (103), create electrical signals associated with (and representing) the brain signals (in response to the measurement), and provide these signals to the processing device (12) so that the processing device (12) can collect the brain signals and process the electrical signals using imaginary-data mapping. In the illustrated embodiment, the coupling portion (20) can be implemented as a wire or cable providing a physical transmission medium on which the electrical signals can propagate to the processing device (12). In certain embodiments, the interface (18) may use a transducer (preferably the microtransducer described above) as part of the interfacing portion (18b) in question, either instead of or in addition to the electrode array (22). The transducer can be used in conjunction with a processing device (12) to convert brain signals into data. For example, the transducer may generate an electrical signal in response to the reception (measurement) of brain signal activity transmitted to a region (103). The generated electrical signal corresponds to a brain signal (i.e., represents a brain signal) (and therefore represents a concept formed by the brain) and is provided to the processing device (12) for processing using an imagination-data mapping.

[0098] In embodiments where the processing device (12) operates to convert data (representing concepts) into brain signals and to provide / transmit brain signals to a region (103) of brain (102) so that the brain signals are interpreted by brain (102) and the concepts (images or sounds) represented by the data are formed by the region (103), the transmission of brain signals may be achieved by stimulating the region (103) (e.g., stimulating one or more neurons of a nerve) by a microdevice, e.g., an electrode array (22) (or transducer). Generally speaking, in such embodiments, the processing device (12) can convert data (using imagination-data mapping) into brain signals (or electrical signals representing brain signal activity) to be transmitted by the region (103). The processing device (12) then provides brain signals to the region (103) to induce the transmission of brain signals (or provides electrical impulses to the region (103) to induce the transmission of brain signals represented by electrical impulses). In certain embodiments, induction of transmission can be achieved by the processing device (12) providing an electrical signal to an electrode array (22) (or transducer), which stimulates neurons in a region (103) according to the electrical signal to induce the transmission of the corresponding nerve impulse.

[0099] Figure 7 shows another embodiment using wireless signal transmission to provide an electrical signal to a microdevice represented here as an electrode array (22). Here, a processing device (12) is connected to a transmitter (Tx) unit (24) via a wire or cable (25), and the electrode array (22) is connected to a receiver (Rx) unit (26) via a wire or cable (27). The Tx unit (24) includes transmitter circuitry and components for transmitting the electrical signal created by the processing device (12) to the Rx unit (26) via a wireless interface. The Rx unit (26) includes receiver circuitry and components for receiving the electrical signal and providing the received signal to the electrode array (22) which stimulates a region (103) to transmit a brain signal corresponding to the electrical signal.

[0100] In certain embodiments, wireless transmission may be RF signal transmission. In such embodiments, the transmitter circuit and components of the Tx unit (24) may include, for example, one or more antennas, digital-to-analog converters, signal modulators, filters, and amplifiers, and the receiver circuit and components of the Rx unit (26) may include, for example, one or more antennas, filters, amplifiers, and demodulators, and so on. In other embodiments, wireless transmission may be inductive signal transmission, thereby the Tx unit (24) and Rx unit (26) operate to transmit and receive, respectively, using inductive signal transmission means. In such embodiments, for example, the Tx unit (24) may include an inductive coil, and the Rx unit (26) may include an inductive receiver.

[0101] As described above, in certain embodiments, the interface (18) can provide non-contact or non-invasive contact between the processing device (12) and the brain region (103). For example, the interface (18) may include, for example, an optical magnetic field sensor configuration or a non-contact modulation configuration employing optical, magnetic, or ultrasonic technology.

[0102] In certain embodiments, the processing device (12) is implemented as a biological processor or biological processing element to be cultured or grown within the subject; in certain embodiments, the interface (18) is the processing device (12) itself.

[0103] It should be noted that in certain embodiments, the interfacing configuration (18) may include multiple interfaces. For example, a first interface can be used to achieve the conversion of data into brain signals. The first interface may use an electrode array (22) or microtransducer (e.g., implemented as an EOSC) connected to or coupled to the processing device (12) via a wired connection (e.g., as shown in Figure 6) or a wireless connection (e.g., as shown in Figure 7). A second interface can be used to achieve the conversion of brain signals into data. The second interface may use an electrode array (22) and / or microtransducer (e.g., implemented as an EOSFET) connected to or coupled to the processing device (12) via a wired connection (e.g., as shown in Figure 5). In other embodiments, the second interface may use non-contact or non-invasive contact means (e.g., an optical magnetic field sensor configuration or a non-contact modulation configuration).

[0104] As another example, a set of four interfaces can be used. The first interface can be used to achieve the conversion of image data into brain signals (for the injection of visual / imaginative images), the second interface can be used to achieve the conversion of brain signals into image data (for the rendering of visual / imaginative images), the third interface can be used to achieve the conversion of data (in the form of auditory signals) into brain signals (for the injection of imaginative sounds), and the fourth interface can be used to achieve the conversion of brain signals into auditory signal data (for the rendering of imaginative sounds).

[0105] Referring again to Figure 1, in certain non-limiting embodiments, the system (10) also includes a control unit (15) that is (electronically) connected to or coupled with the processing device (12) and configured to control the operation of the processing device (12). The control unit (15) preferably includes one or more user input interfaces (e.g., a touchscreen, push buttons, dials, knobs, electronic keypads, (electronic) keyboards, etc.) that allow a user to provide input to the control unit (15). In response to receiving input via the user input interfaces, the control unit (15) preferably operates to provide the processing device (12) with control commands that control or change the operation of the processing device (12).

[0106] For example, the control unit (15) allows the user to define rules or processing criteria that determine at least one operation performed on the generated data by the processing device (12), and to select and / or change from a selected rule to another. For example, the user can define a set of rules on which the processing device (12) will operate. As a further example, the user can select an existing set of rules (e.g., data storage rules, data modification rules, output rules) or a newly defined set of rules (e.g., a set of data storage rules (criteria), a set of data modification (operation) rules, or a set of output rules (criteria)) so that the processing device (12) operates according to the selected rule(s). In addition, the user can select parameters related to the defined rules via the control unit (15). For example, if the user chooses that the processing device (12) operates according to a set of data modification (operation) rules, the user can choose how to modify the generated data, including selecting any additional images or sounds used to modify the generated data. These additional images or sounds can be received from a variety of sources, including, for example, computer memory associated with a processing device (12) that stores images or sounds in digital format, an image capture device, an audio capture device, or an input device such as a microphone or audio player.

[0107] As another example, if the user chooses that the processing device (12) operate according to a set of data storage rules, the user may choose memory devices (e.g., storage / memory (16), external storage / memory (32), server system (34)) to store data representing the generated data, and may also choose which parts (segments or subsamples) of the data are stored in which memory devices (for example, the user may choose some of the data to be stored locally in storage / memory (16) and other parts of the data to be stored remotely in server system (34)).

[0108] Furthermore, the control unit (15) preferably allows the user to select data to be converted into brain signals by the processing device (12). This selection can be applied via a menu, which is part of the user input interface of the control unit (15). This menu may include a list of images or digital audio tracks or sounds stored in memory associated with the processing device (12). In addition, the control unit (15) preferably allows the user to adjust and set the rate at which the brain signals converted from the data by the processing device (12) are provided to the region (103). The rate setting can be applied via the user input interface of the control unit (15).

[0109] In certain preferred embodiments, the control unit (15) selectively switches between different operating modes of the system (10) in response to user input. For example, the control unit (15) can selectively actuate a processing device (12) to retrieve data (images or audio) from memory (e.g., storage / memory (16), storage / memory (32), server system (34)) or an external device (e.g., an image capture device, an audio capture device). Thus, the control unit (15) can enable the user to control whether and when data (e.g., image data or auditory signals) from memory (e.g., storage / memory (16), storage / memory (32), server system (34)) or an external device is converted into brain signals, and / or whether and when such converted brain signals are provided / transmitted to a region (103). In this way, the user can control whether and when an object imagines the injected image or sound.

[0110] The control unit (15) is a computerized control unit comprising one or more computer processors coupled to a computerized storage medium (e.g., memory). The one or more processors can be implemented as any number of computerized processors, including but not limited to microprocessors, microcontrollers, ASICs, FPGAs, DSPs, FPLAs, state machines, biological processors, etc. In a microprocessor implementation, the microprocessor may be a conventional processor, such as those used in servers, computers, and other computerized devices. For example, microprocessors may include AMD and Intel x86 processors, and Intel Xeon® and Pentium® processors. The aforementioned computerized processors may include, or electronically communicate with, a computer-readable medium that stores program code or instruction sets, and these program codes or instruction sets cause the computerized processor to perform actions when executed by the computerized processor. Types of computer-readable media include, but are not limited to, electronic devices, optical devices, magnetic devices, or other storage or transmission devices that can provide computer-readable instructions to the computerized processor. The storage / memory of the control unit (15) can be any conventional storage medium, or any application-specific or dedicated storage medium, and can be implemented in various ways, including, for example, one or more volatile or non-volatile memories, flash memory, read-only memory, random access memory, or any combination thereof. In a particular embodiment, the storage / memory of the control unit (15) can store machine-executable instructions that can be executed by one or more processors of the control unit (15).

[0111] In certain embodiments, the processing device (12) and the control unit (15) share one or more common processors so that the processing device (12) operates to perform both processing and control functions. In other, and possibly more preferred, embodiments, the control unit (15) and the processing device (12) are separate electronic devices electrically connected via wired or wireless connections. In such embodiments, the control unit (15) can be implemented as a user computer device, including, for example, mobile computing devices including, but not limited to, laptops, smartphones, and tablets, and stationary computing devices including, but not limited to, desktop computers.

[0112] In other embodiments, the control unit (15) is implemented via application software running on an electronic device such as a mobile communication device (e.g., a smartphone, tablet, etc.) or a computer device (e.g., a laptop, desktop, etc.). In embodiments where the control unit (15) is implemented on a smartphone, tablet, laptop, etc., the software application can provide a user input interface. In certain embodiments, the control unit (15) provides control to the processing device (12) via a direct wired connection or an indirect wireless connection.

[0113] In certain embodiments, the control functions for the operation of the processing device (12) and the system (10) are algorithmically provided, for example, by an artificial intelligence algorithm implemented by the processing device (12), so that the system (10) operates automatically without requiring any function-specific control units.

[0114] While the embodiments of the present invention described herein so far relate to a single processing device (12), embodiments using multiple such processing devices are contemplated herein. For example, a first processing device may be deployed to interface with input from the parieto-occipital sulcus (109) to the occipital lobe (106) to perform the rendering and injection of visual imagination, and a second processing device may be deployed to interface with the auditory region of the brain (which performs auditory processing) to perform the rendering and injection of auditory / sound imagination.

[0115] In addition, while the embodiments of the present invention described herein relate to a processing device (12) that performs the conversion of brain signals to data and the conversion of data to brain signals, other embodiments are also possible in which the tasks of converting brain signals to data and the conversion of data to brain signals are divided among various processors or processing devices / components. In fact, the processing of brain signals and data described herein can be performed by any number of processors selected from a plurality of distributed processors that together form a processing subsystem. One or more of the processors may be local to the processing device (12) (and the subject (100)), and one or more of the processors may be remote from the processing device (12). Alternatively, only remote processors may be used for processing brain signals and data.

[0116] In one example, in the context of the embodiments shown in Figures 1 and 3, the processing subsystem (200) may include one or more processors of a processing device (12) and one or more processors of a server system (34) or another remote processing system (38) (which may include processing components for processing signals and data) connected to the processing device (12) via a network (36). For example, in a particular embodiment, the processing device (12) may receive brain signals from a region (103) and transmit these signals to the server system (34) (or remote processing system (38)) via the network (36). In such an embodiment, the server system (34) (or remote processing system (38)) may process the signals by applying an imaginary-data mapping to generate data. In another example, the server system (34) (or remote processing system (38)) may modify the generated data. As another example, data (representing concepts formed by brain regions (103) without sensory input) can be processed by a server system (34) (or processing system (38)) by applying an imaginary-data mapping to convert the data into brain signals. The server system (34) (or remote processing system (38)) can then transmit these signals to a processing device (12), which can then provide the signals to brain regions (103).

[0117] Therefore, generally speaking, the processing subsystem (200) (which includes, by definition, at least one processor) is configured to perform the tasks of the processing device (12) described above, including one or more of the following: receiving / acquiring brain signals representing concepts formed by brain regions (103); processing the received / acquired brain signals and transforming the concepts to generate data; performing at least one operation on the generated data according to one or more rules; receiving / acquiring data; processing the received / acquired data and transforming the received / acquired data into brain signals; and providing brain signals to brain regions (103).

[0118] While embodiments of the present invention are particularly useful when applied within the context of human imagination and thought, embodiments of the present disclosure may also be applicable to imagination in non-human animals, including but not limited to other primates (e.g., monkeys, gorillas, etc.), canids, felines, reptiles, birds, marine organisms / aquatic species, etc. In such non-human applications, brain signals can be collected via the same or similar interfacing methods considered above and converted into data by a processing device (12) using secies-specific imagination-data mapping. As described above, for a given animal species, the corresponding brain regions responsible for forming visual or auditory concepts can be identified using brain scanning techniques such as magnetic resonance imaging.

[0119] Any resulting data can be output to another system for further processing or use, for example. For instance, images or auditory signals generated from the brain signals of a dog subject can be provided to a human subject for display or reproduction so that it can see or hear, or they can be converted into brain signals using a human imagination-data mapping function and provided to a human subject's region (103) so that the human subject can imagine images or sounds that would be perceived by the dog subject.

[0120] According to certain embodiments, computer-readable data generated by a processing subsystem (200) from collected brain signals (e.g., nerve impulses) representing a concept formed by a region (103), without sensory input, can be advantageously used to support interaction with and / or control of real-world objects associated with the concept. In the context of this disclosure, the term “real-world object” refers to an object that exists in the physical world of the real world of the subject.

[0121] In general, in such embodiments, the processing subsystem (200) operates to identify data records (e.g., having metadata) associated with one or more data elements of the generated computer-readable data and also associated with real-world objects related to a concept that forms the basis on which the computer-readable data was generated. The processing subsystem (200) can use the identified data records to invoke commands to activate, control, or otherwise interact with the real-world objects related to the concept. For example, if the concept is a mental image of a snowflake, the processing subsystem (200) may convert brain signals representing a snowflake into computer-readable image data (including pixel data) describing a snowflake. The processing subsystem (200) can then identify data records (e.g., one of many data records stored in data record storage, such as a database) associated with one or more pixels of the snowflake image data and also associated with real-world objects related to the concept (i.e., related to "snowflake"). For example, a real-world object could be a freezer, refrigerator, air conditioning unit, or any other object associated with a “cold environment” (i.e., an object “associated” with snowflake). As another example, if the concept is a non-acoustic sound in the form of the word “snowflake,” i.e., an imagined sound, the processing subsystem (200) can convert the brain signal representing the non-acoustic sound into a computer-readable audio signal (i.e., audio data such as an MP3 file) which, when played on an audio playback device (e.g., an MP3 player), the audio playback device emits / generates the sound “snowflake.”The processing subsystem (200) can then identify data records associated with one or more bytes of an audio signal, such as one or more bytes associated with a portion of the audio signal corresponding to "snow", one or more bytes associated with a portion of the audio signal corresponding to "flake", one or more bytes associated with a portion of the audio signal corresponding to "snowflake", and so on.

[0122] Relational data that maintains the relationship between real-world objects and concepts can be pre-programmed into memory associated with the processing subsystem (200) (e.g., storage / memory (32)) or stored in a data record manager along with data records. Note that the relationship between a concept and a real-world object and / or action may be an intrinsic relationship or an arbitrary relationship that may be assigned by a user of the system (100), for example.

[0123] Referring to Figures 8 to 10, and continuing with reference to Figures 1 to 7, the following paragraphs will further describe in detail certain non-limiting embodiments of the present invention that support interaction with and / or control of real-world objects.

[0124] Referring first to Figure 8, a flowchart of process (800) according to an embodiment of the present disclosure is shown. It is understood that process (800) and its subprocesses are executed by a system (10), in particular a processing subsystem (200), and its associated components, such as a processing device (12), a remote processing system (38), and a server system (34). Furthermore, process (800) and its subprocesses are executed, for example, automatically, but can also be executed, for example, manually, and can be executed, for example, in real time.

[0125] Process (800) begins in step (802), in which the processing subsystem (200) receives brain signals representing a concept formed by region (103) without sensory input. In step (804), the processing subsystem (200) processes the received brain signals to convert the concept into computer-readable data representing the tangible form of the concept. In step (806), the processing subsystem (200) identifies data records associated with one or more data elements of the computer-readable data (generated in step (804)) and also associated with real-world objects that are related to (typically, essentially related to) the concept. In step (808), the processing subsystem (200) initiates a response operation according to the data records identified in step (806). As considered, a real-world object may, in some embodiments, be an electronic device (which may include any kind of device, machine, or appliance having one or more processor components (including computerized processor components and / or bioprocessor components)), and a response action typically involves activating, operating, or controlling the real-world object. In certain cases, a response action may include instructing the real-world object to maintain the current state of the action.

[0126] In addition, it should be noted that, even if infrequently, certain computer-readable data (generated in step (804)) may not be associated with any real-world object or action (by design), and therefore, in such cases, the data record associated with one or more data elements of the computer-readable data may not be identifiable. As is obvious, if the data record is not identifiable, the processing subsystem (200) cannot invoke an action on a real-world object.

[0127] The aforementioned data records may typically be data objects with metadata, for example, which may be stored in a database or data table or other data structure storage, along with other similarly structured data records. In the non-limiting example shown in Figure 9, data records may be stored and maintained using a data record manager (60) which can be linked to a processing subsystem (200). The data record manager (60) may be, for example, memory, a server, a database, or any other type of data storage element or medium capable of storing data records in a structured format. In one non-limiting example, the data record manager (60) may be indirectly linked to the processing device (12) of the processing subsystem (200) via a network (36). In another non-limiting example, the data record manager (60) may be directly linked to the processing device (12) of the processing subsystem (200) via a data bus or other wired data communication interface that supports data exchange between the processing device (12) and the data record manager (60). Other deployment configurations of the data record manager (60) are also contemplated herein and include, for example, a direct link between the data record manager (60) and the teleprocessing subsystem (38) and / or the server system (34) via a data bus or other wired data communication interface that supports data exchange between the data record manager (60) and the teleprocessing subsystem (38) and / or the server system (34). In certain embodiments, the data record manager (60) may be implemented as part of the teleprocessing system (38) or the server system (34).

[0128] FIG. 10 schematically shows data types stored in a data record manager (60) and the relationship structure between the stored data types, according to one non-limiting, exemplary embodiment. As shown, and as will be discussed in detail below, data types according to non-limiting, exemplary embodiments include: 1) data records (e.g., DR 1a , DR 1b , DR 2a , etc.) that associate a set of data elements (e.g., a group of pixels, see below) with actions and objects; 2) a set of data elements (e.g., S 1a , S 1b , S 2b , etc.) that generate computer-readable data associated with a data record (the set of data elements can be, for example, pixels, a group of pixels, bytes of audio data, etc.); 3) real-world objects (e.g., ob j1 , ob j2 , etc.) associated with a data record and related to the concept for which the computer-readable data is formed; 4) response actions (e.g., A 1a , A 1b , A 2b , etc.) associated with a data record and applicable to the real-world object.

[0129] As shown in FIG. 10, the data record manager (60) stores a plurality of data records. Data records DR 1a , DR 1b ,... DR 1K represent a first set of data records stored in the data record manager (60), where K is an integer representing the number of data records in the first set. Similarly, data records DR 2a , DR 2b ,... DR 2M represent a second set of data records stored in the data record manager (60), where M is an integer representing the number of data records in the second set. Similarly, data records DR Xa , DR Xb ,... DR XNrepresents the last set of data records stored in the data record manager (60), where N is an integer representing the number of data records in the last set, and X represents the number of sets of data records. As is obvious, the data records can be divided into any suitable number of sets (i.e., X can take any integer value greater than zero), and each set has any suitable number of data records, which may be the same as or different from the number of data records in the other sets (i.e., each of K, M, N, etc., can take any integer value greater than zero).

[0130] Each data record is associated with one or more data elements of the generated computer-readable data. One or more data elements associated with a particular data record constitute a data element set. For example, data record DR 1a is the data element set S 1a Associated with, data record DR 1b is the data element set S 1b Associated with, and data record DR 1K is the data element set S 1K It is associated with, for example, the data record DR. 2a is the data element set S 2a Associated with, data record DR 2b is the data element set S 2b Associated with, and data record DR 2M is the data element set S 2M It is associated with, for example, the data record DR. Xa is the data element set S Xa Associated with, data record DR Xb is the data element set S Xb Associated with, and data record DR XN is the data element set S XN It is associated with.

[0131] The data records in a set are also associated with real-world objects related to concepts (i.e., imagined images, sounds, etc.). Therefore, for example, data record DR 1a , DR 1b ,...DR 1K Each of them is an object ob j1 It is associated with, for example, the data record DR. 2a , DR 2b ,...DR 2M Each of them is an object ob j2 Associated with, data record DR Xa , DR Xb ,...DR XN Each of them is an object ob jX It is associated with.

[0132] The data records in a set are also associated with response behaviors that can be invoked to cause the associated real-world objects to perform one or more actions. For example, a data record DR 1a , DR 1b ,...DR 1K is an object ob j1 Responses associated with the operation 1a , A 1b ,...A 1K Each is associated with the respective. Similarly, for example, data record DR 2a , DR 2b ,...DR 2M is an object ob j2 Response actions A2a and A associated with this behavior 2b ,...A 2M Each is associated with a data record DR Xa , DR Xb ,...DR XN is an object ob jX Responses associated with the operation Xa , A Xb ,...A XN They are associated with each other.

[0133] Accordingly, according to embodiments of the present invention, there is an association between a real-world object (for example, essentially related to a concept), a set of data records, a set of data elements of computer-readable data generated from the concept, and a set of response actions related to the real-world object. In certain embodiments, the aforementioned association (or "linkages") is maintained within a data record manager (60), for example, as relational data.

[0134] Each data record can store various attributes associated with the real-world object it relates to. Such attributes may include, for example, the functional operation parameters of the object. For example, if the object is an electronic device, the attributes may include the activation settings, operation settings, and / or actions of the electronic device. For example, if the object is an air conditioning unit, the attributes stored in the data record (or multiple data records) associated with the air conditioning unit may include temperature settings, fan settings, timer settings, and so on.

[0135] In practice, it should be noted that all data elements of the generated computer-readable data item should be associated with the same object. For example, set S 1a S 2b ,..., S 1K If all data elements of are elements of the same generated computer-readable data item, then set S 1a S 2b ,..., S 1K All data elements within are the same object (for example, ob j1) must be associated with an object. For example, if the concept is a mental image of a remotely controlled television device (i.e., an imagined photograph of the remotely controlled device), then each set of data elements of the same image data may correspond to different groups of pixels in the image (i.e., different parts of the image), but still be associated with the same object (e.g., a television controlled by the remotely controlled device).

[0136] In addition, each set of data elements for a given computer-readable data item needs to correspond to (i.e., be associated with) a different response behavior. Furthermore, in some embodiments, the subject can mentally focus on a particular part of a concept (e.g., a particular region of a mental image, a particular segment of a non-acoustic sound, etc.). The processing subsystem (200) can process brain signals representing the focused portion of the concept to generate data elements that are also part of the computer-readable data generated from the concept. For example, if the concept is a mental image, the subject can mentally focus on a particular region of the mental image so that the processing subsystem (200) generates pixel data associated with that particular region, which is also part of the image data generated from the overall mental image. Therefore, for example, if the concept is a mental image of a television remote-control device, such that computer-readable data is image data describing a remote-control device, then each set of data elements of the same image data can correspond to different pixels (i.e., different parts of the image) of the image, and particular pixels can also be generated (or regenerated) when an object mentally focuses on the part of the mental image corresponding to those particular pixels. Furthermore, each set of data elements of the same image data can correspond to different pixels (i.e., different parts of the image) of the image and can correspond to (i.e., be associated with) different response actions. For example, one part of the image (which can be mentally focused on by an object) could be volume control, which can be associated with a response action that operates or controls the television (the object associated with the image of the remote-control device) to increase or decrease the volume, and another part of the image (which can also be mentally focused on) could be channel control, which can be associated with a response action that operates or controls the television associated with the image of the remote-control device to change channels, etc. Therefore, for example, data records, e.g., DR 1amay be associated with a data element set, such as S 1a (corresponding to the volume control portion of the generated image of the remote control device), and may call a command for an object, such as ob j1 to adjust the volume of the TV, including a first response operation, such as A 1a may be associated with (e.g., associated with the remote control device, i.e., the TV controlled by the remote control device), and another data record, such as DR 1b is another data element set, such as S 1b may be associated with (corresponding to the channel control portion of the generated image of the remote control device), and a second response operation, such as A 1b may be associated with, which may include calling a command for the same object ob j1 (e.g., the TV) to change the channel of the TV.

[0137] It should also be noted that different generated computer-readable data may be associated with the same object but may be associated with different response operations. Thus, for example, a data element set, such as S 1a may be derived from different computer-readable data, and another data element set, such as S 1b may be derived from different computer-readable data, but the data element sets S 1a and S 1b can both be associated with the same object, such as ob j1 As an example, the first concept may be a mental image of a snowflake such that the first computer-readable data is image data depicting a snowflake, and the second concept may be a mental image of a fire such that the second computer-readable data is image data depicting a fire (fire). The first image data (snowflake) may correspond to a first data element set, such as S 1a and the second image data (fire) may correspond to a second data element set, such as S 1b but the data elements S 1a and S 1bBoth sets can be associated with the same object, such as an air conditioning unit, but can be associated with different response actions. Thus, for example, a data record, such as DR 1a can be associated with a set of data elements S 1a corresponding to the generated image of snowflakes, and a first response action, such as A j1 that can include invoking a command for an object ob 1a (such as an air conditioning unit) to lower the output air temperature and can be associated with A 1b and another data record, such as DR 1b can be associated with a set of data elements S j1 corresponding to the generated image of fire, and a second response action, such as A 1b that can include invoking a command for an object ob

[0138] The specific examples of remote control devices, snowflakes, and fire are merely illustrative subsets of images. As will be apparent to those skilled in the art, other images can be generated from a wide range of mental images. For example, mental images such as icons, characters, text strings, and numbers can be converted into corresponding image data for interaction with a computer, smartphone, or other computerized electronic device, and can be used to invoke associated response actions on the relevant electronic device. For example, a subject can be converted by the processing subsystem (200) into image data describing the word "hello," for example, a string of characters (or one character at a time in chronological order), such as "h," "e," "l," "l," "o," can be visualized as a mental image. The processing subsystem (200) can then identify a data record associated with an image of the word "hello" and call an object associated with the identified data record in a data record manager (60), which could be, for example, a computer (e.g., laptop, desktop, etc.) for inputting the word "hello" into a word processing program, an internet browser, or any other computer program with text input, thereby providing virtual keyboard functionality to the object. Note that the processing subsystem (200) can call an action with or without identifying the meaning of the computer-readable data being worked on. Continuing with the example of virtual keyboard functionality, the processing subsystem (200) can call an object to represent the keying of characters or words described by the computer-readable data.

[0139] Although the examples discussed above were explained in the context of image data, as stated in previous sections of this disclosure, computer-readable data can take various forms depending on the type of concept being converted, and therefore, for example, if the concept is a non-acoustic sound (i.e., an imagined sound), it may include audio signals, such as digital sound files like MP3 or WAV files (or any other digital sound files).

[0140] Objects that can initiate one or more response actions include, for example, electronic devices, as well as computers (e.g., laptops, desktops, tablets, etc.) and mobile devices (e.g., smartphones, mobile phones, etc.), computer game consoles (e.g., PlayStation made by Sony Interactive Entertainment LLC in San Mateo, California, USA, Xbox made by Microsoft in Redmond, Washington, USA, etc.), printers, scanners, display devices (e.g., televisions, computer monitors (i.e., electronic displays), head-up displays, head-mounted displays, etc.), household appliances (e.g., washing machines, dryers, refrigerators, air conditioning units, ovens, dishwashers, etc.), robotics, remote control units and devices for controlling other electronic devices (e.g., displays, electrical appliances, etc.), automobiles (e.g., cars, motorcycles, electric bicycles, tractors, tractor-trailers, trains, autonomous vehicles, semi-autonomous vehicles) This may include any device having one or more processor components, including but not limited to machines having electronic components, such as vehicles, aircraft, and computerized components (e.g., medical equipment and medical imaging equipment (e.g., MRI machines, X-ray machines, etc.), assembly lines, production lines, power tools, etc.).

[0141] In order for a system (10) according to embodiments of the present disclosure to enable control of any object, such object must communicate electronically or data-wise with the system (10) so that communication messages, including command-and-control messages and instructions, can be exchanged between the system (10) and the object. In one set of non-limiting examples, the object is a network-enabled device that communicates with the system (10) via a communication network, such as network (36). For example, the object may be fitted with network devices to become a “smart” electronic device, as is well known in the art. In other examples, the object as an electronic device may be pre-configured as a network-enabled device, for example, with wireless network communication hardware such as Bluetooth® compliant hardware. For example, Figure 9 schematically shows an object (40) that is part of a networked system environment in which the system (10) may be deployed according to embodiments of the present disclosure. In the non-limiting embodiment shown in Figure 9, the Tx / Rx unit (30) provides a communication / network interface for sending / receiving data (i.e., exchanging data with network devices) to and from one or more objects (e.g., network devices) (40) over a network (36). However, it should be noted that other networked communication interfaces are possible.

[0142] In certain circumstances, when system (10) initiates a response action, it may be advantageous or beneficial for the subject to receive some type of feedback from system (10) (in particular, the processing subsystem (200)). In certain embodiments, the processing subsystem (200) provides feedback to the subject's sensory system or brain in response to the success or failure of initiating a response action. For example, if the processing subsystem (200) initiates an action that includes controlling or activating an object, the processing subsystem (200) may send one piece of feedback to the subject's sensory system if the action initiated by the processing subsystem is successful, and another piece of feedback to the subject's sensory system if the action initiated by the processing subsystem is unsuccessful. When feedback is provided to the subject's sensory system, the feedback may take one or more forms of visual feedback, auditory feedback, and tactile feedback.

[0143] In certain embodiments, the processing subsystem (200) may provide feedback to the subject as non-sensory feedback by injecting feedback data into appropriate regions of the subject's brain. For example, the processing subsystem (200) may process feedback data indicating the success or failure of a response action and convert the feedback data into one or more brain signals, and then provide one or more brain signals to regions (103) of the subject's brain so that the success or failure of the response action is formed conceptually by the brain region without sensory input. In other embodiments, the processing subsystem (200) may provide feedback to the subject's sensory system by injecting feedback data into appropriate regions of the subject's brain. For example, the processing subsystem (200) may process sensory feedback data indicating the success or failure of a response action (e.g., visual feedback data, auditory feedback data, etc.) and convert the feedback data into one or more brain signals, and then provide one or more brain signals to appropriate regions (e.g., visual cortex, auditory cortex, etc.) of the subject so that the subject perceives the feedback indicating the success or failure of the response action visually or auditorily.

[0144] In certain embodiments, the object may attempt to interact with the object again based on the feedback received by the object.

[0145] In certain embodiments, the processing subsystem (200) is configured to identify data records based solely on computer-readable data generated from concepts. However, in other embodiments, the processing subsystem (200) may be configured to identify data records based on a combination of the generated computer-readable data and one or more detected actions or inputs. These detected actions or inputs may include one or more sensory inputs (e.g., visual perception, auditory perception, touch / touch, smell, etc.) or non-sensory inputs (e.g., additional mental concepts) that differ from the type of mental concept on which the computer-readable data generated by the processing subsystem (200) is based. For example, if the computer-readable data is image data (i.e., the concept is a mental image), the additional non-sensory input may be computer-readable data generated from non-acoustic sounds.

[0146] In certain embodiments, the processing subsystem (200) is configured to initiate a response action to invoke an associated object to execute a command only if the identified data record is associated with one or more specific data elements of computer-readable data pre-programmed in memory associated with the processing subsystem (200). For example, the processing subsystem (200) may be configured to initiate a command only if the data elements associated with the identified data record are pixel data from a pre-configured image or predetermined changes to an image (e.g., color changes, shape changes, etc.).

[0147] In certain embodiments, the processing subsystem (200) can generate computer-readable data from collected brain signals generated while the subject is performing an action, in order to increase the effectiveness of response action calls and object associations. For example, if the subject activates an air conditioning unit to lower the ambient air temperature in a room, the processing subsystem (200) can be configured to collect brain signals representing mental concepts, such as a mental image of a desert formed in a brain region (103), while the subject is activating the air conditioning. The processing subsystem (200) can then generate image data representing the concept (e.g., image data describing a desert). The processing subsystem (200) can then update the data record manager (60) to associate the object (as an air conditioning unit), the desert image, and a response action to activate the object and lower the temperature. As is evident, this process of generating computer-readable data and updating associations in the data record manager (60) while the subject is performing a task or action can be repeated as needed. This process can be advantageously used in several situations, such as in non-human animal training scenarios where animals can understand the relationship between their mental image and the actions performed by an object. For example, a dog can form a mental image of a dog treat dispensing machine, which can then be recalled for dispensing dog treats.

[0148] Returning to Figure 8, it should be understood that the step of identifying data records (step (806)) may be performed by the processing subsystem (200) by, for example, retrieving multiple data records stored / maintained in the data record manager (60), and then marking or flagging the correct data records associated with data elements of the computer-readable data (generated in step (804)). In certain cases, the processing subsystem (200) may perform the retrieval and marking / flagging by remote processing requests over a network (e.g., network (36)) if, for example, the processing components of the data record manager (60) and the processing subsystem (200) are remote from each other (e.g., the processing device (12) and / or the remote processing system (38) and / or the server system (34) are remote from the data record manager (60). Remote processing requests may include, for example, the exchange of network messages over a network (e.g., network (36)). In other cases, for example, if the data record manager (60) is part of a processing subsystem (200) (e.g., maintained in the memory of a processing device (12) or a remote processing system (38) or a server system (34)), the processing subsystem (200) may perform lookup and marking / flacking by local processing requests.

[0149] Furthermore, it should be evaluated that the step of initiating a response operation (step (808)) may be performed by the processing subsystem (200) by, for example, sending a communication message containing command and control messages and instructions that carry a response operation instruction. For example, the processing subsystem (200) may initiate a response operation by sending a communication message that carries a response operation instruction to an object over a network (e.g., network (36)) via a network interface (e.g., Tx / Rx unit (30)).

[0150] It should be noted that the representations of data types and the relationship structures between them shown in Figure 10 are merely illustrative examples to more clearly illustrate the relational structure relationships between actual objects, actions, and computer-readable data generated from concepts. Other data structure relationships that enable the control and / or interaction of real-world objects related to concepts are contemplated herein.

[0151] Implementations of the methods and / or systems of the embodiments of the present invention may include performing or completing selected tasks manually, automatically, or in combination thereof. Furthermore, according to the actual instruments and equipment of the embodiments of the methods and / or systems of the present invention, some selected tasks can be implemented using hardware, software, firmware, or a combination thereof, with respect to an operating system.

[0152] For example, hardware for performing a selected task according to embodiments of the present invention can be implemented as a chip or circuit. As software, a selected task according to embodiments of the present invention can be implemented as a set of software instructions performed by a computer using any suitable operating system. In exemplary embodiments of the present invention, one or more tasks according to exemplary embodiments of the methods and / or systems described herein are performed by a data processor, such as a computing platform that performs a set of instructions. Optionally, the data processor includes volatile memory for storing instructions and / or data, and / or non-volatile storage, such as non-temporary storage media, such as magnetic hard disks and / or removable media, for storing instructions and / or data. Optionally, network connectivity is also provided. A display and / or user input devices, such as a keyboard or mouse, are also provided, as is optional.

[0153] For example, any combination of one or more non-temporary computer-readable (storage) media may be used in accordance with the embodiments listed above of the present invention. Non-temporary computer-readable (storage) media may be, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More specific examples (non-exclusive list) of computer-readable storage media may include electrical connections having one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the context of this document, a computer-readable storage medium can be any tangible medium that contains or can store programs used by or in connection with an instruction execution system, apparatus, or device.

[0154] As can be understood by referring to the above paragraphs and reference drawings, various embodiments of computer implementation methods are provided herein, some of which can be carried out by various embodiments of the apparatus and systems described herein, some of which can be carried out in accordance with instructions stored in non-temporary computer-readable storage media described herein. Furthermore, some embodiments of computer implementation methods provided herein can be carried out by other apparatus or systems, as will be apparent to those skilled in the art by referring to the embodiments described herein, and can be carried out in accordance with instructions stored in computer-readable storage media other than those described herein. Any references to systems and computer-readable storage media in the following computer implementation methods are provided for illustrative purposes only and are not intended to limit any of such systems or any of such non-temporary computer-readable storage media with respect to the embodiments of computer implementation methods described herein. Similarly, any references to systems and computer-readable storage media in the following computer implementation methods are provided for illustrative purposes only and are not intended to limit any of such computer implementation methods disclosed herein.

[0155] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, section, or portion of code containing one or more executable instructions for implementing a specified logical function(s). It should also be noted that in some alternative implementations, the functions shown within a block may be performed in an order different from that shown in the figure. For example, two consecutively shown blocks may actually be performed substantially simultaneously, or in some cases, these blocks may be performed in reverse order depending on the functions they relate to. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system or a combination of dedicated hardware and computer instructions that performs a specified function or operation.

[0156] The descriptions of various embodiments of the present invention are presented for illustrative purposes only and are not intended to be exhaustive or limitful to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the embodiments described. The terms used herein have been chosen to best describe the principles, practical applications, or technical improvements to the technologies available on the market of the embodiments, or to enable those else skilled in the art to understand the embodiments disclosed herein.

[0157] As used herein, the singular forms "a," "an," and "the" include multiple references unless the context clearly indicates otherwise. For example, a reference to a single nerve may refer to both nerves of a nerve pair. Furthermore, a reference to both nerves of a nerve pair may refer to a single nerve unless the context clearly indicates otherwise.

[0158] The term “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described as “exemplary” should not necessarily be construed as being preferable or advantageous to other embodiments, and / or preclude the incorporation of features from other embodiments.

[0159] For clarity, it is understood that certain features of the invention described in relation to separate embodiments may be provided in combination in a single embodiment. Conversely, for brevity, various features of the invention described in relation to a single embodiment may be provided individually, in any suitable subcombination, or as appropriate in any other described embodiment of the invention. Certain features described in relation to various embodiments are not considered essential features of those embodiments unless the embodiments would not function without those elements.

[0160] The processes described above (including parts thereof) can be carried out by software, hardware, and combinations thereof. These processes and parts thereof can be carried out by computers, computer-like devices, workstations, processors, microprocessors, other electronic searching tools and memory, and other non-temporary storage devices associated therewith. Furthermore, the processes and parts thereof can be embodied in programmable non-temporary storage media, such as machine-readable compact discs (CDs), or other discs including magnetic disks and optical disks, or other computer-usable storage media including magnetic storage, optical storage, or semiconductor storage, or other electronic signal sources.

[0161] This specification has described processes (methods) and systems, including their components, with illustrative reference to specific hardware and software. The processes (methods) are described illustratively, so that certain steps and their order may be omitted and / or modified by those skilled in the art to implement these embodiments without excessive experimentation. The processes (methods) and systems are described in a manner sufficient to allow those skilled in the art to readily adapt other hardware and software as necessary to implement any of the embodiments without excessive experimentation and the use of prior art.

[0162] To the extent that the attached claims are written without multiple dependencies, this is done solely to address the formal requirements in jurisdictions that do not permit such multiple dependencies. It should be noted that all possible combinations of features implied by making the claims multiple dependencies are explicitly assumed and should be considered part of the invention.

[0163] While the present invention has been described in relation to its specific embodiments, it will be apparent that many alternative, modified, and variant forms are obvious to those skilled in the art. Therefore, it is intended to encompass all such alternative, modified, and variant forms that fall within the spirit and broad scope of the appended claims.

Claims

1. A method for use in animals having a brain that includes a region responsible for forming concepts without sensory input, wherein the method is A processing subsystem that communicates with the aforementioned brain region receives brain signals representing a concept formed by the aforementioned brain region without sensory input. The process of processing the received brain signals by the processing subsystem in order to convert the aforementioned concept into computer-readable data representing the tangible form of the aforementioned concept, The processing subsystem includes a step of identifying data records associated with one or more data elements of the computer-readable data, wherein the data records are also associated with real-world objects associated with the concept. Methods that include...

2. The method according to claim 1, wherein the real-world object is an electronic device.

3. The method according to claim 1, further comprising the step of initiating at least one response action according to the identified data record.

4. The method according to claim 3, wherein the at least one response action includes activating or controlling the real-world object.

5. The method according to claim 3, further comprising the step of providing feedback to the sensory system in question in response to the success or failure of the initiation of the at least one response action.

6. The method according to claim 3, further comprising the steps of: i) processing feedback data indicating the success or failure of the initiation of the at least one response action by the processing subsystem, thereby converting the feedback data into one or more brain signals; and ii) providing the one or more brain signals to the brain regions such that the success or failure of the initiation of the at least one response action is formed conceptually by the brain regions without sensory input, thereby providing the feedback to the subject.

7. The method according to claim 1, wherein the at least one response operation is selected from a plurality of response operations, each of the plurality of response operations is associated with a corresponding data record among a plurality of data records, and each of the plurality of data records is associated with one or more corresponding elements of the computer-readable data.

8. The method according to claim 1, wherein the real-world object is one of a plurality of real-world objects, the data record is one of a plurality of data records, the plurality of data records consists of a plurality of subsets of data records, and each subset is associated with a corresponding real-world object among the plurality of real-world objects.

9. The method according to claim 1, wherein the concept is a mental image or a non-acoustic sound.

10. The method according to claim 1, wherein the computer-readable data is image data or audio data.

11. A system for use in animals having a brain that includes a region responsible for forming concepts without sensory input, wherein the system is The processing subsystem includes a processing subsystem configured to communicate with the brain region, the processing subsystem is configured to Receiving brain signals that represent concepts formed by the aforementioned brain regions without sensory input, The process involves processing the received brain signals in order to convert the aforementioned concept into computer-readable data representing the tangible form of the aforementioned concept. Identifying a data record associated with one or more data elements of the computer-readable data, wherein the data record is also associated with a real-world object related to the concept. A system configured to perform the following actions.

12. The system according to claim 11, wherein the real-world object includes an electronic device.

13. The system according to claim 11, wherein the processing subsystem is further configured to initiate at least one response operation according to the identified data record.

14. The system according to claim 13, wherein the at least one response action includes activating or controlling the real-world object.

15. The system according to claim 1, wherein the at least one response action is selected from a plurality of response actions, each of the plurality of response actions is associated with a corresponding data record among a plurality of data records, and each of the plurality of data records is associated with one or more corresponding elements of the computer-readable data.

16. The system according to claim 11, wherein the real-world object is one of a plurality of real-world objects, the data record is one of a plurality of data records, and the system further comprises at least one storage medium that communicates with the processing subsystem in order to maintain the plurality of data records, and the plurality of data records consists of a plurality of subsets of data records, each subset being associated with a corresponding real-world object of the plurality of real-world objects.

17. The system according to claim 11, wherein the aforementioned concept is a mental image or a non-acoustic sound.

18. The system according to claim 11, wherein the computer-readable data is image data or audio data.

19. A method for use in animals having a brain that includes a region responsible for forming concepts without sensory input, wherein the method is A processing subsystem that communicates with the aforementioned brain region receives brain signals representing a concept formed by the aforementioned brain region without sensory input. The process of processing the received brain signals by the processing subsystem in order to convert the aforementioned concept into computer-readable data representing the tangible form of the aforementioned concept, The process of activating or controlling a real-world object related to the aforementioned concept and associated with one or more data elements of the computer-readable data, Methods that include...

Images