Processing brain data using autoencoder neural networks

Abstract

Methods, systems, and devices for processing brain data using autoencoder neural networks, including computer programs encoded on computer storage media. One of the methods includes obtaining brain data characterizing brain activity of a patient captured by one or more sensors; processing the brain data to generate modified brain data characterizing a predicted local effect of a future treatment on the brain of the patient; processing the modified brain data using an autoencoder neural network to generate reconstructed brain data; and determining a predicted global effect of the future treatment on the brain of the patient using the reconstructed brain data.

Description

[0001] Cross-reference to related applications

[0002] This application relates to U.S. Patent Application No. 17 / 066171, filed October 8, 2020, the entire contents of which are incorporated herein by reference.

[0003] This application is also related to U.S. Patent Application No. 17 / 066178, filed October 8, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0004] This instruction manual relates to the processing of brain-related data in patients, such as functional magnetic resonance imaging (MRI) data and / or fiber tractography data. Background Technology

[0005] Brain functional connectivity data characterize the degree of correlation of brain activity in each of one or more pairs of locations in a patient's brain.

[0006] Brain-related data can be collected by acquiring and processing images of a patient's brain, for example, using magnetic resonance imaging (MRI), diffusion tensor imaging (DTI), or functional MRI (fMRI). Diffusion tensor imaging uses MRI images to measure the diffusion of water in the human brain. The measured diffusion can be used to generate fiber tract imaging data, which can include images of nerve fiber bundles and corresponding white matter fibers in the subject's brain.

[0007] Data related to a single patient's brain can be highly complex and dimensional, making it difficult for clinicians to manually examine and analyze it for purposes such as planning surgery or diagnosing whether a patient has a brain disease or mental disorder. Summary of the Invention

[0008] This specification describes a system for processing patient brain data using an autoencoder neural network, implemented as a computer program on one or more computers at one or more locations.

[0009] In this specification, an autoencoder neural network is a neural network comprising at least two subnetworks: an encoder subnetwork and a decoder subnetwork. The encoder subnetwork is configured to process the network input of the autoencoder and generate an embedding of that network input. The decoder subnetwork is configured to process the embedding of that network input and generate a reconstructed network input. Typically, the autoencoder neural network is trained to generate a reconstructed network input that is as similar as possible to the original network input.

[0010] In some embodiments described in this specification, the system may use an autoencoder neural network to process "modified" brain data to generate reconstructed brain data. The modified brain data can characterize the predicted local effects of future treatment on the patient's brain, while the reconstructed brain data can characterize the overall effects of future treatment on the patient's brain. Because the local effects of treatment (i.e., the effect of treatment on a brain region local to the treatment target location) are often easier to predict than the overall effects of treatment, the system can utilize the autoencoder neural network to generate accurate predictions of the overall effects of treatment before delivery. Therefore, the system can allow users (e.g., clinicians or other medical professionals) to determine what will be, or is more likely to be, a safe and effective treatment for a patient.

[0011] In some other embodiments described in this specification, the system may use an inverted autoencoder neural network to process “expected” brain data to generate “roadmap” brain data. Expected brain data characterizes the expected overall effect of future treatment on the patient’s brain, while roadmap brain data characterizes the local effect of future treatment on the patient’s brain. Because local effects of treatment are easier to predict than overall effects, the system can utilize an inverted autoencoder neural network to generate roadmap brain data from which future treatments, if provided to the patient, will achieve or are more likely to achieve the expected overall effect of the future treatment. In other words, the system or system users (e.g., clinicians or other medical professionals) can use roadmap brain data to determine parameters for future treatments (e.g., the location in the patient’s brain that serves as the treatment target), making the future treatment safe and more likely to be effective.

[0012] In some other embodiments described in this specification, the system may use an autoencoder neural network to process the patient's brain data to generate reconstructed brain data, and then process the reconstructed brain data to determine whether the patient's original brain data is abnormal, i.e., whether it exceeds the normal range. For example, if the measured difference between the original brain data and the reconstructed brain data exceeds a threshold, the system can determine that the brain data is abnormal. That is, the system can compare the original brain data with the reconstructed brain data (wherein the reconstructed brain data includes information learned from other patients during the training of the autoencoder neural network) to identify abnormalities in the original brain data. In some such embodiments, the system may further identify a subset of the abnormal original brain data, e.g., a subset corresponding to a specific region of the patient's brain. Thus, the system can allow a user (e.g., a clinician or other medical professional) to quickly identify abnormalities in the brain data that may indicate the patient has a brain disease, and optionally, indicate brain regions that may be the target of treatment for the brain disease.

[0013] In this specification, brain data can be any data characterizing the patient's brain. For example, brain data can include one or both of the following: i) direct measurement data of the patient's brain, such as brain images collected using brain imaging techniques; or ii) data derived or generated from initial measurement data of the patient's brain, such as a correlation matrix.

[0014] Specific embodiments of the subject matter described in this specification may be implemented to achieve one or more of the following advantages.

[0015] As mentioned above, brain datasets characterizing the brain of a single patient can often be very large and complex, making it difficult and time-consuming for users to extract useful information from them. By using the techniques described in this specification, the system can process patient brain data to generate clinically relevant web outputs that users can use to provide safe and effective care for the patient. For example, compared to a user manually reviewing large corpora of brain data, the system allows the user to determine patient treatment significantly more accurately and efficiently (e.g., with less time or using fewer computational, storage, or network resources). More specifically, the correlation matrix of a patient's brain (e.g., a correlation matrix of fMRI data) can be a matrix with over one hundred elements, over one thousand elements, or more typically over one hundred thousand elements. When considering a set or series of such correlation matrices, the system used to generate and / or analyze such matrices can handle over one million elements. By using the techniques described in this specification, the system can reduce the number of elements a user is considering to generate clinically relevant data, such as recommendations for specific actions, within 10 minutes, 5 minutes, 3 minutes, 1 minute, 30 seconds, or 5 seconds.

[0016] As stated above, even experienced medical professionals find it difficult to predict the long-term, overall effects of treatment on a patient's brain. By using the techniques described in this specification, the system can train an autoencoder neural network using training data (representing the clinical outcomes of treatments provided to a large number of patients, such as over one hundred, over one thousand, over one hundred thousand, or over one million patients) to accurately predict the outcome of a particular treatment on a particular patient's brain. Similarly, the system can use training data to train an autoencoder neural network to accurately characterize what treatments will produce or are likely to produce the desired outcome on a patient's brain. Therefore, the autoencoder neural network described herein can learn the complex, non-linear relationships between different brain regions and their different treatments, allowing users to seek safe and effective treatments for patients in ways that cannot be achieved by examining a patient's brain data individually.

[0017] By using the techniques described in this specification, the system can quickly identify one or more regions in a patient's brain where brain data deviates from the normal range, such as parcellation, and thus may be an indication of brain disease. The system can then display data representing the identified regions to the user, eliminating the need for the user to search and analyze large amounts of clinically irrelevant data. This significantly reduces the amount of time a user must spend discovering useful portions of brain data, leading to improved outcomes, particularly for patients, users, and / or clinicians, when effective care requires time-sensitive investigations.

[0018] Details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from this description, the drawings, and the claims. Attached Figure Description

[0019] Figure 1A and Figure 1B This is a block diagram illustrating an exemplary computer system for processing medical images.

[0020] Figure 2A This is a diagram of an exemplary autoencoder neural network system.

[0021] Figure 2B This is a diagram of an exemplary inverse autoencoder neural network system.

[0022] Figure 3 This is a diagram of an exemplary anomaly detection system.

[0023] Figure 4 This is a flowchart illustrating an exemplary process for processing brain data using an autoencoder neural network.

[0024] Figure 5 This is a flowchart illustrating an exemplary process for processing brain data using a reverse autoencoder neural network.

[0025] Figure 6 This is a flowchart of an exemplary process for identifying anomalous brain data using an autoencoder neural network.

[0026] Similar reference numerals and names in the various figures indicate similar elements. Detailed Implementation

[0027] This specification describes a system for processing a patient's brain data using an autoencoder neural network.

[0028] Figure 1A and Figure 1BThis is a block diagram of a general-purpose computer system 100 on which the arrangements described in this specification can be implemented. The following description is primarily directed to the computer server module 101. However, this description is equally or equivalently applicable to one or more remote terminals 168.

[0029] like Figure 1A As shown, computer system 100 includes: a server computer module 101; input devices such as a keyboard 102, a pointing device 103 (e.g., a mouse), a scanner 126, a camera 127, and a microphone 180; and output devices including a printer 115, a display device 114, and a speaker 117. An external modem transceiver device 116 may be used by computer server module 101 to communicate with remote terminal 168 over computer communication network 120 via connection 121 and connection 170. This communication can occur between remote terminal 168 and a “cloud,” which, in this description, includes at least one server module 101. Remote terminal 168 typically has input and output devices (not shown) similar to those described with respect to server module 101. Communication network 120 may be a wide area network (WAN), such as the Internet, a cellular telecommunications network, or a dedicated WAN. If connection 121 is a telephone line, modem 116 may be a conventional “dial” modem. Alternatively, if connection 121 is a high-capacity (e.g., cable) connection, modem 116 can be a broadband modem. A wireless modem can also be used for wireless connection to communication network 120.

[0030] Computer server module 101 typically includes at least one processor unit 105 and a memory unit 106. For example, memory unit 106 may have semiconductor random access memory (RAM) and semiconductor read-only memory (ROM). Remote terminal 168 typically includes at least one processor 169 and memory 172. Computer server module 101 also includes multiple input / output (I / O) interfaces, including: an audio-video interface 107 coupled to a video display 114, a speaker 117, and a microphone 180; an I / O interface 113 coupled to a keyboard 102, a mouse 103, a scanner 126, a camera 127, and optionally a joystick or other human-machine interface device (not shown); and an interface 108 for an external modem 116 and a printer 115. In some embodiments, modem 116 may be included within computer module 101, for example, within interface 108. Computer module 101 also has a local network interface 111, which allows computer system 100 to be coupled to a local area communication network 122, called a local area network (LAN), via connection 123. Figure 1AAs shown, the local area communication network 122 can also be coupled to the wide area network 120 via connection 124, which typically includes a so-called "firewall" device or a device with similar functionality. The local network interface 111 may include an Ethernet circuit card, Wireless layout or IEEE 802.11 wireless layout; however, many other types of interfaces can also be used to implement interface 111.

[0031] I / O interfaces 108 and 113 can provide one or both of serial or parallel connections; the former can be implemented according to the Universal Serial Bus (USB) standard and has a corresponding USB connector (not shown). A memory device 109 is provided, which typically includes a hard disk drive (HDD) 110. Other storage devices, such as floppy disk drives and tape drives (not shown), may also be used. An optical disk drive 112 is typically provided for use as a non-volatile data source. For example, portable storage devices such as optical discs (e.g., CD-ROM, DVD, Blu-ray Disc™), USB-RAM, portable external hard disk drives, and floppy disks can be used as suitable data sources for system 100.

[0032] Components 105 to 113 of computer module 101 typically communicate via interconnect bus 104 in a manner consistent with the conventional operating mode of computer system 100 known to those skilled in the art. For example, processor 105 is coupled to system bus 104 via connection 118. Similarly, memory 106 and optical disc drive 112 are coupled to system bus 104 via connection 119.

[0033] The techniques described in this specification can be implemented using computer system 100, for example, as one or more software applications 133 executable within computer system 100. In some embodiments, the one or more software applications 133 execute on computer server module 101 (remote terminal 168 may also perform processing in conjunction with computer server module 101), and browser 171 executes on processor 169 in remote terminal, thereby enabling the user of remote terminal 168 to access the software application 133 executing on server 101 (commonly referred to as the "cloud") using browser 171. Specifically, the techniques described in this specification can be implemented using instructions 131 in software 133 executing within computer system 100 (see...). Figure 1B This can be achieved through [the following]. Software instructions 131 can be formed as one or more code modules, each code module being used to perform one or more specific tasks. The software can also be divided into two separate parts, wherein a first part and its corresponding code modules execute the described techniques, and a second part and its corresponding code modules manage the user interface between the first part and the user.

[0034] The software may be stored on a computer-readable medium, such as the storage devices described below. The software is loaded from the computer-readable medium into computer system 100 and then executed by computer system 100. A computer-readable medium on which such software or a computer program is recorded is a computer program product. Software modules for performing the techniques described herein may also be distributed using a web browser.

[0035] Software 133 is typically stored in HDD 110 or memory 106 (and may be stored, at least to some extent, in memory 172 of remote terminal 168). This software is loaded from a computer-readable medium into computer system 100 and executed by computer system 100. Thus, for example, software 133, which may include one or more programs, may be stored on an optically readable disk storage medium (e.g., CD-ROM) 125 read by optical disk drive 112. A computer-readable medium on which such software or computer programs are recorded is a computer program product.

[0036] In some instances, application 133 may be encoded on one or more CD-ROMs 125 and provided to the user and read via the corresponding drive 112, or may be read by the user from networks 120 or 122. Furthermore, the software may also be loaded into computer system 100 from other computer-readable media. Computer-readable storage media refers to any non-transitory tangible storage medium that provides recorded instructions and / or data to computer system 100 for execution and / or processing. Examples of such storage media include floppy disks, magnetic tapes, CD-ROMs, DVDs, Blu-ray™ discs, hard disk drives, ROMs or integrated circuits, USB storage devices, magneto-optical discs, or computer-readable cards such as PCMCIA cards, regardless of whether such devices are internal or external to computer module 101. Examples of temporary or non-tangible computer-readable transmission media that may also participate in providing software, applications, instructions, and / or data to computer module 101 include radio or infrared transmission channels and network connections to another computer or networked device, as well as the Internet or intranet, including email transmissions and information recorded on websites, etc.

[0037] The second part of the aforementioned application 133 and the corresponding code modules can be executed to implement one or more graphical user interfaces (GUIs) to be rendered or otherwise represented on the display 114. For example, by manipulating the keyboard 102 and mouse 103, users of the computer system 100 and the application can manipulate the interface in a functionally adaptable manner to provide control commands and / or input to the application associated with the GUI. Other forms of functionally adaptable user interfaces can also be implemented, such as an audio interface utilizing voice prompts output via speaker 117 and user voice commands input via microphone 180.

[0038] Figure 1B This is a detailed schematic block diagram of processor 105 and "memory" 134. Memory 134 represents memory that can be... Figure 1A The computer module 101 accesses the logical aggregation of all memory modules (including HDD 109 and semiconductor memory 106).

[0039] When the computer module 101 is initially powered on, the Power-On Self-Test (POST) program 150 can be executed. The POST program 150 can be stored in... Figure 1A The ROM 149 of the semiconductor memory 106 stores the software. The hardware device storing the software, such as ROM 149, is sometimes referred to as firmware. The POST program 150 checks the hardware within the computer module 101 to ensure proper functioning, and typically checks the processor 105, the memories 134 (109, 106), and the Basic Input / Output System Software (BIOS) module 151 (also typically stored in ROM 149) for correct operation. Once the POST program 150 runs successfully, the BIOS 151 can be activated. Figure 1A The hard disk drive 110. Activation of the hard disk drive 110 causes the bootloader 152 residing on the hard disk drive 110 to be executed via the processor 105. This loads the operating system 153 into the RAM memory 106, where the operating system 153 begins operation. The operating system 153 is a system-level application that can be executed by the processor 105 to implement various advanced functions, including processor management, memory management, device management, storage management, software application interface, and general user interface.

[0040] Operating system 153 manages memory 134 (109, 106) to ensure that each process or application running on computer module 101 has sufficient memory to execute within it without conflicting with memory allocated to another process. Furthermore, appropriate use of memory is essential. Figure 1AThe system 100 contains different types of memory so that each process can run efficiently. Therefore, aggregate memory 134 is not intended to describe how specific segments of memory are allocated (unless otherwise stated), but rather to provide a general view of the memory accessible to the computer system 100 and how such memory is used.

[0041] like Figure 1B As shown, processor 105 includes multiple functional modules, including a control unit 139, an arithmetic logic unit (ALU) 140, and local or internal memory 148 (sometimes referred to as a cache). Cache 148 typically includes multiple storage registers 144-146 in a register section. One or more internal buses 141 functionally interconnect these functional modules. Processor 105 also typically has one or more interfaces 142 for communicating with external devices via system bus 104 using connection 118. Memory 134 is coupled to bus 104 using connection 119.

[0042] Application program 133 includes instruction sequence 131, which may include conditional branch and loop instructions. Program 133 may also include data 132 used in the execution of program 133. Instructions 131 and data 132 are stored in memory locations 128, 129, 130 and 135, 136, 137, respectively. Depending on the relative size of instruction 131 and memory locations 128-130, a particular instruction may be stored in a single memory location, as shown in the instruction diagram at memory location 130. Alternatively, instructions may be partitioned into segments, each stored in a separate memory location, as shown in the instruction segments diagram at memory locations 128 and 129.

[0043] Typically, a set of instructions is provided to processor 105 for execution. Processor 105 awaits subsequent input, responding to it by executing another set of instructions. Each input can be provided from one or more of multiple sources, including data generated by one or more of input devices 102, 103, data received on one of networks 120, 122 from an external source 173 (e.g., a brain imaging device 173 such as an MRI or DTI scanner), data retrieved from one of storage devices 106, 109, or data retrieved from storage medium 125 inserted into the corresponding reader 112. Figure 1A As shown in the diagram. The execution of a set of instructions may, in some cases, result in the output of data. Execution may also involve storing data or variables into memory 134.

[0044] Some of the techniques described in this specification use input variables 154, such as a dataset characterizing a patient's brain, stored in corresponding memory locations 155, 156, and 157 in memory 134. These techniques can produce output variables 161, which are stored in corresponding memory locations 162, 163, and 164 in memory 134. Intermediate variables 158 can be stored in memory locations 159, 160, 166, and 167.

[0045] refer to Figure 1B The processor 105, registers 144, 145, 146, arithmetic logic unit (ALU) 140, and control unit 139 work together to execute a sequence of micro-operations required for a "fetch, decode, and execute" cycle for each instruction in the instruction set constituting program 133. Each fetch, decode, and execute cycle may include: i) a fetch operation, which fetches or reads instruction 131 from memory locations 128, 129, 130; ii) a decode operation, in which control unit 139 determines which instruction was fetched; and iii) an execute operation, in which control unit 139 and / or ALU 140 execute the instruction.

[0046] Subsequently, a further extraction, decoding, and execution cycle can be performed for the next instruction. Similarly, a storage cycle can be executed, through which the control unit 139 stores or writes the value to memory location 132.

[0047] Each step or subprocess in the techniques described in this specification may be associated with one or more segments of program 133 and executed by register portions 144, 145, 146, ALU 140, and control unit 139 in processor 105, which work together to perform fetch, decode, and execution cycles for each instruction in the instruction set of the specified segment of program 133. Although cloud-based platforms have been described for practicing the techniques described in this specification, other platform configurations may also be used. Furthermore, other hardware / software configurations and distributions may also be used to practice the techniques described in this specification.

[0048] Figure 2A This is a diagram of an exemplary autoencoder neural network system 200. The autoencoder neural network system 200 is an example of a system implemented as a computer program on one or more computers at one or more locations, in which the systems, components and techniques described below can be implemented.

[0049] The autoencoder neural network system 200 is configured to process patient brain data 202 and generate reconstructed patient brain data 222 corresponding to the patient brain data 202.

[0050] An autoencoder neural network system 200 includes an encoder subnetwork 210 and a decoder subnetwork 220. The encoder subnetwork 210 is configured to process brain data 202 and generate an embedding 212 of the brain data 202. The decoder subnetwork 220 is configured to process the embedding 212 of the brain data 202 and generate reconstructed brain data 222. In this specification, an embedding is an ordered set of numerical values ​​representing an input in a specific embedding space; for example, an embedding can be a floating-point vector or other numerical values ​​with a fixed dimension. In this specification, the reconstructed brain data is brain data estimated using the embeddings of the brain data, for example, by processing the embeddings using the decoder subnetwork of the autoencoder neural network.

[0051] Typically, the embedding 212 has a lower dimensionality than the brain data 202, while the reconstructed brain data 222 has the same dimensionality as the patient's brain data 202. For example, the dimensionality of the embedding 212 could be 1 / 10, 1 / 100, or 1 / 1000 of the dimensionality of the brain data 202. That is, information is lost when the encoder subnetwork 210 processes the brain data 202 to generate the embedding 212, so the reconstructed brain data 222 is only an approximation of the original brain data 202. During training, the encoder subnetwork 202 can learn to encode as much information from the brain data 202 as possible into the embedding 212, and the decoder subnetwork 220 can learn to reconstruct the brain data 202 to generate the reconstructed brain data 222, such that the reconstructed brain data 222 reflects the information encoded into the embedding 212.

[0052] In other words, the autoencoder neural network system 200 can be configured to generate reconstructed patient brain data 222 through training, such that the difference between the patient's brain data 202 and the reconstructed patient brain data 202 is reduced to below a threshold or minimized. For example, the training system can use the autoencoder neural network system 200 to process training examples, where each training example includes brain data corresponding to various patients to generate respective reconstructed brain datasets. For each training example, the training system can determine a reconstruction error, which characterizes i) the difference between the individual patient's brain data and ii) the corresponding reconstructed brain data. For example, the reconstruction error can be an L1 or L2 distance between the brain data and the reconstructed brain data, or a squared version thereof. As another example, the reconstruction error can be the root mean square error between the brain data and the reconstructed brain data.

[0053] The training system can then backpropagate the reconstruction error through the autoencoder neural network system 200 to determine, for example, using gradient descent, updates to the parameter values ​​of the autoencoder neural network system 200. For example, the training system can determine updates to the parameter values ​​of both the encoder subnetwork 210 and the decoder subnetwork 220 (i.e., the encoder subnetwork 210 and the decoder subnetwork 220 can be trained simultaneously). In other words, the training system can train the autoencoder neural network system 200 in an unsupervised manner, i.e., using training examples that do not include a ground truth signal (e.g., a ground truth embedding 212 of the brain data from the training examples).

[0054] Brain data 202 may include any data characterizing the patient's brain. For example, brain data 202 may include one or more of blood oxygenation level-related imaging data, fMRI data, or EEG data captured from the patient's brain. Alternatively or additionally, brain data 202 may include data generated using the various raw data captured from the patient's brain. For example, brain data 202 may include correlation data characterizing the degree of correlation between respective brain activities in each pair of regions of the patient's brain. As a particular example, each region may be a partition of the patient's brain defined by a brain atlas.

[0055] In this specification, a region is a predefined area of ​​the brain. For example, a region can be defined by the boundaries of the brain's three-dimensional volume. A region can be defined such that neurons within the region are functionally similar according to one or more criteria. For example, a set of regions can be defined based on changes in cortical structure, function, connectivity, and / or local anatomy.

[0056] In this specification, a brain atlas is data that defines one or more regions of a patient's brain, for example, by defining the coordinates of the region's outline or the region's volume in a common three-dimensional coordinate system. As another example, brain data 202 may include tractofiber imaging data characterizing tracts of nerve fibers that connect paired regions (e.g., paired regions) in the patient's brain.

[0057] In some implementations, the autoencoder neural network system 200 can simulate the therapeutic effects on a patient's brain.

[0058] For example, the autoencoder neural network system 200 can acquire modified brain data 230, which characterizes the predicted local effect of future treatment on the patient's brain. That is, the modified brain data 230 is generated by modifying real brain data captured from the patient's brain to reflect the predicted local effect of future treatment. In this specification, the local effect of treatment is the local effect at the target location of the treatment. For example, the local effect of treatment could be the therapeutic effect on a region in the patient's brain to which the treatment is targeted. As another example, the local effect of treatment could be the therapeutic effect on a brain region within a threshold distance of the location to which the treatment is targeted (e.g., within one millimeter or one centimeter of the target location).

[0059] A system (e.g., an autoencoder neural network system 200 or an external system) can generate modified brain data 230 by identifying a target location in the patient's brain and modifying the patient's brain data corresponding to the target location and optionally one or more adjacent locations. The system can modify the brain data corresponding to the target location based on parameters of future treatment, according to how the brain data is expected to be modified in the future treatment. As a specific example, the system can modify the data corresponding to a specific region that is the treatment target (e.g., ...) by, for example, increasing or decreasing the identified correlation between brain activity in a specific region of the patient's brain and brain activity in one or more other regions. Figure 2A The correlation matrix represents the values ​​in a set of related data.

[0060] Future treatments can be any appropriate treatment for the patient's brain. For example, a future treatment could be a drug therapy that can be administered to the patient and targets a specific location in the patient's brain. As another example, a future treatment could be surgery targeting a specific location in the patient's brain. As yet another example, a future treatment could be transcranial magnetic stimulation (TMS) targeting a specific location in the patient's brain.

[0061] The autoencoder neural network system 200 can process modified brain data 230 to generate final brain data 240. Specifically, the autoencoder neural network system 200 can i) use an encoder subnetwork 210 to process the modified brain data 230 to generate an embedding of the modified brain data 230, and ii) use a decoder subnetwork 220 to process the embedding of the modified brain data 230 to generate the final brain data 240.

[0062] The final brain data 240 characterizes the predicted overall effect of future treatment on the patient's brain. In this specification, the overall effect of treatment is the effect of treatment on one or more locations in the patient's brain that are not local to the target location of treatment. For example, the overall effect of treatment could be the effect of treatment on the entire patient's brain. As another example, the overall effect of treatment could be the effect of treatment on one or more adjacent locations in the patient's brain to the target location, such as one or more adjacent regions of the target region.

[0063] In other words, the local effect of treatment can be any effect of how an intervention on a specific region of the patient's brain directly affects the function of that specific region. The overall effect of treatment can characterize how the treatment alters the function of one or more networks in a region of the patient's brain.

[0064] As a specific example, if the modified brain data 230 includes the modified related data as described above, then the final brain data 240 may include related data that reflects the correlation between how the treatment will affect brain activity in one or more specific partitions of the patient's brain (e.g., each partition in the patient's brain) and brain activity in one or more other partitions, for one or more specific partitions that are not the target partitions of the treatment.

[0065] Since the autoencoder neural network system 200 has been trained in this way, its output can represent the overall effect of future treatment. During training, the autoencoder neural network system 200 receives brain data 202 corresponding to untreated patients (i.e., patients who have not recently undergone brain treatment, and therefore whose brain data 202 does not reflect the local effects of treatment). The encoder subnetwork 210 learns to encode the most useful information from the brain data 202 into a brain data embedding 212. The decoder subnetwork 210 learns to generate reconstructed brain data 222 using the brain data embedding 212, which simulates the structure of the brain data 202 of these untreated patients. Therefore, when provided with modified brain data 230 that truly reflects the local effects of treatment (e.g., providing immediate effects after treatment), the encoder subnetwork 210 can generate an embedding that encodes information about the local effects at the target location of treatment. The decoder subnetwork 220 can then generate reconstructed brain data 240 that simulates the brain data, i) including the local effects at the target location of treatment and ii) corresponding to untreated patients. In other words, the final brain data 240 reflects how the remaining brain data outside the target location will be constructed for untreated patients, given the presence of a local effect at the target location; that is, how the local effect propagates to the remaining brain data.

[0066] The final brain data 240 can be used to determine what the overall effect of the treatment would be if the future treatment were provided to the patient's brain, such as whether the future treatment is safe and / or effective. For example, a user of the autoencoder neural network system 200 could analyze the final brain data 240 to determine whether the future treatment should be provided to the patient. As another example, the final brain data 240 could be generated by an anomaly detection engine (e.g., Figure 3 The anomaly detection engine 330 shown processes the data to determine whether the final brain data 240 is abnormal. That is, the anomaly detection engine can determine whether providing the treatment will cause one or more abnormalities in the patient's brain. See below. Figure 3 Describe the process in more detail.

[0067] In some cases, the local effects of future treatment on a patient's brain may reflect the short-term effects of the treatment, such as the effects seconds, minutes, hours, or days after the treatment is administered. The overall effects of future treatment on a patient's brain may reflect the long-term effects, such as the effects days, weeks, months, or years after the treatment is administered. Therefore, users can use an autoencoder neural network 200 to predict the long-term effects of future treatments in an effort to ensure the safety and effectiveness of the treatment.

[0068] In some implementations, the autoencoder neural network system 200 can process multiple different modified brain datasets 230, each corresponding to a different future treatment, to generate a separate final brain dataset 240. Each future treatment may have different parameters, such as different intensities, different doses, different schedules, or different target locations. An external system can then help determine, based on the reconstructed brain data from the individual final brain datasets 240, which future treatments would be or are likely to be the safest and / or most effective, and therefore, which future treatment a clinician should consider recommending to the patient.

[0069] As a specific example, the system can use an autoencoder neural network system 200 to process multiple different modified brain datasets 230 in an attempt to generate a specific “desired” brain dataset. That is, the system processes multiple different modified brain datasets 230, generating a separate final brain dataset 240 for each modified brain dataset 230, in order to identify a specific modified brain dataset 230 whose corresponding final brain dataset 240 matches or most closely matches the desired brain dataset. Therefore, this specific modified brain dataset 230 represents the local effect of the treatment of the desired brain dataset on an overall basis. For example, the system can use a “brute-force” method to identify this specific modified brain dataset 230; that is, the system can systematically process many different modified brain datasets 230 (e.g., using grid search) until the specific modified brain dataset 230 is identified.

[0070] As another example, the system can identify an initial modified brain dataset 230 and process it using an autoencoder neural network system 200. The system can determine the difference between i) the final brain data 240 generated using the initial modified brain dataset 230 and ii) the desired brain data. The system can then backpropagate this difference using the autoencoder neural network system 200 to generate an update to the initial modified brain data 230 that reduces the error. That is, instead of updating the parameters of the neural network system 200 during backpropagation, the system updates the input to the autoencoder neural network system 200, generating a new modified brain dataset 230. The system can then process the new modified brain dataset 230 using the autoencoder neural network system 200 and determine another update to the input that will further reduce the difference, i.e., make the corresponding final brain data 240 closer to the desired brain data. The system can iteratively repeat this process until a specific modified brain dataset is identified that generates the desired brain data. For example, the system can perform a predetermined number of iterations, or repeat the process, until the difference between the final brain data 240 and the expected brain data falls below a predetermined threshold.

[0071] In some implementations, the system can perform the above iterative process multiple times using different "seeds" to initially modify the brain dataset 230, and after each iteration, select the specific modified brain dataset that generates the final brain data 240 that is closest to the desired brain data.

[0072] The following is for reference. Figure 2B Other exemplary techniques for identifying modified brain datasets that will, or may, achieve specific desired outcomes in a patient's brain are discussed.

[0073] Figure 2BThis is a diagram of an exemplary inverse autoencoder neural network system 250. The inverse autoencoder neural network system 250 is an example of a system implemented as a computer program on one or more computers at one or more locations, in which the systems, components and techniques described below can be implemented.

[0074] The reverse autoencoder neural network system 250 is Figure 2A This is a reversed version of the autoencoder neural network 200 shown. In this specification, a reversed neural network is a neural network that reverses one or more operations of the main neural network, such that the reversed neural network maps the output of the main neural network to the input of the main neural network. Similar to... Figure 2A The autoencoder neural network system 200 and the inverse autoencoder neural network system 250 depicted are configured to process brain data to generate reconstructed brain data.

[0075] The autoencoder neural network system includes a reverse decoder subnetwork 260 and a reverse encoder subnetwork 270. The reverse decoder subnetwork 260 is configured to process brain data to generate embeddings of the brain data, and the reverse encoder subnetwork 270 is configured to process the embeddings of the brain data to generate reconstructed brain data. The reverse decoder subnetwork 260 can be... Figure 2A The decoder subnetwork 220 shown is a reverse version. Similarly, the reverse encoder subnetwork 270 can be... Figure 2A The reverse version of the encoder subnetwork 210 depicted in the diagram.

[0076] After the autoencoder neural network system 200 has been trained, it can be used to determine the inverse autoencoder neural network system 250.

[0077] The architecture of the inverse autoencoder neural network system 250 can be the inverse of the architecture of the autoencoder neural network system 200. If the autoencoder neural network system 200 has N neural network layers, then the inverse autoencoder neural network system 250 can also have N neural network layers, where the i-th neural network layer of the inverse autoencoder neural network system 250 corresponds to the (N-i+1)-th neural network layer of the autoencoder neural network system 200. The architecture of the i-th neural network layer of the inverse autoencoder neural network system 250 can be the inverse version of the architecture of the (N-i+1)-th neural network layer of the autoencoder neural network system 200. That is, if the (N-i+1)-th neural network layer of the autoencoder neural network system 200 receives the first dimension (e.g., L...), the architecture of the inverse autoencoder neural network system 250 can be the inverse version of the architecture of the (N-i+1)-th neural network layer of the autoencoder neural network system 200. i ×W i ×H i The layer input is used to generate the second dimension (e.g., L).o ×W o ×H o If the i-th layer of the inverse autoencoder neural network system 250 receives the layer input of the second dimension and generates the layer output of the first dimension, then the i-th layer of the inverse autoencoder neural network system 250 receives the layer input of the second dimension and generates the layer output of the first dimension.

[0078] The parameter values ​​of the inverse autoencoder neural network system 250 can be determined using the training parameter values ​​of the autoencoder neural network system 200 through any appropriate processing.

[0079] In some implementations, the inverse autoencoder neural network system 250 can be used to identify treatments that, if provided to a patient, will produce a specific desired effect on the patient's brain.

[0080] For example, the inverse autoencoder neural network system 250 can acquire expected brain data 252, which characterizes the expected overall effect of future treatment on the patient's brain. That is, expected brain data 252 represents the expected state of the patient's brain after the treatment has been provided to the patient.

[0081] In some implementations, the system (e.g., a reverse autoencoder neural network system 250 or an external system) can determine the expected brain data 252 for a patient using brain data corresponding to different second patients that reflects the expected overall effect of future treatment. For example, the expected brain data 252 may be the same as the brain data of the second patient. As another example, the expected brain data 252 can be generated by combining i) the patient's current brain data (i.e., before treatment is provided to the patient) and ii) the brain data of the second patient.

[0082] In some other implementations, the system may use only the patient’s current brain data, i.e., determine the patient’s desired brain data 252 by modifying some or all elements of the patient’s current brain data to reflect the desired outcome.

[0083] The inverse autoencoder neural network system 250 can process the desired brain data 252 to generate reconstructed desired brain data 272. Specifically, the inverse autoencoder neural network system 250 can i) process the desired brain data 252 using the inverse decoder subnetwork 260 to generate an embedding 262 of the desired brain data 252, and ii) process the embedding 262 of the desired brain data 252 using the inverse encoder subnetwork 270 to generate reconstructed desired brain data 272.

[0084] The reconstructed expected brain data 272 can characterize the predicted local effects of future treatments on the patient's brain. In other words, the reconstructed expected brain data 272 can reflect the predicted effects of treatment on the target location, such as the short-term effects of the treatment.

[0085] The reconstructed expected brain data 272 can be used to determine parameters for future treatments, such that the future treatments will achieve the expected overall effect reflected in the expected brain data 252. For example, the reconstructed expected brain data 272 can be used to determine a recommended location in the patient's brain for the future treatment. As another example, the reconstructed expected brain data 272 can be used to determine a recommended intensity or dosage of the future treatment. As yet another example, the reconstructed expected brain data 272 can be used to determine a recommended schedule for delivering the future treatment to the patient.

[0086] For example, the external system can identify one or more differences between i) the reconstructed desired brain data 272 and ii) the patient's current brain data. As a specific example, the external system can identify one or more elements in the reconstructed desired brain data 272 that have the greatest difference among corresponding elements in the patient's current brain data. The external system can then recommend a target location for future treatment in the patient's brain corresponding to the identified one or more differences. Alternatively or additionally, the external system can recommend the intensity or dosage of future treatment based on the magnitude of the identified one or more differences.

[0087] Therefore, the reconstructed expected brain data 172 can also be referred to as “roadmap” brain data, because the reconstructed expected brain data 172 can be used as a roadmap to determine future treatments that, if provided to the patient, will or are likely to achieve the desired overall effect.

[0088] As a specific example, expected brain data 252 may include expected related data 280 (e.g., such as...) Figure 2B (represented by the correlation matrix in the image), for each of one or more pairs of partitions in the patient's brain, the expected correlation data 280 includes the expected correlation between the respective brain activities in that pair of partitions. The inverse autoencoder neural network system 250 can process the expected correlation data 280 to generate roadmap correlation data 290 (e.g., as shown in the image). Figure 2B As represented by the correlation matrix in the diagram, for each of one or more pairs of regions in the patient's brain, the roadmap correlation data 290 includes the "roadmap" correlation between the respective brain activities within that pair of regions. The external system can then identify differences between the expected correlation data 280 and the patient's current brain data to determine parameters for future treatment. For example, the external system can identify specific regions where the correlation values ​​differ between the expected correlation data 280 and the patient's current brain data, and determine the parameters for TMS treatment that will stimulate that specific region.

[0089] Due to the training of the autoencoder neural network 200 (i.e., the network used to determine the inverse autoencoder neural network system 250), the output of the inverse autoencoder neural network system 250 can represent the local effects of future treatment. As described above, the autoencoder neural network 200 can be trained to learn relationships between different parts of the input brain data 202 (e.g., relationships between brain data 202 corresponding to various regions in a patient's brain). Therefore, inverting the autoencoder neural network system 200 can configure the inverse decoder subnetwork 260 to process the input brain data 252 to generate embeddings 262 that encode the relationships between the parts of the brain data 252. The inverse encoder subnetwork 270 can then process the embeddings 262 to generate reconstructed brain data 272 that identifies variations (i.e., local variations) in one or more specific parts of the brain data that can affect the remaining brain data, such that the brain data reflects the desired characteristics of the input brain data 252. Furthermore, typically, the differences between brain data from different patients are continuous and relatively standardized; for example, the values ​​of specific elements of the brain data corresponding to each patient can fall within a finite range. Therefore, the reconstructed brain data 272 generated by the inverse autoencoder neural network 250 can identify standard values ​​of most elements (i.e., elements that do not affect the desired characteristics of the input brain data 252), while identifying relative outliers of elements that will achieve specific features of the input brain data 252 (corresponding to local effects of future treatments).

[0090] In some implementations, the inverse autoencoder neural network system 250 can generate multiple different reconstructed desired brain datasets 272 from a single desired brain dataset 252. Because some information is lost when the autoencoder neural network system 200 generates the embeddings 212 of a specific brain dataset 202, multiple different brain datasets 202 can correspond to a single reconstructed brain dataset 222. Therefore, the inverse autoencoder neural network system 250 can determine multiple different reconstructed desired brain datasets 272, which, if processed by the autoencoder neural network system 200, will generate the same desired brain dataset 252.

[0091] After the inverse autoencoder neural network system 250 has generated multiple different reconstructed desired brain datasets 272, the external system can select a specific set from the multiple different reconstructed desired brain datasets 272 to determine parameters for future treatment. For example, the external system can determine, for each of the multiple different sets, the differences between i) the reconstructed desired brain dataset 272 and ii) the patient's current brain data. The external system can then select a specific set from the multiple different sets based on the determined individual differences. As a specific example, the external system can select a specific set with the smallest difference from the patient's current brain data, i.e., a specific reconstructed desired brain dataset 272 that reflects the smallest local effect that will cause the desired overall effect and thus can correspond to the lowest intensity treatment. As another specific example, the external system can select a specific set with the most localized difference from the patient's current brain data, i.e., a specific reconstructed desired brain dataset 272 from which treatment can be determined for a single location in the patient's brain rather than multiple different locations.

[0092] In some other implementations, the system can generate reconstructed desired brain data 272 by processing desired brain data 252 using a generative neural network that has been trained adversarially. For example, the generative neural network can use one or more feedforward neural network layers to process desired brain data 252 to generate reconstructed brain data 272.

[0093] The training system can use a generative adversarial network (GAN) comprising a generative neural network and one or more discriminator neural networks to train the generative neural network. Each discriminator neural network is configured to process the output of the generative neural network (in this example, the reconstructed desired brain data 272) and generate a prediction of whether the reconstructed desired brain data 272 is real or synthetic, i.e., whether the reconstructed desired brain data 272 is real brain data of the patient (or generated from it) (e.g., real fMRI data captured from the patient's brain) or whether the reconstructed desired brain data 272 was generated by the generative neural network. Typically, the training system determines updates to the parameters of the generative neural network to increase the error in the individual predictions of the discriminator neural network and determines updates to the parameters of the discriminator neural network to reduce its error. An example of a generative adversarial network is discussed in more detail in U.S. Patent Application entitled "Predicting Brain Data using Machine Learning Models" by Michael Sughrue, Stephane Doyen, and Peter Nicholas, filed on the same day as this application and incorporated herein by reference in its entirety.

[0094] Figure 3This is a diagram of an exemplary anomaly detection system 300. The anomaly detection system 300 is an example of a system implemented as a computer program on one or more computers at one or more locations, in which the systems, components and techniques described below can be implemented.

[0095] The anomaly detection system 300 is configured to detect anomalies in brain data 302 characterizing the patient's brain. As described above, brain data 302 may include any data characterizing the patient's brain. For example, brain data 302 may include correlation data characterizing the degree of correlation between the respective brain activities of each pair of regions in the patient's brain. As another example, brain data 302 may include tractography data characterizing the bundles of nerve fibers connecting the pairs of regions in the patient's brain.

[0096] The anomaly detection system includes an autoencoder neural network system 310 and an anomaly detection system 320. The autoencoder neural network system is configured to process brain data 302 and generate reconstructed brain data 312. For example, the autoencoder neural network system 310 may be as described above. Figure 2A The described autoencoder neural network system 200.

[0097] Anomaly detection engine 320 is configured to process reconstructed brain data 312 and optionally process patient brain data 302 to determine whether brain data 302 or 312 is abnormal, i.e., whether it is outside the normal range or whether it is different from the patient's typical brain data.

[0098] In some implementations, the patient's brain data 302 is real brain data captured from the patient's brain (or generated from real brain data captured from the patient's brain), and the anomaly detection engine 320 processes i) the patient's brain data 302 and ii) the reconstructed brain data 312 to determine whether the patient's brain data 302 is abnormal. Specifically, the anomaly detection engine 320 can determine whether the patient's brain data 302 is abnormal based on the differences between the patient's brain data 302 and the reconstructed brain data 312.

[0099] For example, the anomaly detection engine 320 can determine that the difference is the L1 or L2 distance, or its squared version, between brain data 302 and reconstructed brain data 312.

[0100] In some implementations, the anomaly detection engine 320 can determine that the patient's brain data 302 is abnormal if the difference between the determined patient's brain data 302 and the reconstructed brain data 312 exceeds a predetermined threshold. The predetermined threshold can be determined during or after training the autoencoder neural network system 310. For example, after the training system has trained the autoencoder neural network system 310, the training system can use the autoencoder neural network system 310 to process multiple test examples corresponding to various different patients, and for each training example, determine the difference between the input brain data and the reconstructed brain data. The training system can then determine a predetermined threshold such that certain percentages of the calculated difference in the test examples, such as 0.8, 0.9, 0.95, or 0.99, are below the threshold, while the calculated difference in the remaining test examples is above the threshold.

[0101] The autoencoder neural network system 310 has been trained to generate reconstructed brain data 312 similar to the patient's brain data 302 using training examples, which include "normal" brain data corresponding to different individual patients, i.e., non-abnormal brain data. Therefore, if the anomaly detection engine 320 determines that the reconstructed brain data 312 and the patient's brain data 302 are different based on the differences between them, the anomaly detection engine 320 can determine that the patient's brain data 302 is abnormal.

[0102] In some other implementations, as referenced above Figure 2A The patient's brain data 302 is modified brain data characterizing the predicted local effects of future treatment on the patient's brain. An anomaly detection engine can process the corresponding reconstructed brain data 312, which characterizes the predicted overall effects of future treatment on the patient's brain, to determine whether the reconstructed brain data 312 is anomalous. That is, the anomaly detection engine 320 can determine whether providing future treatment to the patient might lead to one or more anomalies in the patient's brain, which could indicate that the treatment may be dangerous for the patient. In other words, when a clinician is considering treatment for a patient, the clinician or another user can modify the patient's brain data to include predicted changes as an outcome of the proposed treatment, thereby generating patient-specific modified brain data. The system can then i) apply an autoencoder to the patient-specific modified brain data to generate reconstructed brain data, and ii) detect potential anomalies in the reconstructed brain data (e.g., using any of a variety of anomaly detection techniques).

[0103] In some such implementations, the anomaly detection engine 320 may use a machine learning model to process the reconstructed brain data 312, the machine learning model being configured to process the reconstructed brain data 312 to determine whether it is anomalous. For example, the machine learning model generates a prediction characterizing the probability of the reconstructed brain data 312 being anomalous, such as a scalar value between 0 and 1. As another example, the machine learning model identifies one or more anomalous elements in the reconstructed brain data 312. The machine learning model may be trained using brain data corresponding to multiple different patients (e.g., real brain data and / or reconstructed brain data).

[0104] In some other implementations, if the reconstructed brain data 312 exceeds a “normal” range of values ​​defined by brain datasets corresponding to other patients, the anomaly detection engine 320 can determine whether the reconstructed brain data 312 is abnormal. For example, the anomaly detection engine 320 can acquire “normal” brain data that identifies a range of values ​​for each of one or more elements in the reconstructed brain data 312 that are considered normal. Normal brain data can be determined from hundreds, thousands, or millions of brain data captured from other patients. As a specific example, normal brain data can identify the mean and standard deviation of values ​​determined from brain data from other patients for each element in the reconstructed brain data 312.

[0105] The anomaly detection engine 320 can use normal brain data to identify one or more anomalous elements in the reconstructed brain data 312. As a specific example, if the value of a particular element is outside the range defined by the mean and standard deviation of the values, the anomaly detection engine 320 can determine that the value is anomalous. For example, if the value of an element exceeds one, two, three, or four standard deviations of the mean, the value of that element can be determined to be anomalous.

[0106] The anomaly detection engine 320 can then determine whether one or more elements identified as anomalous in the reconstructed brain data 312 indicate that future treatment may be unsafe. For example, the anomaly detection system can acquire data to identify one or more specific elements in the reconstructed brain data 312 that, if identified as anomalous, indicate a problem in the patient's brain and thus suggest that future treatment may be unsafe.

[0107] In some implementations, the anomaly detection system 300 may provide data representing one or more elements identified as anomalous in the reconstructed brain data 312 to a downstream system for further processing. For example, the anomaly detection system 300 may provide data to a graphical user interface to display data representing one or more anomalous elements to the user. As another example, the anomaly detection system 300 may provide data to a machine learning system configured to process model inputs including the data and generate clinically relevant model outputs for the user. For example, the machine learning model may process the data to generate a prediction of whether a patient has a specific brain disorder (e.g., autism, depression, or schizophrenia).

[0108] The anomaly detection system is described in more detail in U.S. Patent Application No. 16 / 920078, filed July 2, 2020, the entire contents of which are incorporated herein by reference.

[0109] Figure 4 This is a flowchart of an exemplary process 400 for processing brain data using an autoencoder neural network. Process 400 can be implemented by one or more computer programs installed on one or more computers and programmed according to this specification. For example, process 400 can be implemented by… Figure 1A The computer server module shown is executed. For convenience, the 400 process is described as being executed by a system of one or more computers.

[0110] The system acquires brain data characterizing the patient’s brain activity captured from one or more sensors (step 402).

[0111] The system processes the brain data to generate modified brain data, which characterizes the predicted local effects of future treatment on the patient's brain (step 404). For example, the system may determine the target location of future treatment in the patient's brain, identify one or more elements in the brain data corresponding to the target location, and modify the values ​​of the one or more elements according to one or more parameters of the future treatment.

[0112] The system uses an autoencoder neural network to process the modified brain data to generate reconstructed brain data (step 406). The system can use the encoder subnetwork of the autoencoder neural network to process the network input, which includes the modified brain data, to generate an embedding of the network input, and then use the decoder subnetwork of the autoencoder neural network to process the embedding of the network input to generate reconstructed brain data.

[0113] The system uses the reconstructed brain data to determine the predicted overall future effect of future treatments on the patient's brain (step 408). For example, the system can process the reconstructed brain data to identify one or more abnormalities in the reconstructed brain data. Identified abnormalities may indicate that future treatments are unsafe. As a specific example, the system can acquire normal brain data, which, for each of one or more elements in the reconstructed brain data, identifies the corresponding normal range for that element. The system can then compare i) the reconstructed brain data and ii) the normal brain data to identify the one or more abnormalities.

[0114] In some implementations, the system performs process 400 multiple times for each different future treatment. The system can then select a specific future treatment from the multiple different future treatments using the predicted overall effect of each of the multiple future treatments. For example, the system can select a specific future treatment with an overall effect that reflects the safest and most effective outcome among the multiple different future treatments.

[0115] Figure 5 This is a flowchart of an exemplary process 500 for processing brain data using a reverse autoencoder neural network. Process 500 can be implemented by one or more computer programs installed on one or more computers and programmed according to this specification. For example, process 500 can be implemented by... Figure 1A The computer server module shown is executed. For convenience, the 500 process is described as being executed by a system of one or more computers.

[0116] The system acquires expected brain data characterizing the patient's desired brain activity (step 502). The expected brain data can characterize the overall effect that future treatment may have on the patient's brain. For example, the expected brain data may be identical to (or generated from) brain data characterizing the desired brain activity of a second patient.

[0117] The system provides desired brain data as input to the inverse autoencoder neural network (step 504). The inverse autoencoder neural network can be determined using a trained autoencoder neural network, including inverting one or more operations of the autoencoder neural network.

[0118] An autoencoder can be configured to process patient brain data and generate reconstructed brain data by i) processing network inputs including brain data using an encoder subnetwork to generate an embedding of the network inputs, and ii) processing the embedding of the network inputs using a decoder subnetwork to generate reconstructed brain data.

[0119] The system obtains roadmap brain data from a reverse neural network, which characterizes the predicted local effects of future treatments on the patient's brain (step 506).

[0120] Optionally, the system can determine the values ​​of one or more parameters for future treatment based on the roadmap data (step 508). The values ​​of one or more parameters can be determined to enable the future treatment to achieve the patient's desired brain activity.

[0121] In some implementations, the system can obtain multiple distinct roadmap datasets from a reverse neural network. Each roadmap dataset represents a predicted local effect of the future treatment if the future treatment is provided to the patient based on distinct values ​​of one or more parameters of the future treatment. The system can then use the various roadmap datasets to select specific values ​​for one or more parameters of the future treatment.

[0122] Figure 6 This is a flowchart of an exemplary process 600 for identifying anomalous brain data using an autoencoder neural network. Process 600 can be implemented by one or more computer programs installed on one or more computers and programmed according to this specification. For example, process 600 can be implemented by… Figure 1A The computer server module shown is executed. For convenience, process 600 is described as being executed by a system of one or more computers.

[0123] The system acquires brain data characterizing the patient’s brain activity captured by one or more sensors (step 602).

[0124] The system uses an autoencoder neural network to process brain data to generate reconstructed brain data (step 604). The system can use the encoder subnetwork of the autoencoder neural network to process network inputs including the brain data to generate an embedding of the network inputs, and then use the decoder subnetwork of the autoencoder neural network to process the embedding of the network inputs to generate reconstructed brain data.

[0125] The system uses the reconstructed brain data to determine whether the patient's brain data is abnormal (step 606). For example, the system can determine the error between the brain data and the reconstructed brain data, and determine whether the error meets a predetermined threshold. The predetermined threshold can be determined based on training errors, where each training error represents i) the error between the second brain data of the second patient and ii) the error between the second reconstructed brain data generated by processing the second brain data using an autoencoder neural network.

[0126] Embodiments of the subject matter and functional operation described in this specification may be implemented in digital electronic circuits, in tangibly embodied computer software or firmware, in computer hardware (including the structures disclosed herein and their structural equivalents), or in a combination of one or more of these. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium, for execution by a data processing device or for controlling the operation of a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of these. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals (e.g., machine-generated electrical, optical, or electromagnetic signals) generated for encoding information to be transmitted to a suitable receiver device for execution by the data processing device.

[0127] The term "data processing device" refers to data processing hardware and includes various devices, apparatuses, and machines for processing data, such as programmable processors, computers, or multiple processors or computers. The device may also be or further include special-purpose logic circuitry, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). In addition to hardware, the device may optionally include code for creating an execution environment for computer programs, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, or combinations thereof.

[0128] A computer program, also referred to or described as a program, software, software application, application, module, software module, script, or code, can be written in any programming language (including compiled or interpreted languages, or declarative or procedural languages) and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not necessarily, correspond to a file in a file system. A program may be stored as a portion of a file containing other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). A computer program can be deployed to execute on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected via a data communication network.

[0129] A system of one or more computers configured to perform a specific operation or action means that the system has software, firmware, hardware, or a combination thereof installed thereon, which, in operation, causes the system to perform the operation or action. A system of one or more computer programs configured to perform a specific operation or action means that the one or more programs include instructions that, when executed by a data processing device, cause the device to perform the operation or action.

[0130] As used in this specification, "engine" or "software engine" refers to a software-implemented input / output system that provides outputs other than inputs. An engine can be a functional block of code, such as a library, platform, software development kit ("SDK"), or object. Each engine can be implemented on any suitable type of computing device, including one or more processors and computer-readable media, such as a server, mobile phone, tablet, laptop, music player, e-book reader, laptop or desktop computer, PDA, smartphone, or other fixed or portable device. Furthermore, two or more engines can be implemented on the same computing device or on different computing devices.

[0131] The processing and logic flows described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform functions by manipulating input data and generating outputs. The processing and logic flows can also be executed by dedicated logic circuitry (e.g., FPGA or ASIC) or by a combination of dedicated logic circuitry and one or more programmable computers.

[0132] A computer suitable for executing computer programs can be based on a general-purpose or special-purpose microprocessor, or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are the central processing unit for making or executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by or incorporated into special-purpose logic circuitry. Typically, a computer also includes one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or is operatively coupled to said one or more mass storage devices to receive data from or transfer data to or both. However, a computer need not have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0133] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, such as semiconductor memory devices, such as EPROM, EEPROM and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0134] To provide interaction with a user, embodiments of the subject matter described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, as well as a keyboard and pointing device (e.g., a mouse, trackball) or a presence-sensitive display or other surface that the user can use to provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. Furthermore, the computer can interact with the user by sending and receiving documents to and from the device used by the user; for example, by sending a web page to a web browser on the user's device in response to a request received from a web browser. Additionally, the computer can interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone), running a messaging application, and receiving response messages from the user.

[0135] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes backend components, such as a data server; middleware components, such as an application server; or frontend components, such as a client computer having a graphical user interface, a web browser, or an application through which a user can interact with embodiments of the subject matter described in this specification; or any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0136] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact via a communication network. The client-server relationship is generated by computer programs running on their respective computers and having a client-server relationship with each other. In some embodiments, the server sends data (e.g., HTML pages) to a user device, for example, to display data to a user interacting with the device as a client and to receive user input from that user. Data generated at the user device (e.g., the result of user interaction) can be received from the device at the server.

[0137] In addition to the above embodiments, the following embodiments are also innovative:

[0138] Example 1 is a method comprising:

[0139] Acquire brain data characterizing a patient's brain activity, captured by one or more sensors;

[0140] The brain data is processed to generate modified brain data, which characterizes the predicted local effect of future treatment on the patient's brain, wherein the local effect of the future treatment is the effect of the future treatment on a region of the brain, the region being local to the target location of the future treatment in the brain;

[0141] The modified brain data is processed using an autoencoder neural network to generate reconstructed brain data, wherein:

[0142] The process includes:

[0143] The network input, including the modified brain data, is processed using an encoder subnetwork to generate an embedding of the network input, and

[0144] The embeddings of the network input are processed using a decoder subnetwork to generate the reconstructed brain data; and

[0145] At each of the multiple training time steps, the autoencoder neural network was trained using the following method:

[0146] Based on the current values ​​of multiple network parameters of the autoencoder neural network, the training network input, including second brain data from a second patient, is processed to generate reconstructed second brain data.

[0147] Based on the error between i) the second brain data and ii) the reconstructed second brain data, an update to the current values ​​of the plurality of network parameters is determined; and

[0148] The reconstructed brain data is used to determine the predicted overall effect of the future treatment on the patient's brain, wherein the overall effect of the future treatment is the effect of the future treatment on one or more regions of the brain that are not localized to the target of the future treatment in the brain.

[0149] Example 2 is the method of Example 1, wherein processing the brain data to generate modified brain data characterizing the predicted local effects of treatment on the patient's brain includes:

[0150] Determine the target location in the patient's brain for the future treatment;

[0151] Identify one or more elements of the brain data corresponding to the target location; and

[0152] The values ​​of the one or more elements are modified based on one or more parameters of the future treatment.

[0153] Example 3 is a method of any of Examples 1 or 2, wherein determining the predicted overall effect of the future treatment includes processing the reconstructed brain data to identify one or more abnormalities in the reconstructed brain data.

[0154] Example 4 is the method of Example 3, wherein identifying one or more anomalies in the reconstructed brain data includes:

[0155] Acquire normal brain data, and for each of one or more elements in the reconstructed brain data, the normal brain data identifies the corresponding normal value range for that element; and

[0156] Compare the reconstructed brain data (i) and the normal brain data (ii) to identify the one or more abnormalities.

[0157] Example 5 is a method of any of Examples 1-4, further comprising:

[0158] For each of one or more second future treatments:

[0159] The brain data is processed to generate second modified brain data, which characterizes the predicted local effect of the second future treatment on the patient's brain.

[0160] The second modified brain data is processed using the autoencoder neural network to generate second reconstructed brain data; and

[0161] The second reconstructed brain data is used to determine the predicted overall effect of the second future treatment on the patient's brain; and

[0162] A specific future treatment is selected using the predicted overall effect of the said future treatment and the one or more second future treatments.

[0163] Example 6 is a method of any of Examples 1-5, wherein the future treatment is transcranial magnetic stimulation (TMS) therapy.

[0164] Example 7 is a method of any of Examples 1-6, wherein:

[0165] The brain data includes correlation data for each of a plurality of pairs of partitions formed by a set of partitions in the patient's brain, wherein each pair includes a first partition and a second partition, and the correlation data characterizes the degree of correlation between brain activity in the first partition and brain activity in the second partition in the patient's brain.

[0166] Example 8 is a method comprising:

[0167] Acquire expected brain data characterizing the patient’s expected brain activity, wherein the expected brain data characterizes the possible overall effect of future treatment on the patient’s brain, wherein the overall effect of treatment is the effect of treatment on one or more regions of the brain, which are not local to the target of treatment in the brain;

[0168] The desired brain data is provided as input to the inverse autoencoder neural network, wherein:

[0169] The inverse autoencoder neural network has been determined using a trained autoencoder neural network, the determination including reversing multiple operations of the autoencoder neural network;

[0170] The autoencoder neural network is configured to process the patient's brain data and generate reconstructed brain data of the patient, the processing including:

[0171] The network input, including the brain data, is processed using an encoder subnetwork to generate an embedding of the network input, and

[0172] The embeddings of the network inputs are processed using a decoder subnetwork to generate the reconstructed brain data; and

[0173] At each of the multiple training time steps, the autoencoder neural network was trained using the following method:

[0174] Based on the current values ​​of multiple network parameters of the autoencoder neural network, the training network input, including second brain data from a second patient, is processed to generate reconstructed second brain data.

[0175] Based on the error between i) the second brain data and ii) the reconstructed second brain data, an update to the current values ​​of the plurality of network parameters is determined; and

[0176] Roadmap brain data is obtained from the inverse autoencoder neural network, the roadmap brain data representing the predicted local effect of the future treatment on the patient's brain, wherein the local effect of the future treatment is the effect of the future treatment on a region of the brain, the region being the local area of ​​the target of the treatment.

[0177] Example 9 is the method of Example 8, and further includes:

[0178] Based on the roadmap brain data, values ​​for one or more parameters of the future treatment are determined to achieve the patient's desired brain activity.

[0179] Example 10 is the method of Example 9, wherein the parameters of the future treatment include one or more of the following:

[0180] The recommended target location for future treatment,

[0181] The recommended strength of the future treatment,

[0182] The recommended dose for future treatment; or

[0183] The recommended timeline for future treatments.

[0184] Example 11 is a method of any of Examples 8-10, wherein obtaining expected brain data characterizing the expected brain activity of a patient includes obtaining brain data characterizing the expected brain activity of a second patient.

[0185] Example 12 is a method of any of Examples 8-11, wherein:

[0186] If the future treatment is provided based on a first value of one or more parameters of the future treatment, then the roadmap data characterizes the predicted local effect of the future treatment on the patient's brain; and

[0187] The method also includes:

[0188] One or more second roadmap brain datasets are obtained from the inverse autoencoder neural network, and each second roadmap brain dataset characterizes the predicted local effect of the future treatment on the patient's brain if the future treatment is provided based on respective second values ​​of one or more parameters of the future treatment; and

[0189] Specific values ​​for one or more parameters of the future treatment are selected using the roadmap brain data and the one or more second roadmap datasets.

[0190] Example 13 is a method of any of Examples 8-12, wherein the future treatment is transcranial magnetic stimulation (TMS) therapy.

[0191] Example 14 is a method of any of Examples 8-13, wherein:

[0192] The brain data includes correlation data for each of a plurality of pairs of partitions formed by a set of partitions in the patient's brain, wherein each pair includes a first partition and a second partition, and the correlation data characterizes the degree of correlation between brain activity in the first partition and brain activity in the second partition in the patient's brain.

[0193] Example 15 is a method comprising:

[0194] Acquire brain data generated from data characterizing a patient's brain activity captured by one or more sensors;

[0195] The brain data is processed using an autoencoder neural network to generate reconstructed brain data, wherein:

[0196] The process includes:

[0197] The network input, including the brain data, is processed using an encoder subnetwork to generate an embedding of the network input, and

[0198] The embeddings of the network input are processed using a decoder subnetwork to generate the reconstructed brain data; and

[0199] At each of the multiple training time steps, the autoencoder neural network was trained using the following method:

[0200] Based on the current values ​​of multiple network parameters of the autoencoder neural network, the training network input, including second brain data from a second patient, is processed to generate reconstructed second brain data.

[0201] Based on the error between i) the second brain data and ii) the reconstructed second brain data, an update to the current values ​​of the plurality of network parameters is determined; and

[0202] The reconstructed brain data is used to determine whether the patient's brain data is abnormal.

[0203] Example 16 is the method of Example 15, wherein determining whether the patient's brain data is abnormal includes:

[0204] Determine the error between the brain data and the reconstructed brain data; and

[0205] Determine whether the error meets a predetermined threshold.

[0206] Example 17 is the method of Example 16, wherein the threshold is determined based on a plurality of training errors, wherein each training error represents i) the error between second brain data of a second patient and ii) the error between second reconstructed brain data generated by processing the second brain data using the autoencoder neural network.

[0207] Example 18 is a method of any of Examples 15-17, wherein:

[0208] The brain data includes correlation data for each of a plurality of pairs of partitions formed by a set of partitions in the patient's brain, wherein each pair includes a first partition and a second partition, and the correlation data characterizes the degree of correlation between brain activity in the first partition and brain activity in the second partition in the patient's brain.

[0209] Example 19 is a method of any of Examples 15-18, wherein the brain data is modified brain data characterizing the predicted local effect of future treatment on the patient's brain, wherein the local effect of future treatment is the effect of future treatment on a region of the brain, the region being local to the target location of the future treatment in the brain.

[0210] Example 20 is a method of Example 19, wherein determining whether the patient's brain data is abnormal includes:

[0211] Acquire normal brain data, wherein for each of one or more specific elements of the patient's brain data, the normal data identifies a corresponding normal range for that specific element; and

[0212] Compare the reconstructed brain data described in i) with the normal brain data described in ii) to identify one or more possible abnormalities.

[0213] Example 21 is the method of Example 20, wherein:

[0214] The normal brain data was generated from multiple brain datasets corresponding to various other patients;

[0215] For each of one or more specific elements of the patient's brain data, the normal brain data includes data characterizing i) a measure of the central tendency of the values ​​of the specific element and ii) a measure of the variance of the values ​​of the specific element, wherein the measure of central tendency and the measure of variance have been calculated using the plurality of brain datasets; and

[0216] For each of one or more specific elements in the patient's brain data, the normal range of the specific element is defined by a maximum and a minimum value, wherein the maximum and the minimum value are linear combinations of a corresponding measure of the central tendency of the specific element and a corresponding measure of the variance of the specific element.

[0217] Example 22 is a system including one or more computers and one or more storage devices, wherein instructions stored in the storage devices are operable when executed by the one or more computers to cause the one or more computers to perform the methods of any of Examples 1-21.

[0218] Example 23 is one or more non-transitory computer storage media encoded with a computer program, the program including instructions operable when executed by a data processing device to cause the data processing device to perform the method of any of Examples 1-21.

[0219] While this specification contains numerous specific implementation details, these details should not be construed as limiting the scope of any invention or the scope of the claimed invention, but rather as descriptions of features that may be characteristic of particular embodiments of a particular invention. Certain features described in the context of a single embodiment may also be implemented in combination within that single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described above as functioning in a particular combination and even initially claimed, in certain circumstances one or more features in a claimed combination may be removed from that combination, and the claimed combination may involve sub-combinations or variations thereof.

[0220] Similarly, although operations are described in a specific order in the accompanying drawings, this should not be construed as requiring such operations to be performed in the specific or sequential order shown, or requiring all illustrated operations to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0221] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions listed in the claims may be performed in a different order and still achieve the desired result. As an example, the processes described in the figures do not necessarily require the specific order or sequential order shown to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. A method comprising: Acquire brain data characterizing a patient's brain activity, captured by one or more sensors; The brain data is processed to generate modified brain data, which characterizes the predicted local effect of future treatment on the patient's brain, wherein the local effect of the future treatment is the effect of the future treatment on a region of the brain, the region being local to the target location of the future treatment in the brain; The modified brain data is processed using an autoencoder neural network to generate reconstructed brain data, wherein: The process includes: The network input, including the modified brain data, is processed using an encoder subnetwork to generate an embedding of the network input, and The embeddings of the network input are processed using a decoder subnetwork to generate the reconstructed brain data; and At each of the multiple training time steps, the autoencoder neural network was trained using the following method: Based on the current values ​​of multiple network parameters of the autoencoder neural network, the training network input, including second brain data from a second patient, is processed to generate reconstructed second brain data. Based on the error between i) the second brain data and ii) the reconstructed second brain data, an update to the current values ​​of the plurality of network parameters is determined; and The reconstructed brain data is used to determine the predicted overall effect of the future treatment on the patient's brain, wherein the overall effect of the future treatment is the effect of the future treatment on one or more regions of the brain that are not localized to the target of the future treatment in the brain.

2. The method of claim 1, wherein processing the brain data to generate modified brain data characterizing the predicted local effects of treatment on the patient's brain comprises: Determine the target location in the patient's brain for the future treatment; Identify one or more elements of the brain data corresponding to the target location; and The values ​​of the one or more elements are modified based on one or more parameters of the future treatment.

3. The method of claim 1, wherein determining the predicted overall effect of the future treatment includes processing the reconstructed brain data to identify one or more abnormalities in the reconstructed brain data.

4. The method of claim 3, wherein identifying one or more anomalies in the reconstructed brain data comprises: Acquire normal brain data, and for each of one or more elements in the reconstructed brain data, the normal brain data identifies the corresponding normal value range for that element; and Compare the reconstructed brain data (i) and the normal brain data (ii) to identify the one or more abnormalities.

5. The method according to claim 1, further comprising: For each of one or more second future treatments: The brain data is processed to generate second modified brain data, which characterizes the predicted local effect of the second future treatment on the patient's brain. The second modified brain data is processed using the autoencoder neural network to generate the second reconstructed brain data; and The second reconstructed brain data is used to determine the predicted overall effect of the second future treatment on the patient's brain; and A specific future treatment is selected using the predicted overall effect of the said future treatment and the one or more second future treatments.

6. The method of claim 1, wherein the future treatment is transcranial magnetic stimulation (TMS) therapy.

7. The method according to claim 1, wherein: The brain data includes correlation data for each of a plurality of pairs of partitions formed by a set of partitions in the patient's brain, wherein each pair includes a first partition and a second partition, and the correlation data characterizes the degree of correlation between brain activity in the first partition and brain activity in the second partition in the patient's brain.

8. A system comprising one or more computers and one or more storage devices, wherein instructions stored in the one or more storage devices are operable when executed by the one or more computers to cause the one or more computers to perform operations including: Acquire brain data characterizing a patient's brain activity, captured by one or more sensors; The brain data is processed to generate modified brain data, which characterizes the predicted local effect of future treatment on the patient's brain, wherein the local effect of the future treatment is the effect of the future treatment on a region of the brain, the region being local to the target location of the future treatment in the brain. The modified brain data is processed using an autoencoder neural network to generate reconstructed brain data, wherein: The process includes: The network input, including the modified brain data, is processed using an encoder subnetwork to generate an embedding of the network input, and The embeddings of the network input are processed using a decoder subnetwork to generate the reconstructed brain data; and At each of the multiple training time steps, the autoencoder neural network was trained using the following method: Based on the current values ​​of multiple network parameters of the autoencoder neural network, the training network input, including second brain data from a second patient, is processed to generate reconstructed second brain data. Based on the error between i) the second brain data and ii) the reconstructed second brain data, an update to the current values ​​of the plurality of network parameters is determined; and The reconstructed brain data is used to determine the predicted overall effect of the future treatment on the patient's brain, wherein the overall effect of the future treatment is the effect of the future treatment on one or more regions of the brain that are not localized to the target of the future treatment in the brain.

9. The system of claim 8, wherein processing the brain data to generate modified brain data characterizing the predicted local effects of treatment on the patient's brain comprises: Determine the target location in the patient's brain for the future treatment; Identify one or more elements of the brain data corresponding to the target location; and The values ​​of the one or more elements are modified according to one or more parameters of the future treatment.

10. The system of claim 8, wherein determining the predicted overall effect of the future treatment includes processing the reconstructed brain data to identify one or more abnormalities in the reconstructed brain data.

11. The system of claim 10, wherein identifying one or more anomalies in the reconstructed brain data comprises: Acquire normal brain data, and for each of one or more elements in the reconstructed brain data, the normal brain data identifies the corresponding normal value range for that element; and Compare the reconstructed brain data (i) and the normal brain data (ii) to identify the one or more abnormalities.

12. The system according to claim 8, wherein the operation further comprises: For each of one or more second future treatments: The brain data is processed to generate second modified brain data, which characterizes the predicted local effect of the second future treatment on the patient's brain. The second modified brain data is processed using the autoencoder neural network to generate the second reconstructed brain data; and The second reconstructed brain data is used to determine the predicted overall effect of the second future treatment on the patient's brain; and A specific future treatment is selected using the predicted overall effect of the said future treatment and the one or more second future treatments.

13. The system of claim 8, wherein the future treatment is transcranial magnetic stimulation (TMS) therapy.

14. The system according to claim 8, wherein: The brain data includes correlation data for each of a plurality of pairs of partitions formed by a set of partitions in the patient's brain, wherein each pair includes a first partition and a second partition, the correlation data characterizing the degree of correlation between brain activity in the first partition and brain activity in the second partition in the patient's brain.

15. One or more non-transitory storage media, wherein the instructions stored therein, when executed by one or more computers, cause the one or more computers to perform operations including: Acquire brain data characterizing a patient's brain activity, captured by one or more sensors; The brain data is processed to generate modified brain data, which characterizes the predicted local effect of future treatment on the patient's brain, wherein the local effect of the future treatment is the effect of the future treatment on a region of the brain, the region being local to the target location of the future treatment in the brain. The modified brain data is processed using an autoencoder neural network to generate reconstructed brain data, wherein: The process includes: The network input, including the modified brain data, is processed using an encoder subnetwork to generate an embedding of the network input, and The embeddings of the network inputs are processed using a decoder subnetwork to generate the reconstructed brain data; and At each of the multiple training time steps, the autoencoder neural network was trained using the following method: Based on the current values ​​of multiple network parameters of the autoencoder neural network, the training network input, including second brain data from a second patient, is processed to generate reconstructed second brain data. Based on the error between i) the second brain data and ii) the reconstructed second brain data, an update to the current values ​​of the plurality of network parameters is determined; and The reconstructed brain data is used to determine the predicted overall effect of the future treatment on the patient's brain, wherein the overall effect of the future treatment is the effect of the future treatment on one or more regions of the brain that are not localized to the target of the future treatment in the brain.

16. The non-transitory storage medium of claim 15, wherein processing the brain data to generate modified brain data characterizing the predicted local effects of treatment on the patient's brain comprises: Determine the target location in the patient's brain for the future treatment; Identify one or more elements of the brain data corresponding to the target location; and The values ​​of the one or more elements are modified according to one or more parameters of the future treatment.

17. The non-transitory storage medium of claim 15, wherein determining the predicted overall effect of the future treatment includes processing the reconstructed brain data to identify one or more abnormalities in the reconstructed brain data.

18. The non-transitory storage medium of claim 17, wherein identifying one or more anomalies in the reconstructed brain data comprises: Acquire normal brain data, and for each of one or more elements in the reconstructed brain data, the normal brain data identifies the corresponding normal value range for that element; and Compare the reconstructed brain data (i) and the normal brain data (ii) to identify the one or more abnormalities.

19. The non-transitory storage medium of claim 15, wherein the operation further comprises: For each of one or more second future treatments: The brain data is processed to generate second modified brain data, which characterizes the predicted local effect of the second future treatment on the patient's brain. The second modified brain data is processed using the autoencoder neural network to generate the second reconstructed brain data; and The second reconstructed brain data is used to determine the predicted overall effect of the second future treatment on the patient's brain; and A specific future treatment is selected using the predicted overall effect of the said future treatment and the one or more second future treatments.

20. The non-transitory storage medium according to claim 15, wherein: The brain data includes correlation data for each of a plurality of pairs of partitions formed by a set of partitions in the patient's brain, wherein each pair includes a first partition and a second partition, and the correlation data characterizes the degree of correlation between brain activity in the first partition and brain activity in the second partition in the patient's brain.

Images