Device for assessing schizophrenic tendencies based on near-infrared data

Abstract

This invention provides an apparatus, method, and storage medium for assessing schizophrenic tendency based on near-infrared data. The apparatus includes a processor configured to: acquire near-infrared data of a subject; process the near-infrared data and convert the processed near-infrared data into a test image; input the test image into a classification model to obtain a classification result for the subject, wherein the classification result indicates the subject's tendency to be either healthy or schizophrenic. This invention normalizes near-infrared data based on a preset data set formed by fusing second data sequences from multiple healthy subjects and multiple schizophrenic individuals. This avoids the loss of brain activation information caused by normalization at different scales, resulting in higher accuracy when the classification model classifies the test image converted from the normalized near-infrared data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and more specifically to an apparatus for assessing schizophrenic tendencies based on near-infrared data, a method for assessing schizophrenic tendencies based on near-infrared data, and a storage medium. Background Technology

[0002] Schizophrenia is a severe, chronic, and highly disabling mental disorder. Patients with schizophrenia typically exhibit symptoms such as delusions, hallucinations, inadequate thinking, loss of motivation, and impaired cognitive function (auditory hallucinations, inattention, speech impairment, and memory impairment). These symptoms significantly impact the patient's life and pose potential safety hazards to both the patient and those around them.

[0003] Currently, schizophrenia is typically diagnosed using the Diagnostic and Statistical Manual of Mental Disorders (DSM) in conjunction with the patient's symptom description. However, the diagnostic results obtained in this way are influenced not only by the psychiatrist's clinical experience but also by the cooperation between the patient and the doctor. For example, patients may be unwilling to disclose their symptoms or resist interviews. Furthermore, patients may experience prodromal symptoms, such as depression, that differ from the symptoms of schizophrenia described above, several years before the onset of the illness. These factors may make it difficult for doctors to obtain accurate diagnostic results. Therefore, improving the accuracy of schizophrenia diagnosis is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] This application is made in consideration of the above-mentioned problems. This application provides an apparatus for assessing schizophrenic tendencies based on near-infrared data, a method for assessing schizophrenic tendencies based on near-infrared data, and a storage medium. It can take into account the brain activation of the test subject from a global perspective and the differences between the test subject and healthy subjects and schizophrenic patients. Compared with methods that rely on the subjective judgment of doctors and methods that directly classify near-infrared data, the accuracy of the assessment results is higher.

[0005] According to a first aspect of this application, an apparatus for assessing schizophrenic tendency based on near-infrared data is provided. The apparatus includes a processor. The processor is configured to: acquire near-infrared data of a subject; process the near-infrared data and convert the processed near-infrared data into a test image, wherein the processing includes normalization processing based on maximum and minimum values ​​obtained from a preset data set, the preset data set including a second data sequence of healthy subjects and schizophrenic individuals distributed over time, wherein the second data sequence is obtained based on the near-infrared data of healthy subjects and schizophrenic individuals; and input the test image into a classification model to obtain a classification result for the subject, wherein the classification result indicates the subject's tendency to be either healthy or schizophrenic.

[0006] According to a second aspect of this application, a method for assessing schizophrenic tendencies based on near-infrared data is provided. The method includes: acquiring near-infrared data of a subject; processing the near-infrared data and converting the processed near-infrared data into an image to be tested, wherein the processing includes normalization processing based on maximum and minimum values ​​obtained from a preset data set, the preset data set including a second data sequence distributed over time for healthy subjects and schizophrenic individuals, wherein the second data sequence is obtained based on the near-infrared data of healthy subjects and schizophrenic individuals; and inputting the image to be tested into a classification model to obtain a classification result for the subject, wherein the classification result indicates the subject's tendency to be either healthy or schizophrenic.

[0007] According to a third aspect of this application, a storage medium is provided, storing a computer program / instructions for assessing schizophrenic tendency based on near-infrared data. When executed, the computer program / instructions acquire near-infrared data of a subject; process the near-infrared data and convert the processed near-infrared data into a test image; wherein the processing includes normalization processing based on maximum and minimum values ​​obtained from a preset data set, the preset data set including a second data sequence distributed over time for healthy subjects and schizophrenic individuals, wherein the second data sequence is obtained based on the near-infrared data of the healthy subjects and schizophrenic individuals; and inputting the test image into a classification model to obtain a classification result for the subject, wherein the classification result indicates the subject's tendency to be either a healthy subject or a schizophrenic individual.

[0008] According to a fourth aspect of this application, an apparatus for identifying the mental state of a subject based on near-infrared data is provided, comprising a processor configured to: acquire near-infrared data during a subject's performance of a language fluency task; represent the near-infrared data using an image to be examined, wherein pixels in the image to be examined can characterize the relationship between data at different times in the near-infrared data; and obtain the identification result of the subject's mental state based on the image to be examined using a network model capable of capturing the correlation features between target acquisition channels.

[0009] According to a fifth aspect of this application, a system for identifying the mental state of a subject based on near-infrared data is provided, comprising a data acquisition module and the apparatus for identifying the mental state of a subject based on near-infrared data as described in the embodiments of this application, wherein the data acquisition module is configured to acquire light intensity change data during the subject's performance of a language fluency task to generate near-infrared data of the subject.

[0010] According to a sixth aspect of this application, a computer program / instruction is provided that stores a near-infrared data-based method for identifying the mental state of a subject. When executed, the computer program / instruction performs the following: acquiring near-infrared data during a language fluency task performed by the subject; representing the near-infrared data using an image to be examined, wherein pixels in the image to be examined can characterize the relationship between data at different times in the near-infrared data; and obtaining the identification result of the subject's mental state based on the image to be examined using a network model capable of capturing the correlation features between target acquisition channels.

[0011] According to the above-described scheme of the embodiments of this application, near-infrared data of the subject to be tested can be acquired. Then, the near-infrared data is processed and converted into an image to be tested. Finally, the image to be tested is input into a classification model to obtain the classification result of the subject to be tested, thereby determining whether the subject to be tested is a healthy subject or has a tendency towards schizophrenia. In the above scheme, the near-infrared data is normalized based on the maximum and minimum values ​​obtained from a preset data set formed by fusing second data sequences of healthy subjects and schizophrenic individuals. This considers the brain activation of the subject to be tested globally, avoiding the loss of brain activation information caused by normalization at different scales. This results in a higher accuracy of the classification result obtained when the classification model classifies the image to be tested based on the normalized near-infrared data. Furthermore, the aforementioned maximum and minimum values ​​are obtained based on a preset set of second data sequences distributed over time, including both healthy subjects and schizophrenic individuals. Thus, the differences between the subject to be tested and healthy subjects or schizophrenic patients are considered during the normalization process, improving the accuracy and reliability of the obtained classification result. Compared to methods that rely on doctors' subjective judgment or directly classify near-infrared data, the above method yields more accurate classification results. Attached Figure Description

[0012] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0013] Figure 1 A schematic block diagram of an apparatus for assessing schizophrenic tendencies based on near-infrared data according to an embodiment of this application is shown;

[0014] Figures 2(a) and 3(a) respectively show the oxyhemoglobin concentration change curves of a healthy subject and a patient with schizophrenia according to an embodiment of this application;

[0015] Figure 2(b) shows the results of converting the changes in oxyhemoglobin concentration in healthy subjects and patients with schizophrenia into GADF after applying the normalization method described herein according to one embodiment of this application;

[0016] Figure 2(c) shows the results of converting the changes in oxyhemoglobin concentration in healthy subjects and patients with schizophrenia into GASF after applying the normalization method described herein according to one embodiment of this application;

[0017] Figure 3(b) shows the results of converting the oxyhemoglobin concentration change values ​​of healthy subjects and schizophrenic patients into GADF after using the normalization processing method in the prior art according to an embodiment of this application;

[0018] Figure 3(c) shows the results of converting the changes in oxyhemoglobin concentration in healthy subjects and schizophrenic patients into GASF after using the normalization processing method in the prior art according to an embodiment of this application;

[0019] Figure 4 A schematic diagram illustrating a test object performing a language fluency task according to an embodiment of this application is shown;

[0020] Figure 5 A schematic diagram of a data acquisition channel according to an embodiment of this application is shown;

[0021] Figure 6 A schematic block diagram of the network architecture of a shuffling network according to an embodiment of this application is shown;

[0022] Figure 7 A schematic diagram illustrating the generation process of a first image according to an embodiment of this application is shown;

[0023] Figure 8 A schematic diagram of a Gram angle difference field diagram and a Gram angle sum field diagram according to an embodiment of this application is shown;

[0024] Figure 9 A schematic diagram of a method for assessing schizophrenic tendencies based on near-infrared data according to an embodiment of this application is shown;

[0025] Figure 10 A schematic block diagram of an apparatus for identifying the mental status of a subject based on fNIRS data according to an embodiment of the present invention is shown.

[0026] Figure 11 A schematic block diagram of a system for identifying the mental status of a subject based on fNIRS data according to an embodiment of the present invention is shown. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0028] Functional near-infrared spectroscopy (fNIRS) offers advantages such as being non-invasive, adaptable to various subject groups (e.g., newborns, middle-aged individuals, and the elderly), and suitable for diverse testing scenarios (e.g., both inside and outside the laboratory). Because the main components of blood exhibit good scattering of near-infrared light in the wavelength range of 600-950 nm, fNIRS can be used to obtain information about changes in hemoglobin concentration during brain activity, such as changes in oxyhemoglobin concentration (HBO), deoxyhemoglobin concentration (HBR), and total hemoglobin concentration (HBT).

[0029] Specifically, light intensity variation data acquired by near-infrared spectroscopy can be converted into time-varying optical density data. Furthermore, a modified Beer-Lambert law can be used to obtain hemoglobin concentration variations. In this application, "near-infrared data" can be understood as data characterizing the brain activation level of the subject. It can be light intensity variation data directly acquired using the FNIRS acquisition module, optical density data obtained by converting light intensity variation data, or hemoglobin concentration variation data obtained using the modified Beer-Lambert law. The processor can acquire light intensity variation data for further processing before inputting it into a classification model for classification. Alternatively, it can directly acquire hemoglobin concentration variation data obtained through processing by other devices, convert it into an image to be tested, and then input it into a classification model for classification. This application does not impose specific limitations on this, as long as the "near-infrared data" can characterize the brain activation level of the subject. Those skilled in the art can configure it according to actual needs.

[0030] In practical scenarios for diagnosing schizophrenia, the diagnosis can be made based on the near-infrared data of the subject. Refer to Table 1, which shows features extracted from near-infrared data, such as HBO curves, brain functional connectivity, and dynamic functional connectivity. These features are then classified using machine learning models in Table 1, resulting in various evaluation parameters for each of the resulting models.

[0031] Table 1

[0032]

[0033]

[0034] Accuracy can be the ratio between the number of correctly classified samples and the total number of samples. Precision can be the percentage of samples that are actually positive among those classified as positive (the specific category to be identified, for example, schizophrenia in this embodiment). Recall can be the ratio between the number of samples classified as positive and the number of samples that are actually positive in the total number of samples. The F1 score can be the value obtained by fusing precision and recall. In Table 1, the Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), and Linear Discriminant Analysis (LDA) classifiers do not perform well in assessing schizophrenia predisposition using the above features.

[0035] To at least partially address the aforementioned problems, embodiments of this application provide an apparatus for assessing schizophrenic tendencies based on near-infrared data. Figure 1 A schematic block diagram of an apparatus for assessing schizophrenic tendencies based on near-infrared data according to one embodiment of this application is shown. Figure 1 As shown, the device includes a processor 101.

[0036] The processor 101 can be configured to execute steps S110 to S130.

[0037] In step S110, near-infrared data of the object to be tested is acquired.

[0038] The subjects to be tested can be those whose brains are subjected to near-infrared data collected by a probe for brain function examination; they can also be referred to as examinees.

[0039] Near-infrared data can be acquired from the brain of the subject using a probe. This probe may include at least one pair of emitting and receiving probes. In one example, several emitting and receiving probes can be placed inside a headgear, which is worn by the subject during the acquisition process. The emitting probe in the headgear emits near-infrared light, which is absorbed and scattered by the brain tissue and then received by the corresponding receiving probe. This near-infrared light intensity change data can be directly transmitted as near-infrared data to the processor mentioned above. Alternatively, the intensity change data can be converted into optical density data, or even further converted into hemoglobin concentration change data, before being transmitted as near-infrared data to the processor.

[0040] In some embodiments, the arrangement of the transmitting and receiving probes within the headgear can be determined by the distribution of brain regions in the subject's brain. Optionally, these brain regions may be those of interest to the user of the device described above, related to a diagnosis of schizophrenia. Preferably, these brain regions may include the prefrontal cortex and / or temporal lobe. More specifically, these brain regions may include: the left ventral prefrontal cortex, right ventral prefrontal cortex, left ventrolateral prefrontal cortex, right ventrolateral prefrontal cortex, left dorsolateral prefrontal cortex, right dorsolateral prefrontal cortex, left dorsolateral prefrontal cortex, right dorsolateral prefrontal cortex, left temporal cortex, and right temporal cortex. The inventors have verified, in particular, that the accuracy of assessing schizophrenic tendency based on oxyhemoglobin data from the above brain regions is higher.

[0041] In one example, processor 101 can acquire near-infrared data of the object under test collected by the probe in real time. In another example, the near-infrared data of the object under test collected by the probe can also be saved to a designated storage space. Processor 101 can access the storage space via wired or wireless means to obtain the near-infrared data of the object under test.

[0042] In step S120, the near-infrared data is processed and converted into an image to be measured.

[0043] The above processing includes normalization based on the maximum and minimum values ​​obtained from a preset data set. These maximum and minimum values ​​can be used as normalization boundaries to process the near-infrared data.

[0044] The pre-defined dataset may include a second data sequence distributed over time from healthy individuals and individuals with schizophrenia. The second data sequence may be a sequence of hemoglobin concentration changes obtained by processing optical density data using a modified Beer-Lambert law.

[0045] It should be noted that the "maximum value" and "minimum value" in the "normalization processing of maximum and minimum values ​​obtained based on a preset data set" in this application can be the maximum and minimum values ​​of the data directly obtained from the preset data set, or the maximum and minimum values ​​obtained based on the data in the preset data set. For example, the data in the preset data set can be sorted from smallest to largest, and the maximum and minimum values ​​can be determined based on a preset proportion of the data. For example, the preset proportion can be 5%. The average value of the first 5% (i.e., the low value) of the data can be taken and the average value can be taken as the minimum value, and the average value of the last 5% (i.e., the high value) of the data can be taken and the average value can be taken as the maximum value.

[0046] In some other embodiments, the maximum and minimum values ​​can also be preset. It is understood that the preset values ​​should be correlated with the maximum and minimum values ​​obtained from a large sample including data from healthy individuals and schizophrenia patients to ensure the accuracy of identification. In summary, all acquisition channels of the subject under test use the same maximum and minimum values ​​for normalization to ensure that the normalization scale is the same.

[0047] In one example, processed near-infrared data can be directly converted into an image to be tested. The pixels in the image can characterize the relationships between data at different times in the processed near-infrared data. Specifically, the processed near-infrared data is time-series data and needs to be converted into image data that can be recognized by a classification model. For example, Gram angle field can be used to convert time-series, processed near-infrared data into image data; of course, other methods can also be used for conversion. Exemplarily, the near-infrared data can first be converted into a sequence of hemoglobin concentration change values ​​using a modified Beer-Lambert law. Then, this sequence of hemoglobin concentration change values ​​can be converted into image data. It should be understood that in some embodiments, the acquisition times and / or hemoglobin concentration change values ​​in the sequence of hemoglobin concentration change values ​​can be discarded or supplemented to adapt to the image size of the image to be tested.

[0048] In another example, the near-infrared data can be further processed before being converted into the image to be measured. For instance, if the near-infrared data is optical density data, motion artifact removal, downsampling, and filtering can be performed on the optical density data, and the sequence of hemoglobin concentration changes can be obtained from the processed optical density data using a modified Beer-Lambert law.

[0049] Specifically, in existing technologies, the normalization of near-infrared data before converting it into an image to be tested is performed based on each acquisition channel of each subject. That is, the normalization is performed on each acquisition channel based on its maximum and minimum values. This results in the maximum and minimum values ​​in the normalization process being highly likely to be inconsistent for each subject and even for each acquisition channel. In other words, different scales are used to normalize the near-infrared data of each acquisition channel. The inventors believe that this normalization method not only leads to the loss of brain activation information from the acquisition channels, but also weakens (or even eliminates) the group differences in brain activation between the subject and healthy individuals or patients with schizophrenia, resulting in a low accuracy rate in assessing schizophrenia tendencies.

[0050] The near-infrared data is normalized by using the maximum and minimum values ​​obtained from the preset data set formed by fusing second data sequences from healthy subjects and schizophrenic individuals. This allows for a global consideration of brain activation in the subject, avoiding the loss of brain activation information caused by normalization at different scales. As a result, the classification model achieves higher accuracy when classifying images converted from normalized near-infrared data.

[0051] Furthermore, the aforementioned maximum and minimum values ​​are based on a predefined set of second data sequences distributed over time, including both healthy individuals and those with schizophrenia, rather than from a single second data sequence of healthy individuals or those with schizophrenia. This approach takes into account the differences between the test subjects and healthy individuals or those with schizophrenia during normalization, thereby improving the accuracy and reliability of the obtained classification results.

[0052] In step S130, the image to be tested is input into the classification model to obtain the classification result of the object to be tested.

[0053] For example, the initial model can be trained based on a training dataset to obtain the aforementioned classification model. The training dataset may include several training images and the annotation results corresponding to each training image. The near-infrared data of each first object can be processed according to the relevant content in step S120, and the processed near-infrared data can be converted into training images; this will not be elaborated upon in this embodiment. The aforementioned first object is a subject whose near-infrared data has been pre-collected using a probe. The set of first objects includes healthy individuals and patients with schizophrenia; that is, the training images used for training include both training images converted from near-infrared data of healthy individuals and training images converted from near-infrared data of patients with schizophrenia. The aforementioned initial model can be a deep learning model, a reinforcement learning model, or a similar model. The aforementioned annotation results can be obtained through a specific algorithm or manual annotation. In some examples, the aforementioned annotation results can be used to indicate the tendency of the first object to be a healthy object or a person with schizophrenia, so that the classification results of the trained classification model can be used to indicate the tendency of the test object to be a healthy object or a person with schizophrenia. The classification results may include labels for healthy objects or people with schizophrenia. Alternatively, the classification results may also include the confidence level that the subject is a healthy person or a person with schizophrenia.

[0054] The initial model training can be performed as follows: Divide several training images from the training dataset into multiple batches. Input the training images from the same batch sequentially into the initial model to obtain the prediction results corresponding to the training images. Calculate the loss value by comparing the labeled results and the prediction results corresponding to the training images using a preset loss function. Adjust the model parameters of the initial model based on this loss value. Train the adjusted initial model using another batch of training images until training is complete, obtaining the classification model. The conditions for completing the above training can include the loss value calculated by the loss function stabilizing or the number of iterations reaching a preset number, etc.

[0055] Preferably, the classification model described above can at least capture the correlation features between FNIRS acquisition channels. More preferably, the classification model can be ShuffleNet V1, ShuffleNet V2, etc. In another embodiment, it can also be a classification model including an SE module. Specifically, the inventors have verified that a classification model that can capture the correlation between information about near-infrared data from different acquisition channels in the image under test can effectively improve the accuracy of the classification results for schizophrenic tendencies.

[0056] The inventors verified the recognition effect of the normalization method presented in this paper. 2(a) shows the oxyhemoglobin concentration change curves of a healthy subject (see the figure above, with a high level of activation; the circle indicates the typical value of the selected healthy subject) and a schizophrenic patient (see the figure below, with a low level of activation; the box indicates the typical value of the selected schizophrenic patient). Figure 3(a) and 2(a) The same. To compare the effectiveness of the normalization method presented in this paper with existing normalization methods (i.e., those mentioned above that use the maximum and minimum values ​​of each acquisition channel to normalize the acquisition channel), as a comparison, Figures 2(b)-2(c) and Figures 3(b)-3(c) The results of the Gram angle difference field GADF and the Gram angle sum field GASF obtained by processing with these two methods are shown respectively.

[0057] like Figures 2(b)-2(c) As shown, the GADF results for healthy subjects (see Figure 2(b) above) differed significantly from the GADF results for schizophrenic patients (see Figure 2(b) below), and the GASF results for healthy subjects (see Figure 2(c) above) also differed significantly from the GADF results for schizophrenic patients (see Figure 2(c) below).

[0058] The results of GADF and GASF obtained using existing normalization methods can be found in [reference]. Figures 3(b)-3(c) As can be seen, the GADF results for healthy subjects (see Figure 3(b) top) are relatively similar to those for schizophrenic patients (see Figure 3(b) bottom), and the GASF results for healthy subjects (see Figure 3(c) top) are also less significant than those for schizophrenic patients (see Figure 3(c) bottom). Therefore, the normalization method used in this paper can preserve the differences in brain activation between healthy subjects and schizophrenic patients, preventing these differences from being weakened (or even eliminated), thus improving the recognition performance.

[0059] Furthermore, Table 2 shows the various evaluation parameters of the multiple models obtained by processing near-infrared data using the normalization method in this paper and the normalization method in the prior art, and inputting the transformed test image into three network models, DenseNet, MobileNetV2 and ShuffleNetV2, for training.

[0060] Table 2

[0061]

[0062] In Table 2, "All data" indicates that the normalization method used in this paper is applied, which normalizes the maximum value (i.e., the first preset value) and minimum value (i.e., the second preset value) obtained from a preset time-series dataset composed of healthy subjects and schizophrenic patients. "Each channel" indicates that the normalization method used in the prior art is applied, which normalizes the target acquisition channel by using the maximum and minimum values ​​of each acquisition channel itself.

[0063] The validation results in Table 2 show that the classification performance of the test image obtained by the normalization method in this paper, when input into various network models, is superior to that obtained by the normalization methods in existing technologies. Furthermore, the classification performance of the test image obtained by the normalization method in this paper, when input into the ShuffleNetV2 network which captures the correlation features between target acquisition channels, is significantly better than that of DenseNet and MobileNetV2.

[0064] According to the apparatus described in the embodiments of this application, near-infrared data of the subject to be tested can be acquired. Then, the near-infrared data is processed and converted into an image to be tested. Finally, the image to be tested is input into a classification model to obtain a classification result for the subject to be tested, thereby determining whether the subject to be tested is a healthy subject or has a tendency towards schizophrenia. In the above scheme, the near-infrared data is normalized based on the maximum and minimum values ​​obtained from a preset data set formed by fusing second data sequences of healthy subjects and schizophrenic individuals. This considers the brain activation of the subject to be tested globally, avoiding the loss of activation information caused by normalization at different scales. This results in a higher accuracy of the classification result obtained when the classification model classifies the image to be tested based on the normalized near-infrared data. Furthermore, the maximum and minimum values ​​are obtained based on a preset set of second data sequences distributed over time, including both healthy subjects and schizophrenic individuals. Thus, the differences between the subject to be tested and healthy subjects or schizophrenic patients are considered during the normalization process, improving the accuracy and reliability of the obtained classification result. Compared to methods that rely on doctors' subjective judgment or directly classify near-infrared data, the above method yields more accurate classification results.

[0065] Specifically, near-infrared data can be collected from the test subject while performing tasks specific to schizophrenia, such as resting-state tasks and cognitive tasks. Preferably, near-infrared data is collected from the test subject while performing a language fluency task. Related studies have shown that, compared to other tasks, near-infrared data acquired during language fluency tasks has higher specificity for the identification of mental illnesses, which is beneficial to improving the accuracy of classification results obtained based on near-infrared data.

[0066] When test subjects perform language fluency tasks, they can be guided to perform specific speaking behaviors according to predetermined rules. During this process, the subject's cerebral cortex is activated, leading to changes in hemoglobin concentration.

[0067] The language fluency task may include a first period before the test subject performs the word-building task, a second period after performing the word-building task, and a third period after performing the word-building task. (See also...) Figure 4 , Figure 4 A schematic diagram is shown illustrating a test object performing a language fluency task according to one embodiment of this application. (In conjunction with...) Figure 4 The language fluency task can be sequentially divided into the first phase of performing the task of reading numbers (see reference). Figure 4 The test subject performed a 30-second task of reading numbers "12345" and a second-segment task of forming words (see reference). Figure 4 The test subject performed a 20-second word-building task related to "white", a 20-second word-building task related to "north", a 20-second word-building task related to "big", and a third-segment number reading task (see reference). Figure 4 The test subject performed a 70-second task of reading numbers for "12345".

[0068] The near-infrared data acquisition process is as follows: The subject can first wear the headgear mentioned above. Then, continuous data acquisition begins (or continuous data acquisition begins when the subject starts performing the language fluency task). During the continuous data acquisition process, the subject performs the number reading task and word formation task in the language fluency task. The near-infrared data collected in the first time period before performing the word formation task, the second time period after performing the word formation task, and the third time period after performing the word formation task can be used as the near-infrared data in this embodiment of the application.

[0069] For example, the processor 101 in the device provided in this application embodiment is further configured to execute steps S210 and S220.

[0070] In step S210, the coefficient of variation is calculated for each data sequence of near-infrared data.

[0071] Specifically, the aforementioned data sequence can be either light intensity variation data or light density data. Before performing preprocessing such as motion artifact removal and filtering on the light density data, it is necessary to calculate the coefficient of variation to check the data quality and ensure its accuracy. (See also...) Figure 5 , Figure 5 A schematic diagram of a data acquisition channel according to an embodiment of this application is shown. (In conjunction with...) Figure 5 An example, in which acquisition channels 1 to 48 may be included (see reference). Figure 5 (Ch1 to Ch48). Each data sequence can correspond to near-infrared data acquired through different acquisition channels. One transmitting probe and one receiving probe can form an acquisition channel. In another example, the brain region of interest can correspond to at least one acquisition channel. Alternatively, acquisition channels corresponding to uninterested brain regions can be turned off, specifically, for example, by not setting corresponding transmitting and receiving probes on these uninterested brain regions.

[0072] The coefficient of variation (COP) mentioned above measures the dispersion (or variability) of the data sequence corresponding to each acquisition channel. A higher COP indicates a more unstable data sequence, potentially indicating acquisition anomalies. For example, a hardware malfunction in the transmitting or receiving probe corresponding to the acquisition channel. Another example is a significant relative displacement between the headgear and the subject's head during near-infrared brain function data acquisition. The mean and standard deviation of each data sequence can be calculated first. The ratio of the standard deviation to the mean of each data sequence is used as the COP for that data sequence. In one example, the mean and standard deviation of the data sequence can be calculated based on all data points in the sequence. Alternatively, the mean and standard deviation can be calculated based on representative data points to improve computational efficiency and thus enhance diagnostic efficiency for the subject.

[0073] In step S220, it is determined whether the calculated coefficient of variation is greater than a preset coefficient threshold.

[0074] If the coefficient of variation is greater than a preset threshold, the data sequence corresponding to that coefficient of variation can be considered abnormal. The specific value of the preset threshold can be set by the developer or the user of the device mentioned above. For example, the preset threshold can be 10%, 15%, etc.

[0075] In this embodiment, step S120, generating the image to be tested corresponding to the near-infrared data based on the near-infrared data, can be performed only when the number of coefficients of variation greater than a preset coefficient threshold is less than a preset number threshold.

[0076] If the number of coefficients of variation greater than a preset threshold is greater than or equal to a preset number threshold, it can be considered that there are too many anomalies in the near-infrared data collected for the object under test. In this case, it is necessary to re-collect the near-infrared data of the object under test and then execute the above step S120. The specific value of the preset number threshold can be manually set by the developer or the user of the device mentioned above, or it can be automatically obtained by multiplying a preset ratio by the total number of acquisition channels of the probe. For example, if the coefficient of variation of the data sequence corresponding to a acquisition channel is 20% and the preset threshold is 10%, then the data sequence can be considered to have an anomaly. If the preset ratio is 10% and the total number of acquisition channels of the probe is 48, then the preset number threshold is 5 (48*10% equals 4.8, and is rounded down). If the total number of abnormal data sequences is greater than or equal to 5, it is necessary to re-collect the near-infrared data of the object under test. If the total number of abnormal data sequences is less than 5, step S120 can be executed.

[0077] According to the above-described scheme of this application embodiment, the coefficient of variation can be calculated for the data sequence of near-infrared data from each acquisition channel. Then, it is determined whether the calculated coefficient of variation is greater than a preset threshold, thereby determining whether to generate the image to be tested corresponding to the near-infrared data. The above scheme evaluates the quality of near-infrared data by calculating the coefficient of variation. If it is determined that there are too many abnormal data sequences in the near-infrared data, the quality of the near-infrared data is considered too low, and the accuracy of the generated classification result may also be low. In the above case, the above scheme can choose not to generate the image to be tested, thus saving computing power.

[0078] For example, the processor 101 performs step S120, which processes the near-infrared data and converts the processed near-infrared data into an image to be measured. This may include performing steps S121 and S122.

[0079] In step S121, multiple first data sequences are determined based on near-infrared data.

[0080] Different first data sequences can correspond to different acquisition channels of the probe. By applying a modified Beer-Lambert law, the data acquired through each acquisition channel in the near-infrared data can be processed separately to obtain the first data sequence corresponding to that acquisition channel. The first data sequence can be used to represent the distribution of hemoglobin concentration changes at different acquisition times. In some embodiments, the aforementioned hemoglobin concentration changes can be oxyhemoglobin concentration changes, deoxyhemoglobin concentration changes, and total hemoglobin concentration changes, etc. Preferably, in the embodiments of this application, the hemoglobin concentration changes can be only oxyhemoglobin concentration changes, that is, the first data sequence can be the distribution of oxyhemoglobin concentration changes at different acquisition times. Using the image converted from the oxyhemoglobin concentration changes as input into the classification model can improve the accuracy of the assessment.

[0081] In step S122, the first data sequence A from the plurality of first data sequences is... i The image is converted into a first image to be used as a part of the image to be tested. Here, i is a positive integer less than or equal to the total number of acquisition channels. The conversion process can be referred to in step S120 above, and will not be repeated here.

[0082] In some embodiments, multiple first images can be stitched together along the horizontal and / or vertical dimensions to obtain the image to be tested. For example, if the first image size is 256*256 and the total number of acquisition channels is 4, then two first images can be stitched together horizontally and two first images can be stitched together vertically to obtain an image to be tested with a size of 512*512. Alternatively, four first images can be stitched together only along the horizontal or vertical dimensions to obtain an image to be tested with a size of 1024*256 or 256*1024. Alternatively, multiple first images can be stitched together along a preset dimension to obtain the image to be tested. This preset dimension may be different from the horizontal or vertical dimension. For example, if the first image size is 256*256 and the total number of acquisition channels is 4, then an image to be tested with a size of 256*256*4 can be obtained.

[0083] According to the above-described scheme of the embodiments of this application, multiple first data sequences can be determined based on near-infrared data. Then, the first data sequences are converted into corresponding first images to be used as part of the image to be tested. The first data sequences in the above scheme correspond to different acquisition channels, and different acquisition channels can correspond to different locations in the brain. Therefore, the image to be tested obtained in the above scheme can effectively characterize the brain activation status of the subject, which is beneficial to improving the accuracy of the classification results obtained based on this image to be tested.

[0084] For example, the processor 101 in the apparatus provided in this application embodiment is further configured to: generate a first data sequence based on near-infrared data in a first time period, a second time period, and a third time period, respectively. Then, the first data sequence is calibrated based on the data in the first and third time periods.

[0085] Based on the above, the execution time periods for the language fluency task can include a first time period before the test subject performs the word grouping task, a second time period after performing the word grouping task, and a third time period after performing the word grouping task. The first data sequence can be calibrated based on the hemoglobin concentration change values ​​in the first and third time periods (i.e., the data of the first data sequence in the first time period) and the hemoglobin concentration change values ​​in the third time period (i.e., the data of the first data sequence in the third time period). For example, the mean of the hemoglobin concentration change values ​​in the first and third time periods can be calculated, and then each hemoglobin concentration change value in the first data sequence can be subtracted from this mean to obtain the calibrated first data sequence. In another example, the first data sequence can also be calibrated based on the hemoglobin concentration change values ​​in the first and third time periods using a general linear model, a time derivative distribution repair algorithm, or other methods.

[0086] According to the above-described scheme of the embodiments of this application, the first data sequence is calibrated based on the data in the first time period and the third time period, which can reduce the adverse effects of baseline drift on the first data sequence. The calibrated first data sequence in the above scheme is more representative, and obtaining the classification result based on the test image generated from the calibrated near-infrared data is beneficial to improving the accuracy of the obtained classification result.

[0087] For example, each first data sequence includes multiple hemoglobin concentration variation values ​​distributed over a time series. Preferably, each first data sequence includes multiple oxyhemoglobin concentration (HBO) variation values ​​distributed over a time series.

[0088] Optionally, the aforementioned normalization process can be performed on the first data sequence. This transforms the data in each of the first data sequences into a unified range, facilitating subsequent analysis and processing, and ensuring the accuracy of near-infrared data classification. In other words, the normalization process may include the following operations: based on the maximum and minimum values ​​obtained from a preset data set, normalize the first data sequence A... i Normalization is performed to normalize the first data sequence A. i Convert to the first image.

[0089] Specifically, the maximum and minimum values ​​mentioned above can be used as normalization boundaries for the first data sequence A.i Normalization is performed. The preset dataset may include time-series distributions of hemoglobin concentration changes in healthy individuals and individuals with schizophrenia.

[0090] The aforementioned preset dataset may include hemoglobin concentration changes distributed over time for a preset object. The preset object may include multiple healthy individuals and multiple individuals with schizophrenia; that is, the preset dataset contains hemoglobin concentration changes for both healthy individuals and individuals with schizophrenia. In other words, the preset object is not the object to be tested itself. For example, the preset object may be the first object mentioned in the process of training the initial model. Specifically, developers may pre-collect near-infrared data for multiple first objects and process the near-infrared data using a modified Beer-Lambert law to obtain multiple second data sequences corresponding to each first object. The aforementioned second data sequences may include multiple hemoglobin concentration changes distributed over time. In one example, if the preset dataset only includes multiple second data sequences corresponding to each first object, the largest hemoglobin concentration change among all the second data sequences of the first objects can be taken as the maximum value, and the smallest hemoglobin concentration change among all the second data sequences of the first objects can be taken as the minimum value. In another example, the developer may also filter out the second data sequence, or add the second data sequences of other objects besides the first object to a preset set. It should be understood that in this example, the maximum and minimum values ​​may need to be adjusted accordingly based on the second data sequences of other objects, which will not be elaborated here in the embodiments of this application.

[0091] In some embodiments, training images used for training the initial model can be generated based on the second data sequences corresponding to each of the first objects. Specifically, for example, each second data sequence can be normalized based on the maximum and minimum values ​​to obtain normalized second data sequences. Then, based on the multiple normalized second data sequence groups corresponding to each first object, a training image corresponding to that first object is generated. The process of generating training images can refer to the relevant process of obtaining the test image in step S122 above, and will not be elaborated upon here.

[0092] In one example, the change in hemoglobin concentration x is for either the first data sequence or the second data sequence. j The hemoglobin concentration change values ​​with a distribution range between [-1, 1] can be obtained by normalization using the following formula (1).

[0093]

[0094] Where j is a positive integer less than or equal to the total number of changes in hemoglobin concentration, and X max X is the maximum value mentioned above. min This is the minimum value mentioned above. The changes in hemoglobin concentration in either the first or second data sequence are calculated sequentially using the above formula to obtain the normalized first or second data sequence.

[0095] In some embodiments, the first or second data sequence may be compressed first to reduce computational load. For example, the first or second data sequence can be compressed using a Piecewise Aggregate Approximation (PAA) algorithm or an algorithm based on Huffman coding. Specifically, if the probe's acquisition frequency is 11 Hz and the total acquisition time is 125 seconds, there are a total of 1375 (i.e., 125 * 11) acquisition moments, each corresponding to a hemoglobin concentration change value. The Piecewise Aggregate Approximation algorithm can reduce this to 124 hemoglobin concentration change values.

[0096] According to the above-described scheme of the embodiments of this application, the first data sequence A can be processed based on the maximum and minimum values ​​obtained from the preset data set. i Normalization is performed to normalize the first data sequence A. i The first image is then converted. The above scheme uses maximum and minimum values ​​obtained from a preset dataset for normalization. On one hand, the first data sequences for different test subjects and the first data sequences from different acquisition channels for the same test subject can be represented on the same feature space, which helps improve the accuracy of the classification results obtained by the classification model. Specifically, if each acquisition channel is normalized based on the maximum and minimum values ​​of the hemoglobin concentration change of the current test subject, the obtained first image will lose brain activation information, leading to poorer classification results. However, in the above technical solution, the first data sequence is normalized based on the maximum and minimum values ​​obtained from a preset dataset that integrates the hemoglobin concentration change values ​​of healthy subjects and schizophrenic patients. This considers the brain activation of the test subject globally, avoiding the loss of brain activation information caused by normalization at different scales. This results in higher accuracy when the classification model classifies the test image converted from the normalized near-infrared data. Furthermore, it considers the differences between the test subject and healthy individuals and schizophrenic patients, improving the reliability of the obtained classification results.

[0097] Preferably, the classification model can be a ShuffleNet, a lightweight deep learning architecture that can efficiently perform classification.

[0098] The aforementioned shuffling network may include a feature extraction module, multiple shuffling modules, a convolutional module, and a classification module. It should be understood that the network structure of the aforementioned shuffling network can be added to or removed according to the developer's needs. If the shuffling network is used to generate classification results, the processor 101 executes step S130 to input the image to be tested into the classification model to obtain the classification result of the object to be tested, which may include executing steps S131 to S134.

[0099] In step S131, the image to be tested is input into the feature extraction module to obtain the first image features of the image to be tested.

[0100] The aforementioned feature extraction module is used to perform feature extraction operations on the image to be tested to obtain the first image features of the image to be tested. In some embodiments, the architecture of the aforementioned feature extraction module can be designed with reference to convolutional extraction architecture, autoencoder extraction architecture, etc., and the embodiments of this application are not limited thereto.

[0101] In step S132, the first image features or rearranged features obtained by other shuffling modules are divided into two groups of data by the shuffling module. A convolution operation is performed on one of the two groups of data. The convolved group of data is then concatenated with the other group of data and rearranged to obtain rearranged features.

[0102] The shuffling module can be used to further extract features from the first image features or rearranged features obtained by other shuffling modules. Multiple shuffling modules can be connected sequentially. For example, if there are three shuffling modules 1 to 3, shuffling module 1 can be connected to shuffling module 2, shuffling module 2 can be connected to shuffling module 3, and shuffling module 3 can be connected to the classification module.

[0103] Each shuffling module can use the same convolution parameters (e.g., kernel size, kernel stride) when performing convolution operations to achieve a lightweight shuffling network. The concatenation operation described above can be performed based on the concat operator in related technologies. The rearrangement operation described above can be used to rearrange the feature points in the concatenated features obtained from the concatenation operation. This rearrangement can be achieved by multiplying the concatenated features by a preset transpose matrix, or by rearranging the concatenated features according to specific rules. Through the processing of the shuffling module, information flow of the first image features or rearranged features can be realized, which is beneficial to improving the accuracy of classification results.

[0104] In step S133, the rearranged features obtained by the shuffling module connected to the convolution module are convolved to obtain the first convolution feature.

[0105] The convolution module is used to extract features again from the received rearranged features to improve the representativeness of the rearranged features. The convolution parameters used by the convolution module when performing convolution operations can be determined according to the developer's needs.

[0106] In step S134, the classification module performs a classification operation on the first convolutional features to obtain the classification result of the object to be tested.

[0107] In one example, the classification module can output the confidence level of the test object as either a healthy person or a person with schizophrenia based on the first convolutional feature, and then use the label of the person with the highest confidence level as the classification result.

[0108] Optionally, the training process of the shuffling network can be as follows: Several training images in the training dataset are divided into multiple batches. The training images are labeled, in other words, they have labeling results. Training images from the same batch are sequentially input into the feature extraction module to obtain the second image features of the training images. Then, the shuffling module obtains the rearranged features of the training images. The rearranged features of the training images are then convolved using the convolution module to obtain the first convolutional features of the training images. Finally, the first convolutional features of the training images are classified using the classification module to obtain the prediction result of the first object corresponding to the training image. The labeled results and the prediction results of the training images are calculated using a preset loss function to obtain the loss value. The model parameters of the shuffling network are adjusted based on this loss value. The adjusted shuffling network is trained using another batch of training images until training is complete, thus obtaining the shuffling network used in steps S131 to S133.

[0109] See Figure 6 , Figure 6 A schematic block diagram of the network architecture of a shuffling network according to one embodiment of this application is shown. (In conjunction with...) Figure 6The shuffling network may include an input layer, an initial convolutional layer, a pooling layer, multiple network layers stacked through multiple shuffling modules, a first convolutional layer, a global average pooling layer, and a fully connected layer. The input layer receives the test image or training image and inputs it to the initial convolutional layer and pooling layer. In this example, the feature extraction module mentioned above may include the initial convolutional layer and pooling layer. The convolutional module mentioned above may include the first convolutional layer. The classification module mentioned above may include a global average pooling layer and a fully connected layer. Each shuffling module may include a channel segmentation layer, several 1*1 convolutional layers, 3*3 convolutional layers, a channel concatenation layer, and a channel rearrangement layer. The channel segmentation layer is used to divide any one of the first image features, second image features, or rearranged features obtained from other shuffling modules into two groups of data. The 1*1 convolutional layer and 3*3 convolutional layer are used to perform convolution operations on one of the two groups of data. The channel concatenation layer is used to concatenate the convolutionally processed group of data with the other group of data. The aforementioned channel rearrangement layer is used to rearrange the spliced ​​features obtained from the splicing operation to obtain rearranged features.

[0110] According to the above-described scheme of this application embodiment, the image to be tested can be input into the feature extraction module to obtain the first image features of the image to be tested. Then, through the shuffling module, the first image features or the rearranged features obtained by other shuffling modules can be divided into two groups of data. A convolution operation is performed on one of the two groups of data. The convolved group of data is then concatenated with the other group of data and rearranged again to obtain rearranged features. Then, through the convolution module, the rearranged features obtained by the shuffling module connected to the convolution module are convolved to obtain the first convolutional features. Finally, through the classification module, the first convolutional features are classified to obtain the classification result of the object to be tested. The shuffling network in the above scheme can realize the correlation learning between the relevant features of the hemoglobin concentration change values ​​of different acquisition channels, thus improving the accuracy of the output classification results.

[0111] For example, in step S122, processor 101 selects the first data sequence A from a plurality of first data sequences. i Converting to a first image may include performing steps S1221 and S1222.

[0112] In step S1221, based on the first data sequence A i Generate the first data sequence A i The corresponding Gram angle difference field diagram and Gram angle sum field diagram.

[0113] See Figure 7 , Figure 7 A schematic diagram illustrating the generation process of a first image according to an embodiment of this application is shown. (In conjunction with...) Figure 7In some embodiments, the first data sequence can be converted into data in polar coordinates by the following formula (2).

[0114]

[0115] Where, θ j The change in hemoglobin concentration in the first data sequence The corresponding angular coordinates in the polar coordinate system. It can be understood that, in the example above that includes normalization, this could be the normalized hemoglobin concentration change value in the first data sequence. The corresponding angular coordinates in the polar coordinate system (in this example, The numerical range of is [-1, 1], which is related to the formula used in the normalization operation. arccos is the inverse cosine function. j The acquisition time t in the first data sequence j The corresponding radius coordinates in the polar coordinate system. N is a constant factor for the span of the normal polar coordinate system.

[0116] As described above, in some embodiments, for each first data sequence, the first data sequence can be normalized based on the maximum and minimum values ​​obtained from a preset data set.

[0117] After obtaining the polar coordinates of the first data sequence, the Gramian Angular Difference Field (GADF) and the Gramian Angular Summation Field (GASF) can be determined by the following formula (3).

[0118]

[0119] Where, θ1 to θ s The angle values ​​corresponding to the changes in hemoglobin concentration at different collection times in the first data sequence are obtained by formula (2), where s is the total number. Then, based on the values ​​in the Gram difference field and the Gram sum field, the Gram difference field map and the Gram sum field map are obtained. Specifically, according to the preset correspondence, they can be converted into RGB values ​​to obtain the Gram difference field map and the Gram sum field map. The preset correspondence can be a pre-set correspondence between the values ​​in the Gram difference field and the Gram sum field and the RGB values.

[0120] See Figure 8 , Figure 8 A schematic diagram of a Gram angle difference field map and a Gram angle sum field map according to an embodiment of this application is shown. (In conjunction with...) Figure 8The pixel value distribution of the Gram difference field map and Gram sum field map of healthy individuals differs significantly from that of schizophrenic individuals. Therefore, the test images generated based on the Gram difference field map and Gram sum field map are more representative and can improve the accuracy of the classification results. Specifically, for example, the largest change in hemoglobin concentration (or peak) in the first data sequence of healthy individuals is more prominent in the corresponding regions of the Gram difference field map and Gram sum field map (see reference). Figure 8 (The circular areas in the image are darker). It should be understood that only grayscale Gram difference field maps and Gram sum field maps are shown here. In RGB Gram difference field maps and Gram sum field maps, larger changes in hemoglobin concentration will correspond to redder areas in the Gram difference field map, and smaller changes in hemoglobin concentration will correspond to bluer areas in the Gram sum field map. The specific correspondence between the values ​​in the Gram difference field and Gram sum field maps and the RGB values ​​mentioned above can be used as a reference.

[0121] In some embodiments, the training images used by the classification model in the training phase can also be obtained based on Gram angle difference field map and Gram angle sum field map, which will not be elaborated here in the embodiments of this application.

[0122] In step S1222, the first data sequence A is... i The corresponding Gram angle difference field map and Gram angle sum field map are stitched together to form the first image.

[0123] The above stitching can be performed by stitching the Gram angle difference field map and the Gram angle sum field map in any one of the horizontal, vertical, or preset dimensions mentioned above to obtain the first image.

[0124] Continue reading Figure 7 Taking a first data sequence comprising hemoglobin concentration changes at 124 acquisition times as an example, a Gram difference field map and a Gram sum field map with image sizes of 124*124 can be generated based on this first data sequence. Stitching the Gram difference field map and the Gram sum field map together yields a first image with an image size of 124*124*2. If there are 48 acquisition channels, i.e., 48 first data sequences, then stitching the 48 first images yields a test image with an image size of 124*124*96. Specifically, the GADF and GASF generated by each acquisition channel can be stored alternately to finally obtain the test image.

[0125] Referring to Table 3, Table 3 shows the various evaluation parameters of the multiple models obtained by converting near-infrared data based on normalization processing into Gram angle difference field maps and Gram angle sum field maps in the embodiments of this application, and inputting the images to be tested into deep learning models for training.

[0126] Table 3

[0127]

[0128]

[0129] The validation results in Table 3 show that the evaluation results of ShuffleNet, which can capture the correlation features between acquisition channels, are better than those of DenseNet and MobileNetV2. In other words, ShuffleNet, which can capture the correlation features between acquisition channels, has a better classification effect on schizophrenic patients.

[0130] According to the above-described scheme of the embodiments of this application, a Gram angle difference field map and a Gram angle sum field map can be generated based on the first data sequence. Then, the Gram angle difference field map and the Gram angle sum field map are stitched together to form the first image corresponding to the first data sequence. The rate of increase in hemoglobin concentration in schizophrenic patients is relatively slow, and the degree of brain activation is relatively low (see reference). Figure 8 This data distribution characteristic, which varies with the time of data collection, may be a feature that distinguishes schizophrenic individuals from healthy individuals. In the above scheme, the Gram angle difference field map and the Gram angle sum field map can characterize the temporal correlation between different hemoglobin concentration changes, which is beneficial for preserving the aforementioned features and thus improving the accuracy of the classification results. In addition, the Gram angle difference field map and the Gram angle sum field map contain more spatial information, and there is complementarity in their feature information. The first image obtained by stitching the two together can provide more comprehensive feature information, which is beneficial for improving the accuracy of the classification results obtained by the classification model.

[0131] See Figure 9 , Figure 9 This illustration shows a schematic diagram of a method for assessing schizophrenic tendencies based on near-infrared data according to an embodiment of this application, in conjunction with... Figure 9This application provides a practical application scenario for reference. The subject can first wear the headgear described above. Then, the transmitting probe on the headgear emits near-infrared light, which is received by the receiving probe on the headgear after passing through the brain, thus obtaining the aforementioned near-infrared data. The headgear can be equipped with multiple sets of transmitting and receiving probes to form multiple acquisition channels. After the near-infrared data is processed by the processor 101, a data sequence corresponding to each acquisition channel can be obtained (this data sequence can characterize the distribution of hemoglobin concentration changes in the brain of the subject at different acquisition times). Further, based on the relevant content of calculating the coefficient of variation in steps S210 and S220, it can be determined whether the quality of the data meets the standard. If the quality of the data meets the standard, motion correction, filtering, and other processing can be performed on the data, and then the processed data sequence can be converted into a first data sequence (this first data sequence is used to represent the distribution of hemoglobin concentration changes at different acquisition times) through the modified Beer-Lambert law, and baseline calibration can be performed on the first data sequence to achieve data preprocessing of the first data sequence. The first data sequence can then be normalized using the normalization process described above. The processed first data sequence is then converted to polar coordinates, generating Gram angle difference field map and Gram angle sum field map to form the near-infrared image to be tested. This image is then input into a classification model to output a classification result indicating whether the tested object is a healthy individual or a person with schizophrenia.

[0132] According to another aspect of this application, a method for assessing schizophrenic tendency based on near-infrared data is also provided. The method includes: acquiring near-infrared data of a subject; processing the near-infrared data and converting the processed near-infrared data into a test image, wherein the processing includes normalization processing based on maximum and minimum values ​​obtained from a preset dataset, the preset dataset including a second data sequence distributed over time for healthy subjects and schizophrenic individuals, wherein the second data sequence is obtained based on the near-infrared data of the healthy subjects and schizophrenic individuals; and inputting the test image into a classification model to obtain a classification result for the subject, wherein the classification result indicates the subject's tendency to be either healthy or schizophrenic. The above method can be executed using a processor.

[0133] According to another aspect of this application, a storage medium is also provided, on which program instructions are stored, which, when executed by a computer or processor, cause the computer or processor to perform corresponding steps of the method for assessing schizophrenic tendencies based on near-infrared data according to any embodiment of this application. It can also be used to implement corresponding modules in the apparatus for assessing schizophrenic tendencies based on near-infrared data according to embodiments of this application. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media.

[0134] Currently, there are many types of mental illnesses, and some share similar or identical clinical manifestations, leading to a low diagnostic accuracy rate. The diagnosis of mental illnesses currently relies primarily on psychiatrists using the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) in conjunction with the patient's complaints and the doctor's observations of the patient's behavior. This approach not only considers the psychiatrist's clinical experience but is also influenced by the doctor's subjective perception and the degree of cooperation between the patient and the doctor. For example, the examinee may be unwilling to disclose their true condition or resist the interview. Furthermore, patients may experience nonspecific prodromal symptoms several years before the onset of the illness, all of which can lead to overdiagnosis or underdiagnosis. Therefore, accurately and objectively identifying whether an examinee has a mental health problem is a technical problem that urgently needs to be solved by those skilled in the art.

[0135] In view of the above problems, the present invention also provides an apparatus, method, system, and storage medium for identifying the mental state of a subject based on near-infrared data, which can improve the accuracy of identifying the mental state of the subject.

[0136] To at least partially solve the above problems, embodiments of this application provide a device for identifying the mental state of a subject based on near-infrared data. Figure 10 A schematic block diagram of an apparatus for identifying the mental state of a subject based on near-infrared data, according to one embodiment of this application, is shown. Figure 1 As shown, the identification device may include a processor 102.

[0137] The processor 102 is configured to perform step S2110, acquiring near-infrared data during the subject's performance of a language fluency task.

[0138] In one specific embodiment, optical density time-series data can be obtained by acquiring optical density data at multiple moments during a subject's performance of a language fluency task. Then, using a modified Beer-Lambert law, the optical density data at each acquisition moment in the time-series data is converted into the hemoglobin concentration change value at that acquisition moment, thus obtaining the aforementioned near-infrared data. It is understood that those skilled in the art can perform relevant preprocessing operations during the data processing of the hemoglobin concentration change value obtained from the light intensity change data acquired by the fNIRS acquisition module, as needed. For example, the aforementioned preprocessing may include: removing motion artifacts from the optical density time-series data, filtering the optical density time-series data using a bandpass filter, etc. It should be understood that those skilled in the art can combine various preprocessing operations as needed.

[0139] In one example, during the execution of the above-mentioned language fluency task, specific tasks such as word grouping, counting, and scene description may be performed, and this application embodiment does not impose any limitations.

[0140] The processor 102 can also be configured to execute step S2120, representing near-infrared data using an image to be inspected. Pixels in the image to be inspected can characterize the relationship between data at different times in the near-infrared data. Specifically, methods such as Gram angle field, Markov transition field, recursive graph, short-time Fourier transform, and wavelet transform can be used to obtain the image to be inspected corresponding to the near-infrared data.

[0141] In one example, the acquisition time or hemoglobin concentration change value in the near-infrared data can be filtered or filled to represent the near-infrared data using an image of a preset size. In another example, the pixel values ​​in the image under inspection can also be correlated with the hemoglobin concentration change value. For example, if the image under inspection is a grayscale image, the pixel values ​​in that image can be positively correlated with the hemoglobin concentration change value. Furthermore, if the image under inspection is a color image, the larger the hemoglobin concentration change value corresponding to a pixel, the larger the value of the red channel (or R channel) and the smaller the value of the blue channel (or B channel) for that pixel.

[0142] The processor can also be configured to execute step S2130, which, based on the image to be examined, uses a network model capable of capturing the correlation features between target acquisition channels to obtain the identification result of the examinee's mental state.

[0143] Optionally, the image to be examined can be directly input into the aforementioned network model to obtain the recognition result. Alternatively, the image to be examined can undergo feature enhancement processing before being input into the aforementioned network model to obtain the recognition result. The aforementioned feature enhancement processing may include histogram equalization processing, color space conversion processing, geometric transformation processing, etc., depending on the actual needs. In one example, the recognition result of the subject's mental state can be used to indicate whether the subject's mental state is abnormal. In another example, the recognition result can also be used to indicate whether the subject suffers from a specific type of mental illness, such as depression, anxiety disorder, schizophrenia, etc.

[0144] It should be understood that each brain region of interest can also correspond to multiple data acquisition channels. If more attention is paid to the near-infrared data of one or more brain regions, these brain regions can be designated as brain regions of interest, and the data acquisition channels corresponding to these brain regions of interest can be designated as target acquisition channels.

[0145] The network models that can capture the correlation features between target acquisition channels can be neural network models with compression excitation modules (or Squeeze-and-Excitation, abbreviated as SE), shuffle network models (or ShuffleNet), etc.

[0146] In one specific embodiment, a compression excitation module is taken as an example. The compression excitation module can be embedded in neural network models such as ResNet, Inception (a multi-scale neural network), and MobileNet. After feature extraction, the image to be inspected yields its features. Then, the compression excitation module determines the interdependencies between feature channels in the image features and adaptively adjusts the channel features distributed in each feature channel. Specifically, the compression excitation module can encode each channel feature to integrate it into a global feature. This global feature is then input to two sequentially connected fully connected layers to determine the importance of different feature channels, generating weights for each feature channel. Finally, for each feature channel, the adjusted channel features are determined using the weights and channel features of that feature channel.

[0147] The image to be inspected can be generated based on near-infrared data from multiple target acquisition channels. Therefore, the features of the image to be inspected can include the distribution characteristics of near-infrared data from multiple target acquisition channels. Furthermore, the channel features of the image to be inspected in different feature channels can be considered as including the distribution characteristics of near-infrared data from at least one target acquisition channel. The aforementioned network model can focus on determining the importance between different feature channels; therefore, it can also be considered as the network model being able to capture the correlation features between target acquisition channels. Specifically, the feature channels can include image features corresponding to one or more target acquisition channels in the features of the image to be inspected.

[0148] In real-world scenarios, during a subject's performance of a language fluency task, hemoglobin concentrations in different brain regions of interest may fluctuate to varying degrees, indicating different levels of brain activation. While the fluctuation range of near-infrared (NIIR) data may differ, the NIIR data from different target acquisition channels can all reflect the brain region activation during the subject's language fluency task. Therefore, utilizing a network model capable of capturing the correlation features between target acquisition channels to determine the subject's mental state allows for a comprehensive assessment of brain function, improving the accuracy of the identification results.

[0149] According to the above-described scheme of this application embodiment, the processor 102 acquires near-infrared data during the subject's performance of a language fluency task. Then, it uses an image to be examined that can characterize the relationship between data at different times within the near-infrared data. Based on the image to be examined, the processor 102 uses a network model capable of capturing the correlation features between target acquisition channels to obtain the recognition result of the subject's mental state. This scheme converts near-infrared data into an image to be examined that can characterize the relationship between data at different times within the near-infrared data. This not only facilitates recognition by the network model but also improves the accuracy of recognizing the subject's mental state. Furthermore, the network model in this scheme can capture the correlation features between target acquisition channels in the image to be examined, improving the representativeness of the processed image and further enhancing the accuracy of recognizing the subject's mental state.

[0150] For example, the aforementioned near-infrared data may include near-infrared data from the frontal lobe and / or temporal lobe. Specifically, the near-infrared data comes from the brain region of interest, which is associated with the type of mental illness the network model aims to identify. For instance, for a certain mental illness, near-infrared data from the frontal lobe has high specificity, so only the near-infrared data from the frontal lobe can be acquired. For another mental illness, near-infrared data from the frontal lobe combined with near-infrared data from the temporal lobe has high specificity, so near-infrared data from both the frontal and temporal lobes can be acquired simultaneously.

[0151] According to the above-described scheme of the embodiments of this application, near-infrared data of the frontal lobe and / or temporal lobe of the examinee can be collected to improve the representativeness of the near-infrared data and thus improve the accuracy of the recognition results. Specifically, the temporal lobe is related to language comprehension ability, while the frontal lobe is related to conversational ability; therefore, the frontal and temporal lobes are continuously activated during the examinee's performance of a language fluency task. Related studies have shown that changes in hemoglobin concentration characterized by near-infrared data of the frontal and / or temporal lobes during a language fluency task have high specificity for the recognition of mental status. Therefore, obtaining recognition results based on near-infrared data of at least one brain region, such as the frontal or temporal lobe, can help improve the accuracy of the recognition of the examinee's mental status. Preferably, the near-infrared data includes near-infrared data of the frontal and temporal lobes to achieve more accurate recognition of mental status.

[0152] For example, the identification result of the above identification device can be used to indicate whether the subject is a healthy person or has at least one of the following conditions: schizophrenia, depression, anxiety disorder, bipolar disorder.

[0153] In one example, the network model's output could identify whether the subject is healthy. In other words, in this example, the network model might focus on whether the subject's mental state is normal, rather than whether the subject has a specific disease.

[0154] In a better example, the network model's output can be any one or more of the following as the identification result: whether the subject has schizophrenia, depression, anxiety, or bipolar disorder. Specifically, for example, if the identification result is used to indicate whether the subject has depression, the preset objects used by the network model during the training phase can include patients with depression and healthy individuals, thus the network model can identify whether the subject has depression. As another example, if the identification result is used to indicate whether the subject has depression and anxiety, the preset objects used by the network model during the training phase can include patients with depression, patients with anxiety, and healthy individuals. In other words, the identification module of the network model can output the confidence level that the subject has depression, the confidence level that the subject has anxiety, and the confidence level that the subject is healthy, respectively. Considering practical application scenarios, patients with mental illnesses often have multiple concurrent symptoms, so this example can comprehensively identify the subject's mental state. In another embodiment, the identification result output by the network model can be the probability that the subject has a certain mental illness, rather than an absolute judgment of whether the subject has a certain mental illness.

[0155] It should be understood that the above list of symptoms that affect the mental state of the examinee is not exhaustive and may vary depending on the specific circumstances.

[0156] According to the above-described scheme in the embodiments of this application, the identification results can be finely divided to indicate whether the examinee is a healthy person or has a disease that affects the examinee's mental state, which is helpful to assist doctors in making a more accurate assessment of the examinee's mental state.

[0157] For example, the processor 102 may perform step S2120, which represents near-infrared data using an image to be inspected, and may include performing steps S2121 to S2123.

[0158] In step S2121, for each target acquisition channel, the near-infrared data of each target acquisition channel is normalized using a first preset value and a second preset value as the normalization boundary.

[0159] In step S2122, based on the normalized near-infrared data, a sub-image to be inspected corresponding to each target acquisition channel is generated.

[0160] In step S2123, the image to be inspected is determined based on the sub-image to be inspected corresponding to each target acquisition channel.

[0161] Specifically, both the first and second preset values ​​are pre-set values, not obtained from the near-infrared data of the current subject's target acquisition channel. These first and second preset values ​​can be used as the maximum and minimum values ​​in the normalization process, respectively. This ensures that all target acquisition channels and different subjects use the same maximum and minimum values ​​for normalization, i.e., the same scale is used. Furthermore, this normalization method in this embodiment, which considers the subject's brain activation status globally, avoids losing information about differences in brain activation between different target acquisition channels, the subject, and healthy individuals or patients with mental illnesses. This improves the accuracy of the recognition results.

[0162] In a preferred embodiment, the first and second preset values ​​are obtained based on a preset time-series dataset, which includes near-infrared data of healthy individuals and at least one type of mental illness patient. That is, the first and second preset values ​​obtained based on the preset time-series dataset can be used as the maximum and minimum values ​​in the normalization process, respectively, serving as the normalization boundaries to normalize the near-infrared data of each target acquisition channel. In this way, the differences between the examinee and mental illness patients are considered during the normalization process, improving the accuracy and reliability of the obtained classification results.

[0163] It should be noted that the "first preset value" and "second preset value" in this application can be the maximum and minimum values ​​of data directly obtained from the aforementioned preset time-series dataset, or they can be the maximum and minimum values ​​obtained based on the data in the aforementioned preset time-series dataset. For example, the data in the preset time-series dataset can be sorted from smallest to largest, and the maximum and minimum values ​​can be determined based on a preset proportion of data. For instance, when the preset proportion is 5%, the average value of the first 5% (i.e., low values) of data is taken and this average value is taken as the minimum value (i.e., the second preset value), and the average value of the last 5% (i.e., high values) of data is taken and this average value is taken as the maximum value (i.e., the first preset value). Specifically, when performing the above normalization processing on the hemoglobin concentration change value in the near-infrared data of the examinee, the aforementioned maximum and minimum values ​​should also be the corresponding hemoglobin concentration change values.

[0164] In some other embodiments, the "first preset value" and "second preset value" can also be manually set. It is understood that the manually set values ​​should be correlated with the maximum values ​​obtained from a large sample including healthy individuals and patients with mental illnesses to ensure the accuracy of identification. In summary, all target acquisition channels for the subject use the same first and second preset values ​​for normalization to ensure that the normalization scale is the same.

[0165] Specifically, the pre-defined time-series dataset includes near-infrared data of healthy individuals and at least one type of mental illness patient. It should be understood that the healthy individuals and at least one type of mental illness patient included in the pre-defined time-series dataset are different from the examinee. The examinee is the subject whose mental condition is to be identified, while the healthy individuals and at least one type of mental illness patient are respectively diagnosed as healthy and suffering from a certain mental illness. Specifically, the at least one type of mental illness patient included in the pre-defined time-series dataset should be associated with the mental condition to be identified. For example, when the network model wants to identify a examinee's tendency to have depression, the at least one type of mental illness patient should include at least one patient with depression; when the network model wants to identify a examinee's tendency to have schizophrenia, the at least one type of mental illness patient should include at least one patient with schizophrenia.

[0166] The near-infrared data of the healthy subject and the near-infrared data of the patient with at least one type of mental illness can be obtained based on the optical density data of the healthy subject and the patient with at least one type of mental illness, respectively, through the relevant steps in step S2110 above. This embodiment of the application will not elaborate further here. It should be understood that the data acquisition channel corresponding to the near-infrared data in the preset time-series dataset may include the target acquisition channel used by the subject in step S2120.

[0167] In a preferred example, a change in hemoglobin concentration x in the near-infrared data of the subject. iThe hemoglobin concentration change value with a distribution range between [-1, 1] can be obtained by normalization using formula (1) above, which will not be elaborated further here. In some embodiments, the near-infrared data before normalization can also be compressed to reduce the computational load of normalization calculation. For example, the near-infrared data before normalization can be compressed using the Piecewise Aggregate Approximation (PAA) algorithm or an algorithm based on Huffman coding.

[0168] In step S2122, based on the normalized near-infrared data, a sub-image to be inspected corresponding to each target acquisition channel is generated. Referring to the relevant steps in step S2120 above, for each target acquisition channel, the normalized near-infrared data corresponding to that target acquisition channel is represented in the form of an image, which can then serve as the sub-image to be inspected corresponding to that target acquisition channel. This embodiment of the application will not elaborate further here.

[0169] In step S2123, the image to be inspected is determined based on the sub-images to be inspected corresponding to each target acquisition channel. In one example, multiple sub-images to be inspected can be stitched together along preset dimensions of the image to obtain the image to be inspected. For example, the preset dimensions may include the horizontal dimension, vertical dimension, and depth dimension of the image. For example, if the image size of the sub-images to be inspected is 128*128 (in pixels) and there are 4 target acquisition channels, then stitching along the vertical dimension will result in an image to be inspected with a size of 128*512. Alternatively, stitching along the horizontal dimension will result in an image to be inspected with a size of 512*128. Alternatively, stitching along the depth dimension will result in an image to be inspected with a size of 128*128*4. It should be understood that multiple preset dimensions can also be combined to stitch together the sub-images to be inspected. For example, several sub-images to be inspected can be stitched together along the horizontal dimension and several sub-images to be inspected along the vertical dimension. For example, two sub-images to be inspected can be stitched together in the horizontal dimension and two sub-images to be inspected in the vertical dimension to obtain an image to be inspected with a size of 256*256. The number of images stitched in the above-mentioned preset dimensions can be determined according to the actual needs of the developers, and this embodiment of the application does not impose any limitations.

[0170] According to the above scheme of this embodiment, for each target acquisition channel, the near-infrared data of each target acquisition channel can be normalized using a first preset value and a second preset value obtained based on a preset time-series dataset as normalization boundaries. Then, based on the normalized near-infrared data, a sub-image to be examined corresponding to each target acquisition channel can be generated. Finally, based on the sub-images to be examined corresponding to each target acquisition channel, the image to be examined can be determined. The above-mentioned near-infrared data is normalized using preset normalization boundaries, which can further improve the representativeness of the image to be examined and is conducive to further improving the accuracy of the recognition results. The image to be examined obtained in this embodiment of the present application considers the brain activation information of the subject from a global perspective, thereby improving the accuracy of the recognition results.

[0171] Furthermore, the first and second preset values ​​in this embodiment are not based on near-infrared data from a single healthy subject or a patient with a mental illness. Thus, the differences between the test subject and healthy subjects or patients with mental illnesses (e.g., patients with different types of mental illnesses) are considered during normalization. In particular, when training the network model using training images obtained from a preset time-series dataset, this difference information can be captured by the network model, resulting in higher accuracy in identifying the mental state of the test subject using the trained network model.

[0172] For example, the type of preset time-series data in the preset time-series dataset is the same as the type of near-infrared data acquired during the subject's performance of the language fluency task. Specifically, the type of hemoglobin concentration may include: oxyhemoglobin concentration (HBO), deoxyhemoglobin concentration (HBR), and total hemoglobin concentration (HBT). If the type of preset time-series data in the preset time-series dataset is deoxyhemoglobin concentration, then the type of near-infrared data acquired during the subject's performance of the language fluency task should also be the deoxyhemoglobin concentration change value. Preferably, in the embodiments of this application, the hemoglobin concentration change value may only be the oxyhemoglobin concentration change value. Studies have confirmed that, compared to the other two types of data, the image to be examined converted from the oxyhemoglobin concentration change value during the subject's performance of the language fluency task, when input into a network model capable of capturing the correlation features between target acquisition channels, yields a higher accuracy in recognition results.

[0173] For example, the processor 102 executes step S2122, which generates a sub-image to be inspected corresponding to each target acquisition channel based on the normalized near-infrared data, and may include executing steps S21221 and S21222.

[0174] In step S21221, for the normalized near-infrared data of each target acquisition channel, a Gram angle difference field map and a Gram angle sum field map corresponding to that target acquisition channel are generated.

[0175] In step S1222, for each target acquisition channel, the Gram angle difference field map and the Gram angle sum field map corresponding to that target acquisition channel are stitched together to obtain the sub-image to be inspected corresponding to that target acquisition channel. The specific methods of steps S21221 and S1222 can be found in formulas (2) and (3) above, and will not be repeated here.

[0176] In some embodiments, based on the stitching method of the sub-image to be inspected in step S123 above, for each target acquisition channel, the Gram angle difference field map and the Gram angle sum field map corresponding to the target acquisition channel can be stitched together to obtain the sub-image to be inspected corresponding to the target acquisition channel.

[0177] According to the above-described scheme of the embodiments of this application, for the normalized near-infrared data of each target acquisition channel, a Gram difference field map and a Gram sum field map corresponding to that target acquisition channel can be generated. Then, the Gram difference field map and the Gram sum field map corresponding to that target acquisition channel are stitched together to obtain the sub-image to be examined corresponding to that target acquisition channel. For example, the GADF and GASF generated for each target acquisition channel are stored alternately to finally obtain the sub-image to be examined corresponding to that target acquisition channel.

[0178] In the above scheme, for each target acquisition channel, the temporal correlation between multiple hemoglobin concentration changes in the normalized near-infrared data of that target acquisition channel can be represented by the Gram difference field map and the Gram sum field map. This scheme uses the Gram difference field map and the Gram sum field map, which are more concerned with the temporal correlation between hemoglobin concentration changes, and this has been verified to improve the accuracy of the recognition results. Furthermore, the feature information of the Gram difference field map and the Gram sum field map is complementary. The image obtained by stitching the two together can provide more comprehensive feature information about the subject's mental state, which is beneficial to improving the accuracy of the recognition results.

[0179] For example, when processor 102 executes step S2130, the network model capable of capturing the correlation features between target acquisition channels and used to obtain the recognition result of the examinee's mental state can be a shuffling network model. This shuffling network model includes a sequentially connected feature extraction module, multiple shuffling modules, and a recognition module. For details, please refer to the shuffling network model described above; it will not be repeated here.

[0180] Optionally, the training process of the network model can be as follows: Several training images in the training dataset are divided into multiple batches. The training images may have identification labels, which can be used to indicate whether the mental state of the preset object is good or whether it suffers from a certain mental illness. Near-infrared data of the preset object can be obtained through the relevant steps of step S2110. Then, training images of the preset object are generated through the relevant steps of step S2120 above, which will not be elaborated here. It should be understood that the generation methods of the near-infrared data of the examinee and the near-infrared data of the preset object can be consistent, and the generation methods of the images to be examined and the training images can be consistent. Further, the data acquisition channel corresponding to the near-infrared data of the preset object can at least include the target acquisition channel corresponding to the near-infrared data of the examinee. It should be understood that near-infrared data of the preset object with high specificity for mental state recognition can be used, thus improving the representativeness of the images to be examined generated based on the near-infrared data of the examinee. In some embodiments, the data acquisition channel corresponding to the near-infrared data of the preset object can cover multiple brain regions of interest. Using near-infrared data from different brain regions of interest of the examinee for identification is beneficial for identifying various mental illnesses that affect brain activation changes in different brain regions, and can improve the comprehensiveness of the identification results.

[0181] For example, training images from the same batch can be sequentially input into an untrained network model to obtain prediction results corresponding to the training images. The recognition labels corresponding to the training images and the prediction results corresponding to those training images are calculated using a preset loss function to obtain a loss value. The model parameters of the untrained network model are adjusted based on this loss value. Then, another batch of training images is used to train the adjusted network model until training is complete, resulting in the network model used in step S2130.

[0182] For example, the training images input during network model training are image representations of the training time-series data of a preset object. The training time-series data is obtained by normalizing the near-infrared data of the preset object using a first preset value and a second preset value obtained based on the preset time-series dataset as normalization boundaries. Training images can be generated based on the normalized near-infrared data of the preset object through the relevant steps in step S2120 above. The preset object may include healthy individuals and at least one type of mental illness patient.

[0183] According to the above-described scheme of the embodiments of this application, training time-series data can be obtained by normalizing the near-infrared data of a preset object. Then, based on the training time-series data, more representative training images can be generated, thereby improving the learning effect of the network model and enhancing the accuracy of the recognition results.

[0184] In one specific embodiment, the identification result can be used to indicate whether the subject has a tendency towards schizophrenia. The inventors verified the identification of schizophrenic tendencies using the scheme of the embodiments of this application, and the verification results are specifically shown in Table 2 above.

[0185] The validation results in Table 2 show that the evaluation results of ShuffleNet, which can capture the correlation features between target acquisition channels, are better than those of DenseNet and MobileNetV2. In other words, ShuffleNet, which can capture the correlation features between target acquisition channels, performs better in classifying schizophrenic individuals.

[0186] It is foreseeable that when the identification results are used to represent the mental state of the examinee, or to specifically indicate whether the examinee is a healthy person or has at least one of the above conditions such as schizophrenia, depression, anxiety, bipolar disorder, etc., using a network model that can capture the correlation features between target acquisition channels can improve the accuracy of the identification results. Examples of this application will not be listed here.

[0187] In addition, the inventors also verified the classification results obtained by classifying traditional features such as HBO curves, brain functional connectivity, and dynamic functional connectivity extracted from near-infrared data using classifiers such as SVM and LDA. See Table 1 above. The results show that LDA has a relatively good classification effect on traditional features such as HBO curves, brain functional connectivity, and dynamic functional connectivity. However, the classification effect of the LDA classifier is far inferior to the classification effect of identifying schizophrenic tendencies obtained by converting near-infrared data into the above-mentioned image to be examined and inputting the above-mentioned image to be examined into the shuffle network in the embodiments of this application.

[0188] The inventors verified the recognition effect of the normalization method presented in this paper; the verification results are available in [link to verification]. Figures 2(a)-2(c) as well as Figures 3(a)-3(c) This will not be elaborated further here. The validation results show that the normalization method presented in this paper can preserve the differences in brain activation between healthy subjects and patients with schizophrenia, preventing these differences from being weakened (or even eliminated), thus improving the recognition effect.

[0189] Furthermore, the verification results in Table 2 above show that the classification effect obtained by inputting the image to be examined obtained by the normalization processing method in this paper into the ShuffleNetV2 network, which can capture the correlation features between target acquisition channels, is far superior to the classification effect obtained by inputting the image to be examined obtained by the normalization method in the prior art into the ShuffleNetV2 network.

[0190] According to another aspect of this application, a method for identifying the mental state of a subject based on near-infrared data is also provided. The method includes: acquiring near-infrared data of the subject during a language fluency task; representing the near-infrared data using an image to be examined, wherein pixels in the image to be examined can characterize the relationship between data at different times in the near-infrared data; and obtaining the identification result of the subject's mental state based on the image to be examined using a network model capable of capturing the correlation features between target acquisition channels. Specifically, the method may also include the steps executed by the processor as described in any embodiment of this application, which can be executed using a processor.

[0191] According to another aspect of this application, such as Figure 11 As shown, a system for identifying the mental state of a subject based on near-infrared data is also provided, including a data acquisition module and the aforementioned device for identifying the mental state of a subject based on near-infrared data. The data acquisition module is configured to acquire light intensity change data during the subject's performance of a language fluency task to generate the subject's near-infrared data. The data acquisition module includes, for example, the aforementioned headgear. The headgear may contain a transmitting probe and a receiving probe. Based on the above description, those skilled in the art will understand the structure and function of the data acquisition module; for simplicity, further details are omitted here.

[0192] According to another aspect of this application, a storage medium is also provided, on which program instructions are stored. When the program instructions are executed by a computer or processor, the computer or processor performs the corresponding steps of the method for identifying the mental state of a subject based on near-infrared data according to the embodiments of this application, and is used to implement the corresponding module in the device for identifying the mental state of a subject based on near-infrared data according to the embodiments of this application, or the corresponding module in the electronic device. The storage medium may, for example, include a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0193] According to another aspect of this application, a computer program product is also provided, including computer program instructions, which, when executed by a computer or processor, cause the computer or processor to perform corresponding steps of the method for identifying the mental state of a subject based on near-infrared data in any of the above embodiments.

[0194] The processors 101 and 102 in this application can be processing devices that include more than one general-purpose processing device, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor that runs other instruction sets, or a processor that runs a combination of instruction sets. The processor can also be one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), system-on-a-chip (SoCs), etc.

[0195] Those skilled in the art can understand the method for assessing schizophrenic tendencies based on near-infrared data and the specific implementation scheme of the storage medium by reading the above description of the device for assessing schizophrenic tendencies based on near-infrared data. For the sake of brevity, it will not be described in detail here.

[0196] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0197] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0198] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0199] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0200] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0201] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0202] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0203] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the apparatus for assessing schizophrenic tendencies based on near-infrared data according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0204] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0205] The above description is merely a specific embodiment of the present invention or an explanation of that embodiment. The scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A device for assessing schizophrenic tendencies based on near-infrared data, characterized in that, The device includes a processor. The processor is configured as follows: Acquire near-infrared data of the object under test; The near-infrared data is processed and converted into an image to be measured. The processing includes normalization processing based on the maximum and minimum values ​​obtained from a preset data set. The preset data set includes a second data sequence of healthy subjects and schizophrenic individuals distributed over time. The second data sequence is obtained from the near-infrared data of the healthy subjects and schizophrenic individuals. The image to be tested is input into a classification model to obtain a classification result for the object to be tested, wherein the classification result indicates the tendency of the object to be tested to be a healthy person or a person with schizophrenia.

2. The apparatus as claimed in claim 1, characterized in that, The processor processes the near-infrared data and converts the processed near-infrared data into an image to be measured, including performing the following operations: Based on the near-infrared data, multiple first data sequences are determined, wherein different first data sequences correspond to different acquisition channels; The first data sequence A among the plurality of first data sequences i The image is converted into a first image to be used as part of the image under test, where i is a positive integer less than or equal to the total number of acquisition channels.

3. The apparatus as described in claim 1 or 2, characterized in that, Each first data sequence includes multiple hemoglobin concentration changes distributed over time. The normalization process includes the following operations: Based on the maximum and minimum values ​​obtained from the preset data set, respectively, the first data sequence A is... i Normalization is performed to normalize the first data sequence A. i The data is converted into a first image, wherein the preset data set includes hemoglobin concentration changes distributed over time for healthy individuals and individuals with schizophrenia.

4. The apparatus as claimed in claim 2, characterized in that, The processor will process the first data sequence A from the plurality of first data sequences. i Converting to a first image includes performing the following operations: Based on the first data sequence A i Generate the first data sequence A i The corresponding Gram angle difference field diagram and Gram angle sum field diagram; The first data sequence A i The corresponding Gram angle difference field map and Gram angle sum field map are stitched together to form the first image.

5. The apparatus as claimed in claim 1, characterized in that, The classification model is a shuffling network, which includes a feature extraction module, multiple shuffling modules, a convolutional module, and a classification module. The processor inputs the image to be tested into the classification model to obtain the classification result of the object to be tested, including performing the following operations: The image to be tested is input into the feature extraction module to obtain the first image feature of the image to be tested; The shuffling module divides the first image features or rearranged features obtained by other shuffling modules into two groups of data. A convolution operation is performed on one of the two groups of data. The convolutional group of data is then concatenated with the other group of data before a rearrangement operation is performed to obtain rearranged features. The rearranged features obtained by the shuffling module connected to the convolution module are convolved to obtain the first convolution feature. The classification module performs a classification operation on the first convolutional features to obtain the classification result of the object to be tested.

6. The apparatus according to any one of claims 1 to 5, characterized in that, The near-infrared data is collected when the subject performs a language fluency task, wherein the language fluency task includes a first time period before the subject performs a word grouping task, a second time period after performing the word grouping task, and a third time period after performing the word grouping task.

7. The apparatus as claimed in claim 6, characterized in that, The processor is further configured as follows: Based on the near-infrared data in the first time period, the second time period, and the third time period, the first data sequence is generated for the first time period, the second time period, and the third time period, respectively. The first data sequence is calibrated based on the data from the first time period and the third time period.

8. The apparatus according to any one of claims 1 to 5, characterized in that, The processor is further configured as follows: For each data sequence of the near-infrared data, the coefficient of variation is calculated, where each data sequence corresponds to near-infrared data acquired through different acquisition channels; Determine whether the calculated coefficient of variation is greater than a preset coefficient threshold; The step of generating the image to be tested corresponding to the near-infrared data based on the near-infrared data is performed only when the number of coefficients of variation greater than the preset coefficient threshold is less than the preset number threshold.

9. A method for assessing schizophrenic tendencies based on near-infrared data, characterized in that, The method includes: Acquire near-infrared data of the object under test; The near-infrared data is processed and converted into an image to be measured. The processing includes normalization processing based on the maximum and minimum values ​​obtained from a preset data set. The preset data set includes a second data sequence of healthy subjects and schizophrenic individuals distributed over time. The second data sequence is obtained from the near-infrared data of the healthy subjects and schizophrenic individuals. The image to be tested is input into a classification model to obtain a classification result for the object to be tested, wherein the classification result indicates the tendency of the object to be tested to be a healthy person or a person with schizophrenia.

10. A storage medium storing a computer program / instructions for assessing schizophrenic tendencies based on near-infrared data, characterized in that, The computer program / instructions are used to execute the following during runtime: Acquire near-infrared data of the object under test; The near-infrared data is processed and converted into an image to be measured. The processing includes normalization processing based on the maximum and minimum values ​​obtained from a preset data set. The preset data set includes a second data sequence of healthy subjects and schizophrenic individuals distributed over time. The second data sequence is obtained from the near-infrared data of the healthy subjects and schizophrenic individuals. The image to be tested is input into a classification model to obtain a classification result for the object to be tested, wherein the classification result indicates the tendency of the object to be tested to be a healthy person or a person with schizophrenia.

11. A device for identifying the mental state of a subject based on near-infrared data, characterized in that, Including processors, The processor is configured as follows: Acquire near-infrared data of subjects during the performance of a language fluency task; The near-infrared data is represented using an image to be inspected, wherein the pixels in the image to be inspected can characterize the relationship between data at different times in the near-infrared data; Based on the image to be examined, the identification result of the examinee's mental state is obtained by using a network model that can capture the correlation features between target acquisition channels.

12. The apparatus as claimed in claim 11, characterized in that, The processor represents the near-infrared data using the image to be inspected, including performing the following operations: For each target acquisition channel, the near-infrared data of each target acquisition channel is normalized based on a first preset value and a second preset value as the normalization boundary, wherein the first preset value is greater than the second preset value. Based on the normalized near-infrared data, generate the sub-image to be inspected corresponding to each target acquisition channel; The image to be inspected is determined based on the sub-image to be inspected corresponding to each target acquisition channel.

13. The apparatus as claimed in claim 12, characterized in that, The first preset value and the second preset value are obtained based on a preset time series dataset, wherein the preset time series dataset includes near-infrared data of healthy subjects and at least one type of mental illness patients.

14. The apparatus as claimed in claim 12 or 13, characterized in that, The process of generating the sub-image to be inspected for each target acquisition channel based on the normalized near-infrared data includes the following operations: For the normalized near-infrared data of each target acquisition channel, generate the Gram angle difference field map and Gram angle sum field map corresponding to that target acquisition channel; For each target acquisition channel, the Gram angle difference field map and Gram angle sum field map corresponding to that target acquisition channel are stitched together to obtain the sub-image to be inspected corresponding to that target acquisition channel.

15. The apparatus as claimed in claim 11, characterized in that, The near-infrared data includes near-infrared data from the frontal lobe and / or temporal lobe.

16. The apparatus according to any one of claims 11-15, characterized in that, The network model is a shuffling network model, which includes a sequentially connected feature extraction module, multiple shuffling modules, and a recognition module.

17. The apparatus according to any one of claims 11-16, characterized in that, The training images input during network model training are image representations of training time-series data of preset objects. The preset objects include healthy objects and at least one type of mental illness patient. The training time-series data is obtained by normalizing the near-infrared data of the preset objects based on a first preset value and a second preset value as normalization boundaries.

18. The apparatus according to any one of claims 11-16, characterized in that, The identification result is used to indicate whether the subject is a healthy person or has at least one of the following conditions: schizophrenia, depression, anxiety disorder, and bipolar disorder.

19. A system for identifying the mental state of a subject based on near-infrared data, characterized in that, It includes a data acquisition module and the device for identifying the mental state of a subject based on near-infrared data as described in claim 1, wherein, The acquisition module is configured to acquire light intensity change data during the subject's performance of a language fluency task in order to generate the subject's near-infrared data.

20. A storage medium storing a computer program / instructions for identifying the mental state of a subject based on near-infrared data, characterized in that, The computer program / instructions are used to execute the following during runtime: Acquire near-infrared data of subjects during the performance of a language fluency task; The near-infrared data is represented using an image to be inspected, wherein the pixels in the image to be inspected can characterize the relationship between data at different times in the near-infrared data; Based on the image to be examined, the identification result of the examinee's mental state is obtained by using a network model that can capture the correlation features between target acquisition channels.

Images