Unmanned system brain control method and device based on image semantic segmentation

Abstract

The invention discloses an unmanned system brain control method and device based on image semantic segmentation. The method comprises the steps that unmanned system image information and SSVEP frequency information are acquired; the unmanned system image information comprises unmanned system initial model information and an unmanned system training data set; the SSVEP frequency information comprises a plurality of SSVEP frequencies; processing the unmanned system image information to obtain unmanned system optimization model information; and using the unmanned system optimization model information and the SSVEP frequency information to control the unmanned system. Therefore, the fatigue feeling can be effectively eliminated by using high-frequency visual stimulation, the naturalness of man-machine interaction can be improved by fusing the image semantic segmentation technology, and meanwhile, the high efficiency and the accuracy of a man-machine interaction system can be improved.

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interfaces, and in particular, to a brain control method and device for an unmanned system based on image semantic segmentation. Background Art

[0002] Among all categories of brain-computer interfaces (BCIs), visual brain-computer interfaces perform best in terms of interaction speed and reliability. The highest information transmission rate of visual brain-computer interfaces can reach 376.58 bits per minute, which has a relatively high transmission rate compared with other types of brain-computer interface systems. Usually, a visual BCI (such as a low-frequency SSVEP-BCI system) requires the user to directly gaze at the flashing stimuli within the region of interest. However, long-term gazing at the relevant region is likely to cause problems such as human visual fatigue and excessive consumption of visual memory resources. In addition, the input method of traditional SSVEP-BCI systems is relatively rigid, does not conform to the habits of natural human interaction, has a poor match with users, and the interaction process is not natural enough.

[0003] To solve the above problems, the present invention proposes a brain control method and device for an unmanned system based on image semantic segmentation. Using high-frequency visual stimuli can effectively eliminate the sense of fatigue, and integrating image semantic segmentation technology can improve the naturalness of human-computer interaction, while being beneficial to enhancing the efficiency and accuracy of the human-computer interaction system. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a brain control method and device for an unmanned system based on image semantic segmentation. Using high-frequency visual stimuli can effectively eliminate the sense of fatigue, and integrating image semantic segmentation technology can improve the naturalness of human-computer interaction, while being beneficial to enhancing the efficiency and accuracy of the human-computer interaction system.

[0005] To solve the above technical problem, in the first aspect of an embodiment of the present invention, a brain control method for an unmanned system based on image semantic segmentation is disclosed. The method includes:

[0006] S1, obtaining unmanned system image information and SSVEP frequency information; the unmanned system image information includes unmanned system initial model information and an unmanned system training data set; the SSVEP frequency information includes a plurality of SSVEP frequencies;

[0007] S2, processing the unmanned system image information to obtain unmanned system optimized model information;

[0008] S3, controlling the unmanned system by using the unmanned system optimized model information and the SSVEP frequency information.

[0009] As an alternative implementation, in the first aspect of the embodiments of the present invention, the processing of the unmanned system image information to obtain the unmanned system optimization model information includes:

[0010] S21, preprocess the unmanned system training data set to obtain a preprocessed unmanned system training data set;

[0011] S22, use the preprocessed unmanned system training data set to perform training processing on the initial unmanned system model information to obtain the unmanned system optimization model information.

[0012] As an alternative implementation, in the first aspect of the embodiments of the present invention, the use of the preprocessed unmanned system training data set to perform training processing on the initial unmanned system model information to obtain the unmanned system optimization model information includes:

[0013] S221, use the preprocessed unmanned system training data set to perform training processing on the initial unmanned system model information to obtain the unmanned system training result information and the unmanned system training model information;

[0014] S222, perform calculation processing on the unmanned system training result information to obtain the unmanned system loss function value;

[0015] S223, determine whether the unmanned system loss function value is less than a preset loss function threshold to obtain a first determination result;

[0016] When the first determination result is negative, determine that the unmanned system training model information is the initial unmanned system model information, and execute S221;

[0017] When the first determination result is positive, determine that the unmanned system training model information is the unmanned system optimization model information.

[0018] As an alternative implementation, in the first aspect of the embodiments of the present invention, the use of the unmanned system optimization model information and the SSVEP frequency information to control the unmanned system includes:

[0019] S31, obtain the unmanned system image information; the unmanned system image information includes the first stimulation area information and the second stimulation area information;

[0020] S32, in response to the trigger of the user, obtain the first electroencephalogram signal information;

[0021] S33, perform parsing processing on the first electroencephalogram signal information to obtain the first electroencephalogram parsing signal value;

[0022] S34. Determine whether the first EEG analysis signal value matches the SSVEP frequency corresponding to the first stimulation area information, and obtain a second judgment result;

[0023] When the second judgment result is yes, execute S35;

[0024] When the second judgment result is no, end the control of the unmanned system;

[0025] S35. Use the unmanned system optimization model information, the SSVEP frequency information, and the unmanned system image information to control the unmanned system.

[0026] As an optional implementation manner, in the first aspect of the embodiments of the present invention, the parsing and processing of the first EEG signal information to obtain a first EEG analysis signal value includes:

[0027] S331. Perform first feature extraction on the first EEG signal information to obtain first EEG feature information;

[0028] S332. Perform second feature extraction on the first EEG signal information to obtain second EEG feature information;

[0029] S333. Use the unmanned system fusion calculation model to perform calculation processing on the first EEG feature information and the second EEG feature information to obtain a first EEG analysis signal value;

[0030] Among them, the unmanned system fusion calculation model is:

[0031] ND = θ1·ND1 + θ2·ND2;

[0032] 0 ≤ θ1, θ2 ≤ 1;

[0033] θ1 + θ2 = 1;

[0034] Among them, ND is the first EEG analysis signal value, and θ1 and θ2 are the first weight parameter and the second weight parameter respectively.

[0035] As an optional implementation manner, in the first aspect of the embodiments of the present invention, the using the unmanned system optimization model information, the SSVEP frequency information, and the unmanned system image information to control the unmanned system includes:

[0036] S351. Use the unmanned system optimization model information to perform calculation processing on the unmanned system image information to obtain semantic recognition area information;

[0037] S352. Determine whether the type information corresponding to the semantic recognition area information matches the first type information, and obtain a third judgment result;

[0038] When the third judgment result is yes, execute S353;

[0039] When the third judgment result is no, determine whether the type information corresponding to the semantic recognition area information matches the second type information, and obtain a fourth judgment result;

[0040] When the fourth judgment result is yes, execute S354;

[0041] When the fourth judgment result is no, execute S355;

[0042] S353, using the SSVEP frequency information and the unmanned system image information, perform a first control on the unmanned system, and execute S32;

[0043] S354, using the SSVEP frequency information and the unmanned system image information, perform a second control on the unmanned system, and execute S32;

[0044] S355, using the SSVEP frequency information and the unmanned system image information, perform a third control on the unmanned system, and execute S32.

[0045] As an optional implementation manner, in the first aspect of the embodiments of the present invention, the performing a first control on the unmanned system by using the SSVEP frequency information and the unmanned system image information includes:

[0046] S3531, process the SSVEP frequency information and the unmanned system image information to obtain third stimulation area information and fourth stimulation area information;

[0047] S3532, in response to a user's trigger, obtain second electroencephalogram signal information;

[0048] S3533, perform parsing processing on the second electroencephalogram signal information to obtain a second electroencephalogram parsing signal value;

[0049] S3534, determine whether the second electroencephalogram parsing signal value matches the SSVEP frequency corresponding to the third stimulation area information, and obtain a fourth judgment result;

[0050] When the fourth judgment result is yes, control the unmanned system to perform a takeoff operation;

[0051] When the fourth judgment result is no, control the unmanned system to perform a descending operation.

[0052] The second aspect of the embodiments of the present invention discloses a brain control device for an unmanned system based on image semantic segmentation, and the device includes:

[0053] An acquisition module for acquiring unmanned system image information and SSVEP frequency information; the unmanned system image information includes unmanned system initial model information and an unmanned system training data set; the SSVEP frequency information includes a plurality of SSVEP frequencies;

[0054] A calculation module for processing the unmanned system image information to obtain unmanned system optimized model information;

[0055] A control module for controlling the unmanned system by using the unmanned system optimized model information and the SSVEP frequency information.

[0056] A third aspect of an embodiment of the present invention discloses another brain-controlled device for an unmanned system based on image semantic segmentation, and the device includes:

[0057] A processor;

[0058] A memory coupled to the processor and storing executable program code;

[0059] The processor calls the executable program code stored in the memory to execute some or all of the steps of the brain control method for an unmanned system based on image semantic segmentation disclosed in the first aspect of an embodiment of the present invention.

[0060] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium, and the computer-readable storage medium stores computer instructions, which are used to execute some or all of the steps of the brain control method for an unmanned system based on image semantic segmentation disclosed in the first aspect of an embodiment of the present invention when being called.

[0061] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0062] In the embodiments of the present invention, unmanned system image information and SSVEP frequency information are acquired; the unmanned system image information includes unmanned system initial model information and an unmanned system training data set; the SSVEP frequency information includes a plurality of SSVEP frequencies; the unmanned system image information is processed to obtain unmanned system optimized model information; the unmanned system is controlled by using the unmanned system optimized model information and the SSVEP frequency information. It can be seen that using high-frequency visual stimuli can effectively eliminate the sense of fatigue, and integrating the image semantic segmentation technology can improve the naturalness of human-computer interaction, and at the same time is beneficial to improving the efficiency and accuracy of the human-computer interaction system.

[0063] Through the solution of the present invention, the operation flexibility of the drone can be improved, more natural human-machine interaction can be achieved, and at the same time, the fatigue and operation burden brought by the traditional BCI system can be reduced. This brain control method combining image understanding and visual stimulation opens up new possibilities for the intelligent control of drones, enabling users to interact with drones in a more intuitive and immersive way. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0065] Figure 1 Schematic flowchart of a brain control method for an unmanned system based on image semantic segmentation disclosed in an embodiment of the present invention;

[0066] Figure 2 Schematic diagram of the takeoff and landing control of a drone disclosed in an embodiment of the present invention;

[0067] Figure 3 Schematic diagram of the flight control of a drone disclosed in an embodiment of the present invention;

[0068] Figure 4 Schematic diagram of the structure of a brain control device for an unmanned system based on image semantic segmentation disclosed in an embodiment of the present invention;

[0069] Figure 5 Schematic diagram of the structure of another brain control device for an unmanned system based on image semantic segmentation disclosed in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] In order to enable those skilled in the art to better understand the solution of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0071] In the description, claims and the above-mentioned drawings of the present invention, terms such as "first", "second", etc. are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or equipment.

[0072] Reference to "embodiment" herein means that a particular feature, structure or characteristic described in connection with the embodiment can be included in at least one embodiment of the present invention. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0073] The present invention discloses a brain control method and device for an unmanned system based on image semantic segmentation. Using high-frequency visual stimuli can effectively eliminate fatigue. Integrating image semantic segmentation technology can improve the naturalness of human-computer interaction, and at the same time is beneficial to improving the efficiency and accuracy of the human-computer interaction system. The following will be described in detail respectively.

[0074] Embodiment 1

[0075] Please refer to Figures 1 - 3 , Figure 1 which is a schematic flowchart of a brain control method for an unmanned system based on image semantic segmentation disclosed in an embodiment of the present invention. Among them, Figure 1 the described brain control method for an unmanned system based on image semantic segmentation is applied to a brain control device for an unmanned system based on image semantic segmentation, such as a local server or a cloud server for optimizing management of a brain control for an unmanned system based on image semantic segmentation, etc., which is not limited in the embodiments of the present invention. As Figure 1 shown, the brain control method for an unmanned system based on image semantic segmentation may include the following operations:

[0076] S1, obtaining unmanned system image information and SSVEP frequency information; the unmanned system image information includes unmanned system initial model information and an unmanned system training data set; the SSVEP frequency information includes a plurality of SSVEP frequencies;

[0077] It should be noted that the unmanned system in the present invention may be a drone, specifically, which is not limited in the embodiments of the present invention.

[0078] It should be noted that the initial model information of the unmanned system can be a CNN model, a Transformer model, or a GAN model. Specifically, the embodiments of the present invention do not make any limitations in this regard.

[0079] It should be noted that the training dataset of the unmanned system includes diverse drone images, covering common operation scenarios to ensure the robustness of the system in complex environments. Moreover, it covers different environments and angles to ensure the generalization ability of the model.

[0080] It should be noted that several SSVEP frequencies in the SSVEP frequency information are the flashing frequencies of all the stimulation area information set in the embodiments of the present invention. The specific flashing frequencies and corresponding instructions are as follows in the table:

[0081] Table 1 SSVEP Frequencies and Instructions

[0082] Image category Frequency / HZ Instruction Lower left corner of the window 35 Start semantic segmentation Upper right corner of the window 37 End semantic segmentation Drone (actor) 39 Take off Drone (actor) 55 Land Location 1 43 Fly to this location Location 2 45 Fly to this location Location 3 47 Fly to this location Location 4 49 Fly to this location Object 1 53 Attack Object 1 41 Follow Object 2 57 Attack Object 2 40 Follow … … …

[0083] It should be noted that the frequency range of the SSVEP frequencies in the SSVEP frequency information is 35HZ - 100HZ, and the difference between the SSVEP frequencies in the SSVEP is 2HZ. The specific effects are as follows:

[0084] 1. 35 - 100HZ belongs to the high - frequency range, which can effectively avoid low - frequency interference in electroencephalogram signals, such as heartbeat (~1HZ), blinking (<10HZ), and electromyographic noise (muscle activity is usually concentrated in <30HZ).

[0085] 2. It can effectively stimulate clear SSVEP signals, improve signal decoding accuracy, and enhance the performance of the brain - computer interface.

[0086] 3. High - frequency flickering is more "smooth" perceptually, reducing visual fatigue or discomfort caused by low - frequency flickering (such as 10 - 30HZ). Lower - frequency flickering (especially 3 - 20HZ) can induce epileptic seizures in some sensitive users, while high - frequency flickering in the present invention usually does not trigger.

[0087] 4. A higher frequency range can use more frequency points (such as 35HZ, 37HZ, 55HZ, etc.), thereby improving the resolution of the command channel;

[0088] 5. The frequency interval in the above frequency column is at least 2 Hz. For example, the frequency difference between the lower left corner and the upper right corner of the window is 2 Hz. A smaller frequency interval means that more frequency points need to be processed, increasing the calculation time and resource consumption. At the same time, it will introduce additional details and noise, while a larger frequency interval will cause the characteristics of some frequency bands to be averaged, reducing the frequency resolution ability. By setting the frequency interval to 2 Hz, a good balance can be achieved between resolving signal details and calculation efficiency, meeting the main analysis requirements of EEG signals. If the interval is too low, it may lead to noise amplification and too high computational complexity; if the interval is too high, important frequency characteristics may be lost.

[0089] S2. Process the image information of the unmanned system to obtain the optimized model information of the unmanned system;

[0090] S3. Use the optimized model information of the unmanned system and the SSVEP frequency information to control the unmanned system.

[0091] It can be seen that implementing the brain control method for unmanned systems based on image semantic segmentation described in the embodiments of the present invention, using high-frequency visual stimuli can effectively eliminate the sense of fatigue, integrating image semantic segmentation technology can improve the naturalness of human-computer interaction, and at the same time is conducive to enhancing the efficiency and accuracy of the human-computer interaction system.

[0092] In an optional embodiment, processing the image information of the unmanned system to obtain the optimized model information of the unmanned system includes:

[0093] S21. Preprocess the training data set of the unmanned system to obtain the preprocessed training data set of the unmanned system;

[0094] S22. Use the preprocessed training data set of the unmanned system to perform training processing on the initial model information of the unmanned system to obtain the optimized model information of the unmanned system.

[0095] It can be seen that implementing the brain control method for unmanned systems based on image semantic segmentation described in the embodiments of the present invention, using high-frequency visual stimuli can effectively eliminate the sense of fatigue, integrating image semantic segmentation technology can improve the naturalness of human-computer interaction, and at the same time is conducive to enhancing the efficiency and accuracy of the human-computer interaction system.

[0096] In an optional embodiment, preprocessing the training data set of the unmanned system to obtain the preprocessed training data set of the unmanned system includes:

[0097] S211. Perform data cleaning on the training data set of the unmanned system to obtain the cleaned training data set of the unmanned system;

[0098] It should be noted that the above data cleaning can use tools such as Pandas and NumPy for data cleaning. Specifically, the embodiments of the present invention do not make limitations.

[0099] It should be noted that through data cleaning, images with poor quality, too low resolution or irrelevant to the task can be deleted to ensure that the semantic segmentation labels (masks) of all images are accurate and avoid annotation errors affecting model training.

[0100] S212, perform conversion processing on the cleaned unmanned system training dataset to obtain the converted unmanned system training dataset;

[0101] It should be noted that for the above conversion processing, tools such as OpenCV and ImageMagick can be used. Specifically, the embodiments of the present invention do not make limitations.

[0102] It should be noted that through the conversion processing, all the pictures in the cleaned unmanned system training dataset can be converted into a unified format, such as PNG, JPEG, etc., which is beneficial to improving the robustness of model training.

[0103] S213, perform image enhancement processing on the converted unmanned system training dataset to obtain the enhanced unmanned system training dataset;

[0104] It should be noted that for the above image enhancement processing, tools such as OpenCV and ImageMagick can be used. Specifically, the embodiments of the present invention do not make limitations.

[0105] S214, use the unmanned system color conversion calculation model to perform color conversion processing on the enhanced unmanned system training dataset to obtain the preprocessed unmanned system training dataset;

[0106] Among them, the unmanned system color conversion calculation model is:

[0107]

[0108] MD = max(YSR, YSG, YSB);

[0109] In the formula, SX, BH, and MD are respectively the hue value, saturation value, and lightness value corresponding to any pixel in any image information in the preprocessed unmanned system training dataset, YSR, YSG, and YSB are respectively the red channel value, green channel value, and blue channel value corresponding to any pixel in any image information in the enhanced unmanned system training dataset, δ1, δ2, and δ3 are respectively the first parameter value, the second parameter value, and the third parameter value, and α1, α2, and α3 respectively represent the first hue adjustment factor, the second hue adjustment factor, and the third hue adjustment factor.

[0110] It should be noted that the first parameter value, the second parameter value, and the third parameter value can be set by the user or obtained according to historical data. Specifically, the embodiments of the present invention do not make limitations.

[0111] Exemplarily, the values of the first parameter value, the second parameter value, and the third parameter value are 60, 120, and 240 respectively.

[0112] It should be noted that the first tone adjustment factor, the second tone adjustment factor, and the third tone adjustment factor can be set by the user or obtained according to historical data. Specifically, the embodiments of the present invention do not make any limitations.

[0113] Exemplarily, the value ranges of the first tone adjustment factor, the second tone adjustment factor, and the third tone adjustment factor are all between 0 and 10 to achieve a suitable image enhancement effect.

[0114] It should be noted that through the color conversion calculation model of the unmanned system, the hue, saturation, and lightness values of each pixel in the image are adjusted to convert the RGB mode image into an HSV mode image to optimize the image features and enhance the image quality. By adjusting the hue, saturation, and lightness, combined with the adaptive tone adjustment factors and parameter values, the image quality and the diversity of training data are effectively improved, which can effectively enhance the performance and robustness of the deep learning model, thereby facilitating more accurate identification of the regional information in the unmanned system optimization model information and precisely controlling the unmanned system.

[0115] It can be seen that implementing the brain control method for unmanned systems based on image semantic segmentation described in the embodiments of the present invention, using high-frequency visual stimuli can effectively eliminate the sense of fatigue, and integrating image semantic segmentation technology can improve the naturalness of human-computer interaction. At the same time, it is beneficial to enhance the efficiency and accuracy of the human-computer interaction system.

[0116] In an alternative embodiment, the converted unmanned system training data set is subjected to image enhancement processing to obtain an enhanced unmanned system training data set, including:

[0117] S2131, performing perspective enhancement processing on the converted unmanned system training data set to obtain a first enhanced unmanned system training data set;

[0118] It should be noted that drone images are usually taken from the air from above, and there will be significant perspective changes and perspective distortions in the scene. The above perspective enhancement processing can simulate the shooting perspectives at different heights and tilt angles, generate diverse perspective images, and at the same time make the images look more in line with natural visual perception, helping to highlight certain objects or regions. Especially in applications such as unmanned driving and robot vision, it can improve the detection and recognition accuracy of targets.

[0119] S2132, performing magnification enhancement processing on the first enhanced unmanned system training data set to obtain a second enhanced unmanned system training data set;

[0120] It should be noted that the scenes captured by drones usually contain small targets. Through magnification and enhancement processing, the small target areas in the labels are automatically detected, and local magnification is performed to balance the weights of small targets during training, which can increase the visibility of certain details or local features.

[0121] S2133. Perform environmental enhancement processing on the second enhanced unmanned system training dataset to obtain an enhanced unmanned system training dataset.

[0122] It should be noted that drone images are usually related to specific terrains or scenes. Through environmental enhancement, by combining terrain data (such as DEM, DSM) with the images, the accuracy of the label boundary area is optimized, miscellaneous colors and noises in the images can be removed, the images can be made clearer, and the accuracy of subsequent analysis and recognition can be improved.

[0123] It can be seen that when implementing the brain control method for unmanned systems based on image semantic segmentation described in the embodiments of the present invention, using high-frequency visual stimuli can effectively eliminate the sense of fatigue, and integrating image semantic segmentation technology can improve the naturalness of human-computer interaction, and at the same time is beneficial to improving the efficiency and accuracy of the human-computer interaction system.

[0124] In an optional embodiment, using the preprocessed unmanned system training dataset to perform training processing on the initial unmanned system model information to obtain optimized unmanned system model information, including:

[0125] S221. Use the preprocessed unmanned system training dataset to perform training processing on the initial unmanned system model information to obtain unmanned system training result information and unmanned system training model information;

[0126] S222. Perform calculation processing on the unmanned system training result information to obtain an unmanned system loss function value;

[0127] S223. Determine whether the unmanned system loss function value is less than a preset loss function threshold to obtain a first judgment result;

[0128] It should be noted that the preset loss function threshold can be set by the user or obtained according to historical data. Specifically, the embodiments of the present invention do not make limitations.

[0129] Exemplarily, the value range of the preset loss function threshold is [0.001, 0.005].

[0130] When the first judgment result is negative, determine that the unmanned system training model information is the initial unmanned system model information, and execute S221;

[0131] When the first judgment result is positive, determine that the unmanned system training model information is the optimized unmanned system model information.

[0132] It can be seen that by implementing the brain - controlled method for an unmanned system based on image semantic segmentation described in the embodiments of the present invention, using high - frequency visual stimuli can effectively eliminate the sense of fatigue, and integrating image semantic segmentation technology can improve the naturalness of human - machine interaction, while being beneficial to enhancing the efficiency and accuracy of the human - machine interaction system.

[0133] In an alternative embodiment, calculating and processing the unmanned system training result information to obtain the unmanned system loss function value includes:

[0134] Using the unmanned system loss function calculation model to calculate and process the unmanned system training result information to obtain the unmanned system loss function value;

[0135] Wherein, the unmanned system loss function calculation model is:

[0136] SS = SS1+SS2;

[0137]

[0138] In the formula, SS is the unmanned system loss function value, ZS is the true label information corresponding to the unmanned system training result information, XL is the unmanned system training result information, and M and N are respectively the number of categories in the unmanned system training result information and the number of unmanned system training result values corresponding to each category.

[0139] It should be noted that SS1 provides the main objective of the classification task, ensuring that the model can effectively distinguish different categories and being able to measure the matching degree between the model prediction result and the true label; SS2 measures the deviation of the predicted distribution from the true distribution in weighted similarity, reduces the error sensitivity of the model to low - probability categories, and improves the learning ability for important categories. Through SS, both the classification accuracy is optimized, and the rationality and consistency of the predicted probability distribution are improved, which is beneficial to enhancing the robustness of the model, is beneficial to more accurately identifying the regional information in the unmanned system optimization model information, and thus precisely controlling the unmanned system.

[0140] It can be seen that by implementing the brain - controlled method for an unmanned system based on image semantic segmentation described in the embodiments of the present invention, using high - frequency visual stimuli can effectively eliminate the sense of fatigue, and integrating image semantic segmentation technology can improve the naturalness of human - machine interaction, while being beneficial to enhancing the efficiency and accuracy of the human - machine interaction system.

[0141] In an alternative embodiment, controlling the unmanned system by using the unmanned system optimization model information and SSVEP frequency information includes:

[0142] S31, obtaining the unmanned system image information; the unmanned system image information includes the first stimulus region information and the second stimulus region information;

[0143] It should be noted that the image information of the unmanned system is the scene image information obtained by the AR glasses after the user wears the AR glasses and presented in the AR glasses.

[0144] It should be noted that with reference to Figure 2 , Figure 2 in the lower left corner, the white area marked "Start" is the first stimulation area information; in the upper right corner, the white area marked "End" is the second stimulation area information. When the user wears the AR glasses, the AR glasses will set the stimulation frequency of 35HZ in the SSVEP frequency information on the first stimulation area information, and set the stimulation frequency of 37HZ in the SSVEP frequency information on the second stimulation area information.

[0145] S32. In response to the user's trigger, obtain the first electroencephalogram signal information;

[0146] It should be noted that in response to the user's trigger, obtaining the first electroencephalogram signal information means that the user generates an electroencephalogram signal by gazing at the first stimulation area information or the second stimulation area information, and the first electroencephalogram signal information is obtained through a wireless electroencephalogram cap. All subsequent responses to the user's trigger are in this way.

[0147] S33. Analyze and process the first electroencephalogram signal information to obtain the first electroencephalogram analysis signal value;

[0148] S34. Determine whether the first electroencephalogram analysis signal value matches the SSVEP frequency corresponding to the first stimulation area information to obtain a second judgment result;

[0149] It should be noted that the above matching operation is to determine whether the obtained first electroencephalogram analysis signal value matches the SSVEP frequency (i.e., 35HZ frequency) corresponding to the first stimulation area information. If the difference between the first electroencephalogram analysis signal value and the SSVEP frequency corresponding to the first stimulation area information is within 0.4HZ, it is considered that the two match, indicating that the stimulation area the user gazes at is the first stimulation area information; otherwise, it is the second stimulation area information;

[0150] When the second judgment result is yes, execute S35;

[0151] When the second judgment result is no, end the control of the unmanned system;

[0152] It should be noted that the control in the present invention is to control the unmanned system by sending an unmanned system control instruction to the unmanned system. The control instructions are shown in Table 1 and include takeoff, landing, attack, following, etc.

[0153] S35. Use the unmanned system optimization model information, SSVEP frequency information, and unmanned system image information to control the unmanned system.

[0154] It can be seen that when implementing the brain control method for unmanned systems based on image semantic segmentation described in the embodiments of the present invention, using high-frequency visual stimuli can effectively eliminate the sense of fatigue. Integrating image semantic segmentation technology can improve the naturalness of human-computer interaction, and at the same time is beneficial to enhancing the efficiency and accuracy of the human-computer interaction system.

[0155] In an optional embodiment, parsing and processing the first electroencephalogram signal information to obtain the first electroencephalogram analysis signal value includes:

[0156] S331. Use the first brain control calculation model for the unmanned system to perform first feature extraction on the first electroencephalogram signal information to obtain the first electroencephalogram feature information;

[0157] Among them, the first brain control calculation model for the unmanned system is:

[0158]

[0159] In the formula, ND1 is the first electroencephalogram feature information, L is the length of the first electroencephalogram signal information, XHH j is the j-th signal sampling value in the first electroencephalogram signal information, XHHS is the sum of all signal sampling values in the first electroencephalogram signal information, and jj is the exponential factor;

[0160] It should be noted that the exponential factor can be set by the user or obtained according to historical data. Specifically, the embodiments of the present invention do not make any limitations.

[0161] It should be noted that through the first brain control calculation model for the unmanned system, the extraction result of the electroencephalogram feature information is more accurate, and it can capture details that are difficult to discover by traditional methods. By combining the global statistical information and local characteristics of the electroencephalogram signal, through the methods of nonlinear adjustment and position weight, it can extract more discriminative and robust electroencephalogram feature information, which is beneficial to improving the feature extraction accuracy, enhancing the sensitivity to weak signals, adapting to multi-scale characteristics, etc., and is beneficial to more accurately identifying the regional information in the unmanned system optimization model information, so as to precisely control the unmanned system.

[0162] It should be noted that the value range of the exponential factor is between 0 and 10. By setting the exponential factor, the sensitivity of the model to different signal positions and the feature extraction effect can be optimized, so as to better adapt to the scenarios of the unmanned system in the embodiments of the present invention.

[0163] S332. Use the second brain control calculation model for the unmanned system to perform second feature extraction on the first electroencephalogram signal information to obtain the second electroencephalogram feature information;

[0164] Among them, the brain control calculation model of the second unmanned system is as follows:

[0165]

[0166] In the formula, ND2 is the second electroencephalogram feature information, s j is the Fourier transform of the first electroencephalogram signal information, f j is s j 's center frequency;

[0167] It should be noted that through the brain control calculation model of the second unmanned system, combined with the time-domain amplitude information, the attenuation weight e -j and the frequency-domain characteristics, the key features in the electroencephalogram signal can be comprehensively extracted, which is beneficial to more accurately identify the regional information in the unmanned system optimization model information, so as to precisely control the unmanned system.

[0168] S333. Use the unmanned system fusion calculation model to calculate and process the first electroencephalogram feature information and the second electroencephalogram feature information to obtain the first electroencephalogram analysis signal value;

[0169] Among them, the unmanned system fusion calculation model is as follows:

[0170] ND = θ1·ND1 + θ2·ND2;

[0171] 0 ≤ θ1, θ2 ≤ 1;

[0172] θ1 + θ2 = 1;

[0173] Among them, ND is the first electroencephalogram analysis signal value, and θ1 and θ2 are the first weight parameter and the second weight parameter respectively.

[0174] It should be noted that the first weight parameter and the second weight parameter can be set by the user or obtained according to historical data. Specifically, the embodiments of the present invention do not make limitations.

[0175] It can be seen that implementing the unmanned system brain control method based on image semantic segmentation described in the embodiments of the present invention can effectively eliminate the sense of fatigue by using high-frequency visual stimuli, and the fusion of image semantic segmentation technology can improve the naturalness of human-computer interaction, and at the same time is beneficial to improving the efficiency and accuracy of the human-computer interaction system.

[0176] In an optional embodiment, the unmanned system is controlled by using the unmanned system optimization model information, the SSVEP frequency information, and the unmanned system image information, including:

[0177] S351. Use the unmanned system optimization model information to calculate and process the unmanned system image information to obtain the semantic recognition region information;

[0178] It should be noted that the semantic recognition area information includes the drone area, location area or other object areas (such as buildings, mountains, lakes, etc.) recognized in the unmanned system image information.

[0179] It should be noted that the first type of information is "drone type information", and the second type of information is "other object area type information";

[0180] When the type information corresponding to the semantic recognition area information matches the first type of information, it indicates that the type information corresponding to the recognized semantic recognition area information is drone type information. When the type information corresponding to the semantic recognition area information matches the second type of information, it indicates that the type information corresponding to the recognized semantic recognition area information is other object area type information. Exemplarily, if the type information corresponding to the semantic recognition area information is "drone type information", it matches the first type of information. If the type information corresponding to the semantic recognition area information is "other object area type information", it matches the second type of information. Otherwise, the type information corresponding to the semantic recognition area information is "location area type information".

[0181] S352, determine whether the type information corresponding to the semantic recognition area information matches the first type of information, and obtain a third judgment result;

[0182] When the third judgment result is yes, execute S353;

[0183] When the third judgment result is no, determine whether the type information corresponding to the semantic recognition area information matches the second type of information, and obtain a fourth judgment result;

[0184] When the fourth judgment result is yes, execute S354;

[0185] When the fourth judgment result is no, execute S355;

[0186] S353, use the SSVEP frequency information and the unmanned system image information to perform a first control on the unmanned system, and execute S32;

[0187] S354, use the SSVEP frequency information and the unmanned system image information to perform a second control on the unmanned system, and execute S32;

[0188] S355, use the SSVEP frequency information and the unmanned system image information to perform a third control on the unmanned system, and execute S32.

[0189] It can be seen that by implementing the brain control method for unmanned systems based on image semantic segmentation described in the embodiments of the present invention, using high-frequency visual stimuli can effectively eliminate the sense of fatigue, and integrating the image semantic segmentation technology can improve the naturalness of human-computer interaction, and at the same time is beneficial to improving the efficiency and accuracy of the human-computer interaction system.

[0190] In an optional embodiment, the first control of the unmanned system is performed by using SSVEP frequency information and unmanned system image information, including:

[0191] S3531. Process the SSVEP frequency information and the unmanned system image information to obtain third stimulus area information and fourth stimulus area information;

[0192] It should be noted that referring to Figure 2 , Figure 2 the middle red area in

[0193] is the unmanned aerial vehicle area. At this time, the third stimulus area information and the fourth stimulus area information are set in the unmanned aerial vehicle area through the AR glasses. The third stimulus area information is the white area marked with the character "ascend", and the fourth stimulus area information is the white area marked with the character "descend". Among them, the stimulus frequency with an SSVEP frequency of 39HZ in the SSVEP frequency information is set on the third stimulus area information through the AR glasses, and the stimulus frequency with an SSVEP frequency of 55HZ in the SSVEP frequency information is set on the fourth stimulus area information.

[0194] It should be noted that in response to the user's trigger, the second electroencephalogram signal information is obtained. The user generates an electroencephalogram signal by gazing at the third stimulus area information or the fourth stimulus area information, and the second electroencephalogram signal information is obtained through a wireless electroencephalogram cap.

[0195] S3533. Analyze and process the second electroencephalogram signal information to obtain a second electroencephalogram analysis signal value;

[0196] It should be noted that in the present invention, the analysis and processing of the electroencephalogram signal are all obtained through the steps in S331 - S333.

[0197] S3534. Determine whether the second electroencephalogram analysis signal value matches the SSVEP frequency corresponding to the third stimulus area information to obtain a fourth judgment result;

[0198] When the fourth judgment result is yes, control the unmanned system to perform a take-off operation;

[0199] When the fourth judgment result is no, control the unmanned system to perform a descent operation.

[0200] It should be noted that the above matching operation is to determine whether the obtained second EEG analysis signal value matches the SSVEP frequency corresponding to the third stimulation area information (i.e., the 39HZ frequency). If the difference between the second EEG analysis signal value and the SSVEP frequency corresponding to the third stimulation area information is within 0.4HZ, it is considered a match, indicating that the stimulation area the user is gazing at is the third stimulation area information; otherwise, it is the fourth stimulation area information.

[0201] It can be seen that implementing the brain control method for unmanned systems based on image semantic segmentation described in the embodiments of the present invention can effectively eliminate the sense of fatigue using high-frequency visual stimuli, and integrating image semantic segmentation technology can improve the naturalness of human-computer interaction, while also being beneficial to enhancing the efficiency and accuracy of the human-computer interaction system.

[0202] In an optional embodiment, the unmanned system is secondarily controlled using SSVEP frequency information and unmanned system image information, including:

[0203] S3541, process the SSVEP frequency information and the unmanned system image information to obtain fifth stimulation area information and sixth stimulation area information;

[0204] It should be noted that the fifth stimulation area information and the sixth stimulation area information are respectively set on the left and right sides of other object areas in the unmanned system image information through AR glasses. Among them, the stimulation frequency with an SSVEP frequency of 53HZ in the SSVEP frequency information is set on the fifth stimulation area information through AR glasses, and the stimulation frequency with an SSVEP frequency of 41HZ in the SSVEP frequency information is set on the sixth stimulation area information.

[0205] S3542, in response to the user's trigger, obtain third EEG signal information;

[0206] It should be noted that in response to the user's trigger, obtaining the third EEG signal information means that the user generates an EEG signal by gazing at the fifth stimulation area information or the sixth stimulation area information, and the third EEG signal information is obtained through a wireless EEG cap.

[0207] S3543, perform analysis and processing on the third EEG signal information to obtain a third EEG analysis signal value;

[0208] S3544, determine whether the third EEG analysis signal value matches the SSVEP frequency corresponding to the fifth stimulation area information to obtain a fifth judgment result;

[0209] When the fifth judgment result is yes, control the unmanned system to perform a following operation;

[0210] When the fifth judgment result is no, control the unmanned system to perform an attack operation.

[0211] It should be noted that the above matching operation is to determine whether the third electroencephalogram analysis signal value matches the SSVEP frequency corresponding to the fifth stimulus region information (i.e., the 53HZ frequency). If the difference between the third electroencephalogram analysis signal value and the SSVEP frequency corresponding to the fifth stimulus region information is within 0.4HZ, it is considered a match, indicating that the stimulus region the user is gazing at is the fifth stimulus region information; otherwise, it is the sixth stimulus region information.

[0212] It can be seen that implementing the brain control method for an unmanned system based on image semantic segmentation described in the embodiments of the present invention can effectively eliminate fatigue using high-frequency visual stimuli, and integrating image semantic segmentation technology can improve the naturalness of human-computer interaction, while also being beneficial to enhancing the efficiency and accuracy of the human-computer interaction system.

[0213] In an alternative embodiment, the third control of the unmanned system is performed using SSVEP frequency information and unmanned system image information, including:

[0214] S3551, process the SSVEP frequency information and the unmanned system image information to obtain seventh stimulus region information, eighth stimulus region information, ninth stimulus region information, tenth stimulus region information, and eleventh stimulus region information;

[0215] It should be noted that referring to Figure 3 , Figure 3 , the 5 blank square regions outside the drone area in

[0216] are respectively the seventh stimulus region information, eighth stimulus region information, ninth stimulus region information, tenth stimulus region information, and eleventh stimulus region information, and their sequential distribution positions are at the top, bottom, left, right, and center of the image of the unmanned system image information. Among them, stimulus frequencies with SSVEP frequencies of 43HZ, 45HZ, 47HZ, 49HZ, and 51HZ in the SSVEP frequency information are respectively set on the seventh stimulus region information, eighth stimulus region information, ninth stimulus region information, tenth stimulus region information, and eleventh stimulus region information through AR glasses.

[0217] It should be noted that in response to the user's trigger, the fourth electroencephalogram signal information is obtained. The user gazes at the seventh stimulus region information, eighth stimulus region information, ninth stimulus region information, tenth stimulus region information, or eleventh stimulus region information, thereby generating an electroencephalogram signal, and the second electroencephalogram signal information is obtained through a wireless electroencephalogram cap.

[0218] S3553, perform analysis and processing on the fourth electroencephalogram signal information to obtain the fourth electroencephalogram analysis signal value;

[0219] S3554, using the seventh stimulation area information, the eighth stimulation area information, the ninth stimulation area information, the tenth stimulation area information, the eleventh stimulation area information, the SSVEP frequency information, and the fourth EEG analysis signal value, to control the unmanned system.

[0220] When controlling the unmanned system as described above, when the fourth EEG analysis signal value matches the frequency corresponding to the seventh stimulation area information, the unmanned system is controlled to fly above the location area; when the fourth EEG analysis signal value matches the frequency corresponding to the eighth stimulation area information, the unmanned system is controlled to fly below the location area; when the fourth EEG analysis signal value matches the frequency corresponding to the ninth stimulation area information, the unmanned system is controlled to fly to the left of the location area; when the fourth EEG analysis signal value matches the frequency corresponding to the tenth stimulation area information, the unmanned system is controlled to fly to the right of the location area; when the fourth EEG analysis signal value matches the frequency corresponding to the eleventh stimulation area information, the unmanned system is controlled to fly to the exact middle of the location area. Specifically, the embodiments of the present invention do not make any limitations.

[0221] It should be noted that the above, below, left, and right in the embodiments of the present invention are judged in the direction seen by the user when the user is facing Figure 2 , Figure 3 . Specifically, the embodiments of the present invention do not make any limitations.

[0222] It can be seen that implementing the brain control method for an unmanned system based on image semantic segmentation described in the embodiments of the present invention can effectively eliminate the sense of fatigue by using high-frequency visual stimuli, and the fusion of image semantic segmentation technology can improve the naturalness of human-computer interaction, and at the same time is beneficial to improving the efficiency and accuracy of the human-computer interaction system.

[0223] Embodiment 2

[0224] Please refer to Figure 4 , Figure 4 , which is a schematic structural diagram of a brain control device for an unmanned system based on image semantic segmentation disclosed in the embodiments of the present invention. Among them, Figure 4 The described brain control device for an unmanned system based on image semantic segmentation is applied to an optimization system for brain control of an unmanned system based on image semantic segmentation, such as a local server or a cloud server for brain control of an unmanned system based on image semantic segmentation, etc. The embodiments of the present invention do not make any limitations. As Figure 4 shown, the brain control device for an unmanned system based on image semantic segmentation includes:

[0225] An acquisition module 201, configured to acquire unmanned system image information and SSVEP frequency information; the unmanned system image information includes unmanned system initial model information and an unmanned system training data set; the SSVEP frequency information includes a plurality of SSVEP frequencies;

[0226] The calculation module 202 is configured to process the image information of the unmanned system to obtain the optimized model information of the unmanned system;

[0227] The control module 203 is configured to control the unmanned system by using the optimized model information of the unmanned system and the SSVEP frequency information.

[0228] It can be seen that by implementing the brain-controlled device for unmanned systems based on image semantic segmentation described in the embodiments of the present invention, high-frequency visual stimuli can effectively eliminate the sense of fatigue, and the integration of image semantic segmentation technology can improve the naturalness of human-computer interaction. At the same time, it is beneficial to improve the efficiency and accuracy of the human-computer interaction system.

[0229] Embodiment Three

[0230] Please refer to Figure 5 , Figure 5 which is a schematic structural diagram of another brain-controlled device for unmanned systems based on image semantic segmentation disclosed in the embodiments of the present invention. Among them, Figure 5 The brain-controlled device for unmanned systems based on image semantic segmentation described is applied to the brain-controlled optimization system for unmanned systems based on image semantic segmentation, such as a local server or a cloud server for brain control of unmanned systems based on image semantic segmentation, etc., which is not limited in the embodiments of the present invention. As Figure 5 shown, the brain-controlled device for unmanned systems based on image semantic segmentation includes:

[0231] A processor 301;

[0232] A memory 302 coupled to the processor 301 and storing executable program code;

[0233] The processor 301 calls the executable program code stored in the memory 302 to execute some or all of the steps of the brain-control method for unmanned systems based on image semantic segmentation in Embodiment One.

[0234] It can be seen that by implementing the brain-controlled device for unmanned systems based on image semantic segmentation described in the embodiments of the present invention, high-frequency visual stimuli can effectively eliminate the sense of fatigue, and the integration of image semantic segmentation technology can improve the naturalness of human-computer interaction. At the same time, it is beneficial to improve the efficiency and accuracy of the human-computer interaction system.

[0235] Embodiment Four

[0236] The embodiments of the present invention disclose a computer-readable storage medium. The computer-readable storage medium stores computer instructions, and when the computer instructions are called, they are used to execute some or all of the steps of the brain-control method for unmanned systems based on image semantic segmentation in Embodiment One.

[0237] Embodiment Five

[0238] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps in the brain control method for an unmanned system based on image semantic segmentation described in the first embodiment.

[0239] The system embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative effort.

[0240] Through the specific descriptions of the above embodiments, those skilled in the art can clearly understand that each implementation manner can be realized by means of software plus a necessary general hardware platform, and of course, it can also be realized by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electrically-erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc memories, a magnetic disk memory, a tape memory, or any other computer-readable medium capable of carrying or storing data.

[0241] Finally, it should be noted that: What is disclosed in an unmanned system brain control method and device based on image semantic segmentation disclosed in the embodiments of the present invention is only the preferred embodiments of the present invention, which are only used to illustrate the technical solutions of the present invention, rather than to limit it; Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: They can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A brain control method for an unmanned system based on image semantic segmentation, characterized in that: The method comprises: S1, obtaining unmanned system image information and SSVEP frequency information; the unmanned system image information includes unmanned system initial model information and unmanned system training data set; the SSVEP frequency information includes a plurality of SSVEP frequencies; S2, processing the unmanned system image information to obtain unmanned system optimization model information; S3, controlling the unmanned system by utilizing the unmanned system optimization model information and the SSVEP frequency information.

2. The unmanned system brain control method based on image semantic segmentation according to claim 1 is characterized in that: The step of processing the unmanned system image information to obtain the unmanned system optimization model information includes: S21, preprocessing the unmanned system training data set to obtain a preprocessed unmanned system training data set; S22, using the pre-processed unmanned system training data set, training the unmanned system initial model information to obtain unmanned system optimized model information.

3. The unmanned system brain control method based on image semantic segmentation according to claim 2 is characterized in that: The method of using the pre-processed unmanned system training data set to train the unmanned system initial model information to obtain the unmanned system optimization model information includes: S221, using the pre-processed unmanned system training data set, performing training processing on the unmanned system initial model information to obtain unmanned system training result information and unmanned system training model information; S222, calculating and processing the unmanned system training result information to obtain the unmanned system loss function value; S223, determining whether the loss function value of the unmanned system is less than a preset loss function threshold, and obtaining a first determination result; When the first judgment result is no, determining that the unmanned system training model information is the unmanned system initial model information, and executing S221; When the first judgment result is yes, it is determined that the unmanned system training model information is unmanned system optimization model information.

4. The unmanned system brain control method based on image semantic segmentation according to claim 1, characterized in that: The controlling the unmanned system by using the unmanned system optimization model information and the SSVEP frequency information includes: S31, acquiring unmanned system image information; the unmanned system image information includes first stimulation area information and second stimulation area information; S32, in response to a trigger from the user, obtaining first electroencephalogram signal information; S33, analyzing and processing the first EEG signal information to obtain a first EEG analysis signal value; S34, determining whether the first electroencephalogram analysis signal value matches the SSVEP frequency corresponding to the first stimulation area information, and obtaining a second determination result; When the second judgment result is yes, executing S35; When the second judgment result is no, ending the control of the unmanned system; S35, controlling the unmanned system by utilizing the unmanned system optimization model information, the SSVEP frequency information and the unmanned system image information.

5. The unmanned system brain control method based on image semantic segmentation according to claim 4 is characterized in that: The step of analyzing the first EEG signal information to obtain a first EEG analysis signal value includes: S331, performing first feature extraction on the first EEG signal information to obtain first EEG feature information; S332, performing second feature extraction on the first EEG signal information to obtain second EEG feature information; S333, using the unmanned system fusion calculation model, calculating and processing the first EEG feature information and the second EEG feature information to obtain a first EEG analysis signal value; Among them, the unmanned system fusion calculation model is: ND=θ1·ND1+θ2·ND2; 0≤θ1,θ2≤1; θ1+θ2=1; Wherein, ND is the first EEG analysis signal value, θ1 and θ2 are the first weight parameter and the second weight parameter respectively.

6. The unmanned system brain control method based on image semantic segmentation according to claim 4 is characterized in that: The controlling the unmanned system by using the unmanned system optimization model information, the SSVEP frequency information and the unmanned system image information includes: S351, using the unmanned system optimization model information, calculating and processing the unmanned system image information to obtain semantic recognition area information; S352, determining whether the type information corresponding to the semantic recognition area information matches the first type information, and obtaining a third determination result; When the third judgment result is yes, execute S353; When the third judgment result is no, determining whether the type information corresponding to the semantic recognition area information matches the second type information, and obtaining a fourth judgment result; When the fourth judgment result is yes, execute S354; When the fourth judgment result is no, executing S355; S353, performing a first control on the unmanned system by using the SSVEP frequency information and the unmanned system image information, and executing S32; S354, performing a second control on the unmanned system by using the SSVEP frequency information and the unmanned system image information, and executing S32; S355, using the SSVEP frequency information and the unmanned system image information, perform a third control on the unmanned system and execute S32.

7. The unmanned system brain control method based on image semantic segmentation according to claim 6, characterized in that: The first controlling the unmanned system by using the SSVEP frequency information and the unmanned system image information includes: S3531, processing the SSVEP frequency information and the unmanned system image information to obtain third stimulation area information and fourth stimulation area information; S3532, in response to a trigger from the user, obtaining second electroencephalogram signal information; S3533, analyzing and processing the second EEG signal information to obtain a second EEG analysis signal value; S3534, determining whether the second electroencephalogram analysis signal value matches the SSVEP frequency corresponding to the third stimulation area information, and obtaining a fourth determination result; When the fourth judgment result is yes, controlling the unmanned system to perform a take-off operation; When the fourth judgment result is no, the unmanned system is controlled to perform a descending operation.

8. An unmanned system brain control device based on image semantic segmentation, characterized in that: The device comprises: An acquisition module, used to acquire unmanned system image information and SSVEP frequency information; the unmanned system image information includes unmanned system initial model information and unmanned system training data set; the SSVEP frequency information includes a plurality of SSVEP frequencies; A computing module, used for processing the unmanned system image information to obtain unmanned system optimization model information; A control module is used to control the unmanned system by utilizing the unmanned system optimization model information and the SSVEP frequency information.

9. An unmanned system brain control device based on image semantic segmentation, characterized in that: The device comprises: processor; a memory coupled to the processor and storing executable program code; The processor calls the executable program code stored in the memory to execute the unmanned system brain control method based on image semantic segmentation as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, which, when called, are used to execute the unmanned system brain control method based on image semantic segmentation as described in any one of claims 1-7.

Images