SSVEP instruction coding method and device

Abstract

The invention discloses an SSVEP instruction coding method and device, and the method comprises the steps: obtaining a real-time scene image through a preset camera of AR glasses; performing coloring processing on the real-time scene image to obtain a colored scene image; processing the semantic segmentation result set and the colored scene image to obtain an SSVEP (Steady-State Visual Evoked Potential) stimulation image; using the AR glasses to present the SSVEP stimulation image, and synchronously collecting an electroencephalogram signal; and processing the electroencephalogram signal to obtain an instruction code. According to the method, a user can be helped to more intuitively distinguish different types of stimulation targets, the SSVEP instruction can be quickly and accurately obtained, and the recognition accuracy and real-time performance during long-time wearing are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of brain-computer interface, and specifically relates to a SSVEP instruction encoding method and device. Background Art

[0002] Steady-state visual evoked potential (SSVEP) is the brain's response to periodic repetitive stimulation within a specific frequency range. When used, this signal has the characteristics of short training time, high information transmission rate, and simple use. When collecting signals, the corresponding signal can be obtained from the occipital region of the human brain. The characteristics of SSVEP are more obvious in the frequency domain. After obtaining the original signal, through filtering and Fourier transform, a more obvious peak phenomenon can be seen in the frequency domain at the same frequency as the stimulus frequency.

[0003] In the elderly care and disability assistance scenarios, the real-time and accuracy of command recognition are required to be high. However, the existing SSVEP classification method focuses on the decoding algorithm of EEG signals, ignoring that the division of stimulation areas during the SSVEP command generation process is not intuitive and has a certain delay, resulting in the user needing to maintain a high level of concentration for a long time when they want to accurately locate the SSVEP stimulation area. Therefore, after wearing it for a long time, it is easy to cause a decrease in attention, making it impossible to obtain EEG signals that can reflect the user's true intentions in time, thereby reducing the real-time and accuracy of the command encoding acquisition process. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a SSVEP instruction encoding method and device, which is beneficial to improving the recognition accuracy and real-time performance when worn for a long time.

[0005] In order to solve the above technical problems, an embodiment of the present invention discloses a SSVEP instruction encoding method, the method comprising:

[0006] S1. Using the camera of the preset AR glasses to obtain real-time scene images;

[0007] S2, performing semantic segmentation processing on the real-time scene image to obtain a semantic segmentation result set;

[0008] The semantic segmentation result set includes N semantic segmentation results; N is the number of pixels in the real-time scene image; the semantic segmentation result includes a pixel position and an object type number; the pixel position includes a pixel horizontal coordinate and a pixel vertical coordinate;

[0009] S3, performing coloring processing on the real-time scene image to obtain a colored scene image;

[0010] S4, processing the semantic segmentation result set and the colored scene image to obtain an SSVEP stimulus image;

[0011] S5. Using the AR glasses, present the SSVEP stimulation image and synchronously collect EEG signals;

[0012] S6. Process the EEG signal to obtain instruction code.

[0013] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the coloring process is performed on the real-time scene image to obtain a colored scene image, including:

[0014] S31, presetting a first mapping table, and initializing a query set as the semantic segmentation result set; the first mapping table includes a plurality of first mapping items; the first mapping item includes the object type number and color value;

[0015] S32, obtaining any of the semantic segmentation results in the query set to obtain a first segmentation result; determining a first position and a first type number, which are respectively the pixel position and the object type number of the first segmentation result;

[0016] S33, in the first mapping table, query the object type number that is the same as the first type number, the corresponding color value, and obtain the current color value;

[0017] S34, replacing the pixel value of the first position in the real-time scene image with the current color value to obtain an updated real-time scene image;

[0018] S35, deleting the first segmentation result from the query set to obtain an updated query set;

[0019] S36, repeat S32 to S35 until the query set is empty; determine that the shaded scene image is the updated real-time scene image.

[0020] As an optional implementation, in the first aspect of the embodiment of the present invention, the processing of the semantic segmentation result set and the colored scene image to obtain the SSVEP stimulus image includes:

[0021] S41, presetting a second mapping table and a transparency value; the second mapping table includes M second mapping items; M is the number of the first mapping items in the first mapping table; the second mapping item includes the object type number and the frequency value;

[0022] S42, processing the semantic segmentation result set and the second mapping table to obtain a flicker frequency matrix; the number of rows and the number of columns of the flicker frequency matrix are the number of pixel rows and the number of pixel columns of the colored scene image, respectively;

[0023] S43, setting the transparency of each pixel in the colored scene image to the transparency value to obtain a scene presentation image;

[0024] S44. Process the flicker frequency matrix and the scene presentation image to obtain the SSVEP stimulation image.

[0025] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the semantic segmentation result set and the second mapping table to obtain a flicker frequency matrix includes:

[0026] S421, initializing the frequency matrix;

[0027] S422, obtaining any of the semantic segmentation results in the semantic segmentation result set to obtain a second segmentation result; determining a second type number, which is the object type number of the second segmentation result; determining a first number p and a second number q, which are the pixel abscissa and pixel ordinate of the pixel position of the second segmentation result, respectively;

[0028] S423, in the second mapping table, query the object type number that is the same as the second type number, the corresponding frequency value, and obtain the current frequency value;

[0029] S424, replacing the element value in the p-th row and the q-th column in the frequency matrix with the current frequency value to obtain the updated frequency matrix;

[0030] S425, deleting the second segmentation result from the semantic segmentation result set to obtain an updated semantic segmentation result set;

[0031] S426, repeat S422 to S425 until the semantic segmentation result set is empty; determine that the flicker frequency matrix is ​​the frequency matrix.

[0032] As an optional implementation, in the first aspect of the embodiment of the present invention, the processing of the EEG signal to obtain the instruction code includes:

[0033] S61, analyzing and processing the EEG signal to obtain a target frequency;

[0034] S62: Process the target frequency and the second mapping table to obtain instruction code.

[0035] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the processing of the target frequency and the second mapping table to obtain the instruction code includes:

[0036] S621, preset a third mapping table; the third mapping table includes M third mapping items; M is the number of the first mapping items in the first mapping table; the third mapping item includes the frequency value and the code value;

[0037] S622. In the third mapping table, search for the instruction frequency closest to the target frequency and the corresponding encoding value to obtain the instruction encoding.

[0038] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the querying of the instruction frequency closest to the target frequency in the third mapping table and the corresponding encoding value to obtain the instruction encoding includes:

[0039] S6221. Calculate the absolute value of the difference between the target frequency and the M frequency values ​​in the third mapping table respectively to obtain M distance values;

[0040] S6222. Obtain the frequency value corresponding to the minimum value among the M distance values ​​to obtain a query frequency;

[0041] S6223. Obtain the encoding value corresponding to the query frequency in the third mapping table to obtain the instruction encoding.

[0042] A second aspect of an embodiment of the present invention discloses an SSVEP instruction encoding device, the device comprising AR glasses, a semantic segmentation module, a coloring module, a stimulus image generation module, an EEG acquisition device and an instruction encoding acquisition module;

[0043] The AR glasses are used to acquire real-time scene images and present the SSVEP stimulation images;

[0044] The semantic segmentation module is used to perform semantic segmentation processing on the real-time scene image to obtain a semantic segmentation result set;

[0045] The coloring module is used to perform coloring processing on the real-time scene image to obtain a colored scene image;

[0046] The stimulus image generation module is used to process the semantic segmentation result set and the colored scene image to obtain an SSVEP stimulus image;

[0047] The EEG acquisition device is used to acquire EEG signals;

[0048] The instruction code acquisition module is used to process the EEG signal and acquire the instruction code.

[0049] A third aspect of an embodiment of the present invention discloses another SSVEP instruction encoding device, the device comprising:

[0050] A memory storing executable program code;

[0051] a processor coupled to the memory;

[0052] The processor calls the executable program code stored in the memory to execute part or all of the steps in the SSVEP instruction encoding method disclosed in the first aspect of the embodiment of the present invention.

[0053] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute part or all of the steps in the SSVEP instruction encoding method disclosed in the first aspect of the embodiment of the present invention.

[0054] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0055] The present invention performs semantic segmentation and coloring on the real-time scene images acquired by AR glasses, and further generates SSVEP stimulation images, and obtains instruction codes after analyzing the synchronously collected EEG signals. The present invention can help users more intuitively distinguish different categories of stimulation targets, quickly and accurately obtain SSVEP instruction codes, and effectively improve recognition accuracy and real-time performance when worn for a long time. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 It is a flowchart of a SSVEP instruction encoding method disclosed in an embodiment of the present invention.

[0058] Figure 2 It is a schematic diagram of a colored scene image of an SSVEP instruction encoding method disclosed in an embodiment of the present invention.

[0059] Figure 3 It is a structural diagram of a SSVEP instruction encoding device disclosed in an embodiment of the present invention.

[0060] Figure 4 It is a schematic diagram of a colored scene image of another SSVEP instruction encoding method disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0063] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0064] Embodiment 1

[0065] See also Figure 1 , Figure 1 : is a flow chart of a SSVEP instruction encoding method disclosed in an embodiment of the present invention. Figure 1 The described SSVEP instruction encoding method is applied to the field of brain-computer interface, such as SSVEP instruction encoding in elderly care and disability assistance scenarios, and is not limited in the embodiments of the present invention. Figure 1 As shown, the method includes:

[0066] S1. Using the camera of the preset AR glasses to obtain real-time scene images;

[0067] It should be noted that the above-mentioned AR glasses are worn on the user's eyes;

[0068] S2, performing semantic segmentation processing on the real-time scene image to obtain a semantic segmentation result set;

[0069] The above semantic segmentation result set includes N semantic segmentation results; the above N is the number of pixels of the real-time scene image; the above semantic segmentation result includes a pixel position and an object type number; the above pixel position includes a pixel horizontal coordinate and a pixel vertical coordinate;

[0070] It should be noted that the above object type number is used to indicate the specific type of the identified target, such as a wall, a bed, a curtain or a television, etc., which is not limited in the embodiment of the present invention;

[0071] It should be noted that semantic segmentation is a computer vision technology that assigns each pixel in an image to a corresponding category to achieve pixel-level semantic understanding. This technology not only distinguishes different objects in an image, but also understands the semantic content of each pixel. It is related to object detection and instance segmentation, but different.

[0072] It should be noted that the above semantic segmentation can use a model of an encoder-decoder network structure (such as DeepLab, PSPNet, etc.), which is not limited in the embodiment of the present invention. This type of model is usually divided into an encoding stage and a decoding stage. The encoding stage is used to extract feature information from the image, and the decoding stage uses the encoded feature information to classify pixels;

[0073] S3, performing coloring processing on the real-time scene image to obtain a colored scene image;

[0074] S4, processing the semantic segmentation result set and the colored scene image to obtain the SSVEP stimulus image;

[0075] S5. Using AR glasses, present the SSVEP stimulation image and simultaneously collect EEG signals;

[0076] It should be noted that while the SSVEP stimulation image is presented in front of the user through the AR glasses, the EEG signal is synchronously collected by using a collection device worn on the user's head; the above-mentioned collection device can be an OpenBCI EEG collection cap, which is not limited in the embodiment of the present invention;

[0077] S6. Process the EEG signal to obtain the instruction code.

[0078] It can be seen that obtaining and displaying SSVEP stimulation images through semantic segmentation and coloring processing can help users distinguish different categories of stimulation targets more intuitively, reduce the consumption of users' energy, and thus still obtain EEG signals that reflect the user's true intentions during long-term use, thereby improving the real-time and accuracy of the instruction encoding acquisition process.

[0079] In an optional embodiment, the above-mentioned coloring process is performed on the real-time scene image to obtain the colored scene image, including:

[0080] S31, presetting a first mapping table, and initializing a query set as a semantic segmentation result set; the first mapping table includes a plurality of first mapping items; the first mapping items include an object type number and a color value;

[0081] It should be noted that there is no identical color value in the first mapping table;

[0082] S32, obtaining any semantic segmentation result in the query set to obtain a first segmentation result; determining a first position and a first type number, which are respectively a pixel position and an object type number of the first segmentation result;

[0083] S33, searching the first mapping table for the object type number that is the same as the first type number and the corresponding color value to obtain the current color value;

[0084] S34, replacing the pixel value at the first position in the real-time scene image with the current color value to obtain an updated real-time scene image;

[0085] S35, deleting the first segmentation result from the query set to obtain an updated query set;

[0086] S36, repeat S32 to S35 until the query set is empty; determine that the colored scene image is the updated real-time scene image.

[0087] It should be noted that in the above colored scene images, objects of the same category have the same color, while objects of different categories generally have different colors. Figure 2 As shown, the curtains on the left and right sides are both red, and the ground is white, which is not limited in this embodiment of the present invention.

[0088] In another optional embodiment, the above processing of the semantic segmentation result set and the colored scene image to obtain the SSVEP stimulus image includes:

[0089] S41, presetting a second mapping table and a transparency value; the second mapping table includes M second mapping items; the M is the number of first mapping items in the first mapping table; the second mapping item includes an object type number and a frequency value;

[0090] S42, processing the semantic segmentation result set and the second mapping table to obtain a flicker frequency matrix; the number of rows and the number of columns of the flicker frequency matrix are the number of pixel rows and the number of pixel columns of the colored scene image, respectively;

[0091] S43, setting the transparency of each pixel in the colored scene image to a transparency value to obtain a scene presentation image;

[0092] S44. Process the flicker frequency matrix and the scene presentation image to obtain a SSVEP stimulus image.

[0093] It should be noted that the SSVEP stimulus image is obtained by flashing the pixels in the i-th row and j-th column of the scene presentation image at the frequency value of the i-th row and j-th column of the flashing frequency matrix, where 1≤i≤N1, 1≤j≤N2, N1 and N2 are the number of rows and columns of the flashing frequency matrix, respectively.

[0094] In yet another optional embodiment, the frequency values ​​in the second mapping table constitute an arithmetic progression; the maximum value and the minimum value of the frequency values ​​are respectively the preset first threshold and the second threshold;

[0095] Preferably, the first threshold and the second threshold are 8 Hz and 15 Hz respectively, and the frequency values ​​form an arithmetic progression with a tolerance of 0.5 Hz.

[0096] It should be noted that by distributing the frequency values ​​at equal intervals, the interpolation between adjacent frequency values ​​can be maximized, thus avoiding interference between adjacent frequency value signals during EEG signal analysis.

[0097] In yet another optional embodiment, the semantic segmentation result set and the second mapping table are processed to obtain a flicker frequency matrix, including:

[0098] S421, initializing the frequency matrix;

[0099] It should be noted that the frequency matrix is ​​initialized as a matrix of N1×N2, where N1 and N2 are the number of pixel rows and the number of pixel columns of the colored scene image, respectively; the elements in the frequency matrix can be initialized to any value, which is not limited in the embodiment of the present invention;

[0100] S422, obtaining any semantic segmentation result in the semantic segmentation result set to obtain a second segmentation result; determining a second type number, which is the object type number of the second segmentation result; determining a first number p and a second number q, which are the pixel abscissa and pixel ordinate of the pixel position of the second segmentation result, respectively;

[0101] S423, searching in the second mapping table for the object type number that is the same as the second type number and the corresponding frequency value to obtain the current frequency value;

[0102] S424, replacing the element value in the p-th row and q-th column in the frequency matrix with the current frequency value to obtain an updated frequency matrix;

[0103] S425, deleting the second segmentation result from the semantic segmentation result set to obtain an updated semantic segmentation result set;

[0104] S426, repeat S422 to S425 until the semantic segmentation result set is empty; determine that the flicker frequency matrix is ​​a frequency matrix.

[0105] In yet another optional embodiment, the above-mentioned processing of the EEG signal to obtain the instruction code includes:

[0106] S61, analyzing and processing the EEG signal to obtain a target frequency;

[0107] S62, processing the target frequency and the second mapping table to obtain instruction code.

[0108] Optionally, the above analysis and processing of the EEG signal adopts the FBCCA algorithm;

[0109] It should be noted that the FBCCA algorithm is used to analyze and process EEG signals, including:

[0110] S611, decompose the EEG into multiple sub-band components through multiple filtering

[0111] S612: Compare each subband component with the corresponding sine-cosine reference template (where S represents the number of stimuli) and uses the canonical correlation analysis (CCA) method to calculate each sub-band component The correlation coefficient matrix ρ between the reference template k , ρ k is defined as follows:

[0112]

[0113] The above CCA can be used to analyze the correlation between two signals. When classifying the collected EEG signals, a signal with different frequencies and fixed frequency can be used as a reference signal, and the collected signal and the reference signal with different frequencies can be used to calculate the correlation coefficient using canonical correlation analysis. The one with the highest correlation can be considered as the frequency transmitted by the EEG signal. This method does not require prior training when processing EEG signals, and is simple and effective.

[0114] S613, using the above correlation coefficient matrix, construct classification features corresponding to S stimulation frequencies Its definition is as follows:

[0115]

[0116] Where n is the serial number of each subband component, and w(n) is the relevant weight of each subband, which is defined as follows:

[0117] w(n)=n -a +b,n∈[1,N]

[0118] Wherein, a and b are preset constants.

[0119] S614, Get The stimulation frequency corresponding to the largest component is the target frequency.

[0120] In yet another optional embodiment, the target frequency and the second mapping table are processed to obtain the instruction code, including:

[0121] S621, preset a third mapping table; the third mapping table includes M third mapping items; the M is the number of first mapping items in the first mapping table; the third mapping item includes a frequency value and a coding value;

[0122] It should be noted that the third mapping item in the third mapping table corresponds one-to-one to the second mapping item in the second mapping table, and the frequency value in the corresponding third mapping item and the second mapping item is the same;

[0123] It should be noted that each of the above-mentioned coding values ​​corresponds to a control instruction, and the corresponding relationship between the function of the control instruction and the frequency value is shown in the following subscript, which is not limited in the embodiment of the present invention.

[0124] Frequency / Hz Function Frequency / Hz Function 15Hz Turn on the ceiling light 11 Draw the curtains 14.5 Wipe the wall 10.5 Move the chair 14 Open the cupboard 10 Turn on the desk lamp 13.5 Turn on the lights 9.5 Turn on the TV 13 sleep 9 Crawl 12.5 Open the cabinet 8.5 Change shoes 12 Window 8 Mopping 11.5 Have a meal

[0125] S622. In the third mapping table, search for the instruction frequency closest to the target frequency and the corresponding encoding value to obtain the instruction encoding.

[0126] In yet another optional embodiment, the third mapping table is searched for the instruction frequency closest to the target frequency, and the corresponding encoding value is obtained to obtain the instruction encoding, including:

[0127] S6221, respectively calculating the absolute value of the difference between the target frequency and the M frequency values ​​in the third mapping table to obtain M distance values;

[0128] S6222. Obtain a frequency value corresponding to the minimum value among the M distance values ​​to obtain a query frequency;

[0129] S6223. Obtain the encoding value corresponding to the query frequency in the third mapping table to obtain the instruction encoding.

[0130] It can be seen that the implementation of the SSVEP instruction encoding method described in the embodiment of the present invention can help users more intuitively distinguish different categories of stimulation targets, quickly and accurately obtain SSVEP instructions, and effectively improve the recognition accuracy and real-time performance during long-term wear.

[0131] Embodiment 2

[0132] See also Figure 3 , Figure 3 : is a schematic diagram of the structure of a SSVEP instruction encoding device disclosed in an embodiment of the present invention. Figure 3The described SSVEP instruction encoding device is applied to the field of brain-computer interface, such as SSVEP instruction encoding in the elderly care and disability assistance scenario, and the embodiments of the present invention do not limit this. Figure 3 As shown, the SSVEP instruction encoding device comprises:

[0133] AR glasses 201, used to acquire real-time scene images and present SSVEP stimulus images;

[0134] The semantic segmentation module 202 is used to perform semantic segmentation processing on the real-time scene image to obtain a semantic segmentation result set;

[0135] A coloring module 203 is used to perform coloring processing on the real-time scene image to obtain a colored scene image;

[0136] A stimulus image generation module 204 is used to process the semantic segmentation result set and the colored scene image to obtain an SSVEP stimulus image;

[0137] An electroencephalogram (EEG) acquisition device 205 is used to acquire EEG signals;

[0138] The instruction code acquisition module 206 is used to process the EEG signal and acquire the instruction code.

[0139] It can be seen that the implementation of the SSVEP command encoding device described in the embodiment of the present invention can help users more intuitively distinguish different categories of stimulation targets, quickly and accurately obtain SSVEP command codes, and effectively improve the recognition accuracy and real-time performance when worn for a long time.

[0140] Embodiment 3

[0141] See also Figure 4 , Figure 4 : is a schematic diagram of the structure of another SSVEP instruction encoding device disclosed in an embodiment of the present invention. Figure 4 The described SSVEP instruction encoding device can be applied to the field of brain-computer interface, such as SSVEP instruction encoding in elderly care and disability assistance scenarios, and the embodiments of the present invention are not limited thereto. Figure 4 As shown, the SSVEP instruction encoding device may include the following parts:

[0142] A memory 301 storing executable program codes;

[0143] a processor 302 coupled to the memory 301;

[0144] The processor 302 calls the executable program code stored in the memory 301 to execute the steps in the SSVEP instruction encoding method described in the first embodiment.

[0145] Embodiment 4

[0146] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps in the SSVEP instruction encoding method described in the first embodiment.

[0147] The device embodiments described above are only illustrative, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, i.e., they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0148] Through the specific description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution can be essentially or partly contributed to the prior art in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, a magnetic disk storage, a magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0149] Finally, it should be noted that the SSVEP instruction encoding method and device disclosed in the embodiment of the present invention discloses only the preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A SSVEP instruction encoding method, characterized in that: include: S1. Using the camera of the preset AR glasses to obtain real-time scene images; S2, performing semantic segmentation processing on the real-time scene image to obtain a semantic segmentation result set; The semantic segmentation result set includes N semantic segmentation results; N is the number of pixels in the real-time scene image; the semantic segmentation result includes a pixel position and an object type number; the pixel position includes a pixel horizontal coordinate and a pixel vertical coordinate; S3, performing coloring processing on the real-time scene image to obtain a colored scene image; S4, processing the semantic segmentation result set and the colored scene image to obtain an SSVEP stimulus image; S5. Using the AR glasses, present the SSVEP stimulation image and synchronously collect EEG signals; S6. Process the EEG signal to obtain instruction code.

2. The SSVEP instruction encoding method according to claim 1, characterized in that: The coloring process is performed on the real-time scene image to obtain a colored scene image, including: S31, presetting a first mapping table, and initializing a query set as the semantic segmentation result set; the first mapping table includes a plurality of first mapping items; the first mapping item includes the object type number and color value; S32, obtaining any of the semantic segmentation results in the query set to obtain a first segmentation result; determining a first position and a first type number, which are respectively the pixel position and the object type number of the first segmentation result; S33, in the first mapping table, query the object type number that is the same as the first type number, the corresponding color value, and obtain the current color value; S34, replacing the pixel value of the first position in the real-time scene image with the current color value to obtain an updated real-time scene image; S35, deleting the first segmentation result from the query set to obtain an updated query set; S36, repeat S32 to S35 until the query set is empty; determine that the shaded scene image is the updated real-time scene image.

3. The SSVEP instruction encoding method according to claim 2, characterized in that: The processing of the semantic segmentation result set and the colored scene image to obtain the SSVEP stimulus image includes: S41, presetting a second mapping table and a transparency value; the second mapping table includes M second mapping items; M is the number of the first mapping items in the first mapping table; the second mapping item includes the object type number and the frequency value; S42, processing the semantic segmentation result set and the second mapping table to obtain a flicker frequency matrix; the number of rows and the number of columns of the flicker frequency matrix are the number of pixel rows and the number of pixel columns of the colored scene image, respectively; S43, setting the transparency of each pixel in the colored scene image to the transparency value to obtain a scene presentation image; S44. Process the flicker frequency matrix and the scene presentation image to obtain the SSVEP stimulation image.

4. The SSVEP instruction encoding method according to claim 3, characterized in that: The processing of the semantic segmentation result set and the second mapping table to obtain a flicker frequency matrix includes: S421, initializing the frequency matrix; S422, obtaining any of the semantic segmentation results in the semantic segmentation result set to obtain a second segmentation result; determining a second type number, which is the object type number of the second segmentation result; determining a first number p and a second number q, which are the pixel abscissa and pixel ordinate of the pixel position of the second segmentation result, respectively; S423, in the second mapping table, query the object type number that is the same as the second type number, the corresponding frequency value, and obtain the current frequency value; S424, replacing the element value in the p-th row and the q-th column in the frequency matrix with the current frequency value to obtain the updated frequency matrix; S425, deleting the second segmentation result from the semantic segmentation result set to obtain an updated semantic segmentation result set; S426, repeat S422 to S425 until the semantic segmentation result set is empty; determine that the flicker frequency matrix is ​​the frequency matrix.

5. The SSVEP instruction encoding method according to claim 3, characterized in that: The step of processing the EEG signal to obtain the instruction code includes: S61, analyzing and processing the EEG signal to obtain a target frequency; S62: Process the target frequency and the second mapping table to obtain instruction code.

6. The SSVEP instruction encoding method according to claim 5, characterized in that: The processing of the target frequency and the second mapping table to obtain instruction code includes: S621, preset a third mapping table; the third mapping table includes M third mapping items; M is the number of the first mapping items in the first mapping table; the third mapping item includes the frequency value and the code value; S622. In the third mapping table, search for the instruction frequency closest to the target frequency and the corresponding encoding value to obtain the instruction encoding.

7. The SSVEP instruction encoding method according to claim 6, characterized in that: In the third mapping table, querying the instruction frequency closest to the target frequency and the corresponding encoding value to obtain the instruction encoding includes: S6221. Calculate the absolute value of the difference between the target frequency and the M frequency values ​​in the third mapping table respectively to obtain M distance values; S6222. Obtain the frequency value corresponding to the minimum value among the M distance values ​​to obtain a query frequency; S6223. Obtain the encoding value corresponding to the query frequency in the third mapping table to obtain the instruction encoding.

8. An SSVEP instruction encoding device, characterized in that: The device includes AR glasses, a semantic segmentation module, a coloring module, a stimulus image generation module, an EEG acquisition device, and an instruction code acquisition module; The AR glasses are used to acquire real-time scene images and present the SSVEP stimulation images; The semantic segmentation module is used to perform semantic segmentation processing on the real-time scene image to obtain a semantic segmentation result set; The coloring module is used to perform coloring processing on the real-time scene image to obtain a colored scene image; The stimulus image generation module is used to process the semantic segmentation result set and the colored scene image to obtain an SSVEP stimulus image; The EEG acquisition device is used to acquire EEG signals; The instruction code acquisition module is used to process the EEG signal and acquire the instruction code.

9. An SSVEP instruction encoding device, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the SSVEP instruction encoding method as described in any one of claims 1-7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer instructions, and when the computer instructions are called, they are used to execute the SSVEP instruction encoding method as described in any one of claims 1-7.

Images