EEG and Eye Movement Fusion Method, Medium and Device for AR Target Recognition

Abstract

The present invention discloses an electroencephalogram and eye movement fusion method, medium and device for AR target recognition. The method includes synchronously collecting eye movement data, electroencephalogram data and Trigger signals when gazing at an AR glasses; performing denoising preprocessing on the eye movement data and electroencephalogram data according to the stimulation time indicated by the Trigger signals; using a distance weighted average algorithm for the preprocessed eye movement data to determine four adjacent target points of the AR glasses stimulation module; taking the label information of the four adjacent target points as a reference frequency and inputting it into an electroencephalogram decoding program to calculate an AR target. The present invention reduces the electroencephalogram fusion calculation amount, solves the requirement for electroencephalogram and eye movement fusion under an AR device, and reduces problems such as high occupancy, high calculation amount and unstable output brought by existing fusion algorithms, improving the recognition efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the field of bioelectric signal processing, and in particular to a method, device, medium and equipment for fusing electroencephalogram and eye movement for AR target recognition. Background Art

[0002] AR (augmented reality) technology is a technology that cleverly integrates virtual information with the real world. Placing the SSVEP stimulation module and eye movement stimulation on the AR device can not only improve the portability of brain control devices, but also improve the accuracy of EEG decoding.

[0003] However, the existing eye movement and EEG fusion algorithm first extracts features, then inputs them into the machine learning pattern recognition network for intent recognition, and finally fuses the results through a decision-making strategy. This fusion method has problems such as complex decoding methods and low decoding accuracy. Summary of the invention

[0004] In view of the problems existing in the prior art, the first purpose of the present invention is to provide an EEG and eye movement fusion method for AR target recognition, which reduces the data processing amount of the eye movement and EEG fusion algorithm and improves the efficiency and accuracy of AR target recognition.

[0005] The second object of the present invention is to provide a brain wave and eye movement fusion device for AR target recognition.

[0006] The third object of the present invention is to provide an electronic device for implementing the above-mentioned fusion method.

[0007] A fourth object of the present invention is to provide a computer-readable medium for executing the above-mentioned fusion method.

[0008] To achieve the above object, a first aspect of the present invention provides a method for fusion of electroencephalogram and eye movement for AR target recognition, comprising the following steps:

[0009] Synchronously collect eye movement data, EEG data and trigger signals when looking at AR glasses;

[0010] Performing denoising preprocessing on the eye movement data and the EEG data according to the stimulation time indicated by the Trigger signal;

[0011] The distance-weighted average algorithm is used on the preprocessed eye movement data to determine the four neighboring target points of the AR glasses stimulation module;

[0012] The label information of four neighboring targets is input into the EEG decoding program as the reference frequency to calculate the AR target.

[0013] Further, the distance weighted average algorithm is used for the preprocessed eye movement data to determine the four adjacent target points of the AR glasses stimulation module, including:

[0014] The preprocessed eye movement data is equally divided into three groups of eye movement sub-data at equal intervals according to the order of the 3-second stimulation time, and the average coordinates (X 1 , Y 1 ), (X 2 , Y 2 ), (X 3 , Y 3 ) of the three groups of eye movement sub-data are calculated respectively;

[0015] The weighted average coordinates (X m , Y m ) are calculated for the average coordinates of the three groups of eye movement sub-data according to a predetermined weighting ratio;

[0016] The distance Dw between the calculated weighted average coordinates (X m , Y m ) and the center coordinates (X i , Y i ) of each stimulation module of the AR glasses is calculated;

[0017] The sort function in Matlab is used to sort each calculated distance Dw from smallest to largest to obtain distances Ds1, Ds2, Ds3, Ds4;

[0018] The four target points corresponding to the distances Ds1, Ds2, Ds3, Ds4 are determined as the four adjacent target points.

[0019] Further, the weighting ratio of the average coordinates of the first group of eye movement sub-data is 10%, the weighting ratio of the average coordinates of the second group of eye movement sub-data is 20%, and the weighting ratio of the average coordinates of the third group of eye movement sub-data is 70%.

[0020] Further, the calculation formula for calculating the weighted average coordinates (X m , Y m ) for the average coordinates of the three groups of eye movement sub-data according to a predetermined weighting ratio is:

[0021] (X m , Y m ) = 10% × (X 1 , Y 1 ) + 20% × (X 2 , Y 2 ) + 70% × (X 3 , Y 3 ).

[0022] The second aspect of the present invention provides an electroencephalogram and eye movement fusion device for AR target recognition, including:

[0023] An acquisition module, configured to synchronously acquire eye movement data, electroencephalogram data, and Trigger signals when looking at an AR glasses;

[0024] A preprocessing module, configured to perform denoising preprocessing on the eye movement data and electroencephalogram data according to the stimulation time indicated by the Trigger signal;

[0025] A screening module, configured to determine four adjacent target points of the AR glasses stimulation module for the preprocessed eye movement data by using a distance weighted average algorithm;

[0026] An identification module, configured to input the label information of the four adjacent target points as a reference frequency into an electroencephalogram decoding program to calculate an AR target.

[0027] Furthermore, the screening module includes:

[0028] A segmentation sub-module, configured to equally segment the preprocessed eye movement data into 3 groups of eye movement sub-data at equal intervals according to the order of the 3-second stimulation time, and respectively calculate the average coordinates (X 1 , Y 1 ) of the 3 groups of eye movement sub-data, (X 2 , Y 2 ), (X 3 , Y 3 )

[0029] A coordinate calculation sub-module, configured to calculate a weighted average coordinate (X m , Y m ) for the average coordinates of the 3 groups of eye movement sub-data according to a predetermined weighting ratio;

[0030] A distance calculation sub-module, configured to calculate the distance Dw between the weighted average coordinate (X m , Y m ) and the central coordinate (X i , Y i ) of each stimulation module of the AR glasses;

[0031] A sorting sub-module, configured to sort each calculated distance Dw from smallest to largest by using the sort function in Matlab to obtain distances Ds1, Ds2, Ds3, Ds4;

[0032] A determination sub-module, configured to determine the four target points corresponding to the distances Ds1, Ds2, Ds3, Ds4 as the four adjacent target points.

[0033] The third aspect of the present invention provides an electronic device, including:

[0034] One or more processors; and

[0035] A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the fusion method according to the first aspect.

[0036] A fourth aspect of the present invention provides a computer-readable medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the fusion method according to the first aspect.

[0037] The present invention performs segmented processing on eye movement data based on the stimulation time marked by electroencephalogram data, and screens out the eye movement data that best meets the fusion requirements from the eye movement data for electroencephalogram and eye movement fusion, reducing the electroencephalogram fusion calculation amount, solving the need for electroencephalogram and eye movement fusion under AR devices, and reducing problems such as high occupancy, high calculation amount, and unstable output brought by existing fusion algorithms, improving the recognition efficiency and accuracy.

[0038] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0040] Figure 1 It is a flowchart of an electroencephalogram and eye movement fusion method for AR target recognition according to an embodiment of the present invention;

[0041] Figure 2 It is a flowchart of an electroencephalogram and eye movement fusion method for AR target recognition according to another embodiment of the present invention;

[0042] Figure 3 It is a structural block diagram of an electroencephalogram and eye movement fusion device for AR target recognition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] Now, the exemplary embodiments will be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present invention will be more complete and comprehensive, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art.

[0044] In addition, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present invention. However, those skilled in the art will realize that the technical solutions of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present invention.

[0045] The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0046] The flowcharts shown in the drawings are merely illustrative and do not necessarily include all the content and operations / steps, nor are they necessarily executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined, so the actual execution order may change according to the actual situation.

[0047] As Figure 1 shown, the electroencephalogram and eye movement fusion method for AR target recognition of the present invention includes the following steps:

[0048] Step S110: Synchronously collect eye movement data, electroencephalogram data, and Trigger signal when gazing at the AR glasses;

[0049] Step S120: Perform denoising preprocessing on the eye movement data and electroencephalogram data according to the stimulation time indicated by the Trigger signal;

[0050] Step S130: Use the distance weighted average algorithm for the preprocessed eye movement data to determine four adjacent target points of the AR glasses stimulation module;

[0051] Step S140: Take the label information of the four adjacent target points as the reference frequency and input it into the electroencephalogram decoding program to calculate the AR target.

[0052] The present invention performs segmented processing on the eye movement data according to the stimulation time indicated by the electroencephalogram data, and screens out the eye movement data that best meets the fusion requirements from the eye movement data for electroencephalogram and eye movement fusion, reducing the electroencephalogram fusion calculation amount, solving the need for electroencephalogram and eye movement fusion under AR devices, and reducing problems such as high occupancy, high calculation amount, and unstable output brought by existing fusion algorithms, improving the recognition efficiency and accuracy.

[0053] In an embodiment of the present invention, in step S110, Neuroscan electroencephalogram (EEG) acquisition equipment is used to collect EEG data. The EEG acquisition electrode cap has 64 channels. During the implementation process, the main control machine realizes signal synchronization through a parallel port cable and receives the EEG signals collected by Neuroscan through a serial port. The eye movement data and Trigger signal are sent to the main control machine by Hololens through the UDP method, and the eye movement and the main control machine achieve synchronization by means of UDP.

[0054] In an embodiment of the present invention, in step S120, after the main control machine collects the Trigger, EEG, and eye movement data, it determines the stimulation time of each EEG stimulation module according to the Trigger signal, whereby the sampled eye movement and EEG can be preprocessed in segments. In addition, the EEG analysis plugin EEGLAB is used to perform preprocessing such as filtering, baseline correction, and downsampling on the EEG data.

[0055] As Figure 2 shown, the above step S130 may include steps S210 to S250.

[0056] Step S210: The preprocessed eye movement data is equally divided into 3 groups of eye movement sub-data according to the chronological order of the 3-second stimulation time, and the average coordinates (X 1 , Y 1 )、(X 2 , Y 2 )、(X 3 , Y 3 ) of the 3 groups of eye movement sub-data are calculated respectively. In this embodiment, the stimulation time is 3 s, the sampling frequency is 1000 Hz, and the number of sampling points after stimulation is 3000. The 3000 sampling points after stimulation are divided into three groups, with 1000 sampling points in each group. The average coordinates (X 1 , Y 1 )、(X 2 , Y 2 )、(X 3 , Y 3 ) of the 3 groups of eye movement sub-data are calculated by averaging the sampling points.

[0057] Step S220: Calculate the weighted average coordinates (X m , Y m ) of the average coordinates of the 3 groups of eye movement sub-data according to a predetermined weighting ratio. Among them, the correlation between each group of eye movement sub-data and the target data is different according to the chronological order of the stimulation time, and different weights are assigned to the average coordinates of each group of eye movement sub-data to calculate the weighted average coordinates (X m , Y m) is closer to the target data, thus improving the accuracy of the final target recognition. Usually, the weight ratio of the average coordinates of the first group of eye movement sub-data is set to 10%, the weight ratio of the average coordinates of the second group of eye movement sub-data is set to 20%, and the weight ratio of the average coordinates of the third group of eye movement sub-data is 70%. The first group of eye movement sub-data corresponds to the eye movement data at the 1st second of the stimulation time, the second group of eye movement sub-data corresponds to the eye movement data at the 2nd second of the stimulation time, and the third group of eye movement sub-data corresponds to the eye movement data at the 3rd second of the stimulation time. According to the average coordinates (X 1 , Y 1 )、(X 2 , Y 2 )、(X 3 , Y 3 ) of the three groups of eye movement sub-data and the corresponding weight ratios, the weighted average coordinates (X m , Y m ) can be calculated according to the following calculation formula:

[0058] (X m , Y m ) = 10%×(X 1 , Y 1 ) + 20%×(X 2 , Y 2 ) + 70%×(X 3 , Y 3 ).

[0059] Step S230: Calculate the distance Dw between the weighted average coordinates (X m , Y m ) and the center coordinates (X i , Y i ) of each stimulation module of the AR glasses. In this embodiment, the number of stimulation modules of the AR glasses is 12, the width and height of the stimulation module are 410 and 290 respectively, and the entire projection range is -669~669, -369~261. It should be noted that the number of stimulation modules of the AR glasses is not limited to this, and the present invention is not limited thereto.

[0060] Step S240: Use the sort function in Matlab to sort each calculated distance Dw from smallest to largest to obtain distances Ds1, Ds2, Ds3, Ds4.

[0061] Step S250: Determine the four target points corresponding to the distances Ds1, Ds2, Ds3, and Ds4 as the four adjacent target points. Screen out the four smallest distances Ds1, Ds2, Ds3, and Ds4 from the numerous distances Dw calculated above. The target points corresponding to the four smallest distances Ds1, Ds2, Ds3, and Ds4 are recognized as the closest to the target. After initially screening out the four adjacent target points closest to the target, electroencephalogram (EEG) fusion is performed through the four adjacent target points. In this way, the data processing volume of the EEG fusion algorithm is greatly reduced, the high occupancy and unstable output during the fusion calculation process are reduced, and the recognition efficiency and accuracy are improved.

[0062] In an embodiment of the present invention, in step S140, a CCA decoding algorithm is used for EEG decoding, and the reference frequencies corresponding to these four adjacent target points are used as the frequency parameters of the sine and cosine functions. Each reference frequency forms a signal template composed of a fundamental frequency and sine and cosine signals of its different multiple frequencies. The correlation value analysis is performed on the stimulus-segment EEG signal and these four groups of signal templates one by one, and the corresponding frequency of the template signal with the largest correlation value is the result frequency. Thus, the AR target can be obtained through the result frequency. It should be noted that the decoding algorithm used in this step is not limited to this, and it can also use the FBCCA decoding algorithm for EEG decoding.

[0063] As Figure 3 shown, a second aspect of the present invention provides an EEG and eye movement fusion device 300 for AR target recognition, including an acquisition module 310, a preprocessing module 320, a screening module 330, and an identification module 340.

[0064] The acquisition module 310 is used to synchronously acquire eye movement data, EEG data, and Trigger signals when looking at the AR glasses.

[0065] The preprocessing module 320 is used to perform denoising preprocessing on the eye movement data and EEG data according to the stimulus time indicated by the Trigger signal.

[0066] The screening module 330 is used to determine four adjacent target points of the AR glasses stimulation module for the preprocessed eye movement data by using a distance weighted average algorithm;

[0067] The identification module 340 is used to input the label information of the four adjacent target points as the reference frequency into the EEG decoding program to calculate the AR target.

[0068] In an embodiment of the present invention, the screening module 330 includes a segmentation sub-module, a coordinate calculation sub-module, a distance calculation sub-module, a sorting sub-module, and a determination sub-module.

[0069] A segmentation sub-module, configured to equally segment the preprocessed eye movement data into three groups of eye movement sub-data according to the chronological order of the 3-second stimulation time, and calculate the average coordinates (X 1 , Y 1 )、(X 2 , Y 2 )、(X 3 , Y 3 ) of the three groups of eye movement sub-data respectively

[0070] A coordinate calculation sub-module, configured to calculate the weighted average coordinates (X m , Y m ) of the average coordinates of the three groups of eye movement sub-data according to a predetermined weighting ratio;

[0071] A distance calculation sub-module, configured to calculate the distance Dw between the weighted average coordinates (X m , Y m ) and the central coordinates (X i , Y i ) of each stimulation module of the AR glasses;

[0072] A sorting sub-module, using the sort function in Matlab to sort the calculated distances Dw from smallest to largest to obtain distances Ds1, Ds2, Ds3, Ds4;

[0073] A determination sub-module, configured to determine the four target points corresponding to the distances Ds1, Ds2, Ds3, Ds4 as the four adjacent target points.

[0074] According to an embodiment of the present invention, the device 300 can implement Figures 1 - 2 the fusion method described in the embodiment.

[0075] Since each module of the electroencephalogram and eye movement fusion device 300 according to the exemplary embodiments of the present invention can be used to implement the steps of the exemplary embodiments of the above-mentioned 1- Figure 2 described fusion method, for details not disclosed in the device embodiments of the present invention, please refer to the embodiments of the above-mentioned fusion method of the present invention.

[0076] It can be understood that the acquisition module 310, the preprocessing module 320, the screening module 330, the recognition module 340, the segmentation sub-module, the coordinate calculation sub-module, the distance calculation sub-module, the sorting sub-module, and the determination sub-module can be combined and implemented in one module, or any one of them can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the acquisition module 310, the preprocessing module 320, the screening module 330, and the recognition module 340 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or can be implemented in any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or in an appropriate combination of software, hardware, and firmware. Alternatively, at least one of the acquisition module 310, the preprocessing module 320, the screening module 330, and the recognition module 340 can be at least partially implemented as a computer program module, and when the program is run on a computer, it can execute the functions of the corresponding module.

[0077] A third aspect of the present invention provides an electronic device, including: one or more processors; and

[0078] a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the above Figure 1 and Figure 2 fusion method described in the embodiments.

[0079] A fourth aspect of the present invention provides a computer-readable medium, on which a computer program is stored, and characterized in that when the program is executed by a processor, it implements the above Figure 1 and Figure 2 fusion method described in the embodiments.

[0080] According to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it executes the above functions defined in the system of the present application.

[0081] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the above two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0083] The modules involved in the embodiments of the present invention can be implemented in software or in hardware, and the described modules can also be provided in a processor. Among them, the names of these modules do not constitute a limitation to the modules themselves in certain cases.

[0084] It should be noted that although several modules or units of a device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present invention, the features and functions of two or more of the above-described modules or units can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by a plurality of modules or units.

[0085] From the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present invention.

[0086] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present invention. This application is intended to cover any variations, uses, or adaptations of the present invention, which follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed by the present invention. The specification and examples are only considered as exemplary, and the true scope and spirit of the present invention are pointed out by the following claims.

[0087] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. An electroencephalogram (EEG) and eye movement fusion method for AR target recognition, characterized in that, it includes the following steps: Synchronously collect eye movement data, EEG data, and Trigger signals when gazing at an AR glasses; Perform denoising preprocessing on the eye movement data and EEG data according to the stimulation time indicated by the Trigger signal; Use the distance weighted average algorithm for the preprocessed eye movement data to determine four adjacent target points of the AR glasses stimulation module; Take the label information of the four adjacent target points as the reference frequency and input it into the EEG decoding program to calculate the AR target; Using the distance weighted average algorithm for the preprocessed eye movement data to determine four adjacent target points of the AR glasses stimulation module includes: The preprocessed eye movement data is evenly segmented into 3 groups of eye movement sub-data according to the chronological order of the 3-second stimulation time, and the average coordinates (X 1 , Y 1 ), (X 2 , Y 2 ), (X 3 , Y 3 ) of the 3 groups of eye movement sub-data are calculated respectively; Calculate the weighted average coordinates (X m , Y m ) for the average coordinates of three groups of eye movement data according to a predetermined weighting ratio; Calculate the distance Dw between the weighted average coordinates (X m , Y m ) and the center coordinates (X i , Y i ) of each stimulation module of the AR glasses; Use the sort function in Matlab to sort each calculated distance Dw from smallest to largest to obtain distances Ds1, Ds2, Ds3, Ds4; Determine the four target points corresponding to the distances Ds1, Ds2, Ds3, Ds4 as the four adjacent target points.

2. The EEG and eye movement fusion method according to claim 1, characterized in that, The weight ratio of the average coordinates of the first group of eye movement sub-data is 10%, the weight ratio of the average coordinates of the second group of eye movement sub-data is 20%, and the weight ratio of the average coordinates of the third group of eye movement sub-data is 70%.

3. The EEG and eye movement fusion method according to claim 2, characterized in that, Calculate the weighted average coordinates (X m , Y m ) of the average coordinates of the three groups of eye movement data according to the predetermined weighting ratio. The calculation formula is as follows: (X m , Y m ) = 10%×(X 1 , Y 1 ) + 20%×(X 2 , Y 2 ) + 70%×(X 3 , Y 3 ).

4. An EEG and eye movement fusion device for AR target recognition, characterized in that, it includes: An acquisition module for synchronously collecting eye movement data, EEG data, and Trigger signals when gazing at an AR glasses; A preprocessing module for performing denoising preprocessing on the eye movement data and EEG data according to the stimulation time indicated by the Trigger signal; A screening module for using the distance weighted average algorithm for the preprocessed eye movement data to determine four adjacent target points of the AR glasses stimulation module; An identification module for taking the label information of the four adjacent target points as the reference frequency and inputting it into the EEG decoding program to calculate the AR target; The screening module includes: A segmentation sub-module, which is used to equally segment the preprocessed eye movement data into 3 groups of eye movement sub-data according to the chronological order of 3-second stimulation time, and calculate the average coordinates (X 1 , Y 1 ), (X 2 , Y 2 ), (X 3 , Y 3 ) A coordinate calculation sub-module, which is used to calculate the weighted average coordinates (X m , Y m ) of the average coordinates of three groups of eye movement sub-data according to a predetermined weighting ratio; A distance calculation sub-module, configured to calculate the distance Dw between the weighted average coordinates (X m , Y m ) and the central coordinates (X i , Y i ) of each stimulation module of the AR glasses; A sorting sub-module that uses the sort function in Matlab to sort each calculated distance Dw from smallest to largest to obtain distances Ds1, Ds2, Ds3, Ds4; A determination sub-module for determining the four target points corresponding to the distances Ds1, Ds2, Ds3, Ds4 as the four adjacent target points.

5. An electronic device, including: One or more processors; and A storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the fusion method according to any one of claims 1 to 3.

6. A computer-readable medium having a computer program stored thereon, characterized in that, when the program is executed by a processor, it implements the fusion method according to any one of claims 1 to 3.

Images