ACV autonomous movement method based on brain-computer fusion control and related device

Abstract

The invention discloses an ACV autonomous movement method and related device based on brain-computer fusion control, and the method comprises the steps: carrying out the collection and processing of a brain wave signal of an operator based on a brain wave cap, and obtaining a brain wave signal; preprocessing the brain wave signal based on a brain wave processing unit to form a preprocessed brain wave signal; performing identification processing on the preprocessed brain wave signal based on a machine learning engine, and converting the identified specific brain wave signal into an operation instruction; the operation instruction is issued to a corresponding AGV, after the AGV receives the operation instruction, the operation instruction is analyzed, an operation task in the operation instruction is obtained, and the operation task comprises the step of controlling the AGV to steer and / or move; and executing the operation task in the operation execution based on the current environment data information of the AGV. In the embodiment of the invention, the intuition and the efficiency of man-machine interaction can be effectively improved, and meanwhile, the safety and the reliability are improved.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to an ACV autonomous movement method based on brain-computer fusion control and related devices. Background Art

[0002] Traditional automated guided vehicle (AGV) systems rely on lasers, magnetic strips, optical markers or various sensors for navigation and positioning, and they complete tasks according to preset paths or map information. In recent years, with the advancement of artificial intelligence, AGVs have begun to adopt more complex algorithms, such as deep learning and reinforcement learning, to enhance their environmental perception and autonomous decision-making capabilities. These intelligent algorithms, combined with advanced sensor technologies such as LiDAR and visual sensors, enable AGVs to operate more accurately in complex environments.

[0003] At the same time, non-invasive Brain-Computer Interface (BCI) technology is also developing rapidly. Non-invasive BCI systems record electroencephalograms (EEGs) through electrodes on the scalp and decode brain wave patterns to identify the user's intentions. They have been used in game control and prosthetic operation. Although invasive BCIs provide more sophisticated neural signal capture, their application is limited due to surgical risks and the possibility of infection. Regardless of the type of BCI, signal processing and pattern recognition algorithms are crucial for filtering noise, extracting features, and converting EEG signals into executable instructions through machine learning models.

[0004] Although BCI technology has made some progress in controlling small robots or drones, it still faces problems such as signal reliability, stability, latency, and user fatigue when applied to AGVs. In addition, how to maintain the effectiveness and safety of BCI control in complex industrial environments is also a challenge that needs to be solved. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art. The present invention provides an ACV autonomous movement method and related devices based on brain-computer fusion control, which can effectively improve the intuitiveness and efficiency of human-computer interaction while improving safety and reliability.

[0006] In order to solve the above technical problems, an embodiment of the present invention provides an ACV autonomous movement method based on brain-machine fusion control, the method comprising:

[0007] The operator's brain wave signals are collected and processed based on the EEG cap to obtain brain wave signals;

[0008] Preprocessing the brain wave signal based on the brain wave processing unit to form a preprocessed brain wave signal;

[0009] Recognize and process the preprocessed brain wave signal based on a machine learning engine, and convert the recognized specific brain wave signal into an operation instruction;

[0010] The operation instruction is sent to the corresponding AGV. After receiving the operation instruction, the AGV analyzes the operation instruction to obtain the operation task in the operation instruction, where the operation task includes controlling the AGV to turn and / or move;

[0011] The operation task in the operation execution is performed based on the current environmental data information of the AGV.

[0012] Optionally, the EEG cap has a built-in non-invasive brain-computer interface;

[0013] The operator's brain wave signal is detected based on the non-invasive brain-computer interface built into the EEG cap, and when the brain wave signal is detected, acquisition processing is performed to obtain the brain wave signal.

[0014] Optionally, the preprocessing of the brain wave signal by the brain wave processing unit to form a preprocessed brain wave signal includes:

[0015] Based on the brain wave processing unit, the brain wave signal is filtered and denoised in sequence to generate a processed brain wave signal;

[0016] The processed brain wave signal is subjected to feature extraction processing based on a preset algorithm to obtain feature data corresponding to the brain wave signal.

[0017] Optionally, the performing feature extraction processing on the processed brain wave signal based on a preset algorithm to obtain feature data corresponding to the brain wave signal includes:

[0018] The operator's operation intention is expressed in the form of discrete signal states as follows:

[0019] S = {s1, s2, ..., s n};

[0020] Each discrete signal state s corresponding to the discrete signal state set S i , i = 1, 2, ..., n corresponds to an action instruction of the operator, and the processed brain wave signal is subjected to feature extraction processing according to the discrete signal state set by using the posterior probability algorithm to obtain feature data corresponding to the brain wave signal;

[0021] The posterior probability algorithm is as follows:

[0022]

[0023] Among them, P(s i |X) is the processed brain wave signal X in the discrete signal state s i The posterior probability under ; is the estimated optimal state.

[0024] Optionally, the machine learning engine includes a convolutional neural network and a long short-term memory network;

[0025] The method of identifying and processing the pre-processed brain wave signal based on the machine learning engine and converting the identified specific brain wave signal into an operation instruction includes:

[0026] The pre-processed brain wave signal is input into the convolutional neural network in the machine learning engine for signal feature recognition processing to obtain recognition signal features, and the recognition signal features are converted into feature maps:

[0027] Reshaping the feature graph into a sequence feature, and inputting the sequence feature into the long short-term memory network to perform an operation intention recognition process on the operator, thereby obtaining the operation intention of the operator;

[0028] The corresponding brain wave signal is converted into an operation instruction according to the operation intention.

[0029] Optionally, executing the operation task in the operation execution based on the current environmental data information of the AGV includes:

[0030] The AGV obtains current environmental data information in real time based on the laser radar sensor and camera sensor:

[0031] Execute the operation task in the operation execution based on the current environment data information, and at the same time, the AGV obtains the current position information in real time based on the positioning sensor;

[0032] The position accuracy adjustment control is performed using a root mean square error algorithm based on the current position information.

[0033] Optionally, the performing position accuracy adjustment control based on the current position information by using a root mean square error algorithm includes:

[0034] Based on the current position information, the current position error value is calculated using the position error formula;

[0035] Based on the current position error value, the root mean square error algorithm is used to perform evaluation processing, and the position accuracy is adjusted and controlled according to the evaluation result;

[0036] The position error formula is as follows:

[0037] E(t)=X d(t)-X a (t);

[0038] The root mean square error algorithm is as follows:

[0039]

[0040] Among them, X d (t) represents the expected position; X a (t) represents the actual position; E(t) represents the position error; N represents the number of position errors, t=1, 2, 3, ..., N; RMSE represents the evaluation result of the root mean square error.

[0041] In addition, an embodiment of the present invention further provides an ACV autonomous mobile device based on brain-machine fusion control, the device comprising:

[0042] Signal acquisition module: used to collect and process the operator's brain wave signals based on the EEG cap to obtain brain wave signals;

[0043] Signal processing module: used for preprocessing the brain wave signal based on the brain wave processing unit to form a preprocessed brain wave signal;

[0044] Signal recognition module: used to recognize and process the pre-processed brain wave signal based on the machine learning engine, and convert the recognized specific brain wave signal into an operation instruction;

[0045] Instruction analysis module: used to send the operation instruction to the corresponding AGV. After receiving the operation instruction, the AGV analyzes the operation instruction to obtain the operation task in the operation instruction, and the operation task includes controlling the AGV to turn and / or move;

[0046] Instruction execution module: used to execute the operation task in the operation execution based on the current environmental data information of the AGV.

[0047] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the ACV autonomous movement method as described in any one of the above is implemented.

[0048] In addition, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0049] one or more processors;

[0050] Memory:

[0051] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to: execute the ACV autonomous mobility method according to any one of the above.

[0052] In an embodiment of the present invention, the brain-computer interface technology is used to directly capture the operator's intentions, thereby achieving a more natural and intuitive control method. The operator does not need to manually operate the remote control or use voice commands, but only needs to think about the corresponding actions to control the movement of the AGV, which greatly simplifies the control process and improves work efficiency. Combined with environmental data information, the AGV can make autonomous decisions and adapt to complex and changeable working environments, reducing dependence on external manual intervention, and ensuring that the AGV can detect and avoid obstacles in a timely manner during operation, effectively preventing the occurrence of collision accidents. Control through the brain-computer interface reduces the possibility of human operational errors and further improves the safety performance of the overall system. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 is a flow chart of an ACV autonomous movement method based on brain-machine fusion control in an embodiment of the present invention;

[0055] Figure 2 Schematic diagram of the structure of an ACV autonomous mobile device based on brain-machine fusion control in an embodiment of the present invention;

[0056] Figure 3 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] For example, see Figure 1 , Figure 1 It is a flow chart of the ACV autonomous movement method based on brain-machine fusion control in an embodiment of the present invention.

[0059] like Figure 1 As shown, an ACV autonomous movement method based on brain-machine fusion control, the method comprising:

[0060] S11: collecting and processing the operator's brain wave signal based on the EEG cap to obtain the brain wave signal;

[0061] In the specific implementation process of the present invention, the EEG cap is equipped with a non-invasive brain-computer interface; the operator's brain wave signals are detected based on the non-invasive brain-computer interface built into the EEG cap, and when the brain wave signals are detected, acquisition processing is performed to obtain the brain wave signals.

[0062] Specifically, a non-invasive brain-computer interface (BCI) is built into the EEG cap. BCI is a technology that allows the human brain to communicate directly with external devices without implanting any physical devices in the brain. It detects electrical signals generated by brain activity and converts these signals into commands that can be recognized by the computer. BCI is used to capture the operator's intention signals, namely brain wave signals.

[0063] S12: preprocessing the brain wave signal based on the brain wave processing unit to form a preprocessed brain wave signal;

[0064] In the specific implementation process of the present invention, the brain wave signal is preprocessed based on the brain wave processing unit to form a preprocessed brain wave signal, including: filtering and denoising the brain wave signal in sequence based on the brain wave processing unit to generate a processed brain wave signal; performing feature extraction processing on the processed brain wave signal based on a preset algorithm to obtain feature data corresponding to the brain wave signal.

[0065] Furthermore, the feature extraction process is performed on the processed brain wave signal based on a preset algorithm to obtain feature data corresponding to the brain wave signal, including: expressing the operation intention of the operator in the form of discrete signal states, as follows:

[0066] S = {s1, s2, ..., s n};

[0067] Each discrete signal state s corresponding to the discrete signal state set S i , i=1, 2, ..., n corresponds to an action instruction of the operator, and the processed brain wave signal is subjected to feature extraction processing according to the discrete signal state set by using the posterior probability algorithm to obtain feature data corresponding to the brain wave signal; the posterior probability algorithm is as follows:

[0068]

[0069] Among them, P(s i}X) is the processed brain wave signal X in the discrete signal state s i The posterior probability under ; is the estimated optimal state.

[0070] Specifically, after obtaining the operator's brain wave signal, in order to facilitate subsequent processing, the brain wave signal must first be filtered and denoised in sequence by a brain wave processing unit to generate a processed brain wave signal; then, feature extraction is performed on the processed brain wave signal through a preset algorithm to obtain feature data corresponding to the brain wave signal.

[0071] That is, the brain wave signals obtained by BCI are used to identify the operator's operating intention. This process usually involves signal processing, etc. It is assumed that the operator's intention can be represented by a series of discrete signal states, such as:

[0072] S = {s1, s2, ..., s n};

[0073] Each discrete signal state s corresponding to the discrete signal state set S i , i = 1, 2, ..., n corresponds to an action instruction of the operator, and the processed brain wave signal is subjected to feature extraction processing according to the discrete signal state set by using the posterior probability algorithm to obtain feature data corresponding to the brain wave signal;

[0074] The posterior probability algorithm is as follows:

[0075]

[0076] Among them, P(s i}X) is the processed brain wave signal X in the discrete signal state s i The posterior probability under ; is the estimated optimal state.

[0077] S13: Recognize and process the pre-processed brain wave signal based on a machine learning engine, and convert the recognized specific brain wave signal into an operation instruction;

[0078] In the specific implementation process of the present invention, the machine learning engine includes a convolutional neural network and a long short-term memory network; the pre-processed brain wave signal is identified and processed based on the machine learning engine, and the identified specific brain wave signal is converted into an operation instruction, including: inputting the pre-processed brain wave signal into the convolutional neural network in the machine learning engine for signal feature recognition processing to obtain the identified signal feature, and converting the identified signal feature into a feature map; reshaping the feature map into a sequence feature, and inputting the sequence feature into the long short-term memory network for operator's operation intention recognition processing to obtain the operator's operation intention; converting the corresponding brain wave signal into an operation instruction according to the operation intention.

[0079] Specifically, the machine learning engine can identify and understand the operator's thinking patterns, and then predict and execute corresponding actions. This enables the AGV to respond to the operator's intentions with minimal human intervention; in this embodiment, the machine learning engine contains two deep learning models - convolutional neural network (CNN) and long short-term memory network (LSTM).

[0080] CNN is good at processing inputs with grid structures, such as images and time series data, which makes it perform well in processing EEG signals, which are a series of voltage fluctuations that change over time. CNN automatically learns local and global features in these signals through convolutional and pooling layers, preparing for the next step of identifying different types of operator intentions.

[0081] CNN usually contains multiple convolutional layers for feature extraction and fully connected layers for classification or regression. Assume that the input EEG signal is X∈R T×C , where T is the time step and C is the number of channels. After the convolution layer and the pooling layer, the output feature map Z∈R H×w×D , where H and W are the height and width of the feature map, and D is the depth (number of channels) of the feature map.

[0082] The training process of a CNN involves a back-propagation algorithm that minimizes a loss function, such as the cross-entropy loss, via gradient descent to update the weight matrix W:

[0083]

[0084] LSTM is a special recurrent neural network (RNN) designed to solve long-term dependency problems and is very suitable for processing brain wave signals with time series properties. Through the gating mechanism, LSTM can retain important historical information while discarding irrelevant information, which is crucial for understanding and predicting the operator's continuous intention sequence.

[0085] The LSTM unit contains an input gate, a forget gate, an output gate, and a cell state for storing information. Suppose the input sequence X = [x1, x2, ..., x T ]; LSTM calculates the hidden state h through the following steps t and cell state c t :

[0086] i t =σ(W xi x t +W hi h t-1 +b i );

[0087] f t =σ(W xf x t +W hf h t-1 +b f );

[0088]

[0089]

[0090] o t =σ(W xo x t +W ho h t-1 +b o );

[0091] h t =o t ⊙tanh(c t );

[0092] Among them, x t represents the input vector of the current time step; h t-1 represents the hidden state vector of the previous time step; c t-1 represents the unit state vector of the previous time step; W xi Represents the weight matrix, which is used to transform the input x of the current time step t Converted to the input part of the input gate; W hi Represents the weight matrix, which is used to transform the hidden state c of the current time step t-1 Converted to the input part of the input gate; W xf Represents the weight matrix, which is used to transform the input x of the current time step t Converted to the input part of the forget gate; W hf Represents the weight matrix, which is used to transform the hidden state h of the previous time step t-1 Converted to the input part of the forget gate; W xcRepresents the weight matrix, which is used to transform the input x of the current time step t The input part that is converted into the candidate cell state; W hc Represents the weight matrix, which is used to transform the hidden state h of the previous time step t-1 The input part that is converted into the candidate cell state; W xo Represents the weight matrix, which is used to transform the input x of the current time step t Converted to the input part of the output gate; W ho Represents the weight matrix, which is used to transform the hidden state h of the previous time step t-1 Converted to the input part of the output gate; b i , b f , b c , b o Represent the bias items of input gate, forget gate, candidate cell state and output gate respectively; i t represents the output of the input gate, which determines how much information of the current time step is written into the cell state; f t Represents the output of the forget gate, which determines how much information in the cell state of the previous time step is retained; represents the candidate cell state, which is calculated from the current input and the previous hidden state; c t represents the updated cell state, which is a weighted combination of the old cell state and the new candidate cell state; o t Represents the output of the output gate, which determines how much information is output in the hidden state of the current time step; h t represents the hidden state of the current time step, which is calculated based on the updated cell state and the output of the output gate; σ is a given parameter.

[0093] The flow of information is controlled through a gating mechanism to effectively handle long-term dependencies, so that the operator's continuous intention sequence can be more accurately understood and predicted in applications such as AGV control.

[0094] Once the features are extracted from CNN, they can be used as input to LSTM to capture dependencies in time series. Since LSTM requires serialized input, the feature map Z output by CNN needs to be reshaped into a sequence. This is achieved by flattening the feature map into a series of vectors, each of which represents the features of a time point. Assuming that the shape of Z is H×W×D, it can be reshaped into HW vectors, each of which has a length of D. Then a sequence [z1, z2, ..., z HW ].

[0095] The above sequence can be directly used as the input of LSTM; LSTM receives a feature vector at each time step and tracks the temporal dependency through its internal state; LSTM receives the feature sequence from CNN and uses its internal gating mechanism to process these sequence data.

[0096] The final output of the LSTM is the hidden state h at the last time step t ; Classify or regress the LSTM output through one or more fully connected layers to predict the operator's intention; the final intention prediction Generated by activation function.

[0097] The whole process is as follows:

[0098] EEG signal → CNN → feature map Z;

[0099] Feature map → reshape into sequence → LSTM input sequence;

[0100] LSTM → captures temporal dependencies → final hidden state h t ;

[0101] Fully connected layer → intent prediction

[0102] When the operator's operation intention is obtained, that is, when the operation intention corresponding to the brain wave signal is identified, the specific brain wave signal corresponding to the operation intention is converted into a corresponding operation instruction.

[0103] S14: Sending the operation instruction to the corresponding AGV. After receiving the operation instruction, the AGV analyzes the operation instruction to obtain the operation task in the operation instruction, where the operation task includes controlling the AGV to turn and / or move.

[0104] In the specific implementation process of the present invention, it is necessary to send the operation instruction to the corresponding AGV. After receiving the operation instruction, the AGV will parse the operation instruction accordingly and obtain the operation task in the operation instruction, which mainly includes controlling the AGV to turn and / or move.

[0105] S15: Execute the operation task in the operation execution based on the current environment data information of the AGV.

[0106] In the specific implementation process of the present invention, the operation task in the operation execution is executed based on the current environmental data information of the AGV, including: the AGV obtains the current environmental data information in real time based on the lidar sensor and the camera sensor; the operation task in the operation execution is executed based on the current environmental data information, and at the same time, the AGV obtains the current position information in real time based on the positioning sensor; and the position accuracy is adjusted and controlled by using the root mean square error algorithm based on the current position information.

[0107] Further, the position accuracy adjustment control based on the current position information using the root mean square error algorithm includes: obtaining the current position error value by calculating the position error formula based on the current position information; performing evaluation processing based on the current position error value using the root mean square error algorithm, and performing position accuracy adjustment control according to the evaluation result; the position error formula is as follows:

[0108] E(t)=X d (t)-X a (t);

[0109] The root mean square error algorithm is as follows:

[0110]

[0111] Among them, X d (t) represents the expected position; X a (t) represents the actual position; E(t) represents the position error; N represents the number of position errors, t=1, 2, 3, ..., N; RMSE represents the evaluation result of the root mean square error.

[0112] Specifically, AGV is equipped with an environmental perception system and actuators. The environmental perception system continuously monitors the surrounding environment. If an obstacle is detected, the AGV control system will intervene first, and the path planning and obstacle avoidance module will recalculate the path to avoid safety accidents and ensure the safe driving of the AGV. Among them, the environmental perception system includes lidar sensors and camera sensors, which enable it to move autonomously on a preset path while having obstacle avoidance and positioning functions. Combined with BCI and machine learning, AGV can better adapt to complex environmental changes.

[0113] While executing the operation tasks in the operation execution according to the current environmental data information, the AGV will use the positioning sensor to obtain the current position information in real time; and then use the root mean square error algorithm to adjust the position accuracy according to the current position information.

[0114] That is, in order to ensure the control accuracy of AGV, the real-time feedback mechanism is crucial. The position error E(t) of AGV is expressed as follows:

[0115] E(t)=X d (t)-X a (t);

[0116] The root mean square error algorithm is as follows:

[0117]

[0118] Among them, X d (t) represents the expected position; X a (t) represents the actual position; E(t) represents the position error; N represents the number of position errors, t=1, 2, 3, ..., N; RMSE represents the evaluation result of the root mean square error.

[0119] In an embodiment of the present invention, the brain-computer interface technology is used to directly capture the operator's intentions, thereby achieving a more natural and intuitive control method. The operator does not need to manually operate the remote control or use voice commands, but only needs to think about the corresponding actions to control the movement of the AGV, which greatly simplifies the control process and improves work efficiency. Combined with environmental data information, the AGV can make autonomous decisions and adapt to complex and changeable working environments, reducing dependence on external manual intervention, and ensuring that the AGV can detect and avoid obstacles in a timely manner during operation, effectively preventing the occurrence of collision accidents. Control through the brain-computer interface reduces the possibility of human operational errors and further improves the safety performance of the overall system.

[0120] For example 2, please refer to Figure 2 , Figure 2 It is a schematic diagram of the structural composition of the ACV autonomous mobile device based on brain-machine fusion control in an embodiment of the present invention.

[0121] like Figure 2 As shown, an ACV autonomous mobile device based on brain-machine fusion control, the device comprises:

[0122] Signal acquisition module 21: used to collect and process the operator's brain wave signals based on the EEG cap to obtain brain wave signals;

[0123] In the specific implementation process of the present invention, the EEG cap is equipped with a non-invasive brain-computer interface; the operator's brain wave signals are detected based on the non-invasive brain-computer interface built into the EEG cap, and when the brain wave signals are detected, acquisition processing is performed to obtain the brain wave signals.

[0124] Specifically, a non-invasive brain-computer interface (No n -invasiv eBrain-Computer Interface (BCI), BCI is a technology that allows the human brain to communicate directly with external devices without implanting any physical devices in the brain; it detects the electrical signals generated by brain activity and converts these signals into commands that can be recognized by the computer. BCI is used to capture the operator's intention signals, namely brain wave signals.

[0125] Signal processing module 22: used for preprocessing the brain wave signal based on the brain wave processing unit to form a preprocessed brain wave signal;

[0126] In the specific implementation process of the present invention, the brain wave signal is preprocessed based on the brain wave processing unit to form a preprocessed brain wave signal, including: filtering and denoising the brain wave signal in sequence based on the brain wave processing unit to generate a processed brain wave signal; performing feature extraction processing on the processed brain wave signal based on a preset algorithm to obtain feature data corresponding to the brain wave signal.

[0127] Furthermore, the feature extraction process is performed on the processed brain wave signal based on a preset algorithm to obtain feature data corresponding to the brain wave signal, including: expressing the operation intention of the operator in the form of discrete signal states, as follows:

[0128] S = {s1, s2, ..., s n};

[0129] Each discrete signal state s corresponding to the discrete signal state set S i , i=1, 2, ..., n corresponds to an action instruction of the operator, and the processed brain wave signal is subjected to feature extraction processing according to the discrete signal state set by using the posterior probability algorithm to obtain feature data corresponding to the brain wave signal; the posterior probability algorithm is as follows:

[0130]

[0131] Among them, P(s i}X) is the processed brain wave signal X in the discrete signal state s i The posterior probability under ; is the estimated optimal state.

[0132] Specifically, after obtaining the operator's brain wave signal, in order to facilitate subsequent processing, the brain wave signal must first be filtered and denoised in sequence by a brain wave processing unit to generate a processed brain wave signal; then, feature extraction is performed on the processed brain wave signal through a preset algorithm to obtain feature data corresponding to the brain wave signal.

[0133] That is, the brain wave signals obtained by BCI are used to identify the operator's operating intention. This process usually involves signal processing, etc. It is assumed that the operator's intention can be represented by a series of discrete signal states, such as:

[0134] S = {s1, s2, ..., s n};

[0135] Each discrete signal state s corresponding to the discrete signal state set S i , i = 1, 2, ..., n corresponds to an action instruction of the operator, and the processed brain wave signal is subjected to feature extraction processing according to the discrete signal state set by using the posterior probability algorithm to obtain feature data corresponding to the brain wave signal;

[0136] The posterior probability algorithm is as follows:

[0137]

[0138] Among them, P(s i}X) is the processed brain wave signal X in the discrete signal state s i The posterior probability under ; is the estimated optimal state.

[0139] Signal recognition module 23: used to recognize and process the pre-processed brain wave signal based on a machine learning engine, and convert the recognized specific brain wave signal into an operation instruction;

[0140] In the specific implementation process of the present invention, the machine learning engine includes a convolutional neural network and a long short-term memory network; the pre-processed brain wave signal is identified and processed based on the machine learning engine, and the identified specific brain wave signal is converted into an operation instruction, including: inputting the pre-processed brain wave signal into the convolutional neural network in the machine learning engine for signal feature recognition processing to obtain the identified signal feature, and converting the identified signal feature into a feature map; reshaping the feature map into a sequence feature, and inputting the sequence feature into the long short-term memory network for operator's operation intention recognition processing to obtain the operator's operation intention; converting the corresponding brain wave signal into an operation instruction according to the operation intention.

[0141] Specifically, the machine learning engine can identify and understand the operator's thinking patterns, and then predict and execute corresponding actions. This enables the AGV to respond to the operator's intentions with minimal human intervention; in this embodiment, the machine learning engine contains two deep learning models - convolutional neural network (CNN) and long short-term memory network (LSTM).

[0142] CNN is good at processing inputs with grid structures, such as images and time series data, which makes it perform well in processing EEG signals, which are a series of voltage fluctuations that change over time. CNN automatically learns local and global features in these signals through convolutional and pooling layers, preparing for the next step of identifying different types of operator intentions.

[0143] CNN usually contains multiple convolutional layers for feature extraction and fully connected layers for classification or regression. Assume that the input EEG signal is X∈R T×C , where T is the time step and C is the number of channels. After the convolution layer and the pooling layer, the output feature map Z∈R H×W×D , where H and W are the height and width of the feature map, and D is the depth (number of channels) of the feature map.

[0144] The training process of a CNN involves a back-propagation algorithm that minimizes a loss function, such as the cross-entropy loss, via gradient descent to update the weight matrix W:

[0145]

[0146] LSTM is a special recurrent neural network (RNN) designed to solve long-term dependency problems and is very suitable for processing brain wave signals with time series properties. Through the gating mechanism, LSTM can retain important historical information while discarding irrelevant information, which is crucial for understanding and predicting the operator's continuous intention sequence.

[0147] The LSTM unit contains an input gate, a forget gate, an output gate, and a cell state for storing information. Suppose the input sequence X = [x1, x2, ..., x T ]; LSTM calculates the hidden state h through the following steps t and cell state c t :

[0148] i t =σ(W xi x t +W hi h t-1 +b i );

[0149] f t =σ(W xf x t +W hf h t-1 +b f );

[0150]

[0151]

[0152] o t =σ(W xo x t +W ho h t-1 +b o );

[0153] h t =o t ⊙tanh(c t );

[0154] Among them, x t represents the input vector of the current time step; h t-1 represents the hidden state vector of the previous time step; c t-1 represents the unit state vector of the previous time step; W xi Represents the weight matrix, which is used to transform the input x of the current time step t Converted to the input part of the input gate; W hi Represents the weight matrix, which is used to transform the hidden state c of the current time step t-1 Converted to the input part of the input gate; W xf Represents the weight matrix, which is used to transform the input x of the current time step t Converted to the input part of the forget gate; W hf Represents the weight matrix, which is used to transform the hidden state h of the previous time step t-1 Converted to the input part of the forget gate; W xc Represents the weight matrix, which is used to transform the input x of the current time step t The input part that is converted into the candidate cell state; W hc Represents the weight matrix, which is used to transform the hidden state h of the previous time step t-1 The input part that is converted into the candidate cell state; W xo Represents the weight matrix, which is used to transform the input x of the current time step t Converted to the input part of the output gate; W ho Represents the weight matrix, which is used to transform the hidden state h of the previous time step t-1 Converted to the input part of the output gate; b i , b f , b c , b o Represent the bias items of input gate, forget gate, candidate cell state and output gate respectively; i t represents the output of the input gate, which determines how much information of the current time step is written into the cell state; f t Represents the output of the forget gate, which determines how much information in the cell state of the previous time step is retained; represents the candidate cell state, which is calculated from the current input and the previous hidden state; c t represents the updated cell state, which is a weighted combination of the old cell state and the new candidate cell state; o t Represents the output of the output gate, which determines how much information is output in the hidden state of the current time step; h t represents the hidden state of the current time step, which is calculated based on the updated cell state and the output of the output gate; σ is a given parameter.

[0155] The flow of information is controlled through a gating mechanism to effectively handle long-term dependencies, so that the operator's continuous intention sequence can be more accurately understood and predicted in applications such as AGV control.

[0156] Once the features are extracted from CNN, they can be used as input to LSTM to capture dependencies in time series. Since LSTM requires serialized input, the feature map Z output by CNN needs to be reshaped into a sequence. This is achieved by flattening the feature map into a series of vectors, each of which represents the features of a time point. Assuming that the shape of Z is H×W×D, it can be reshaped into HW vectors, each of which has a length of D. Then a sequence [z1, z2, ..., z HW ].

[0157] The above sequence can be directly used as the input of LSTM; LSTM receives a feature vector at each time step and tracks the temporal dependency through its internal state; LSTM receives the feature sequence from CNN and uses its internal gating mechanism to process these sequence data.

[0158] The final output of the LSTM is the hidden state h at the last time step t ; Classify or regress the LSTM output through one or more fully connected layers to predict the operator's intention; the final intention prediction Generated by activation function.

[0159] The whole process is as follows:

[0160] EEG signal → CNN → feature map Z;

[0161] Feature map → reshape into sequence → LSTM input sequence;

[0162] LSTM → captures temporal dependencies → final hidden state h t ;

[0163] Fully connected layer → intent prediction

[0164] When the operator's operation intention is obtained, that is, when the operation intention corresponding to the brain wave signal is identified, the specific brain wave signal corresponding to the operation intention is converted into a corresponding operation instruction.

[0165] Instruction analysis module 24: used to send the operation instruction to the corresponding AGV. After receiving the operation instruction, the AGV analyzes the operation instruction to obtain the operation task in the operation instruction, and the operation task includes controlling the AGV to turn and / or move;

[0166] In the specific implementation process of the present invention, it is necessary to send the operation instruction to the corresponding AGV. After receiving the operation instruction, the AGV will parse the operation instruction accordingly and obtain the operation task in the operation instruction, which mainly includes controlling the AGV to turn and / or move.

[0167] Instruction execution module 25: used to execute the operation task in the operation execution based on the current environmental data information of the AGV.

[0168] In the specific implementation process of the present invention, the operation task in the operation execution is executed based on the current environmental data information of the AGV, including: the AGV obtains the current environmental data information in real time based on the lidar sensor and the camera sensor; the operation task in the operation execution is executed based on the current environmental data information, and at the same time, the AGV obtains the current position information in real time based on the positioning sensor; and the position accuracy is adjusted and controlled by using the root mean square error algorithm based on the current position information.

[0169] Further, the position accuracy adjustment control based on the current position information using the root mean square error algorithm includes: obtaining the current position error value by calculating the position error formula based on the current position information; performing evaluation processing based on the current position error value using the root mean square error algorithm, and performing position accuracy adjustment control according to the evaluation result; the position error formula is as follows:

[0170] E(t)=X d (t)-X a (t);

[0171] The root mean square error algorithm is as follows:

[0172]

[0173] Among them, X d (t) represents the expected position; X a (t) represents the actual position; E(t) represents the position error; N represents the number of position errors, t=1, 2, 3, ..., N; RMSE represents the evaluation result of the root mean square error.

[0174] Specifically, AGV is equipped with an environmental perception system and actuators. The environmental perception system continuously monitors the surrounding environment. If an obstacle is detected, the AGV control system will intervene first, and the path planning and obstacle avoidance module will recalculate the path to avoid safety accidents and ensure the safe driving of the AGV. Among them, the environmental perception system includes lidar sensors and camera sensors, which enable it to move autonomously on a preset path while having obstacle avoidance and positioning functions. Combined with BCI and machine learning, AGV can better adapt to complex environmental changes.

[0175] While executing the operation tasks in the operation execution according to the current environmental data information, the AGV will use the positioning sensor to obtain the current position information in real time; and then use the root mean square error algorithm to adjust the position accuracy according to the current position information.

[0176] That is, in order to ensure the control accuracy of AGV, the real-time feedback mechanism is crucial. The position error E(t) of AGV is expressed as follows:

[0177] E(t)=X d (t)-X a (t);

[0178] The root mean square error algorithm is as follows:

[0179]

[0180] Among them, X d (t) represents the expected position; X a (t) represents the actual position; E(t) represents the position error; N represents the number of position errors, t=1, 2, 3, ..., N; RMSE represents the evaluation result of the root mean square error.

[0181] In an embodiment of the present invention, the brain-computer interface technology is used to directly capture the operator's intentions, thereby achieving a more natural and intuitive control method. The operator does not need to manually operate the remote control or use voice commands, but only needs to think about the corresponding actions to control the movement of the AGV, which greatly simplifies the control process and improves work efficiency. Combined with environmental data information, the AGV can make autonomous decisions and adapt to complex and changeable working environments, reducing dependence on external manual intervention, and ensuring that the AGV can detect and avoid obstacles in a timely manner during operation, effectively preventing the occurrence of collision accidents. Control through the brain-computer interface reduces the possibility of human operational errors and further improves the safety performance of the overall system.

[0182] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the ACV autonomous movement method of any one of the above embodiments is implemented. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the storage device includes any medium that can store or transmit information in a readable form by a device (for example, a computer, a mobile phone), which can be a read-only memory, a disk or an optical disk, etc.

[0183] An embodiment of the present invention further provides a computer application program, which runs on a computer and is used to execute the ACV autonomous movement method of any one of the above embodiments.

[0184] also, Figure 3 It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention.

[0185] The embodiment of the present invention further provides an electronic device, such as Figure 3 The electronic device includes a processor 302, a memory 303, an input unit 304, a display unit 305 and other devices. Those skilled in the art will understand that Figure 3 The electronic device structure device shown does not constitute a limitation on all devices, and may include more or fewer components than shown, or combine certain components. The memory 303 can be used to store the application 301 and various functional modules, and the processor 302 runs the application 301 stored in the memory 303, thereby executing various functional applications and data processing of the device. The memory can be an internal memory or an external memory, or include both internal and external memories. The internal memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, or a random access memory. The external memory may include a hard disk, a floppy disk, a ZIP disk, a U disk, a magnetic tape, etc. The memory disclosed in the present invention includes but is not limited to these types of memories. The memory disclosed in the present invention is only used as an example and not as a limitation.

[0186] The input unit 304 is used to receive the input of the signal and the keyword input by the user. The input unit 304 may include a touch panel and other input devices. The touch panel can collect the user's touch operation on or near it (such as the user's operation on or near the touch panel using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program; other input devices may include but are not limited to one or more of a physical keyboard, a function key (such as a playback control key, a switch key, etc.), a trackball, a mouse, a joystick, etc. The display unit 305 can be used to display the information input by the user or the information provided to the user and various menus of the terminal device. The display unit 305 can be in the form of a liquid crystal display, an organic light emitting diode, etc. The processor 302 is the control center of the terminal device, which uses various interfaces and lines to connect the various parts of the entire device, and executes various functions and processes data by running or executing software programs and / or modules stored in the memory 303, and calling the data stored in the memory.

[0187] As an embodiment, the electronic device includes: one or more processors 302, a memory 303, and one or more applications 301, wherein the one or more applications 301 are stored in the memory 303 and are configured to be executed by the one or more processors 302, and the one or more applications 301 are configured to execute the ACV autonomous mobility method in any one of the above-mentioned embodiments.

[0188] In an embodiment of the present invention, the brain-computer interface technology is used to directly capture the operator's intentions, thereby achieving a more natural and intuitive control method. The operator does not need to manually operate the remote control or use voice commands, but only needs to think about the corresponding actions to control the movement of the AGV, which greatly simplifies the control process and improves work efficiency. Combined with environmental data information, the AGV can make autonomous decisions and adapt to complex and changeable working environments, reducing dependence on external manual intervention, and ensuring that the AGV can detect and avoid obstacles in a timely manner during operation, effectively preventing the occurrence of collision accidents. Control through the brain-computer interface reduces the possibility of human operational errors and further improves the safety performance of the overall system.

[0189] In addition, the above is a detailed introduction to an ACV autonomous movement method based on brain-machine fusion control and related devices provided by an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. An ACV autonomous movement method based on brain-machine fusion control, characterized in that: The method comprises: The operator's brain wave signals are collected and processed based on the EEG cap to obtain brain wave signals; Preprocessing the brain wave signal based on the brain wave processing unit to form a preprocessed brain wave signal; Recognize and process the preprocessed brain wave signal based on a machine learning engine, and convert the recognized specific brain wave signal into an operation instruction; The operation instruction is sent to the corresponding AGV. After receiving the operation instruction, the AGV analyzes the operation instruction to obtain the operation task in the operation instruction, where the operation task includes controlling the AGV to turn and / or move; The operation task in the operation execution is performed based on the current environmental data information of the AGV.

2. The ACV autonomous movement method according to claim 1, characterized in that: The EEG cap has a built-in non-invasive brain-computer interface; The operator's brain wave signal is detected based on the non-invasive brain-computer interface built into the EEG cap, and when the brain wave signal is detected, acquisition processing is performed to obtain the brain wave signal.

3. The ACV autonomous movement method according to claim 1, characterized in that: The preprocessing of the brain wave signal by the brain wave processing unit to form a preprocessed brain wave signal includes: Based on the brain wave processing unit, the brain wave signal is filtered and denoised in sequence to generate a processed brain wave signal; The processed brain wave signal is subjected to feature extraction processing based on a preset algorithm to obtain feature data corresponding to the brain wave signal.

4. The ACV autonomous movement method according to claim 3, characterized in that: The step of performing feature extraction processing on the processed brain wave signal based on a preset algorithm to obtain feature data corresponding to the brain wave signal includes: The operator's operation intention is expressed in the form of discrete signal states as follows: S={s1,s2,…,s n }; Each discrete signal state s corresponding to the discrete signal state set S i , i=1,2,…,n corresponds to an action instruction of the operator, and the feature extraction processing of the processed brain wave signal is performed according to the discrete signal state set by using the posterior probability algorithm to obtain the feature data corresponding to the brain wave signal; The posterior probability algorithm is as follows: Among them, P(s i |X) is the processed brain wave signal X in the discrete signal state s i The posterior probability under ; is the estimated optimal state.

5. The ACV autonomous movement method according to claim 1, characterized in that: The machine learning engine includes a convolutional neural network and a long short-term memory network; The method of identifying and processing the pre-processed brain wave signal based on the machine learning engine and converting the identified specific brain wave signal into an operation instruction includes: Inputting the preprocessed brain wave signal into the convolutional neural network in the machine learning engine for signal feature recognition processing to obtain recognition signal features, and converting the recognition signal features into a feature map; Reshaping the feature graph into a sequence feature, and inputting the sequence feature into the long short-term memory network to perform an operation intention recognition process on the operator, thereby obtaining the operation intention of the operator; The corresponding brain wave signal is converted into an operation instruction according to the operation intention.

6. The ACV autonomous movement method according to claim 1, characterized in that: The performing of the operation task in the operation execution based on the current environment data information of the AGV includes: The AGV obtains current environmental data information in real time based on the laser radar sensor and the camera sensor; Execute the operation task in the operation execution based on the current environment data information, and at the same time, the AGV obtains the current position information in real time based on the positioning sensor; The position accuracy adjustment control is performed using a root mean square error algorithm based on the current position information.

7. The ACV autonomous movement method according to claim 6, characterized in that: The position accuracy adjustment control is performed based on the current position information using a root mean square error algorithm, including: Based on the current position information, the current position error value is calculated using the position error formula; Based on the current position error value, the root mean square error algorithm is used to perform evaluation processing, and the position accuracy is adjusted and controlled according to the evaluation result; The position error formula is as follows: E(t)=X d (t)-X a (t); The root mean square error algorithm is as follows: Among them, X d (t) represents the expected position; X a (t) represents the actual position; E(t) represents the position error; N represents the number of position errors, t = 1, 2, 3, …, N; RMSE represents the evaluation result of the root mean square error.

8. An ACV autonomous mobile device based on brain-machine fusion control, characterized in that: The device comprises: Signal acquisition module: used to collect and process the operator's brain wave signals based on the EEG cap to obtain brain wave signals; Signal processing module: used for preprocessing the brain wave signal based on the brain wave processing unit to form a preprocessed brain wave signal; Signal recognition module: used to recognize and process the pre-processed brain wave signal based on the machine learning engine, and convert the recognized specific brain wave signal into an operation instruction; Instruction analysis module: used to send the operation instruction to the corresponding AGV. After receiving the operation instruction, the AGV analyzes the operation instruction to obtain the operation task in the operation instruction, and the operation task includes controlling the AGV to turn and / or move; Instruction execution module: used to execute the operation task in the operation execution based on the current environmental data information of the AGV.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the ACV autonomous movement method as described in any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to: execute the ACV autonomous mobility method according to any one of claims 1 to 7.

Images