Intelligent car control system

Abstract

The application relates to an intelligent trolley control system, which comprises a signal processor, the signal processor comprises a steering control device, the steering control device comprises a predicted deflection module, a target deflection module and a steering instruction generation module; the predicted deflection module is used for acquiring a real scene image signal to be processed, and a predicted deflection amount is obtained according to the real scene image signal; the target deflection module is used for acquiring an actual deflection amount, and the predicted deflection amount and the actual deflection amount are respectively weighted and then added to obtain a target deflection amount; and the steering instruction generation module is used for generating a corresponding steering control instruction according to the target deflection amount. Compared with the prior art, the difficulty of trolley steering control can be reduced, and an operator can easily control the intelligent trolley.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent car control, and particularly to an intelligent car control system. BACKGROUND

[0002] In the intelligent era, intelligent cars, also known as wheeled robots, have been widely used in various industries and fields such as industry, service industry, and family. At present, the control system of intelligent mobile cars is usually composed of a remote controller containing left and right control keys, an intelligent car control board, and a direct current motor. The remote controller and the car control board are mainly connected wirelessly. The operator realizes the steering control of the intelligent car by operating the left and right keys on the remote controller. However, for some people with physical disabilities, manual control is not friendly. Although the operator can wear a head-mounted gyroscope, and the head-mounted gyroscope sends a steering control signal by rotating the head, and the steering control of the intelligent car is realized according to the steering control signal, this control method requires high reaction ability of the operator and has high control difficulty in complex turning scenarios. SUMMARY

[0003] The present application aims to overcome the shortcomings and deficiencies of the prior art, and provides an intelligent car control system which can reduce the difficulty of car steering control and enable the operator to easily control the intelligent car.

[0004] The present application is realized by the following technical solutions: an intelligent car control system, comprising a signal processor, wherein the signal processor comprises a steering control device, and the steering control device comprises a predicted deflection module, a target deflection module, and a steering instruction generation module.

[0005] The predicted deflection module comprises:

[0006] A preliminary feature extraction submodule is configured to obtain a real scene image signal to be processed, perform maximum pooling after convolution of the real scene image signal, and obtain preliminary image features.

[0007] A first deep feature extraction submodule is configured to perform first deep feature extraction on the preliminary image features to obtain first deep image features.

[0008] A first attention calculation submodule is configured to calculate attention weights based on the first deep image features, and weight the first deep image features by the attention weights to obtain first deep attention features.

[0009] A second deep feature extraction submodule is configured to perform second deep feature extraction on the first deep attention features to obtain second deep image features.

[0010] a second attention calculation submodule configured to calculate an attention weight according to the second depth image feature and weight the second depth image feature by the attention weight to obtain a second depth attention feature;

[0011] a third depth feature extraction submodule configured to perform third depth feature extraction on the second depth attention feature to obtain a third depth image feature;

[0012] a third attention calculation submodule configured to calculate an attention weight according to the third depth image feature and weight the third depth image feature by the attention weight to obtain a third depth attention feature;

[0013] a deflection amount prediction submodule configured to perform global average pooling on the third depth attention feature and then perform deflection amount prediction to obtain a predicted deflection amount;

[0014] the target deflection module is configured to obtain an actual deflection amount, weight and add the predicted deflection amount and the actual deflection amount to obtain a target deflection amount;

[0015] the steering instruction generation module is configured to generate a corresponding steering control instruction according to the target deflection amount.

[0016] Compared with the prior art, the intelligent trolley control system of the present application extracts image features of real scene images in the front direction of the intelligent trolley, predicts a deflection amount according to the image features of the real scene images, and combines the predicted deflection amount with an actual deflection amount generated according to the head movement of the operator to assist the operator in controlling the steering of the trolley, thereby reducing the difficulty of trolley steering control, enabling the operator to more easily control the intelligent trolley to successfully turn in the face of different turning scenarios. Moreover, in the extraction of image features of real scene images, the real scene images are subjected to multiple depth feature extractions, and the extracted depth features are subjected to feature enhancement by attention weights after each depth feature extraction, thereby improving the mutual dependence relationship between feature maps to improve the accuracy of image feature extraction, thereby improving the accuracy of assisted driving.

[0017] Further, the first depth feature extraction submodule is configured to perform 3 consecutive grouped convolutions on the down-sampled preliminary image features to obtain first depth image features;

[0018] the second depth feature extraction submodule is configured to perform 7 grouped convolutions on the down-sampled first depth attention features to obtain second depth image features;

[0019] the third depth feature extraction submodule is configured to perform 3 grouped convolutions on the down-sampled second depth attention features.

[0020] Further, the downsampling comprises the following steps:

[0021] The preliminary image features or the first deep attention features or the second deep attention features are divided into two different channels to obtain image features of two different channels; the image features of one channel are subjected to deep separable convolution with a step of 2 and batch normalization, and then subjected to standard convolution and batch normalization; after the image features of another channel are subjected to standard convolution and batch normalization, they are subjected to deep separable convolution with a step of 2 and batch normalization, and then subjected to standard convolution and batch normalization; the image features extracted from the two different channels are spliced in the channel; and the spliced image features are subjected to channel shuffling to obtain down-sampled image features.

[0022] Further, the grouped convolution comprises the following steps:

[0023] The image features obtained by the downsampling are divided into two different channels to obtain image features of two different channels; after the image features of one channel are subjected to standard convolution and batch normalization, they are subjected to deep separable convolution with a step of 1 and batch normalization, and then subjected to standard convolution and batch normalization; the image features extracted from the channel are spliced with the image features of another channel in the channel; and the spliced image features are subjected to channel shuffling to obtain grouped convolution image features.

[0024] Further, the attention weight calculation comprises the following steps:

[0025] The first deep image features or the second deep image features or the third deep image features are subjected to global average pooling in the channel dimension; the pooled image features are subjected to one-dimensional convolution; and the convolved image features are normalized by a response function to obtain the attention weight of each channel.

[0026] Further, the expression of the target deflection amount is:

[0027] D = αD pred + (1-α)D deflect

[0028] wherein D is the target deflection amount; D pred is the predicted deflection amount; D deflect is the actual deflection amount; α is a weight parameter, and the weight parameter α increases as the value range of the predicted deflection amount D pred increases.

[0029] Further, the device further comprises a concentration control device, and the concentration control device comprises:

[0030] A frequency division module is configured to acquire an EEG signal to be processed, perform frequency division on the EEG signal, and obtain a plurality of EEG frequency division signals.

[0031] a power feature calculation module configured to calculate a power spectrum density of each of the EEG frequency signals to obtain a power feature of each of the EEG frequency signals;

[0032] an energy feature calculation module configured to calculate an energy value of each of the EEG frequency signals according to the power feature of each of the EEG frequency signals to obtain an energy feature of each of the EEG frequency signals;

[0033] a concentration calculation module configured to calculate a concentration value by using a support vector machine with a linear function as a kernel function to calculate the energy features of all the EEG frequency signals;

[0034] a vehicle speed instruction generation module configured to calculate a target vehicle speed according to the concentration value and generate a corresponding vehicle speed control instruction according to the target vehicle speed.

[0035] Further, the device further comprises a blink control device, and the blink control device comprises:

[0036] a band-pass filtering module configured to obtain an EOG signal to be processed, and perform band-pass filtering on the EOG signal to obtain an EOG noise-reduced signal;

[0037] a first-order difference module configured to perform difference processing on an amplitude value of each sampling point in the EOG noise-reduced signal to obtain a first-order difference signal;

[0038] a normalization module configured to normalize the first-order difference signal;

[0039] a second-order difference module configured to perform difference processing on the normalized first-order difference signal again to obtain a second-order difference signal;

[0040] a peak identification module configured to determine that a next sampling point of a sampling point corresponding to a minimum value in the second-order difference signal is a peak point;

[0041] a single-blink identification module configured to determine that the EOG signal is a single-blink signal if there is only one peak point and an amplitude value of the peak point is greater than an amplitude threshold value;

[0042] a double-blink identification module configured to determine that the EOG signal is a double-blink signal if there are multiple peak points, a sampling time interval of adjacent peak points is less than an interval time threshold value, and energy values between the two peak points are both greater than an energy threshold value;

[0043] an instruction generation module configured to generate a corresponding control instruction according to the single-blink signal or the double-blink signal.

[0044] Furthermore, the normalization module is used to normalize to 1 if the amplitude value corresponding to a sampling point in the first-order differential signal is greater than 0; to normalize to 0 if the amplitude value corresponding to a sampling point in the first-order differential signal is equal to 0; and to normalize to -1 if the amplitude value corresponding to a sampling point in the first-order differential signal is less than 0.

[0045] For a sampling point whose amplitude value is normalized to 0, if the value corresponding to the next sampling point is greater than or equal to 0, then the amplitude value corresponding to the current sampling point is normalized to 1; if the amplitude value corresponding to the next sampling point is less than 0, then the amplitude value corresponding to the current sampling point is normalized to -1.

[0046] Furthermore, the instruction generation module is used to generate start control instructions or stop control instructions based on the double blink signal.

[0047] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the structure of an exemplary intelligent vehicle control system in one of the embodiments;

[0049] Figure 2 This is a schematic diagram of the steering control device 10 in the embodiment;

[0050] Figure 3 for Figure 2 A schematic flowchart of the steering control method executed by the steering control device 10 shown.

[0051] Figure 4 This is a schematic diagram of the focus control device 20 in the embodiment;

[0052] Figure 5 for Figure 4 A flowchart illustrating the attention control method executed by the attention control device 20 shown.

[0053] Figure 6 This is a schematic diagram of the blink control device 30 in the embodiment;

[0054] Figure 7 for Figure 6 A flowchart illustrating the blink control method executed by the blink control device 30 shown. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0056] Please see Figure 1It is a structural schematic diagram of an exemplary intelligent car control system in the embodiment. The intelligent car control system includes an intelligent car A, a mobile terminal B, a display C, an electroencephalogram signal acquisition device D, and a signal processor E. The intelligent car A is a four-wheel robot, including a motor and a control unit. The control unit receives control instructions from the signal processor E and sends control signals to the motor according to the control instructions. The motor drives the wheels to rotate under the control of the control signals, so that the intelligent car A realizes actions such as moving forward, moving backward, decelerating, decelerating, turning left, turning right, starting, and stopping. In the embodiment, the control unit of the intelligent car A uses Raspberry Pi 4B, and the motor uses a four-phase DC motor driven by an L298N motor control chip.

[0057] The mobile terminal B is loaded on the execution device 1. The mobile terminal B includes a camera and a processor. The camera is used to collect a real scene image signal in the forward direction of the execution device 1. The mobile terminal B transmits the real scene image signal to the signal processor E. At the same time, the processor receives the real scene image signal and runs an AR client to render an AR picture for the real scene image signal to obtain a virtual reality image signal. The AR client also changes the AR picture according to the control instructions transmitted by the signal processor E to realize the control of the virtual reality image signal by the operator. For example, the AR picture can be a racing game picture. The operator controls the real intelligent car A to play a virtual racing game, and the signal processor E transmits control instructions using game props in the racing game.

[0058] The display C is used to display the real scene image signal collected by the mobile terminal B to provide control feedback for the operator. At the same time, the display C is used to display a virtual display image signal. The display C can be a VR display such as VR glasses.

[0059] The electroencephalogram signal acquisition device D is used to collect EEG (electroencephalogram) signals and EOG (electro-oculogram) signals of the system user. The electroencephalogram signal acquisition device D is provided with a gyroscope for collecting the deflection angle of the head rotation of the operator and outputting the corresponding gyroscope signal. The electroencephalogram signal acquisition device D transmits the collected signals to the signal processor E. The signal transmission between the electroencephalogram signal acquisition device D and the signal processor E can be performed through Bluetooth.

[0060] The signal processor E comprises a steering control device 10, a concentration control device 20 and a blinking control device 30. The steering control device 10 is configured to obtain a target deflection amount according to the gyroscope signal and the real scene image signal, and generate a steering control instruction according to the target deflection amount, the steering control instruction controlling the left or right turning and the turning angle of the smart car A. The concentration control device 20 is configured to obtain a concentration value according to the EEG signal, and generate a vehicle speed control instruction according to the concentration value, the vehicle speed control instruction controlling the moving speed of the smart car A. The blinking control device 30 is configured to identify whether the eye movement of the operator is a single blinking action or a double blinking action according to the EOG signal, and generate different control instructions according to the single blinking action or the double blinking action. For example, a prop triggering control instruction is generated according to the single blinking action, the prop triggering control instruction controlling the use of the game prop of the racing game performed by the AR client. A start-stop control instruction is generated according to the double blinking action, the start-stop control instruction controlling the start and stop of the smart car A.

[0061] Specifically, refer to Figure 2 and Figure 3 wherein Figure 2 is a structural schematic diagram of the steering control device 10 of the embodiment; Figure 3 is a flowchart of a steering control method performed by the steering control device 10. The steering control device 10 comprises a predicted deflection module 11, a target deflection module 12 and a steering instruction generation module 13.

[0062] The predicted deflection module 11 is configured to perform step S11: obtaining a real scene image signal to be processed, and predicting a predicted deflection amount according to the real scene image signal.

[0063] Further, the predicted deflection module 11 comprises a preliminary feature extraction submodule 111, a first deep feature extraction submodule 112, a first attention calculation submodule 113, a second deep feature extraction submodule 114, a second attention calculation submodule 115, a third deep feature extraction submodule 116, a third attention calculation submodule 117 and a deflection amount prediction submodule 118. The preliminary feature extraction submodule 111 is configured to perform step S111: performing maximum pooling after convolution on the real scene image signal to be processed to obtain preliminary image features.

[0064] The first deep feature extraction submodule 112 is configured to perform step S112: performing first deep feature extraction on the preliminary image features to obtain first deep image features.

[0065] The first deep feature extraction submodule 112 is configured to perform step S112: performing first deep feature extraction on the preliminary image features to obtain first deep image features.

[0066] The preliminary image features are down-sampled to reduce the spatial size of the preliminary image features and extract higher-level features, including the steps of: channel division on the preliminary image features to obtain image features of two different channels; depth separable convolution with a step size of 2 and batch normalization are performed on the image features of one of the channels, and then standard convolution and batch normalization are performed, and the activation function of the standard convolution is a RELU function; after standard convolution and batch normalization are performed on the image features of the other channel, depth separable convolution with a step size of 2 and batch normalization are performed, and then standard convolution and batch normalization are performed, and the activation function of the standard convolution is a RELU function; the image features extracted from the two different channels are spliced in the channel; and the spliced image features are mixed in the channel to obtain down-sampled image features.

[0067] Each grouping convolution includes the steps of: channel division on the image features to be convolved to obtain image features of two different channels; after standard convolution and batch normalization are performed on the image features of one of the channels, depth separable convolution with a step size of 1 and batch normalization are performed, and then standard convolution and batch normalization are performed, and the activation function of the standard convolution is a RELU function; the image features extracted from the channel are spliced with the image features of the other channel in the channel; and the spliced image features are mixed in the channel to obtain grouping convolution image features. Wherein, the image features to be convolved are down-sampled image features or grouping convolution image features obtained by the previous grouping convolution, and the grouping convolution image features obtained by the last grouping convolution are the first depth image features.

[0068] The first attention calculation sub-module 113 is configured to perform step S113: performing attention weight calculation according to the first depth image features, and weighting the first depth image features by the attention weight to obtain first depth attention features.

[0069] Wherein, the attention weight calculation according to the first depth image features includes the steps of: performing global average pooling on the first depth image features in the channel dimension; performing one-dimensional convolution on the pooled image features; and normalizing the convolved image features by a response function to obtain the attention weight of each channel.

[0070] The first depth attention features are obtained by element-wise multiplication of the feature elements in the corresponding channel of the first depth image features by the attention weight of each channel.

[0071] The second depth feature extraction sub-module 114 is configured to perform step S114: performing second depth feature extraction on the first depth attention features to obtain second depth image features.

[0072] The second deep feature extraction on the first deep attention feature comprises downsampling and 7 times of grouped convolution on the first deep attention feature, and the steps of downsampling and each time of grouped convolution on the first deep attention feature are the same as the steps of downsampling and grouped convolution described in the step S112, and will not be repeated here.

[0073] The second attention calculation sub-module 115 is configured to perform a step S115 of performing attention weight calculation according to the second depth image feature, and weighting the second depth image feature by the attention weight to obtain a second deep attention feature.

[0074] The step of performing attention weight calculation according to the second depth image feature is the same as the step described in the step S113, and will not be repeated here.

[0075] The third deep feature extraction sub-module 116 is configured to perform a step S116 of performing third deep feature extraction on the second deep attention feature to obtain a third depth image feature.

[0076] The third deep feature extraction on the second deep attention feature comprises downsampling and 3 times of grouped convolution on the second deep attention feature, and the steps of downsampling and each time of grouped convolution on the second deep attention feature are the same as the steps of downsampling and grouped convolution described in the step S112, and will not be repeated here.

[0077] The third attention calculation sub-module 117 is configured to perform a step S117 of performing attention weight calculation according to the third depth image feature, and weighting the third depth image feature by the attention weight to obtain a third deep attention feature.

[0078] The step of performing attention weight calculation according to the third depth image feature is the same as the step described in the step S113, and will not be repeated here.

[0079] The deflection amount prediction sub-module 118 is configured to perform a step S118 of performing deflection amount prediction on the third deep attention feature after global average pooling to obtain a predicted deflection amount.

[0080] The deflection amount prediction on the third deep attention feature after global average pooling is performed by a classifier, which can be selected as a fully connected layer or a softmax function. The predicted deflection amount has a value range of -180°-180°.

[0081] The target deflection module 12 is configured to perform a step S12 of obtaining an actual deflection amount, weighting and adding the predicted deflection amount and the actual deflection amount respectively to obtain a target deflection amount.

[0082] Wherein, the actual deflection amount is the gyro signal, that is, the rotation angle in the horizontal direction detected by the gyro.

[0083] The expression of adding the predicted deflection amount and the actual deflection amount after weighting is as follows:

[0084] D = aD pred + (1-a)D deflect

[0085] Wherein, D is the target deflection amount, D pred is the predicted deflection amount, D deflect is the actual deflection amount, and a is the weight parameter.

[0086] Therefore, the steering control of the operator can be assisted by the predicted deflection amount, and different steering control effects can be obtained by setting different weight parameters a, and the specific data can be set according to the actual needs of the operator.

[0087] In order to improve the auxiliary effect under sharp turns, different weight parameters a can be set for the predicted deflection amount in different value intervals, and the weight parameter a is greater when the predicted deflection amount interval is greater. In a specific implementation, when the absolute value of D pred is less than 30 degrees, the weight parameter a is 0.55; when the absolute value of D pred is greater than 30 degrees and less than 60 degrees, the weight parameter a is 0.7; and when the absolute value of D pred is greater than 60 degrees, the weight parameter a is 0.9.

[0088] The steering instruction generation module 13 is used to execute step S13: generating a corresponding steering control instruction according to the target deflection amount.

[0089] Please refer to Figure 4 and Figure 5 , wherein Figure 4 is a structural schematic diagram of the concentration control device 20 of the embodiment; Figure 5 is a flow schematic diagram of the concentration control method executed by the concentration control device 20. The concentration control device 20 includes a frequency division module 21, a power feature calculation module 22, an energy feature calculation module 23, a concentration calculation module 24, and a vehicle speed instruction generation module 25, wherein the frequency division module 21 is used to execute step S21: obtaining an EEG signal to be processed, and performing frequency division on the EEG signal to be processed to obtain a plurality of EEG division signals.

[0090] The EEG signal to be processed is low-pass filtered by a third-order cutoff frequency Butterworth filter with a cutoff frequency of 60 Hz and high-pass filtered by a third-order cutoff frequency Butterworth filter with a cutoff frequency of 0.1 Hz to achieve frequency segmentation of the EEG signal to be processed, and a plurality of EEG frequency signals are obtained, including a first EEG frequency signal δ with a frequency band of (1-3 Hz), a second EEG frequency signal θ with a frequency band of (4-7 Hz), a third EEG frequency signal α with a frequency band of (8-13 Hz), a fourth EEG frequency signal β with a frequency band of (14-30 Hz), and a fifth EEG frequency signal γ with a frequency band of (31-48 Hz).

[0091] The power feature calculation module 22 is configured to perform step S22: performing power spectral density calculation on each EEG frequency signal to obtain the power feature of each EEG frequency signal.

[0092] The power spectral density calculation on each EEG frequency signal is performed according to the following formula:

[0093]

[0094] The power spectral density of the i-th EEG frequency signal is PSD i (freq), the length of the i-th EEG frequency signal is M, and the calculation power of the Hamming window function is U. x i The i-th EEG frequency signal is x(n), the Hamming window function value of the n-th sampling point of the i-th EEG frequency signal is ω(n), and the frequency is freq.

[0095] The energy feature calculation module 23 is configured to perform step S23: calculating the energy value of each EEG frequency signal according to the power feature of each EEG frequency signal to obtain the energy feature of each EEG frequency signal.

[0096] The energy value of each EEG frequency signal is calculated according to the power feature of each EEG frequency signal, and the energy value expression of the first EEG frequency signal δ is:

[0097]

[0098] The energy value expression of the second EEG frequency signal θ is:

[0099]

[0100] The energy value expression of the third EEG frequency signal α is:

[0101]

[0102] The energy value expression of the fourth EEG frequency signal β is:

[0103]

[0104] The energy value expression of the fifth EEG frequency signal γ is:

[0105]

[0106] The concentration calculation module 24 is configured to perform step S24: performing concentration calculation according to the energy features of all EEG frequency signals to obtain a concentration value.

[0107] The concentration value is obtained by performing concentration calculation on the energy features of all EEG frequency signals by a support vector machine with a linear function as a kernel function, and the concentration value ranges from -2 to 2.

[0108] The vehicle speed instruction generation module 25 is configured to perform step S25: calculating a target vehicle speed according to the concentration value, and generating a corresponding vehicle speed control instruction according to the target vehicle speed.

[0109] The target vehicle speed is calculated according to the concentration value, and a corresponding target vehicle speed is set for different concentration value intervals. The greater the concentration value, the greater the target vehicle speed. In a specific implementation, the relationship between the speed percentage of the target vehicle speed and the concentration value is as follows:

[0110]

[0111] Please refer to Figure 6 and Figure 7 , wherein Figure 6 is a structure diagram of the blink control device 30 of the embodiment; Figure 7 is a flowchart of a blink control method performed by the blink control device 30. The blink control device 30 includes a band-pass filtering module 31, a first-order difference module 32, a normalization module 33, a second-order difference module 34, a wave peak identification module 35, a single blink identification module 36a, a double blink identification module 36b, and an instruction generation module 37. The band-pass filtering module 31 is configured to perform step S31: obtaining an EOG signal to be processed, and performing band-pass filtering on the EOG signal to be processed to obtain an EOG denoising signal.

[0112] The EOG signal to be processed is band-pass filtered by a band-pass filter of 1-10 Hz to eliminate the influence of high-frequency noise and baseline drift, and the EOG denoising signal is obtained.

[0113] The first-order difference module 32 is configured to perform step S32: performing difference processing on the amplitude value of each sampling point in the EOG denoising signal to obtain a first-order difference signal.

[0114] wherein, for the denoising EOG signal V = [v1, v2, v3, …, vn] containing n sampling points, the first-order difference signal Diff n v (i) = V(i+1) - V(i) is obtained, wherein V(i+1) is the amplitude value of the (i+1)th sampling point, and V(i) is the amplitude value of the ith sampling point.

[0115] The normalization module 33 is configured to perform step S33: normalizing the first-order difference signal.

[0116] wherein, the first-order difference signal is normalized, that is, the data in the first-order difference signal is normalized to binary data with only two values. Specifically, if the amplitude value corresponding to a sampling point in the first-order difference signal is greater than 0, it is normalized to 1; if the amplitude value corresponding to a sampling point in the first-order difference signal is equal to 0, it is normalized to 0; if the amplitude value corresponding to a sampling point in the first-order difference signal is less than 0, it is normalized to -1, which can be expressed as the following formula:

[0117]

[0118] For the sampling point normalized to 0, that is, sign(Diff v ) = 0, if the amplitude value corresponding to the next sampling point is greater than or equal to 0, the amplitude value corresponding to the current sampling point is normalized to 1; if the amplitude value corresponding to the next sampling point is less than 0, the amplitude value corresponding to the current sampling point is normalized to -1, which can be expressed as the following formula:

[0119]

[0120] The second-order difference module 34 is configured to perform step S34: performing difference processing on the normalized first-order difference signal again to obtain a second-order difference signal.

[0121] wherein, for the normalized first-order difference signal T(i), the second-order difference signal R = Diff(T) is obtained by difference processing.

[0122] The peak identification module 35 is configured to perform step S35: determining that the next sampling point of the sampling point corresponding to the minimum value in the second-order difference signal is a peak point.

[0123] The single-blink identification module 36a is configured to perform step S36a: if there is only one peak point, and the amplitude value of the peak point is greater than the amplitude threshold, determining that the EOG signal to be processed is a single-blink signal.

[0124] ​The double-blinking recognition module 36b is configured to perform step S36b: if there are multiple peak points, and the sampling time interval of adjacent peak points is less than the interval time threshold, and the energy value between the two peak points is greater than the energy threshold, it is determined that the EOG signal to be processed is a double-blinking signal.

[0125] wherein the sampling time interval of adjacent peak points is G = t peak2 -t peak1 , t peak2 and t peak1 are the sampling time points corresponding to the two adjacent peak points, respectively.

[0126] The energy value between adjacent peak points is wherein V t is the amplitude value corresponding to the sampling time point t.

[0127] The interval time threshold and the energy threshold are obtained by the following steps:

[0128] The EOG signals generated by 5 double blinks are collected, each with an interval of 3 seconds. The collected EOG signals are sliced through a 3-second time window, and the time interval G and the energy value E of the double peaks in each sliced signal are calculated. The smallest time interval and the largest time interval are taken as the minimum interval time threshold G min and the maximum interval time threshold G max ; and the smallest energy value and the largest energy value are taken as the minimum energy threshold E min and the maximum energy threshold E max .

[0129] The instruction generation module 37 is configured to perform step S37: generating a corresponding control instruction according to the single-blinking signal or the double-blinking signal.

[0130] The prop trigger control instruction is generated according to the single-blinking signal.

[0131] The start control instruction or the stop control instruction is generated according to the double-blinking signal. When the intelligent car is in a stationary state, the start control instruction is generated according to the double-blinking signal; when the intelligent car is in a moving state, the stop control instruction is generated according to the double-blinking signal.

[0132] Compared with the prior art, the intelligent trolley control system of the application can reduce the difficulty of trolley turning control, so that the operator can more easily control the intelligent trolley to successfully turn in the face of different turning scenarios. Moreover, in the image feature extraction of the real scene image, the real scene image is subjected to multiple deep feature extractions, and the extracted deep features are subjected to feature enhancement through attention weights after each deep feature extraction, so as to improve the mutual dependence relationship between feature maps, improve the accuracy of image feature extraction, and thus improve the accuracy of assisted driving.

[0133] Meanwhile, the intelligent trolley control system of the application can predict the concentration value according to the characteristics of the EEG signal of the operator, control the speed of the intelligent trolley through the concentration value, and be beneficial to the speed control of the intelligent trolley by the operator.

[0134] In addition, the intelligent trolley control system of the application also performs blink recognition on the operator, generates different control instructions using the recognized single blink signal and double blink signal, and realizes multiple control of the intelligent trolley.

[0135] Based on the same inventive concept, the present application also provides an electronic device, which can be a server, a desktop computing device or a mobile computing device (for example, a laptop computer, a handheld computing device, a tablet computer, a netbook, etc.) and the like terminal device. The device includes one or more processors and memories, wherein the processor is used to execute the program to realize the turning control method and / or the concentration control method and / or the blink control method of the method embodiment; the memory is used to store the computer program executable by the processor.

[0136] Based on the same inventive concept, the present application also provides a computer readable storage medium, which corresponds to the foregoing turning control method and / or concentration control method and / or blink control method embodiment, and the computer readable storage medium has a computer program stored thereon, which is executed by the processor to realize the steps of the turning control method and / or concentration control method and / or blink control method described in any of the foregoing embodiments.

[0137] The application can take the form of a computer program product accessible from a storage medium (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code for use by or in connection with a computer. Computer-usable storage media include permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0138] The above-described embodiments only express several embodiments of the present application, which are described in detail and specifically, but cannot be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, and the present application also intends to include these modifications and improvements.

Claims

1. A smart car control system, comprising a signal processor, characterized in that: The signal processor includes a steering control device, which includes a prediction deflection module, a target deflection module, and a steering command generation module. The predicted deflection module includes: The preliminary feature extraction submodule is used to acquire the real-scene image signal to be processed, and to perform max pooling on the real-scene image signal after convolution to obtain preliminary image features; The first depth feature extraction submodule is used to perform first depth feature extraction on the preliminary image features to obtain first depth image features; The first attention calculation submodule is used to calculate attention weights based on the first depth image features, and to weight the first depth image features using the attention weights to obtain the first depth attention features. The second deep feature extraction submodule is used to extract second deep features from the first deep attention features to obtain second deep image features. The second attention calculation submodule is used to calculate attention weights based on the second depth image features, and to weight the second depth image features using the attention weights to obtain the second depth attention features; The third deep feature extraction submodule is used to extract the third deep features from the second deep attention features to obtain the third deep image features. The third attention calculation submodule is used to calculate attention weights based on the third depth image features, and to weight the third depth image features using these attention weights to obtain the third depth attention features. The deflection prediction submodule is used to perform global average pooling on the third deep attention feature and then predict the deflection to obtain the predicted deflection. The target deflection module is used to obtain the actual deflection amount by weighting the predicted deflection amount and the actual deflection amount respectively and then adding them together to obtain the target deflection amount. The steering command generation module is used to generate corresponding steering control commands based on the target deflection amount.

2. The intelligent vehicle control system according to claim 1, characterized in that: The first depth feature extraction submodule is used to perform three consecutive group convolutions on the preliminary image features after downsampling to obtain the first depth image features; The second depth feature extraction submodule is used to perform 7 group convolutions on the first depth attention features after downsampling to obtain the second depth image features; The third deep feature extraction submodule is used to perform three grouped convolutions after downsampling the second deep attention features.

3. The intelligent vehicle control system according to claim 2, characterized in that, The downsampling includes the following steps: The preliminary image features, the first depth attention features, or the second depth attention features are divided into channels to obtain image features in two different channels. One channel's image features are subjected to depthwise separable convolution with a stride of 2 and batch normalization, followed by standard convolution and batch normalization. The other channel's image features are subjected to standard convolution and batch normalization, followed by depthwise separable convolution with a stride of 2 and batch normalization, and then standard convolution and batch normalization. The image features extracted from the two different channels are then concatenated along the channels. The concatenated image features are then subjected to channel shuffling to obtain downsampled image features.

4. The intelligent vehicle control system according to claim 2, characterized in that, The grouped convolution includes the following steps: The downsampled image features are divided into channels to obtain image features in two different channels. After standard convolution and batch normalization of the image features in one channel, depthwise separable convolution and batch normalization with a stride of 1 are performed, followed by standard convolution and batch normalization. The image features extracted from this channel are concatenated with the image features from the other channel. The concatenated image features are then shuffled by channel to obtain grouped convolutional image features.

5. The intelligent vehicle control system according to claim 4, characterized in that, The attention weight calculation includes the following steps: Global average pooling is performed on the first, second, or third depth image features along the channel dimension; one-dimensional convolution is performed on the pooled image features; and the convolutional image features are normalized using a response function to obtain the attention weight for each channel.

6. The intelligent vehicle control system according to any one of claims 1-5, characterized in that: The expression for the target deflection is: D=αD pred +(1-α)D deflect Where D is the target deflection amount; D pred D is the predicted deflection amount; deflect The actual deflection is denoted as α; α is a weighting parameter, which varies with the predicted deflection D. pred The value range increases as the value range increases.

7. The intelligent vehicle control system according to claim 1, characterized in that, It also includes a focus control device, which comprises: The frequency division module is used to acquire the EEG signal to be processed and to perform frequency division on the EEG signal to obtain multiple EEG frequency-divided signals. The power characteristic calculation module is used to calculate the power spectral density of each of the EEG frequency division signals to obtain the power characteristics of each EEG frequency division signal. The energy characteristic calculation module is used to calculate the energy value of each EEG frequency division signal based on the power characteristics of each EEG frequency division signal, and obtain the energy characteristics of each EEG frequency division signal. The focus calculation module is used to calculate the focus value by using a support vector machine with a linear function as the kernel function to perform focus calculation on the energy characteristics of all the EEG frequency division signals. The vehicle speed command generation module is used to calculate the target vehicle speed based on the attention value and generate a corresponding vehicle speed control command based on the target vehicle speed.

8. The intelligent vehicle control system according to claim 1, characterized in that, It also includes a blink control device, which comprises: A bandpass filter module is used to acquire the EOG signal to be processed, perform bandpass filtering on the EOG signal, and obtain an EOG noise-reduced signal. The first-order differential module is used to perform differential processing on the amplitude value of each sampling point in the EOG noise-reduced signal to obtain a first-order differential signal. The normalization module is used to normalize the first-order difference signal; The second-order difference module is used to perform difference processing on the normalized first-order difference signal again to obtain the second-order difference signal; The peak identification module is used to determine the next sampling point of the sampling point corresponding to the minimum value in the second-order difference signal as the peak point; The single blink recognition module is used to determine that the EOG signal is a single blink signal if there is only one peak point and the amplitude value of the peak point is greater than the amplitude threshold. The double blink recognition module is used to determine that the EOG signal is a double blink signal if there are multiple peak points, the sampling time interval between adjacent peak points is less than the interval time threshold, and the energy value between the two peak points is greater than the energy threshold. The instruction generation module is used to generate corresponding control instructions based on the single blink signal or the double blink signal.

9. The intelligent vehicle control system according to claim 8, characterized in that: The normalization module is used to normalize the amplitude value corresponding to a sampling point in the first-order differential signal to 1 if the amplitude value corresponding to a sampling point in the first-order differential signal is greater than 0; normalize the amplitude value corresponding to a sampling point in the first-order differential signal to 0 if the amplitude value corresponding to a sampling point in the first-order differential signal is equal to 0; and normalize the amplitude value corresponding to a sampling point in the first-order differential signal to -1 if the amplitude value corresponding to a sampling point in the first-order differential signal is less than 0. For a sampling point whose amplitude value is normalized to 0, if the value corresponding to the next sampling point is greater than or equal to 0, then the amplitude value corresponding to the current sampling point is normalized to 1. If the amplitude value corresponding to the next sampling point is less than 0, then the amplitude value corresponding to the current sampling point will be normalized to -1.

10. The intelligent vehicle control system according to claim 9, characterized in that: The instruction generation module is used to generate start control instructions or stop control instructions based on the double blink signal.

Images