Man-machine co-driving control right conversion method based on quantum deep learning

Abstract

The invention discloses a man-machine co-driving control right conversion method based on quantum deep learning, and relates to the technical field of automatic driving intelligent decision control, and the method comprises the steps: collecting an in-vehicle driver state picture and an out-vehicle state picture, and processing the collected pictures through a quantum color image algorithm; inputting the state picture of the driver in the vehicle into the space-time fusion convolutional neural network model; inputting the image into an optimized residual neural network for classification, and judging the distracted driving type of the driver; inputting the picture of the condition outside the vehicle into a SiamMask network, and converting the obtained image into a bird's-eye view; analyzing the aerial view, carrying out settlement on the state information of the traffic participants, and predicting the position and collision possibility of the vehicle; and performing driving risk degree quantification according to the distraction driving type of the driver and the vehicle collision possibility, and performing vehicle control right conversion according to a quantification result. According to the method, the operation efficiency and accuracy are greatly improved, and the rapidity and accuracy of control right conversion are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving intelligent decision-making and control technology, and in particular to a method for transferring control rights between humans and machines based on quantum deep learning. Background Art

[0002] In the early days, the human-machine co-driving control transfer system used classic machine learning algorithms to build a risk assessment mechanism and feedback mechanism. This type of model can identify driving behavior to a certain extent, but it has obvious limitations when faced with complex and changeable actual driving scenarios.

[0003] At present, there are two main sources of limitations in the human-machine co-driving control transfer system. One is the incomplete risk assessment problem. It only identifies, evaluates or predicts distracted driving behaviors without considering their actual impact on vehicle risks. Even if the driver is distracted, if the vehicle itself is in a low-risk state, the actual safety risk may be low. The other is the lack of an effective feedback mechanism. When distracted driving behavior is detected, the existing system will issue a warning or take some preventive measures, but these feedbacks are based on incomplete risk assessments, which may lead to excessive warnings or ineffective interventions, reducing the reliability of the system. In addition, in the process of extracting image information inside and outside the vehicle in the existing technology, the extraction calculation efficiency and accuracy are low, resulting in low calculation efficiency. It can be seen that how to improve the risk assessment mechanism and how to adopt an effective and timely feedback mechanism are issues that need to be solved urgently. Summary of the invention

[0004] In order to overcome the above-mentioned problems existing in the prior art, the present invention proposes a method for transferring human-machine co-driving control rights based on quantum deep learning.

[0005] The technical solution adopted by the present invention to solve the technical problem is: a method for converting human-machine co-driving control rights based on quantum deep learning, comprising: S1, collects the driver status pictures inside the car and the conditions outside the car, and processes the collected pictures through the quantum color image algorithm; S2, input the in-car driver status picture obtained in S1 into the spatiotemporal fusion convolutional neural network model for feature extraction; S3, inputting the image obtained in S2 into the optimized residual neural network for classification to determine the driver's distracted driving type; S4, input the vehicle exterior condition picture obtained in S1 into the SiamMask network for feature extraction, and convert the obtained image into a bird's-eye view; S5, analyzing the bird's-eye view obtained in S4, calculating the status information of the traffic participants, and predicting the position and collision possibility of the vehicle according to the two-dimensional Gaussian function; S6: quantify the degree of driving risk based on the driver's distracted driving type obtained in S3 and the possibility of vehicle collision obtained in S5, and transfer the vehicle control right based on the quantification result.

[0006] In the above-mentioned method for transferring control rights between human and machine co-driving based on quantum deep learning, the collected images are processed by the quantum color image algorithm in S1, which specifically includes: S11, use 2n H gates and 3q identity gates I to prepare the initial 2n quantum bits into a superposition state, and use NCQI color setting submodule Prepare all pixel grayscale information on the classical image into the prepared superposition state. And the grayscale range of RGB three channels is Color image, its NCQI model It is expressed as: ; in, Respectively represent the grayscale information of the R, G, and B channels of the color image; Represents a pixel set of a color image. For a filter window of size N, the pixel subset included is , the center pixel is ; and Respectively represent the pixel position information in the vertical and horizontal directions; S12, preparing the color information of the neighboring pixels into the quantum superposition state of the central pixel, and calculating the aggregation distance of the neighboring pixels; S13, searching for the minimum aggregation distance and the corresponding pixel through the sorting module, and taking the pixel corresponding to the minimum aggregation distance as the vector median pixel; S14, using a set of Swap gates to center pixels Pixel with vector target Exchange is performed to complete the vector median filtering task and filter out the salt and pepper noise of the quantum color image.

[0007] In the above-mentioned method for transferring control rights of human-machine co-driving based on quantum deep learning, the specific calculation process of the aggregation distance of the neighborhood pixels in S12 is: taking the Manhattan distance as the vector distance of the pixel element : ; in, represents the red pixel subset, represents the green pixel subset, represents the blue pixel subset; Sum the vector distances and calculate the aggregate distance for each pixel in the window : ; Where N represents the size of the filter window, .

[0008] In the above-mentioned method for transferring control rights between human and machine co-driving based on quantum deep learning, S2 specifically includes: when the spatiotemporal fusion convolutional network performs feature recognition on the driver status picture, the spatial domain convolutional neural network extracts the current spatial domain features consisting of the depth information of a single image, and the temporal domain convolutional neural network extracts the temporal domain features consisting of the optical flow sequence of past pictures; The optical flow iteration information contains ten past image frames, which are extracted through a 5-layer convolutional neural network. The 5-layer network structure is the same as the CONV1-5 layers in the VGG-16 network. The information of a single image is also extracted through the 5-layer network structure, the same as the CONV1-5 layers in the VGG-16 network. The optical flow features and the depth features of a single image are fused into a 224×224 two-dimensional fusion feature through convolution and pooling, and input into the reverse VGG-16 network for upsampling to obtain the saliency features of the final image, thereby judging whether the driver is in a distracted driving state.

[0009] In the above-mentioned method for transferring control rights of human-machine co-driving based on quantum deep learning, the convolutional layers in the optimized residual neural network in S3 are connected by Mish activation functions; when the optimized residual neural network is trained, multiple deep learning networks with the same structure are constructed for a data set, and the gradient descent method is used for training to obtain multiple different local optimal solutions, and the final judgment result is obtained by averaging the local optimal solutions.

[0010] In the above-mentioned method for transferring control rights of human-machine co-driving based on quantum deep learning, the specific method for predicting the position of the vehicle in S5 is: to associate the current vehicle status with the future vehicle position through a two-dimensional Gaussian distribution function; the longitudinal motion and lateral motion states of the vehicle are independent of each other, and the two-dimensional Gaussian function distribution density function of the vehicle motion position is calculated. for: ; ; in, , The vehicle takes the current position as the origin and passes through the The coordinates of the location after Indicates the vehicle unit time interval The density function of possible positions within; is the covariance matrix; , are the lateral and longitudinal acceleration values ​​of the vehicle respectively; , is the weight value.

[0011] In the above-mentioned method for transferring control rights of human-machine co-driving based on quantum deep learning, the prediction of the possibility of vehicle collision in S5 can be described by the following formula: ; in, is the similarity between the position distribution of other vehicles and the position distribution of the vehicle; For your car The position distribution after For another motor vehicle on the road The position distribution after and Represents the correlation coefficient between the lateral and longitudinal motions of the vehicle. Since the lateral and longitudinal motions of the vehicle are independent of each other, and All are taken as 0; Represents the variance of the vehicle's lateral motion; represents the variance of the vehicle's longitudinal motion; and Represents the expected value of the distribution Components in two dimensions, where represents the component of the horizontal dimension, The component representing the longitudinal dimension, which describes the most likely position of the vehicle in the corresponding dimension. Indicates the variance of the lateral motion of the road vehicle; Indicates the variance of the longitudinal motion of the road vehicle; and Represents the expected value of the distribution Components in two dimensions, where represents the component of the horizontal dimension, Represents the component of the longitudinal dimension, which is used to describe the most likely position of a road vehicle in the corresponding dimension.

[0012] The more similar the distribution of the ego vehicle is to that of other motor vehicles, The closer it is to 0, the greater the potential trajectory conflict; The larger the value, the greater the difference between the distribution functions and the smaller the possibility of potential trajectory conflict.

[0013] In the above-mentioned method for transferring control rights of human-machine co-driving based on quantum deep learning, the S6 is specifically: quantifying the risk level of the current driving state, and the quantified index The formula is: ; in, is the sampling time; The number of calculations of distracted driving risk within a sampling period; Indicates the category of distracted driving at the current moment, using 1, 2, and 3 to indicate the degree of distracted driving, where 1 indicates severe distraction, 2 indicates mild distraction, and 3 indicates normal driving; The information divergence representing the similarity of vehicle trajectories at the current moment ; Based on the analysis of the driving status of the driver inside the car and the prediction of the external situation information, the degree of danger of the vehicle is evaluated and divided into three risk levels: mild risk, moderate risk and severe risk; based on experience, Set to severe risk, For medium risk, It is a mild risk.

[0014] When the vehicle is at a mild risk, it is determined that the system is at a system sensitivity risk caused by the driver's driving habits and external vehicle conditions, and is therefore not considered; when the vehicle is at a moderate risk, the system warning function is activated; when the vehicle is at a severe risk, the car's automatic driving system is forced to intervene, take over control of the vehicle, complete the transfer of control, and guide the car into a safe driving route; after the automatic driving system gains control of the car, the driver can use subjective judgment to manually shut down the automatic driving system and gain control of the vehicle.

[0015] The beneficial effects of the present invention are: (1) quantum algorithm optimization for images. The original image is encoded with pixel values ​​by using the NCQI quantum color digital image algorithm, and the image is operated by the quantum algorithm, and finally a quantum image is constructed and output to realize the quantization of the image. Subsequently, the image is enhanced by using the quantum color image vector median filter, which effectively removes the noise in the image, improves the clarity and quality of the image, helps to extract and further operate the saliency features of the images inside and outside the vehicle, and is beneficial to the accuracy of the decision-making of the control right conversion system.

[0016] (2) Quantum algorithms improve image processing efficiency. Classical computers and their algorithms still have problems with real-time image detection and recognition, such as low operating efficiency, large space occupation, and incomplete recognition. Compared with traditional classical computing, quantum computing has stronger feature space and probability distribution generation capabilities, and can generate more accurate feature maps in image processing. By introducing quantum circuits and utilizing the parallelism of quantum algorithms, the accuracy of target detection algorithms can be improved, making up for the lack of accuracy in real-time image recognition. At the same time, the speed of image processing can be greatly accelerated, which can reduce the corresponding delay of the human-machine co-driving conversion system and effectively improve system efficiency.

[0017] (3) Improvement of the human-machine co-driving control transfer system. The system combines the status information inside and outside the vehicle to comprehensively assess the risk level of the vehicle. At the same time, the risk is divided into three levels: mild risk, moderate risk and severe risk. The system provides feedback based on the risk level, greatly improving the accuracy of the control transfer system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the present invention; Figure 2 It is a flow chart of NCQI quantum image preparation and enhancement of the present invention; Figure 3 Schematic diagram of the spatiotemporal fusion convolutional neural network of the present invention; Figure 4 It is a comparison diagram of the traditional convolutional neural network and the residual neural network of the present invention; Figure 5 It is a road environment feature information extraction map of the present invention; Figure 6 It is a schematic diagram of driving risk assessment of the present invention; Figure 7 It is a schematic diagram of control right decision at different driving risk stages of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0020] This embodiment discloses a method for converting human-machine co-driving control rights based on quantum deep learning, and the flow chart is as follows Figure 1 As shown, the driver's behavior inside the car and the vehicle's condition outside the car are first detected to evaluate the risk of the vehicle. The clarity of the image has an important impact on the detection of the inside and outside of the car. In order to solve the problem of misjudgment caused by low precision when shooting images, the image is first preprocessed, and the new quantum color image (NCQI) algorithm is used to encode the pixel value of the original image. The image is operated using a quantum algorithm, and finally a quantum image is constructed and output. Then, the quantum color image vector median filtering is used to process the salt and pepper noise of the quantum image for image enhancement.

[0021] The processed quantum image is input into the spatiotemporal fusion convolutional neural network to extract the significant features of the image, and then input into the optimized residual neural network (ResNet) to classify the significant features and determine the specific category of the driver's distracted driving. The SiamMask network is used to extract the road state features based on the quantum image, and the perspective conversion module in opencv is used to convert the road environment information image into a bird's-eye view effect. The state information of the traffic participants is then solved through the monocular ranging method, and finally the vehicle's position and collision possibility are predicted based on the two-dimensional Gaussian function.

[0022] The driving risk level is quantified based on the driver's specific distraction state and the possibility of vehicle collision, and a system for transferring the control of the car's automatic driving is constructed accordingly. The system effectively handles three different situations: mild risk, moderate risk, and severe risk. The car control technology based on human-machine co-driving can greatly improve the safety of road driving, reduce driving risks, and protect the driver's physical, mental, and property safety. The specific process is:

[0023] S1, collect the driver status picture inside the car and the condition picture outside the car. In order to better extract the corresponding features from the driver status picture inside the car and the condition picture outside the car, it is necessary to pay special attention to image processing to improve its accuracy. In this embodiment, the NCQI quantum image is first constructed, and then the quantum color image vector median filtering is used to improve the image clarity; the quantum color image spatial domain filtering method based on median calculation can filter out the salt and pepper noise on the quantum color image.

[0024] The collected images are processed by quantum color image algorithm. The quantum image preparation and enhancement flow chart is as follows: Figure 2 shown.

[0025] S11, use 2n H gates and 3q identity gates I to prepare the initial 2n quantum bits into a superposition state, and use NCQI color setting submodule Prepare all pixel grayscale information on the classical image into the prepared superposition state. Depend on indivual The matrix representation of the H gate and the identity gate I can be expressed as: .

[0026] for And the grayscale range of RGB three channels is Color image, its NCQI model It is expressed as: ; in, Respectively represent the grayscale information of the R, G, and B channels of the color image; Represents a pixel set of a color image. For a filter window of size N, the pixel subset included is , the center pixel is ; and Respectively represent the pixel position information in the vertical and horizontal directions.

[0027] S12, prepare the color information of the neighboring pixels into the quantum superposition state of the central pixel, that is: ; Calculate the aggregation distance of neighborhood pixels. To calculate the aggregation distance of neighborhood pixels, we must first complete the distance calculation between the three-dimensional vectors between pixels, and use the Manhattan distance shown in the following formula as the vector distance between pixels : ; in, represents the red pixel subset, represents the green pixel subset, represents the blue pixel subset. Sum the vector distances to calculate the aggregate distance for each pixel in the window: ; From the above formula, we can see that the aggregation distance of each pixel is obtained by adding the vector distances between it and other pixels. Therefore, the temporary aggregation distance after each vector distance calculation is As the input of the next vector distance calculation module , in order to achieve The summation operation of the vector distance is performed on the quantum bit sequence. At this time, the quantum state It is expressed as: .

[0028] S13, searching for the minimum aggregation distance and the corresponding pixel through the sorting module, and taking the pixel corresponding to the minimum aggregation distance as the vector median pixel.

[0029] The vector median selection is to find the pixel corresponding to the minimum aggregation distance. The output of this module determines the final effect of image filtering. The minimum aggregation distance and its corresponding pixel are searched through the sorting module, that is, the color information is manipulated using two exchange modules and a comparison module. And the corresponding aggregation distance The corresponding quantum state at this time is for: ; in, The output of the vector median filter is: .

[0030] S14, using a set of Swap gates to center pixels Pixel with vector target Exchange is performed to complete the vector median filtering task and filter out the salt and pepper noise of the quantum color image.

[0031] Median pixel replacement uses a set of swap gates to replace the center pixel Pixel with vector target Exchange is performed to complete the vector median filtering task.

[0032] The quantum state after processing can be expressed as: .

[0033] After the quantum color image vector median filtering process, the image noise is significantly reduced and the pixels are clearer, which is helpful for the subsequent extraction and further operation of the saliency features of the images inside and outside the vehicle, and is more conducive to the decision-making of the human-machine co-driving conversion system.

[0034] S2, input the driver status picture obtained in S1 into the spatiotemporal fusion convolutional neural network model for feature extraction. The schematic diagram of the spatiotemporal fusion convolutional neural network model is shown in Figure 3 shown.

[0035] When the spatiotemporal fusion convolutional network performs feature recognition on the driver status image, the spatial domain convolutional neural network extracts the current spatial domain features consisting of the depth information of a single image, and the temporal domain convolutional neural network extracts the temporal domain features consisting of the optical flow sequence of past images; The optical flow iteration information contains ten past image frames, which are extracted through a 5-layer convolutional neural network. The 5-layer network structure is the same as the CONV1-5 layers in the VGG-16 network. The information of a single image is also extracted through the 5-layer network structure, the same as the CONV1-5 layers in the VGG-16 network. The optical flow features and the depth features of a single image are fused into a 224×224 two-dimensional fusion feature through convolution and pooling, and input into the reverse VGG-16 network for upsampling to obtain the saliency features of the final image, thereby judging whether the driver is in a distracted driving state.

[0036] S3, inputs the image obtained in S2 into the optimized residual neural network for classification to determine the driver's distracted driving type.

[0037] The image saliency features extracted by the spatiotemporal fusion convolutional network are input into the ResNet network for identification, and the specific type of distracted driving is obtained. In order to improve the classification accuracy of the ResNet network, further optimization is performed by optimizing the network activation function and training strategy. Figure 4 As shown in Figure 1, compared with the traditional convolutional neural network structure, the residual neural network (ResNet network) adds a short-circuit connection part. The residual neural network adds a short-circuit connection from the first layer to the second layer activation function. The input of the activation function is the output of the traditional network. Transformed to To ensure Can and To perform addition operations, the two tensors need to have the same shape, usually using The convolution operation makes the output dimension consistent with the input dimension. This cross-layer identity mapping effectively solves the problems of gradient explosion or gradient disappearance caused by the increase of network depth, speeds up the training speed, and effectively improves network performance. The specific optimization points of the residual neural network in this embodiment are as follows: On the one hand, the network activation parameters are optimized. The convolutional layers in the residual network are connected by the Mish activation function, which allows negative gradients within a certain range to pass through. For other activation functions, the smoothness of the Mish function is full range, which allows negative information features to enter the deep network. Compared with traditional Leaky ReLU and ELU, it does not contain fixed parameters that need to be adjusted. The mathematical expression of Mish is as follows: .

[0038] On the other hand, the network training strategy is optimized. The training process of deep learning is a parameter optimization process, and the gradient descent method (SGD) is generally used for parameter optimization. However, for deep networks with a large number of weight parameters, the global optimal point is generally difficult to find. In order to solve the problem that a single network obtained by SGD may only be a local optimal point, this study constructs multiple deep learning networks with the same structure for a data set, uses SGD for training, obtains multiple different local optimal solutions, and obtains the final judgment result by averaging the results of each network, making the identification of distracted driving status more accurate.

[0039] The ResNet network is a network built using residual modules and residual connections. By adding cross-layer connections, it is less likely to have gradient vanishing and gradient exploding problems caused by the increase in the number of network layers, and can better meet the needs of visual recognition classification. The model is trained through a large number of pictures of different distracted driving states, and image recognition is performed based on the extracted image saliency features based on the trained ResNet network to determine whether the current state is distracted and what kind of distracted state it is in.

[0040] S4, input the vehicle exterior condition picture obtained in S1 into the SiamMask network for feature extraction, and convert the obtained image into a bird's-eye view. Figure 5 shown.

[0041] SiamMask network is a fast target tracking network, whose input is a single video frame and output is the position information of the object in each video frame. This method has a very fast recognition speed and is more suitable for systems that respond to road dynamic features in a timely manner, so as to obtain information on traffic participants such as motor vehicles and pedestrians on the road in a timely manner. Through the SiamMask network, the position information of motor vehicles and pedestrians in the images collected by the front camera can be tracked to obtain 17×17 dimensional traffic participant position features.

[0042] Road environment information is traffic participants extracted from a single image frame, which reflects the position distribution in a two-dimensional coordinate system. In order to associate traffic participants with the vehicle position, it is necessary to map the information obtained by SiamMask into a Gaussian two-dimensional distribution map. The perspective transformation module in opencv is used to transform the road environment information image into a bird's-eye view effect.

[0043] S5, analyzing the bird's-eye view obtained in S4, calculating the status information of traffic participants, and predicting the position and collision possibility of vehicles based on a two-dimensional Gaussian function.

[0044] Through the monocular ranging method in the mobieye driving assistance system, the speed of other traffic participants relative to the vehicle can be obtained through the ranging information between multiple consecutive frames, thereby calculating the speed and acceleration of other traffic participants.

[0045] Distracted driving can cause significant changes in the vehicle state. Visual distraction can cause the vehicle to frequently deviate from the road. Operational distraction can cause the driver's control of the vehicle to deteriorate, and the variance of the vehicle's steering angle, steering wheel angle, and speed can increase. Therefore, analyzing the danger level of distracted driving requires effective feature extraction of vehicle state parameters to predict the future position of the vehicle.

[0046] The specific method for predicting the position of a vehicle is as follows: the current vehicle status is associated with the future vehicle position through a two-dimensional Gaussian distribution function; the longitudinal and lateral motion states of the vehicle are independent of each other, and the two-dimensional Gaussian function distribution density function of the vehicle motion position is calculated. for: ; ; in, , The vehicle takes the current position as the origin and passes through the The coordinates of the location after Indicates the vehicle unit time interval The density function of the possible positions within; is the covariance matrix; , are the lateral and longitudinal acceleration values ​​of the vehicle respectively; , is the weight value. Covariance determines the distribution range uncertainty of probability density. Taking the acceleration of the vehicle as the covariance value of the two-dimensional Gaussian distribution represents the uncertainty brought by the acceleration of the vehicle to the vehicle position. The larger the variable range of acceleration, the larger the potential range of vehicle motion state changes and the larger the position distribution range. , The factor that represents the influence of acceleration on position uncertainty. The smaller the value, the greater the The vehicle tends to maintain the current state of motion during the time. The larger the value, the The vehicle tends to deviate from the current motion state within a certain time period, and its value is related to the driver's specific driving style. The distribution represents the time interval of each vehicle Possible location distribution.

[0047] The specific method for predicting the possibility of vehicle collision is as follows: In order to provide a quantitative index for the mutual influence relationship between the ego vehicle and other vehicles, the similarity of the position distribution of the ego vehicle and other vehicles is analyzed by information divergence.

[0048] First, define the vehicle The position distribution after and another motor vehicle on the road The position distribution after , and They are all two-dimensional Gaussian distributions, as shown below: ; ; In the formula, and is the expected value of the distribution, and is the covariance matrix. When calculating information divergence, the position distribution function of the vehicle itself is used as the benchmark to calculate the similarity between the position distribution of other vehicles and the position distribution of the vehicle itself. The process is as follows: D KL ( P ego || Q else )=∫ log( P ego ( x ))−log( Q else ( x ))] P ego ( x ) dx . =∫[ 1 2 log | Z | |Σ| − 1 2 ( x − μ ) T Σ −1 ( x − μ )+ 1 2 ( x − θ ) T Z −1 ( x − θ )] P ego ( x ) dx = 1 2 [ log | Z | |Σ| − d + tr { Z −1 Σ}+( θ − μ ) T Z −1 ( θ − μ )] .

[0049] because and They are both two-dimensional Gaussian distributions, The specific parameters in are as follows: ; ; Since the lateral and longitudinal motions of a vehicle are independent of each other, and are all taken as 0. Therefore can be converted to: ; in, is the similarity between the position distribution of other vehicles and the position distribution of the vehicle; For your car The position distribution after For another motor vehicle on the road The position distribution after and Represents the correlation coefficient between the lateral and longitudinal motions of the vehicle. Since the lateral and longitudinal motions of the vehicle are independent of each other, and All are taken as 0; Represents the variance of the vehicle's lateral motion; represents the variance of the vehicle's longitudinal motion; and Represents the expected value of the distribution Components in two dimensions, where represents the component of the horizontal dimension, The component representing the longitudinal dimension, which describes the most likely position of the vehicle in the corresponding dimension. Indicates the variance of the lateral motion of the road vehicle; Indicates the variance of the longitudinal motion of the road vehicle; and Represents the expected value of the distribution Components in two dimensions, where represents the component of the horizontal dimension, Represents the component of the longitudinal dimension, which is used to describe the most likely position of a road vehicle in the corresponding dimension.

[0050] The more similar the distribution of the ego vehicle is to that of other motor vehicles, The closer it is to 0, the greater the potential trajectory conflict; The larger the value, the greater the difference between the distribution functions and the smaller the possibility of potential trajectory conflict.

[0051] S6, quantify the degree of driving risk based on the driver's distracted driving type obtained in S3 and the possibility of vehicle collision obtained in S5. The driving risk assessment diagram is shown in Figure 6 As shown in Figure 2, the vehicle control right is transferred according to the quantified results.

[0052] The distracted driving category is combined with the possibility of vehicle trajectory conflict based on information divergence to quantify the risk level of the current driving state. The formula is: ; in, is the sampling time; The number of calculations of distracted driving risk within a sampling period; Indicates the category of distracted driving at the current moment, using 1, 2, and 3 to indicate the degree of distracted driving, where 1 indicates severe distraction, 2 indicates mild distraction, and 3 indicates normal driving; The information divergence representing the similarity of vehicle trajectories at the current moment ; Based on the analysis of the driving status of the driver inside the car and the prediction of the external situation information, the degree of danger of the vehicle is evaluated and divided into three risk levels: mild risk, moderate risk and severe risk; based on experience, Set to severe risk, For medium risk, It is a mild risk.

[0053] The control decision at different driving risk stages is as follows: Figure 7 As shown in the figure, when the vehicle is at a mild risk, it is determined that the system is at a system sensitivity risk caused by the driver's driving habits and external vehicle conditions, and therefore is not considered; when the vehicle is at a moderate risk, the system warning function is activated; when the vehicle is at a severe risk, the car's automatic driving system is forced to intervene, take over control of the vehicle, complete the transfer of control, and guide the car into a safe driving route; after the automatic driving system obtains control of the car, the driver can manually shut down the automatic driving system and obtain control of the vehicle through subjective judgment.

[0054] The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present invention, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. A method for transferring control rights between human and machine co-driving based on quantum deep learning, characterized in that: include: S1, collects the driver status pictures inside the car and the conditions outside the car, and processes the collected pictures through the quantum color image algorithm; S2, input the in-car driver status picture obtained in S1 into the spatiotemporal fusion convolutional neural network model for feature extraction; S3, inputting the image obtained in S2 into the optimized residual neural network for classification to determine the driver's distracted driving type; S4, input the vehicle exterior condition picture obtained in S1 into the SiamMask network for feature extraction, and convert the obtained image into a bird's-eye view; S5, analyzing the bird's-eye view obtained in S4, calculating the status information of the traffic participants, and predicting the position and collision possibility of the vehicle according to the two-dimensional Gaussian function; S6: quantify the degree of driving risk based on the driver's distracted driving type obtained in S3 and the possibility of vehicle collision obtained in S5, and transfer the vehicle control right based on the quantification result.

2. According to the method of claim 1, the method of transferring control rights between human and machine based on quantum deep learning is characterized in that: The processing of the collected images by the quantum color image algorithm in S1 specifically includes: S11, use 2n H gates and 3q identity gates I to prepare the initial 2n quantum bits into a superposition state, and use NCQI color setting submodule Prepare all pixel grayscale information on the classical image into the prepared superposition state. And the grayscale range of the RGB three channels is Color image, its NCQI model It is expressed as: in, Respectively represent the grayscale information of the R, G, and B channels of the color image; Represents a pixel set of a color image. For a filter window of size N, the pixel subset included is , the center pixel is ; and Respectively represent the pixel position information in the vertical and horizontal directions; S12, preparing the color information of the neighboring pixels into the quantum superposition state of the central pixel, and calculating the aggregation distance of the neighboring pixels; S13, searching for the minimum aggregation distance and the corresponding pixel through the sorting module, and taking the pixel corresponding to the minimum aggregation distance as the vector median pixel; S14, using a set of Swap gates to center pixels Pixel with vector target Exchange is performed to complete the vector median filtering task and filter out the salt and pepper noise of the quantum color image.

3. According to claim 2, a method for transferring control rights between human and machine based on quantum deep learning is characterized in that: The specific calculation process of the aggregation distance of the neighborhood pixels in S12 is: Manhattan distance is used as the vector distance of the pixel : in, represents the red pixel subset, represents the green pixel subset, represents the blue pixel subset; Sum the vector distances and calculate the aggregate distance for each pixel in the window : Where N represents the size of the filter window, .

4. According to the method of claim 1, the method of transferring control rights between human and machine based on quantum deep learning is characterized in that: The S2 specifically includes: when the spatiotemporal fusion convolutional network performs feature recognition on the driver status picture, the spatial domain convolutional neural network extracts the current spatial domain features consisting of the depth information of a single image, and the temporal domain convolutional neural network extracts the temporal domain features consisting of the optical flow sequence of past pictures; The optical flow iteration information contains ten past image frames, which are extracted through a 5-layer convolutional neural network. The 5-layer network structure is the same as the CONV1-5 layers in the VGG-16 network. The information of a single image is also extracted through the 5-layer network structure, the same as the CONV1-5 layers in the VGG-16 network. The optical flow features and the depth features of a single image are fused into a 224×224 two-dimensional fusion feature through convolution and pooling, and input into the reverse VGG-16 network for upsampling to obtain the saliency features of the final image, thereby judging whether the driver is in a distracted driving state.

5. According to the method of claim 1, the method of transferring control rights between human and machine based on quantum deep learning is characterized in that: The convolutional layers in the optimized residual neural network in S3 are connected by the Mish activation function. When the optimized residual neural network is trained, multiple deep learning networks with the same structure are constructed for a data set, and the gradient descent method is used for training to obtain multiple different local optimal solutions, and the local optimal solutions are averaged to obtain the final judgment result.

6. According to the method of claim 1, the method of transferring control rights between human and machine based on quantum deep learning is characterized in that: The specific method for predicting the position of the vehicle in S5 is: to associate the current vehicle status with the future vehicle position through a two-dimensional Gaussian distribution function; the longitudinal motion and lateral motion states of the vehicle are independent of each other, and the two-dimensional Gaussian function distribution density function of the vehicle motion position is calculated for: in, , The vehicle takes the current position as the origin and passes through the The coordinates of the location after Indicates the vehicle unit time interval The density function of the possible positions within; is the covariance matrix; , are the lateral and longitudinal acceleration values ​​of the vehicle respectively; , is the weight value.

7. According to the method of claim 1, the method of transferring control rights between human and machine based on quantum deep learning is characterized in that: The prediction of the possibility of vehicle collision in S5 can be described by the following formula: in, is the similarity between the position distribution of other vehicles and the position distribution of the vehicle; For your car The position distribution after For another motor vehicle on the road The position distribution after and Represents the correlation coefficient between the lateral and longitudinal motions of the vehicle. Since the lateral and longitudinal motions of the vehicle are independent of each other, and All are taken as 0; Represents the variance of the vehicle's lateral motion; represents the variance of the vehicle's longitudinal motion; and Represents the expected value of the distribution Components in two dimensions, where represents the component of the horizontal dimension, The component representing the longitudinal dimension is used to describe the most likely position of the vehicle in the corresponding dimension; Indicates the variance of the lateral motion of the road vehicle; Indicates the variance of the longitudinal motion of the road vehicle; and Represents the expected value of the distribution Components in two dimensions, where represents the component of the horizontal dimension, The component representing the longitudinal dimension is used to describe the most likely position of a road vehicle in the corresponding dimension; The more similar the distribution of the ego vehicle is to that of other motor vehicles, The closer it is to 0, the greater the potential trajectory conflict; The larger the value, the greater the difference between the distribution functions and the smaller the possibility of potential trajectory conflict.

8. According to the method of claim 1, the method of transferring control rights between human and machine based on quantum deep learning is characterized in that: The S6 is specifically: quantifying the risk level of the current driving state, and the quantified index The formula is: in, is the sampling time; The number of calculations of distracted driving risk within a sampling period; Indicates the category of distracted driving at the current moment, using 1, 2, and 3 to indicate the degree of distracted driving, where 1 indicates severe distraction, 2 indicates mild distraction, and 3 indicates normal driving; The information divergence representing the similarity of vehicle trajectories at the current moment ; Based on the analysis of the driving status of the driver inside the car and the prediction of the external situation information, the degree of danger of the vehicle is evaluated and divided into three risk levels: mild risk, moderate risk and severe risk; based on experience, Set to severe risk, For medium risk, It is a mild risk.

9. When the vehicle is at a mild risk, it is determined that the system is at a system sensitivity risk caused by the driver's driving habits and external vehicle conditions, so it is not considered; When the vehicle is at medium risk, the system warning function is activated; When the vehicle is at serious risk, the car's autonomous driving system intervenes forcibly, takes over control of the vehicle, completes the transfer of control, and guides the car into a safe driving route; after the autonomous driving system takes control of the car, the driver can use subjective judgment to manually shut down the autonomous driving system and take control of the vehicle.

Images