Convolutional neural network automobile lane changing auxiliary system

Abstract

The invention relates to the technical field of automobile safety, and discloses a convolutional neural network automobile lane changing auxiliary system, which comprises an image acquisition module adopting at least two low-light night vision cameras and an automobile body attitude sensor mounted at the front end of the gravity center of an automobile, a carrying AI chip, a vehicle-mounted AI chip, a vehicle-mounted AI chip and a vehicle-mounted AI chip, the data processing unit is internally provided with a target detection algorithm based on a convolutional neural network; and the early warning prompt module comprises a display screen and a voice prompt device. According to the convolutional neural network automobile lane changing auxiliary system, the convolutional neural network is trained by using not less than 50,000 actual road image data sets of a complex environment scene, and the annotation content is detailed; each residual unit in the ResNet-101 network structure is meticulously designed, dimensionality reduction is carried out firstly, then dimensionality increase is carried out, and features are effectively extracted; in combination with a feature pyramid network (FPN), multi-scale features are fused in a top-down and transverse connection mode, so that the model can better adapt to target detection requirements of different sizes and distances, and a more effective lane changing auxiliary function is provided.

Description

Technical Field

[0001] The present invention relates to the field of automobile safety technology, and in particular to a convolutional neural network automobile lane change assistance system. Background Art

[0002] With the continuous increase in the number of cars, road traffic safety issues have become increasingly prominent. During the driving process, lane changing is one of the important factors that cause traffic accidents. Traditional lane change assistance systems mainly rely on millimeter-wave radar for ranging and speed measurement, but there are certain limitations. For example, millimeter-wave radars are not effective in detecting smaller and slow-speed hazardous sources, and some products have electromagnetic radiation problems, which may have potential effects on the health of people in the car. In addition, some existing systems are costly and have single functions, and cannot meet users' diverse needs for vehicle safety performance.

[0003] In recent years, deep learning technology has made significant progress in the field of computer vision, and convolutional neural networks have shown strong capabilities in target detection and recognition. Therefore, applying convolutional neural networks to automobile lane change assistance systems is expected to overcome the shortcomings of traditional technologies, improve the accuracy and reliability of the system, and provide drivers with more effective lane change assistance. Therefore, we propose a convolutional neural network automobile lane change assistance system. Summary of the invention

[0004] 1. Technical issues to be resolved In view of the shortcomings of the prior art, the present invention provides a convolutional neural network automobile lane change assistance system, which has the advantage of being able to accurately identify potential dangerous targets on the side and rear of the vehicle, and provide timely and accurate warning information to the driver during the lane change process, thereby improving the safety of automobile driving. It solves the problem that in recent years, deep learning technology has made significant progress in the field of computer vision, and convolutional neural networks have shown powerful capabilities in target detection and recognition; therefore, applying convolutional neural networks to automobile lane change assistance systems is expected to overcome the shortcomings of traditional technologies, improve the accuracy and reliability of the system, and provide drivers with more effective lane change assistance.

[0005] (II) Technical solution In order to achieve the above-mentioned purpose of being able to accurately identify potential dangerous targets behind the side of the vehicle and provide timely and accurate warning information to the driver during the lane change process, thereby improving the safety of automobile driving, the present invention provides the following technical solutions: a convolutional neural network automobile lane change assistance system, comprising: Image acquisition module: Use at least two low-light-level night vision cameras and a body posture sensor installed at the front end of the vehicle's center of gravity. The two low-light-level night vision cameras are installed on both sides of the vehicle body to collect images of the side and rear environment of the vehicle. The camera resolution is not less than 1280×720 pixels and the frame rate is not less than 30fps; Data processing unit: Equipped with an AI chip with a computing power of no less than 5tops and a built-in target detection algorithm based on a convolutional neural network. The convolutional neural network uses ResNet-101 as the backbone network and integrates a feature pyramid network (FPN) to process and analyze the collected image data and identify potential approaching moving targets in the image, including pedestrians, motor vehicles, and non-motor vehicles; Early warning prompt module: includes a display screen and a voice prompt device. The display screen is a high-definition display screen of no less than 7 inches, which is used to display processed images and mark potential dangerous targets. The voice prompt device can emit prompt sounds of different frequencies and voice contents according to different types of dangerous targets. The voice prompt volume can be adjusted within the range of 30-80 decibels.

[0006] Preferably, the target detection algorithm based on the convolutional neural network uses no less than 50,000 actual road image data sets containing various complex environmental scenes during the training process, with no less than 30,000 training times, and the images in the data sets are annotated, and the annotated content includes the target category, location and size information.

[0007] Preferably, the ResNet-101 backbone network uses a 3×3 convolution kernel in the convolution layer, with a step size of 1 and a padding of 1, and in each residual unit, it first passes through a 1×1 convolution layer for dimensionality reduction, then passes through a 3×3 convolution layer for feature extraction, and finally passes through a 1×1 convolution layer for dimensionality increase to achieve effective feature extraction and transmission.

[0008] Preferably, the data processing unit is connected to the vehicle's body posture sensor. When the body posture sensor detects that the vehicle intends to change lanes, the data processing unit starts analyzing and processing the side rear image and completes the detection and judgment of dangerous targets within 0.5 seconds.

[0009] Preferably, the low-light-level night vision camera of the image acquisition module has an automatic gain control function, which can automatically adjust the sensitivity according to the ambient light intensity and maintain clear imaging within the light intensity range of 0.1-10 lux.

[0010] Preferably, the data processing unit adopts the Linux operating system, runs and optimizes the algorithm based on the TensorFlow deep learning framework, and has at least 4GB of running memory and 32GB of storage space.

[0011] Preferably, the display screen of the early warning prompt module adopts the MIPI interface, which can display the processed image data in real time, and the image display delay does not exceed 50ms; the voice prompt device adopts a multi-channel audio chip and supports Chinese and English voice prompt languages.

[0012] Preferably, it also includes a wireless communication module, which adopts one of the communication protocols including Bluetooth 5.0 or Wi-Fi 6, and is used for data transmission with the car's on-board system and the driver's mobile device to realize remote monitoring and software upgrades of the system.

[0013] Preferably, the data processing unit further includes a collision risk assessment module, which calculates the speed and distance of the target by the following formula: The speed estimation formula is: ,in Indicates the pixel displacement of the target in two consecutive frames of images, Indicates the distance between the target and the vehicle. Indicates the focal length of the camera. Indicates a time interval; For methods with known target size, the distance estimation formula is: ,in Indicates the actual width of the target, Indicates the width of the target in the image.

[0014] Preferably, the collision risk assessment module assesses whether there is a collision risk by using the following formula: The collision risk conditions are: in Indicates the preset minimum safe distance threshold. Indicates the preset maximum allowed approach speed threshold, Indicates the actual calculated distance between the target and the vehicle. Indicates the speed of the target.

[0015] Preferably, the data processing unit further includes an optical flow estimation algorithm module, which calculates the speed direction and size of the target object by calculating the optical flow field between consecutive frames, and provides a motion vector for each pixel, thereby helping to estimate the speed of the target.

[0016] (III) Beneficial effects Compared with the prior art, the present invention provides a convolutional neural network vehicle lane change assistance system, which has the following beneficial effects: 1. The convolutional neural network vehicle lane change assistance system uses a data set of no less than 50,000 actual road images of complex environment scenes to train the convolutional neural network, with detailed annotations covering target category, location and size information; each residual unit in the ResNet-101 network structure is carefully designed to first reduce the dimension and then increase the dimension to effectively extract features; combined with the feature pyramid network (FPN), multi-scale features are integrated through top-down and lateral connections, so that the model can better adapt to the detection needs of targets of different sizes and distances, significantly enhancing the system's generalization ability and target recognition accuracy, thereby providing a more effective lane change assistance function.

[0017] 2. The convolutional neural network vehicle lane change assistance system, the image acquisition module captures the side and rear environment of the vehicle in real time through at least two low-light night vision cameras with a resolution of 1280×720 pixels and a frame rate of not less than 30fps; when the body posture sensor detects the intention to change lanes, the data processing unit starts the built-in convolutional neural network-based target detection algorithm. The algorithm uses ResNet-101 as the backbone network and integrates the feature pyramid network (FPN). It can complete the detection and judgment of potential dangerous targets including pedestrians, motor vehicles and non-motor vehicles within 0.5 seconds, greatly improving the speed and accuracy of target recognition, and providing drivers with timely and reliable warning information.

[0018] 3. The convolutional neural network vehicle lane change assistance system can calculate and display the distance and speed of the rear vehicle in real time through the built-in collision risk assessment module; when a potential dangerous target is detected, the system uses the pixel displacement of the target in continuous image frames, the camera focal length and other information to estimate the vehicle speed, and accurately measures the distance from the target to the vehicle based on its actual size; in addition, the optical flow estimation algorithm module further enhances the accuracy of speed estimation, and provides the motion vector of each pixel by analyzing the changes in the optical flow field between continuous frames; this information is presented in an intuitive way on the high-definition display screen of the early warning prompt module, marking the location of the potential dangerous target and its speed and distance relative to the vehicle, helping the driver to make safe decisions more quickly, thereby effectively reducing the risk of traffic accidents when changing lanes.

[0019] 4. The convolutional neural network vehicle lane change assistance system, the early warning prompt module includes a high-definition display screen of no less than 7 inches and a voice prompt device. The screen uses the MIPI interface to ensure that the image display delay does not exceed 50ms, and supports real-time visual feedback; the voice prompt adjusts the frequency and content according to different types of dangerous targets, and the volume range is adjustable from 30-80 decibels to ensure that the information is conveyed clearly and unambiguously; in addition, it supports Chinese and English languages ​​to meet the needs of different user groups; the efficient early warning mechanism not only improves the user experience, but also helps the driver respond to potential risks more quickly, enhances driving safety, and optimizes human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic diagram of the architecture of the vehicle lane change assistance system of the present invention; Figure 2 This is a flow chart of the operation of the vehicle lane change assistance system of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the embodiments of the present invention and the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] See also Figure 1-2 , a convolutional neural network vehicle lane change assistance system, comprising: Image acquisition module: Use at least two low-light-level night vision cameras and a body posture sensor installed at the front end of the vehicle's center of gravity. The two low-light-level night vision cameras are installed on both sides of the vehicle body to collect images of the side and rear environment of the vehicle. The camera resolution is not less than 1280×720 pixels and the frame rate is not less than 30fps; Data processing unit: Equipped with an AI chip with a computing power of no less than 5tops and a built-in target detection algorithm based on a convolutional neural network. The convolutional neural network uses ResNet-101 as the backbone network and integrates a feature pyramid network (FPN) to process and analyze the collected image data and identify potential approaching moving targets in the image, including pedestrians, motor vehicles, and non-motor vehicles; Early warning prompt module: includes a display screen and a voice prompt device. The display screen is a high-definition display screen of no less than 7 inches, which is used to display processed images and mark potential dangerous targets. The voice prompt device can emit prompt sounds of different frequencies and voice contents according to different types of dangerous targets. The voice prompt volume can be adjusted within the range of 30-80 decibels.

[0023] Embodiment 1: Image acquisition module 1. Camera selection and installation Two high-resolution low-light-level night vision cameras, model GC4653, are selected, with a resolution of 1280×720 pixels and a frame rate of up to 30fps. This combination of resolution and frame rate can ensure that clear and smooth images of the vehicle's rear-side environment are captured during driving, providing a high-quality data basis for subsequent target recognition. The cameras are installed on both sides of the car body close to the rearview mirrors, and the installation angles are carefully designed. Through analysis and testing of the body structures and field of view of more than 50 different models, it is determined that the best installation angle of the camera is 30 degrees with the center axis of the vehicle body, so that it can cover a field of view of at least 65 degrees to the rear of the vehicle, ensuring that potential danger areas are not missed.

[0024] 2. Automatic Gain Control (AGC) Technology The two high-resolution low-light night vision cameras have built-in advanced automatic gain control (AGC) function, which works based on real-time monitoring of ambient light intensity. Light data is collected through photosensors, and when the light intensity varies in the range of 0.1-10lux, the AGC system automatically adjusts the camera's sensitivity. In low-light environments, such as rural roads without street lights at night (light intensity of about 0.3lux), AGC will increase the sensitivity to 5 times the standard value to improve image brightness. In strong-light environments, such as highways with direct sunlight during the day (light intensity of about 8lux), AGC will reduce the sensitivity to 0.5 times the standard value to prevent overexposure of the image, thereby always maintaining clear imaging.

[0025] 3. Body posture sensor The body posture sensor uses a high-precision MEMS (micro-electromechanical system) gyroscope and accelerometer combination, which can accurately measure the vehicle's yaw rate with a measurement accuracy of ±0.5° / s, a longitudinal acceleration accuracy of ±0.2m / s², and a lateral acceleration accuracy of ±0.15m / s². The sensor is installed at the front end of the vehicle's center of gravity, and its measurement data is ensured to be accurate and stable through a specially designed mounting bracket and calibration procedure. When the vehicle intends to change lanes, the sensor can detect posture changes within 0.08 seconds and transmit the signal to the data processing unit.

[0026] Embodiment 2: Data processing unit 1. AI Chip and Algorithm Architecture It is equipped with an AI chip designed for deep learning tasks developed by advanced domestic Aixin Yuanzhi, with a computing power of 3.6tops, which can efficiently process image data and run complex convolutional neural network algorithms; the built-in convolutional neural network-based target detection algorithm uses ResNet-101 as the backbone network and integrates the feature pyramid network (FPN); the ResNet-101 network structure uses a 3×3 convolution kernel in the convolution layer, with a step size of 1 and a padding of 1, and in each residual unit, it first passes through a 1×1 convolution layer for dimensionality reduction, then passes through a 3×3 convolution layer for feature extraction, and finally passes through a 1×1 convolution layer for dimensionality increase to achieve effective feature extraction and transmission; FPN fuses features of different scales through top-down and lateral connections, allowing the algorithm to better detect targets of different sizes and distances.

[0027] During the training process, more than 60,000 actual road image data sets containing various complex environmental scenes were used. These images cover different situations such as urban congested roads (accounting for 30%), highways (accounting for 25%), rural roads (accounting for 20%), rainy days (accounting for 15%), and snowy days (accounting for 10%). Each image has been carefully annotated by professionals, and the annotation content includes the category, location and size information of the target. The location information is accurate to the pixel level, and the size information is accurate to 0.1 meter. The number of training times is no less than 35,000 times. Through a large amount of training data and iterations, the performance of the algorithm is continuously optimized to improve the accuracy of target recognition. The specific convolutional neural network training process is as follows: Step 1: First, divide the labeled image dataset into training set, validation set and test set in a ratio of 8:1:1; during the training process, the stochastic gradient descent (SGD) algorithm is used as the optimizer, and the initial learning rate is set to 0.001. After every 1000 iterations, the learning rate is reduced to 0.9 times the original value, and the momentum value is 0.9 and the weight decay value is 0.0005 to accelerate the training process and improve the generalization ability of the model; Step 2: In each training batch, 32 images are randomly selected and input into the network for forward propagation calculation. After the prediction results are obtained, the loss value between the prediction results and the true labels is calculated through the cross entropy loss function. Then, the gradient is calculated using the back propagation algorithm and the network parameters are updated. After every 500 iterations, the performance of the model is evaluated on the validation set, including indicators such as accuracy, recall, and F1 value. If the performance of the model on the validation set does not improve for three consecutive evaluation cycles, the training is stopped in advance to avoid overfitting. Step 3. Finally, the trained model is evaluated on the test set to ensure the reliability and effectiveness of the model in practical applications.

[0028] 2. ResNet-101 network structure ResNet-101 consists of multiple convolutional layers, pooling layers, and fully connected layers, and its core lies in the design of residual units. In each residual unit, a 1×1 convolutional layer is first used for dimensionality reduction to compress the number of channels of the input feature map, for example, from 256 channels to 64 channels, and the number of convolution kernels depends on the settings of the specific layer. Then a 3×3 convolutional layer is used for the main feature extraction, with a step size of 1 and a padding of 1 to ensure that the size of the feature map remains unchanged and rich local features are extracted. Finally, a 1×1 convolutional layer is used for dimensionality increase to restore the number of channels of the feature map to the same number as the input or the number required by the network structure, so as to connect or add to subsequent layers. The entire ResNet-101 network contains 101 layers, in which a large number of residual units are stacked. This structure enables the network to learn more complex feature representations, effectively alleviating the gradient vanishing and gradient exploding problems of deep neural networks during training, thereby improving the performance and generalization ability of the model.

[0029] During the training process, the He initialization method is used for the parameter initialization of ResNet-101. This method can reasonably initialize the weight parameters according to the number of input and output neurons of the layer, so that the network can converge faster in the early stage of training; at the same time, batch normalization technology is used after each convolution layer to normalize the input data, speed up the training speed and improve the stability of the model; for example, in a typical residual unit, after the 1×1 convolution layer dimensionality reduction, batch normalization operation is immediately performed to adjust the mean and variance of the data to the appropriate range, and then input it into the 3×3 convolution layer.

[0030] 3. Feature Pyramid Network (FPN) implementation details: Feature Pyramid Network (FPN) fuses features of different scales through top-down and lateral connections. First, feature maps are extracted at different stages of the basic convolutional neural network (such as ResNet-101). The output of these stages is used as the input of FPN, where Represents the feature map of the shallower layer of the network, which has higher resolution but relatively weaker semantic information. It represents the feature map of the deeper layer of the network, which has stronger semantic information but lower resolution; Then, for the top-down path, from the highest level feature map First, the resolution is doubled by upsampling (e.g., using bilinear interpolation) to make it consistent with the feature map of the next layer. Match in spatial dimensions and then add the two together; for example, After upsampling Add the elements at corresponding positions to get a new feature map ; Then Then upsample and Fusion , and so on, until we get the final feature maps of multiple different scales , , , ; In the process of horizontal connection, in order to ensure the consistency of the number of channels, a 1×1 convolution layer is used to adjust the channels of the feature maps before addition; for example, After upsampling Before adding, pass it through a 1×1 convolutional layer The number of channels is adjusted to The same amount after upsampling; In this way, FPN constructs a feature pyramid with different resolutions and semantic strengths, enabling the model to better detect objects of different sizes and distances; in the object detection process, feature maps of different scales will be used to predict objects of different sizes. For example, smaller objects will be detected in feature maps with higher resolutions (such as ), while larger objects are detected on feature maps with lower resolution but richer semantic information (such as ) is detected, and the specific feature map correspondence is shown in the following formula: Overall algorithm training: The built-in convolutional neural network-based target detection algorithm uses ResNet-101 as the backbone network and integrates the feature pyramid network (FPN). During the training process, more than 60,000 actual road image data sets containing various complex environmental scenes were used. These images cover different situations such as urban congested roads (accounting for 30%), highways (accounting for 25%), rural roads (accounting for 20%), rainy days (accounting for 15%), and snowy days (accounting for 10%). Each image has been carefully annotated by professionals, and the annotation content includes the category, location and size information of the target. The location information is accurate to the pixel level, and the size information is accurate to 0.1 meter. The number of training times is no less than 35,000 times. Through a large amount of training data and iterations, the performance of the algorithm is continuously optimized to improve the accuracy of target recognition. The specific convolutional neural network training process is as follows: Step 1: First, divide the labeled image dataset into training set, validation set and test set in a ratio of 8:1:1; during the training process, the stochastic gradient descent (SGD) algorithm is used as the optimizer, and the initial learning rate is set to 0.001. After every 1000 iterations, the learning rate is reduced to 0.9 times the original value, and the momentum value is 0.9 and the weight decay value is 0.0005 to accelerate the training process and improve the generalization ability of the model; Step 2: In each training batch, 32 images are randomly selected and input into the network for forward propagation calculation. After the prediction results are obtained, the loss value between the prediction results and the true labels is calculated through the cross entropy loss function. Then, the gradient is calculated using the back propagation algorithm and the network parameters are updated. After every 500 iterations, the performance of the model is evaluated on the validation set, including indicators such as accuracy, recall, and F1 value. If the performance of the model on the validation set does not improve for three consecutive evaluation cycles, the training is stopped in advance to avoid overfitting. Step 3. Finally, the trained model is evaluated on the test set to ensure the reliability and effectiveness of the model in practical applications. 4. Operating System and Framework The data processing unit uses the Linux operating system. Linux was chosen because it is highly stable and customizable, and can meet the system's requirements for real-time and resource management. The algorithm is run and optimized based on the TensorFlow deep learning framework. TensorFlow provides a wealth of tools and function libraries to facilitate developers to build, train and deploy models. At the same time, in order to ensure the smooth operation of the system, it is equipped with at least 4GB of running memory and 32GB of storage space. The running memory uses high-speed DDR4 memory, and the storage space uses a solid-state drive (SSD), with a read and write speed of up to 500MB / s to improve data reading and writing speeds.

[0031] Example 3: Early warning module 1. Display screen design The display screen uses a 7-inch high-definition MIPI interface display screen, which adopts advanced IPS liquid crystal display technology, has the characteristics of high contrast ratio of 1000:1, high brightness of 400cd / m² and wide viewing angle of 170°, and can clearly display images under different lighting conditions; the MIPI interface of the display screen ensures high-speed data transmission, and the image display delay does not exceed 40ms; in the design of the driving circuit of the display screen, a high-performance graphics processor (GPU) is used to render and process the image, which can quickly display the image data transmitted by the data processing unit and mark it according to the type and degree of danger of the target; for example, a green rectangular frame is used to mark pedestrians, and the thickness of the frame is 3 pixels; a red rectangular frame is used to mark motor vehicles, and the thickness of the frame is 5 pixels; a yellow rectangular frame is used to mark non-motor vehicles, and the thickness of the frame is 4 pixels. The flashing frequency of the frame is dynamically adjusted according to the degree of danger of the target. When the degree of danger is high, the flashing frequency is 3Hz, and when the degree of danger is low, the flashing frequency is 1Hz. At the same time, the approximate distance and relative speed information of the target are displayed on the edge of the screen. The display accuracy of the distance information is ±3 meters, and the display accuracy of the relative speed information is ±3km / h.

[0032] Voice prompt device The voice prompt device uses a multi-channel audio chip and supports Chinese and English voice prompt languages. The chip integrates a high-quality speech synthesis engine that can generate specific voice content and frequency according to different types of dangerous targets. For example, when a motor vehicle is detected approaching quickly from behind, a high-frequency and urgent voice is issued, "There is a vehicle approaching quickly from behind, please pay attention to lane change safety and keep a safe distance!" The voice frequency is between 1200-1800Hz; when a pedestrian approaches, a medium-frequency and gentle voice is issued, "Pedestrians are approaching, please change lanes carefully and avoid them!" The voice frequency is between 600-1000Hz; the voice prompt volume can be adjusted by the driver within the range of 30-80 decibels through the setting interface, which can be entered through the touch button on the vehicle system or display; the volume adjustment adopts digital potentiometer technology, which can achieve precise volume control and maintain voice clarity and intelligibility at different volume levels.

[0033] Embodiment 4: Wireless communication module Communication protocol selection The wireless communication module uses Bluetooth 5.0 or Wi-Fi 6 communication protocols. These two protocols are chosen because they are high-speed, stable and low-power. Bluetooth 5.0 is suitable for short-distance, low-power data transmission scenarios, such as connecting with the driver's mobile device to achieve some simple control and information interaction functions. Wi-Fi 6 is used for high-speed data transmission with the car's on-board system, which can meet the needs of system software upgrades and large-scale data transmission.

[0034] Module design and function implementation In terms of hardware design, the wireless communication module uses high-performance RF chips and antennas to ensure the strength and stability of the signal. At the software level, it implements secure connection and data transmission protocols with the vehicle system and mobile devices. For example, the transmitted data is encrypted using the AES-256 encryption algorithm to prevent the data from being stolen or tampered with. At the same time, a corresponding application programming interface (API) is developed to facilitate data interaction between the vehicle system and mobile devices and this system. Through the wireless communication module, the vehicle system can obtain system operating status information, such as camera working status, target recognition results, warning history records, etc., and display them on the central control screen. The driver's mobile device can receive push notifications of abnormalities in the vehicle's surrounding environment, such as scratches on the vehicle, abnormal objects approaching nearby, etc., and can remotely control some functions of the system, such as adjusting the voice prompt volume and setting the warning sensitivity.

[0035] In summary, the convolutional neural network car lane change assistance system, the image acquisition module captures the vehicle's side and rear environment in real time through at least two low-light night vision cameras with a resolution of 1280×720 pixels and a frame rate of not less than 30fps; when the vehicle body posture sensor detects the intention to change lanes, the data processing unit starts the built-in convolutional neural network-based target detection algorithm. The algorithm uses ResNet-101 as the backbone network and integrates the feature pyramid network (FPN). It can complete the detection and judgment of potential dangerous targets including pedestrians, motor vehicles and non-motor vehicles within 0.5 seconds, greatly improving the speed and accuracy of target recognition, and providing drivers with timely and reliable warning information.

[0036] In addition, the convolutional neural network vehicle lane change assistance system uses a data set of no less than 50,000 actual road images of complex environment scenes to train the convolutional neural network, with detailed annotations covering target categories, locations and size information; each residual unit in the ResNet-101 network structure is carefully designed, first reducing the dimension and then increasing the dimension to effectively extract features; combined with the feature pyramid network (FPN), multi-scale features are fused through top-down and lateral connections, so that the model can better adapt to the detection needs of targets of different sizes and distances, significantly enhancing the system's generalization ability and target recognition accuracy, thereby providing a more effective lane change assistance function.

[0037] In addition, the convolutional neural network vehicle lane change assistance system, through the built-in collision risk assessment module, can calculate and display the distance and speed of the rear vehicle in real time; when a potential dangerous target is detected, the system uses the pixel displacement of the target in continuous image frames, the camera focal length and other information to estimate the vehicle speed, and accurately measures the distance from the target based on the actual size of the target; in addition, the optical flow estimation algorithm module further enhances the accuracy of speed estimation, and provides the motion vector of each pixel by analyzing the changes in the optical flow field between continuous frames; this information is presented in an intuitive way on the high-definition display screen of the early warning prompt module, marking the location of the potential dangerous target and its speed and distance relative to the vehicle, helping the driver to make safe decisions more quickly, thereby effectively reducing the risk of traffic accidents when changing lanes.

[0038] In addition, the convolutional neural network car lane change assistance system, the warning prompt module includes a high-definition display screen of no less than 7 inches and a voice prompt device. The screen uses the MIPI interface to ensure that the image display delay does not exceed 50ms, and supports real-time visual feedback; the voice prompt adjusts the frequency and content according to different types of dangerous targets, and the volume range is adjustable from 30 to 80 decibels to ensure that the information is conveyed clearly and unambiguously; in addition, it supports both Chinese and English languages ​​to meet the needs of different user groups; the efficient warning mechanism not only improves the user experience, but also helps the driver respond to potential risks more quickly, enhances driving safety, and optimizes human-computer interaction, solving the problem that deep learning technology has made significant progress in the field of computer vision in recent years, and convolutional neural networks have shown strong capabilities in target detection and recognition; therefore, applying convolutional neural networks to car lane change assistance systems is expected to overcome the shortcomings of traditional technologies, improve the accuracy and reliability of the system, and provide drivers with more effective lane change assistance.

[0039] The relevant modules involved in this system are all hardware system modules or functional modules that combine computer software programs or protocols with hardware in the prior art. The computer software programs or protocols involved in the functional modules are themselves technologies that are well known to those skilled in the art and are not improvements of this system. The improvements of this system are the interaction or connection relationships between the modules, that is, improvements to the overall structure of the system to solve the corresponding technical problems to be solved by this system.

[0040] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A convolutional neural network vehicle lane change assistance system, characterized in that: include: Image acquisition module: Use at least two low-light-level night vision cameras and a body posture sensor installed at the front end of the vehicle's center of gravity. The two low-light-level night vision cameras are installed on both sides of the vehicle body to collect images of the side and rear environment of the vehicle. The camera resolution is not less than 1280×720 pixels and the frame rate is not less than 30fps; Data processing unit: Equipped with an AI chip with a computing power of no less than 5tops and a built-in target detection algorithm based on a convolutional neural network. The convolutional neural network uses ResNet-101 as the backbone network and integrates a feature pyramid network (FPN) to process and analyze the collected image data and identify potential approaching moving targets in the image, including pedestrians, motor vehicles, and non-motor vehicles; Early warning prompt module: includes a display screen and a voice prompt device. The display screen is a high-definition display screen of no less than 7 inches, which is used to display processed images and mark potential dangerous targets. The voice prompt device can emit prompt sounds of different frequencies and voice contents according to different types of dangerous targets. The voice prompt volume can be adjusted within the range of 30-80 decibels.

2. The convolutional neural network vehicle lane change assistance system according to claim 1, characterized in that: The target detection algorithm based on the convolutional neural network uses no less than 50,000 actual road image data sets containing various complex environmental scenes during the training process, with no less than 30,000 training times, and the images in the data sets are annotated, and the annotated content includes the category, location and size information of the target.

3. The convolutional neural network vehicle lane change assistance system according to claim 1, characterized in that: The ResNet-101 backbone network uses a 3×3 convolution kernel in the convolution layer, with a step size of 1 and a padding of 1. In each residual unit, a 1×1 convolution layer is first used for dimensionality reduction, a 3×3 convolution layer is then used for feature extraction, and finally a 1×1 convolution layer is used for dimensionality increase to achieve effective feature extraction and transmission.

4. The convolutional neural network vehicle lane change assistance system according to claim 1, characterized in that: The data processing unit is connected to the vehicle body posture sensor. When the vehicle body posture sensor detects that the vehicle intends to change lanes, the data processing unit starts analyzing and processing the rear image and completes the detection and judgment of dangerous targets within 0.5 seconds.

5. The convolutional neural network vehicle lane change assistance system according to claim 1, characterized in that: The low-light-level night vision camera of the image acquisition module has an automatic gain control function, which can automatically adjust the sensitivity according to the ambient light intensity and maintain clear imaging within the light intensity range of 0.1-10 lux.

6. The convolutional neural network vehicle lane change assistance system according to claim 1, characterized in that: The data processing unit adopts the Linux operating system and runs and optimizes the algorithm based on the TensorFlow deep learning framework, and has at least 4GB of running memory and 32GB of storage space.

7. The convolutional neural network vehicle lane change assistance system according to claim 1, characterized in that: It also includes a wireless communication module, which uses one of the communication protocols including Bluetooth 5.0 or Wi-Fi 6, for data transmission with the car's on-board system and the driver's mobile device to achieve remote monitoring and software upgrades of the system.

8. The convolutional neural network vehicle lane change assistance system according to claim 1, characterized in that: The data processing unit further includes a collision risk assessment module, which calculates the speed and distance of the target by the following formula: The speed estimation formula is: ,in Indicates the pixel displacement of the target in two consecutive frames of images. Indicates the distance between the target and the vehicle. Indicates the focal length of the camera. Indicates a time interval; For methods with known target size, the distance estimation formula is: ,in Indicates the actual width of the target, Indicates the width of the target in the image.

9. The convolutional neural network vehicle lane change assistance system according to claim 9, characterized in that: The collision risk assessment module assesses whether there is a collision risk using the following formula: The collision risk conditions are: in Indicates the preset minimum safe distance threshold. Indicates the preset maximum allowed approach speed threshold, Indicates the actual calculated distance between the target and the vehicle. Indicates the speed of the target.

10. The convolutional neural network vehicle lane change assistance system according to claim 9, characterized in that: The data processing unit further includes an optical flow estimation algorithm module, which calculates the speed direction and size of the target object by calculating the optical flow field between consecutive frames, and provides a motion vector for each pixel, thereby helping to estimate the speed of the target.

Images