Vehicle lane changing detection method and device

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a vehicle lane changing detection method and device. When vehicle lane change detection is carried out, a road video collected by an image collection device of a vehicle is obtained, a plurality of image frames in the road video are input into the lane line detection model, and the image frames are processed through a feature extraction layer, a feature fusion layer, a first full connection layer and a second full connection layer of the lane line detection model. The method comprises the steps of obtaining position information and target types of lane lines included in a plurality of image frames in a road video, and then determining whether a vehicle has illegal lane changing behaviors or not according to the position information and the target types of the lane lines included in the plurality of image frames and central points of the image frames. According to the method, the existing video monitoring resources are fully utilized, high-precision identification of the type and the position of the lane line is realized, automatic judgment of illegal lane changing behaviors is further realized, and the detection precision and the robustness in a complex environment are effectively improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for detecting vehicle lane changes. Background Technology

[0002] With the continuous expansion of urban transportation networks and the sustained growth in the number of motor vehicles, intelligent detection and management of vehicle lane-changing behavior has become a crucial technological aspect for improving road safety and traffic efficiency. Especially on key sections such as bus lanes and main urban roads, illegal lane-changing not only disrupts normal traffic order but can also lead to traffic accidents and create safety hazards.

[0003] Therefore, how to detect vehicles' illegal lane-changing behavior has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a vehicle lane change detection method and apparatus to solve the safety hazards caused by illegal lane changes by vehicles in the prior art.

[0005] In a first aspect, this application provides a vehicle lane change detection method, the method comprising: Acquire road video captured by the vehicle's image acquisition device; For multiple image frames in the road video, the image frame is input into the feature extraction layer of the lane detection model to extract features and obtain a first feature tensor. The first feature tensor is then input into the feature fusion layer of the lane detection model to concatenate data from different dimensions in the first feature tensor to obtain a first fused feature tensor. The first fused feature tensor is then input into the first fully connected layer and the second fully connected layer of the lane detection model to obtain the position information of the lane lines included in the image frame output by the first fully connected layer and the target type of the lane lines included in the image frame output by the second fully connected layer. Based on the lane line position information, target type, and center point of the image frames included in multiple image frames, it is determined whether the vehicle has engaged in illegal lane changing behavior.

[0006] Furthermore, after acquiring the road video captured by the vehicle's image acquisition device, and before inputting multiple image frames from the road video into the feature extraction layer of the lane detection model, the method further includes: The image region information of lane lines included in multiple image frames in the road video is determined using an object detection model; For multiple image frames in the road video, the image frame is input into the feature extraction layer of the lane detection model to extract features from the image frame, resulting in a first feature tensor, including: For multiple image frames, the image frame and the image region information of the lane line corresponding to the image frame are input into the feature extraction layer of the lane line detection model. The feature extraction layer performs feature extraction on the image region corresponding to the image region information in the image frame to obtain the first feature tensor.

[0007] Furthermore, the method also includes: If no image region information is detected in any image frame, then determine whether image region information is detected in other image frames within a set time period after that image frame; If image region information is detected in any other image frame, it is determined that the lane line is temporarily obscured, and the image region information corresponding to the previous image frame is used as the image region information of this image frame.

[0008] Further, determining whether the vehicle has engaged in illegal lane-changing behavior based on the lane line position information, target type, and center point of the image frames included in multiple image frames includes: Based on the positional information of the lane lines included in multiple image frames in the road video, determine the trajectory of the relative position change between the lane lines and the center point; If the center point is determined to cross the lane line based on the change trajectory, it is then determined whether the target type of the crossed lane line is a preset type. If so, a first prompt message is output, wherein the preset type includes solid line and double solid line.

[0009] Furthermore, the method also includes: If the center point is determined to have crossed the lane line N times consecutively within a preset time period based on the change trajectory, then a second prompt message is output, where N is a positive integer.

[0010] Furthermore, the training process of the lane detection model includes: Obtain the target sample image in the sample set, as well as the standard position information and standard type of the lane lines corresponding to the target sample image; The target sample image is input into the feature extraction layer of the original lane detection model to extract features from the target sample image, resulting in a second feature tensor. The second feature tensor is then input into the feature fusion layer of the original lane detection model to concatenate data from different dimensions in the second feature tensor, resulting in a second fused feature vector. The second fused feature vector is then input into the first fully connected layer and the second fully connected layer of the original lane detection model, respectively, to obtain the predicted position information of the lane lines included in the target sample image output by the first fully connected layer, and the predicted type of the lane lines included in the target sample image output by the second fully connected layer. A first loss value is determined based on the deviation between the predicted location information and the standard location information; A second loss value is determined based on the deviation between the predicted type and the standard type; Based on the first loss value and the second loss value, a target loss value is determined, and the parameters of the original lane detection model are adjusted based on the target loss value.

[0011] Furthermore, the process of constructing the sample set includes: Obtain the original image and the corresponding label of the original image, wherein the label includes the standard location information and standard type of the lane lines included in the original image; The original image and the label are input into the augmented model to obtain the augmented image corresponding to the original image; The enhanced image is stored in the sample set as a sample image corresponding to the label.

[0012] Furthermore, after acquiring the original image and before inputting the original image and the label into the augmented large model, the method further includes: The original image is preprocessed, including rotation, scaling, cropping, flipping, brightness adjustment, contrast adjustment, noise reduction, and sharpening.

[0013] Furthermore, the image acquisition device displays a crosshair in the acquisition screen, and the crosshair coincides with a calibration rod set on the vehicle, which is located on the central axis of the vehicle.

[0014] Secondly, this application provides a vehicle lane change detection device, the device comprising: The acquisition module is used to acquire road videos captured by the vehicle's image acquisition device; The detection module is used to take multiple image frames from the road video, input the image frame into the feature extraction layer of the lane line detection model, extract features from the image frame to obtain a first feature tensor; input the first feature tensor into the feature fusion layer of the lane line detection model, concatenate the data of different dimensions in the first feature tensor to obtain a first fused feature tensor; input the first fused feature tensor into the first fully connected layer and the second fully connected layer of the lane line detection model respectively to obtain the position information of the lane lines included in the image frame output by the first fully connected layer, and the target type of the lane lines included in the image frame output by the second fully connected layer; and determine whether the vehicle has engaged in illegal lane changing behavior based on the position information of the lane lines included in multiple image frames, the target type, and the center point of the image frames.

[0015] Furthermore, the detection module is also used to determine the image region information of lane lines included in multiple image frames in the road video using a target detection model; The detection module is specifically used to input the image region information of the image frame and the lane line corresponding to the image frame into the feature extraction layer of the lane line detection model for multiple image frames. The feature extraction layer performs feature extraction on the image region corresponding to the image region information in the image frame to obtain a first feature tensor.

[0016] Furthermore, the detection module is also used to determine whether image region information is detected in other image frames within a set time period after the image frame if no image region information is detected in any image frame; if image region information is detected in any other image frame, it is determined that the lane line is temporarily obscured, and the image region information corresponding to the previous image frame of the image frame is used as the image region information of the image frame.

[0017] Furthermore, the detection module is specifically used to determine the trajectory of the relative position change between the lane line and the center point based on the position information of the lane lines included in multiple image frames in the road video; if the trajectory of change determines that the center point crosses the lane line, then it is determined whether the target type of the crossed lane line is a preset type; if so, a first prompt message is output, wherein the preset type includes solid line and double solid line.

[0018] Furthermore, the detection module is also configured to output a second prompt message if the center point is determined to have crossed the lane line N times consecutively within a preset time period based on the change trajectory, where N is a positive integer.

[0019] Furthermore, the acquisition module is also used to acquire target sample images in the sample set, as well as the standard position information and standard type of the lane lines corresponding to the target sample images; The training module is used to input the target sample image into the feature extraction layer of the original lane detection model to extract features from the target sample image, obtaining a second feature tensor; input the second feature tensor into the feature fusion layer of the original lane detection model to concatenate data of different dimensions in the second feature tensor to obtain a second fused feature vector; input the second fused feature vector into the first fully connected layer and the second fully connected layer of the original lane detection model respectively to obtain the predicted position information of the lanes included in the target sample image output by the first fully connected layer, and the predicted type of the lanes included in the target sample image output by the second fully connected layer; determine a first loss value based on the deviation between the predicted position information and the standard position information; determine a second loss value based on the deviation between the predicted type and the standard type; determine a target loss value based on the first loss value and the second loss value, and adjust the parameters of the original lane detection model based on the target loss value.

[0020] Furthermore, the acquisition module is also used to acquire the original image and the label corresponding to the original image, wherein the label includes the standard position information and standard type of the lane lines included in the original image; The construction module is used to input the original image and the label into the augmented large model to obtain the augmented image corresponding to the original image; and to save the augmented image as a sample image corresponding to the label in the sample set.

[0021] Furthermore, the building module is also used to preprocess the original image, the preprocessing including rotation, scaling, cropping, flipping, brightness adjustment, contrast adjustment, noise reduction, and sharpening.

[0022] Furthermore, the image acquisition device displays a crosshair in the acquisition screen, and the crosshair coincides with a calibration rod set on the vehicle, which is located on the central axis of the vehicle.

[0023] Thirdly, this application also provides an electronic device including a processor, which executes a computer program stored in a memory to implement the steps of any of the vehicle lane change detection methods described above.

[0024] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the vehicle lane change detection methods described above.

[0025] In this embodiment, when detecting lane changes, road video captured by the vehicle's image acquisition device is obtained. Multiple image frames from the road video are input into the lane detection model. The model processes these frames through a feature extraction layer, a feature fusion layer, a first fully connected layer, and a second fully connected layer to obtain the position information and target type of the lane lines included in the multiple image frames. Then, based on the position information, target type, and center point of the image frames, it is determined whether the vehicle has engaged in an illegal lane change. This fully utilizes existing video surveillance resources to achieve high-precision identification of lane line types and positions, thereby enabling automatic determination of illegal lane changes and effectively improving detection accuracy and robustness in complex environments. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A schematic flowchart illustrating a vehicle lane change detection process provided in an embodiment of this application; Figure 2 A schematic flowchart illustrating another lane change detection process provided in an embodiment of this application; Figure 3 A schematic flowchart illustrating another vehicle lane change detection process provided in this application embodiment; Figure 4 A schematic flowchart illustrating another vehicle lane change detection process provided in this application embodiment; Figure 5 This is a schematic diagram illustrating a sample set construction process provided in an embodiment of this application; Figure 6 This is a schematic diagram of an image acquisition device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the installation of an image acquisition device provided in an embodiment of this application; Figure 8 This is a schematic diagram of a vehicle lane change detection device provided in an embodiment of this application; Figure 9 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art are within the scope of protection of this application.

[0029] Traditional vehicle lane change detection methods primarily rely on manual video surveillance or roadside monitoring equipment analysis, which suffers from high workload, low efficiency, and poor real-time performance, failing to meet the demands of modern intelligent traffic management for high efficiency, accuracy, and automation. While some roads utilize physical sensors (such as geomagnetism and radar) for auxiliary judgment, the hardware investment is substantial, maintenance is complex, and costs are high. With the development of computer vision and deep learning technologies, multi-frame fusion detection algorithms based on video streams have gradually become the industry mainstream. By integrating continuous frame information, they improve the stability and accuracy of vehicle trajectory and lane line recognition. Deep learning-based target detection methods, such as the YOLO series algorithms, can significantly improve recognition accuracy and robustness while maintaining detection speed, adapting to complex road conditions and diverse scenarios. However, existing technologies still face problems such as high false alarm rates and insufficient adaptability in special scenarios such as blurred lane lines, severe occlusion, and left turns in bus lanes, affecting the effectiveness of practical applications.

[0030] Based on this, embodiments of this application provide a vehicle lane change detection method and apparatus. The method involves acquiring road video captured by a vehicle image acquisition device; inputting multiple image frames from the road video into the feature extraction layer of a lane detection model to extract features from the image frames, obtaining a first feature tensor; inputting the first feature tensor into the feature fusion layer of the lane detection model to concatenate data from different dimensions in the first feature tensor, obtaining a first fused feature tensor; inputting the first fused feature tensor into the first fully connected layer and the second fully connected layer of the lane detection model respectively, obtaining the position information of the lane lines included in the image frame output by the first fully connected layer, and the target type of the lane lines included in the image frame output by the second fully connected layer; and determining whether the vehicle has engaged in illegal lane change behavior based on the position information of the lane lines included in the multiple image frames, the target type, and the center point of the image frames.

[0031] Figure 1 This application provides a schematic flowchart of a vehicle lane change detection process, as shown in the embodiments. Figure 1 As shown, the process includes the following steps: S101: Acquire the road video captured by the vehicle's image acquisition device.

[0032] The vehicle lane change detection method provided in this application is applied to an electronic device, which may be a server, PC, vehicle controller, etc.

[0033] In order to detect lane-changing behavior of vehicles, this embodiment of the application can acquire road video captured by the vehicle's image acquisition device. That is, video data of the road in front of the vehicle is acquired through an onboard camera.

[0034] In this embodiment of the application, the vehicle image acquisition device can be installed in any part of the vehicle, as long as it can ensure that the acquired video data includes a complete road view.

[0035] S102: For multiple image frames in the road video, the image frame is input to the feature extraction layer of the lane detection model to extract features from the image frame and obtain a first feature tensor; the first feature tensor is input to the feature fusion layer of the lane detection model to concatenate data of different dimensions in the first feature tensor to obtain a first fused feature tensor; the first fused feature tensor is input to the first fully connected layer and the second fully connected layer of the lane detection model respectively to obtain the position information of the lane lines included in the image frame output by the first fully connected layer, and the target type of the lane lines included in the image frame output by the second fully connected layer.

[0036] After obtaining the road video, a pre-trained lane detection model can be used to determine the location information and target type of lane lines in multiple image frames within the video. In other words, the lane detection model identifies the locations of multiple lane lines and their corresponding target types. The target type can be a solid line, dashed line, double line, white line, or yellow line, etc.

[0037] In this embodiment, the lane detection model may include a feature extraction layer, a feature fusion layer, a first fully connected layer, and a second fully connected layer.

[0038] In this embodiment, multiple image frames from a road video can be input into a feature extraction layer. This feature extraction layer is the backbone network of a lane detection model; exemplarily, it can be a convolutional neural network. The feature extraction layer can quickly extract effective features, such as edges and textures, from the received image frames.

[0039] In this embodiment, after receiving an image frame, the feature extraction layer can perform feature extraction on the image frame to obtain a first feature tensor. How to perform feature extraction on an image frame is existing technology, and this embodiment will not elaborate on this process.

[0040] After obtaining the first feature tensor, it can be input into the feature fusion layer, which then concatenates the data from different dimensions within the tensor. In other words, it concatenates feature maps from different levels output by the backbone network to achieve dimensionality reduction and decrease the number of channels in the feature maps. For example, when concatenating data from different dimensions, the data from different dimensions can be joined end-to-end to obtain the first fused feature tensor.

[0041] After obtaining the first fused feature tensor, it can be input into a first fully connected layer and a second fully connected layer. The first fully connected layer is used to detect the position of lane lines included in the image frame. The second fully connected layer is used to detect the type of lane lines included in the image frame. In this embodiment, the position information of the lane lines can be obtained after the first fully connected layer performs a first fully connected process on the received first fused feature tensor. Simultaneously, the second fully connected layer performs a second fully connected process on the received first fused feature tensor, and can output the target type of the lane lines included in the image frame.

[0042] In one possible implementation, to make the output location information and target type intuitive, in this embodiment of the application, an output layer can be connected after the first fully connected layer and the second fully connected layer. The output layer processes the feature tensors output by the multiple fully connected layers to obtain the final location information and target type.

[0043] S103: Based on the position information of lane lines included in multiple image frames, the target type, and the center point of the image frames, determine whether the vehicle has engaged in illegal lane changing behavior.

[0044] Since the vehicle's image acquisition equipment is generally fixed at a certain position on the vehicle, in order to determine the relative position between the vehicle and the lane lines, in this embodiment of the application, a preset position in the image frame can be designated as a fixed reference point to determine whether the vehicle has changed lanes. For example, the center point of the image frame can be used as the reference point.

[0045] After determining the positional information of lane lines and target types in multiple image frames of a road video, it is possible to determine whether a vehicle has engaged in illegal lane changing based on the positional information of lane lines, target types, and the center point of the image frames. In other words, the center point of the image frame is considered the vehicle.

[0046] Specifically, if in the first 20 seconds of the road video, the center point is consistently located to the right of the lane line, but from the 21st second onwards, the center point begins to move to the left of the lane line, then the vehicle can be considered to have changed lanes. Further analysis can be conducted to determine whether the lane line crossed by the center point during this change is a dashed line. If so, it can be determined that the vehicle did not illegally change lanes. If the lane line crossed by the center point is not a dashed line, it can be determined that the vehicle illegally changed lanes.

[0047] In this embodiment, when detecting lane changes, road video captured by the vehicle's image acquisition device is obtained. Multiple image frames from the road video are input into the lane detection model. The model processes these frames through a feature extraction layer, a feature fusion layer, a first fully connected layer, and a second fully connected layer to obtain the position information and target type of the lane lines included in the multiple image frames. Then, based on the position information, target type, and center point of the image frames, it is determined whether the vehicle has engaged in an illegal lane change. This fully utilizes existing video surveillance resources to achieve high-precision identification of lane line types and positions, thereby enabling automatic determination of illegal lane changes and effectively improving detection accuracy and robustness in complex environments.

[0048] To further improve the accuracy of vehicle lane change detection, based on the above embodiments, in this embodiment, after acquiring the road video captured by the vehicle's image acquisition device, and before inputting multiple image frames from the road video into the feature extraction layer of the lane detection model, the method further includes: The image region information of lane lines included in multiple image frames in the road video is determined using an object detection model; For multiple image frames in the road video, the image frame is input into the feature extraction layer of the lane detection model to extract features from the image frame, resulting in a first feature tensor, including: For multiple image frames, the image frame and the image region information of the lane line corresponding to the image frame are input into the feature extraction layer of the lane line detection model. The feature extraction layer performs feature extraction on the image region corresponding to the image region information in the image frame to obtain the first feature tensor.

[0049] Since image acquisition devices capture road videos that include not only the road surface but also the views of both sides of the road, there is considerable redundancy in the video for lane detection models. Therefore, in this embodiment, after acquiring the road video, before determining the location information and target type based on the pre-trained lane detection model, the image regions containing lane lines in multiple image frames can be pre-identified. This allows the subsequent lane detection model to detect only the lane lines within these image regions, further improving the accuracy of vehicle lane change detection.

[0050] In the embodiments of this application, the image region information of lane lines included in multiple image frames in a road video can be determined using an object detection model.

[0051] Specifically, the YOLO (You Only Look Once) model can be used to initially extract the area where the lane lines are located.

[0052] In this embodiment, multiple image frames from a road video can be input into a YOLO model to obtain the lane_region output by the YOLO model. This lane_region can be understood as the bounding box coordinates of the image region where the lane lines are located, i.e., the image region information. In this embodiment, the bounding box used to identify the image region can be rectangular, and the lane_region can include xmin, ymin, xmax, and ymax. Here, xmin represents the x-coordinate of the lower left corner of the bounding box, ymin represents the y-coordinate of the lower left corner of the bounding box, xmax represents the x-coordinate of the upper right corner of the bounding box, and ymax represents the y-coordinate of the upper right corner of the bounding box. Of course, the shape of the image region corresponding to the image region information can also be other shapes, and those skilled in the art can set it as needed.

[0053] After determining the image region information of lane lines included in multiple image frames, when using the lane line detection model to determine the position information and target type of lane lines, the image region information of each image frame and its corresponding lane lines can be input into the lane line detection model. The feature extraction layer of the lane line detection model then determines the image region in the corresponding image frame that needs lane detection based on the image region information, and performs feature extraction on that image region to obtain the first feature tensor.

[0054] To further improve the accuracy of vehicle lane change detection, based on the above embodiments, the method in this application embodiment further includes: If no image region information is detected in any image frame, then determine whether image region information is detected in other image frames within a set time period after that image frame; If image region information is detected in any other image frame, it is determined that the lane line is temporarily obscured, and the image region information corresponding to the previous image frame is used as the image region information of this image frame.

[0055] Since foreign objects may obstruct the image acquisition device during vehicle operation, or the image acquisition device may be unclear due to light illumination, in this embodiment of the application, the problem of temporary occlusion of road video can be eliminated before using the lane line detection model to identify the position information and target type of lane lines.

[0056] In this embodiment, it can be determined whether there are image frames in the road video where no image region information is detected. If so, for any image frame where no image region information is detected, it can be determined whether image region information is detected in other image frames within a set time length after that image frame. The set time length can be any value, such as 1 second, 2 seconds, 5 seconds, etc., and can be configured as needed by those skilled in the art.

[0057] If it is determined that image region information is detected in any other image frame within a set time period following the current image frame, then it can be concluded that the failure to detect image region information in the previous few image frames was due to temporary occlusion of the lane lines. In this embodiment, the image region information corresponding to the previous image frame can be used as the image region information of the current image frame. That is, when it is determined that the lane lines are temporarily occluded, the image region information corresponding to the previous image frame can be used.

[0058] In one possible implementation, the target detection model can output not only image region information but also the corresponding confidence level of the image region information. Before determining the location information and target type based on the pre-trained lane line detection model, the image region information can be filtered according to the confidence level to avoid false detections, identifying non-lane line regions as lane line image regions. These non-lane line regions can be regions corresponding to objects such as guardrails or billboards. In this embodiment, the deletion of corresponding region information can be determined based on the relationship between the confidence levels of multiple image region information and a confidence threshold. For example, image region information with a confidence level less than 0.7 can be deleted.

[0059] The following is combined Figure 2 This is a schematic flowchart illustrating another vehicle lane change detection process provided in an embodiment of this application, as shown below. Figure 2As shown, after obtaining any image frame, it can be input into the YOLO model for lane detection to obtain the image region information of the lanes included in the image frame. During lane detection, the YOLO model can employ a model architecture of CSPDarknet-53 backbone network + PANet feature fusion to ensure the detection accuracy of lane line regions. Simultaneously, to guarantee the accuracy of lane detection, the output of the YOLO model can be filtered during lane detection to reduce invalid input to subsequent lane detection models. In this embodiment, the lane detection model can be referred to as UFLDV2.

[0060] After obtaining the filtered image region information, the image frame and image region information can be input into the trained UFLDV2 model to perform lane line localization and type recognition, obtaining the lane line location information and target type. In this embodiment, the UFLDV2 model can perform multi-task processing through multi-layer feature extraction and fusion, including classification branches and localization branches, and a joint loss function is used for collaborative optimization during model training.

[0061] In this embodiment, the YOLO object detection network is first used to process the ADAS image frame to detect areas that may contain lane lines, thus achieving preliminary target extraction of lane lines. Then, the relevant information obtained after YOLO detection is input into the improved UFLDV2 model. This model adds a multi-task head design, with the localization branch outputting the localization of each lane line and the classification branch outputting the type of each lane line. The model uses a joint loss function to coordinately optimize the localization and classification tasks, making feature sharing and information fusion more sufficient, and further enhancing the detection performance in complex road environments.

[0062] To further improve the accuracy of vehicle lane change detection, based on the above embodiments, in this embodiment, determining whether the vehicle has engaged in illegal lane change behavior based on the lane line position information, target type, and center point of the image frames included in multiple image frames includes: Based on the positional information of the lane lines included in multiple image frames in the road video, determine the trajectory of the relative position change between the lane lines and the center point; If the center point is determined to cross the lane line based on the change trajectory, it is then determined whether the target type of the crossed lane line is a preset type. If so, a first prompt message is output, wherein the preset type includes solid line and double solid line.

[0063] In this embodiment of the application, when determining whether a vehicle has engaged in illegal lane changing based on the position information of lane lines included in multiple image frames, the target type, and the center point of the image frames, the trajectory of the relative position change between the lane lines and the center point can be determined based on the position information of lane lines included in multiple image frames in the road video. That is, it determines which side of the lane line the center point in each image frame is located on.

[0064] After determining the trajectory of the change, it can be determined whether the center point crosses a lane line. If so, it can be determined whether the target type of the crossed lane line is a preset type. This preset type can include solid lines, double solid lines, etc., and can be configured as needed by those skilled in the art.

[0065] If the target type of the lane line being crossed is determined to be a preset type, it can be determined that the vehicle has engaged in illegal lane changing, and a first warning message can be output. This first warning message can be in text form, audio form, or other forms, and can be configured as needed by those skilled in the art.

[0066] In this embodiment, if the center point is determined to have crossed the lane lines N times consecutively within a preset time period based on the changing trajectory, a second prompt message is output, where N is a positive integer. This second prompt message can be in text format, audio format, or other formats, and can be configured as needed by those skilled in the art. To facilitate user identification of the current violation through the prompt message, in this embodiment, the first and second prompt messages can be different.

[0067] Specifically, after identifying lane line types and obtaining the coordinates of each type of lane line, a warning can be issued for illegal lane changes based on the following rules by comparing them laterally with the vehicle's position. In this embodiment, the center point of the image frame can be used as the vehicle's position.

[0068] Solid line lane change detection: When the target type of the lane line crossed by the center point is determined to be a solid line, double solid lines, or other prohibited lane lines, it can be determined that the current behavior is a high-risk operation, requiring the triggering of a level one alarm for emergency alert. For example, the first prompt message will be output.

[0069] Continuous lane change detection: If the target type of the lane line crossed by the center point is determined to be a dashed line, and the vehicle crosses the lane line twice in a short period of time, the current behavior can be determined to be a medium-risk operation, requiring the triggering of a level two alarm and a routine reminder. For example, a second prompt message can be output.

[0070] The following is combined Figure 3The process of detecting vehicle lane changes will be explained. Figure 3 This is a flowchart illustrating another vehicle lane change detection process provided in this application embodiment. During vehicle operation, a video stream is loaded in real time. If it is determined that all cameras of the Advanced Driving Assistance System (ADAS) can normally acquire the video stream, it can be determined that the cameras are working properly and can acquire vehicle data in real time. Based on the acquired vehicle data, it is determined whether the vehicle's current speed exceeds a preset speed threshold. If so, it can be considered that the vehicle is currently performing a lane change. The image frame corresponding to the current moment can be preprocessed and input into the lane line detection model. The lane line detection model infers and obtains lane line data, which may include location information and target type. Preprocessing the image frame corresponding to the current moment can be understood as using a target detection model to determine the image region information of the lane lines included in the image frame and verifying this image region information. If the vehicle's current speed does not exceed the preset speed threshold, the video stream can continue to be loaded to monitor the vehicle.

[0071] After determining the lane line data, the relative position between the center point of the image frame and the lane line can be used to determine whether the vehicle is currently changing lanes across a solid line. If so, a first-level alarm can be triggered, and the first prompt message can be output. If not, it can be further determined whether the vehicle is currently engaging in continuous lane changes. If so, a second-level alarm can be triggered, and the second prompt message can be output. If it is determined that the vehicle is not currently engaging in continuous lane changes, the video stream can continue to be loaded for monitoring of the vehicle.

[0072] The following is combined Figure 4 The process of vehicle lane change detection provided in the embodiments of this application will be described. Figure 4 This is a schematic flowchart illustrating another vehicle lane change detection process provided in an embodiment of this application, as shown below. Figure 4 As shown, video data is first collected to obtain road video. ADAS cameras installed on the windshield of buses can collect real-time video data of the road ahead.

[0073] The UNICE large-scale model is used to enhance image frames in road videos, resulting in enhanced image frames. The model's image generation and optimization capabilities are leveraged to enhance blurred lane lines, improving their clarity and continuity.

[0074] The lane line detection model is used to detect lane lines in the enhanced image frame to determine the location information and target type of the lane lines included in the image frame.

[0075] Finally, based on pre-defined rules for illegal lane changes, lane line location information, target type, and the center point of the image frame, it is determined whether the vehicle has engaged in illegal lane changing and an alert is issued. The pre-defined rules for illegal lane changing may include crossing double solid lines, making multiple lane changes in a short period, etc.

[0076] To further improve the accuracy of vehicle lane change detection, based on the above embodiments, in this embodiment, the training process of the lane line detection model includes: Obtain the target sample image in the sample set, as well as the standard position information and standard type of the lane lines corresponding to the target sample image; The target sample image is input into the feature extraction layer of the original lane detection model to extract features from the target sample image, resulting in a second feature tensor. The second feature tensor is then input into the feature fusion layer of the original lane detection model to concatenate data from different dimensions in the second feature tensor, resulting in a second fused feature vector. The second fused feature vector is then input into the first fully connected layer and the second fully connected layer of the original lane detection model, respectively, to obtain the predicted position information of the lane lines included in the target sample image output by the first fully connected layer, and the predicted type of the lane lines included in the target sample image output by the second fully connected layer. A first loss value is determined based on the deviation between the predicted location information and the standard location information; A second loss value is determined based on the deviation between the predicted type and the standard type; Based on the first loss value and the second loss value, a target loss value is determined, and the parameters of the original lane detection model are adjusted based on the target loss value.

[0077] In order to obtain a lane detection model with high lane detection accuracy, in this embodiment of the application, the original lane detection model can be trained to obtain a lane detection model.

[0078] In this embodiment, to obtain a lane detection model with high accuracy, a sample set is pre-configured, and the original lane detection model is trained based on multiple sample images contained in the sample set. To facilitate subsequent training of the original lane detection model, the sample set contains not only each sample image but also a label corresponding to each sample image. This label is used to identify the standard location information and standard type of the lane lines contained in the sample image.

[0079] After obtaining the sample set, multiple sample images from the sample set are sequentially input into the original lane detection model. The original lane detection model determines the predicted position information and prediction type of the lane lines included in the target sample image. The model architecture of this original lane detection model is consistent with the model architecture of the lane detection models described in the above embodiments. The model obtained after training the original lane detection model is the lane detection model described in the above embodiments. In this embodiment, the sample image can be input into the feature extraction layer of the original lane detection model. The feature extraction layer extracts features from the target sample image to obtain a second feature tensor. This second feature tensor is input into the feature fusion layer of the original lane detection model. The feature fusion layer concatenates the data from different dimensions of the second feature tensor to obtain a second fused feature vector. Then, this second fused feature vector is input into the first fully connected layer and the second fully connected layer respectively to obtain the predicted position information of the lane lines included in the target sample image output by the first fully connected layer, and the predicted type of the lane lines included in the target sample image output by the second fully connected layer.

[0080] After obtaining the predicted location information and prediction type, a first loss value can be determined based on the deviation between the predicted location information and the standard location information. A second loss value is then determined based on the deviation between the predicted type and the standard type. Finally, a target loss value can be determined based on these first and second loss values. For example, the sum of the first and second loss values ​​can be used as the target loss value. Alternatively, different weights can be pre-assigned to the first and second loss values, and then the target loss value can be determined based on the first loss value, the second loss value, and their corresponding weights. Specifically, the loss value can be determined based on the following formula:

[0081] in, This indicates the weight corresponding to the first loss value; This indicates the weight corresponding to the second loss value; Indicates the target loss value; This represents the first loss value, also known as the positioning loss, which is used to measure the accuracy of lane line positioning. This represents the second loss value, which is the classification loss and is used to measure the accuracy of lane line type classification.

[0082] After obtaining the target loss value, the parameters of the original lane detection model can be adjusted based on the target loss value.

[0083] In this embodiment, a convergence condition is preset. This convergence condition may be that the number of times the predicted position information and predicted type in the training sample set are consistent with the standard position information and standard type is greater than a set number, or that the number of iterations of training the prediction model reaches a set maximum number of iterations, etc. Specific embodiments of this application do not limit this. When the convergence condition is met, the training of the original lane detection model can be considered complete, resulting in a trained lane detection model that can be used for vehicle lane change detection.

[0084] In this embodiment, the obtained lane detection model can perform fine-grained lane line localization and type identification. It innovatively introduces a multi-task head structure, including a localization branch and a classification branch, and employs a joint loss function to collaboratively optimize the localization and classification tasks. Specifically, the localization branch outputs the position of each lane line. The classification branch outputs the type label for each lane line, including solid lines, dashed lines, double lines, white lines, yellow lines, etc. The classification branch uses convolutional layers to output the probability distribution of the corresponding category for each lane line.

[0085] In this embodiment, the multi-task head structure realizes the joint output of localization and classification, which improves the model's comprehensive understanding of lane lines. The joint loss function promotes the synergistic optimization of localization and classification tasks, and enhances the model's detection performance and robustness in complex road environments.

[0086] To further improve the accuracy of the lane detection model, based on the above embodiments, in this embodiment, the process of constructing the sample set includes: Obtain the original image and the corresponding label of the original image, wherein the label includes the standard location information and standard type of the lane lines included in the original image; The original image and the label are input into the augmented model to obtain the augmented image corresponding to the original image; The enhanced image is stored in the sample set as a sample image corresponding to the label.

[0087] To improve the accuracy of the lane detection model obtained through training, in this embodiment of the application, when constructing the sample set, the original image and the corresponding label of the original image can be obtained. The label includes the standard location information and standard type of the lane lines included in the original image.

[0088] Specifically, the original images can be image frames included in sample road videos captured by image acquisition devices installed on vehicles. In this embodiment, the location and type of lane lines in multiple original images can be manually labeled. Alternatively, automatic or semi-automatic labeling tools can be used to accurately label the lane line regions in the original images. Afterwards, manual verification and correction are performed to ensure the accuracy of the labeled data, providing high-quality label information for model training.

[0089] After obtaining the original image, the original image and its corresponding labels can be input into the augmentation model. The augmentation model then enhances the blurred lane lines in the original image, resulting in an enhanced image. In other words, the powerful image generation and optimization capabilities of the augmentation model can effectively enhance the originally blurred lane lines, making them clearer and more continuous in the image.

[0090] For example, the UNICE (Universal Image Contrast Enhancer) model can be used to enhance the obtained original image. This model employs techniques such as multi-scale feature extraction and attention mechanisms to further enhance its ability to capture lane line details.

[0091] After obtaining the enhanced image, the enhanced image is saved as a sample image and its corresponding label in the sample set.

[0092] In this embodiment, the original image is generated and optimized using an enhanced large model. The model automatically enhances the clarity and continuity of lane lines, repairs broken or incomplete lane line areas, making them more obvious and coherent in the image. This effectively improves the quality of the sample images included in the sample set and further improves the accuracy of model training.

[0093] To further improve the accuracy of lane detection model detection, based on the above embodiments, in this embodiment, after acquiring the original image and before the original image and the label are input into the enhanced large model, the method further includes: The original image is preprocessed, including rotation, scaling, cropping, flipping, brightness adjustment, contrast adjustment, noise reduction, and sharpening.

[0094] Since the sample set contains a large number of sample images, image enhancement can be performed by preprocessing the original images after they are obtained. In this embodiment, preprocessing may include operations such as rotation, scaling, cropping, flipping, brightness adjustment, contrast adjustment, noise reduction, and sharpening.

[0095] Specifically, during the preprocessing of the original images, various data augmentation techniques can be employed, including image rotation, scaling, cropping, flipping, and brightness and contrast adjustments, to expand the sample size and diversity, thereby improving the model's generalization ability. Simultaneously, denoising and sharpening are performed on some blurry or low-quality lane lines in the original images, laying the foundation for subsequent model training.

[0096] The following is combined Figure 5 The process of constructing the sample set is explained. Figure 5 This is a schematic diagram illustrating a sample set construction process provided in an embodiment of this application. The process of constructing a sample set can be divided into two parts: data preparation and data output.

[0097] During data preparation, video data actually collected by the vehicle can be acquired, and multiple image frames can be extracted from the video data, which can then be used as the original images. After acquiring any original image, preprocessing can be performed on it, such as flipping, rotating, adjusting brightness, adjusting contrast, cropping, stitching, denoising, and sharpening.

[0098] After preprocessing the original image, the lane line areas included in the preprocessed original image can be accurately labeled to obtain labels.

[0099] After annotation, the preprocessed original image and its corresponding labels can be input into the UNICE model, allowing the UNICE model to automatically enhance the clarity and continuity of lane lines and repair broken or incomplete lane line areas. To improve the accuracy of model enhancement, the UNICE model can be a pre-trained and optimized model.

[0100] To further improve the accuracy of vehicle lane change detection, based on the above embodiments, in this embodiment, the image acquisition device displays a crosshair in the acquisition screen, the crosshair coincides with a calibration rod set on the vehicle, and the calibration rod is set on the central axis of the vehicle.

[0101] Since the detection of vehicle lane changes is based entirely on road video captured by the image acquisition equipment installed on the vehicle, the image acquisition equipment of the vehicle needs to be calibrated in this embodiment.

[0102] In this embodiment of the application, the image acquisition device may include crosshairs in the acquisition screen. Figure 6 This is a schematic diagram of an image acquisition device provided in an embodiment of this application, such as... Figure 6 As shown, the crosshairs included in the image are the crosshairs included in the captured image. It should be noted that... Figure 6The example is only for the purpose of understanding the "crosshair"; the "crosshair" is not included in the road video captured by the image acquisition device.

[0103] When installing the image acquisition device, it can be ensured that the crosshairs included in the image coincide with the calibration rod set on the vehicle. This calibration rod is located in the horizontal direction of the vehicle, and the image acquisition device can be set on the vehicle's central axis. In this embodiment, for the vertical direction of the vehicle, the image acquisition device can be set at any position in the vertical direction, as long as it can capture the complete ground image.

[0104] For example, Figure 7 This is a schematic diagram of the installation of an image acquisition device provided in an embodiment of this application, such as... Figure 7 As shown, the image acquisition device can be set at the intersection of the vehicle's centerline and the horizontal line 2-5 cm above the vehicle's windshield wipers.

[0105] It should be noted that the site must be flat and the vehicle must be in a level position when installing the image acquisition equipment to ensure that the image from the installed equipment is horizontal and not tilted.

[0106] In one possible implementation, the image acquisition device used to acquire road video can have a resolution of 1080P or higher.

[0107] It should be noted that the parameters involved in the vehicle lane change detection method provided in this application embodiment can be adjusted according to the actual use scenario (such as urban roads, highways, and rural roads) or user needs (such as passenger transport company safety standards and private car personalized settings). The adjustment method can be achieved by directly inputting values ​​or selecting preset parameter templates through the ADAS backend visual interface, without modifying the core code logic.

[0108] The vehicle lane change detection method provided in this application embodiment can be used for any type of vehicle, such as buses, passenger buses, subway trains, intercity trains, cars, sport utility vehicles (SUVs), new energy vehicles, etc.

[0109] To ensure the accuracy of vehicle lane change detection, in this embodiment, the lane line crossing judgment benchmark can be calibrated during initialization based on parameters such as the vehicle model's wheelbase, wheel size, and body width, thus adapting to the detection needs of different vehicle models and limiting detection to YOLO model acquisition.

[0110] Compared to traditional illegal lane change detection methods, the vehicle lane change detection method provided in this application enhances lane line samples through the UNICE large model and combines YOLO and UFLDV2 models to achieve high-precision identification of lane line type and location, effectively improving the accuracy and robustness of lane change detection under complex road conditions. Compared to traditional methods, it can significantly reduce the false detection rate, improve the illegal lane change detection rate, realize automatic and real-time alarm for vehicle illegal lane change behavior, and enhance the level of intelligent traffic safety management.

[0111] Based on the same inventive concept, this application also provides a vehicle lane change detection device. Figure 8 This is a schematic diagram of a vehicle lane change detection device provided in an embodiment of this application, as shown below. Figure 8 As shown, the device includes: The acquisition module 801 is used to acquire road video captured by the vehicle's image acquisition device; The detection module 802 is used to: input the image frame into the feature extraction layer of the lane line detection model for multiple image frames in the road video, extract features from the image frame to obtain a first feature tensor; input the first feature tensor into the feature fusion layer of the lane line detection model, concatenate the data of different dimensions in the first feature tensor to obtain a first fused feature tensor; input the first fused feature tensor into the first fully connected layer and the second fully connected layer of the lane line detection model respectively, to obtain the position information of the lane lines included in the image frame output by the first fully connected layer, and the target type of the lane lines included in the image frame output by the second fully connected layer; and determine whether the vehicle has engaged in illegal lane changing behavior based on the position information of the lane lines included in multiple image frames, the target type, and the center point of the image frames.

[0112] In one possible implementation, the detection module 802 is further configured to use a target detection model to determine image region information of lane lines included in multiple image frames in the road video; The detection module 802 is specifically used to input the image region information of the image frame and the lane line corresponding to the image frame into the feature extraction layer of the lane line detection model for multiple image frames. The feature extraction layer performs feature extraction on the image region corresponding to the image region information in the image frame to obtain a first feature tensor.

[0113] In one possible implementation, the detection module 802 is further configured to determine whether image region information is detected in other image frames within a set time period after the image frame if no image region information is detected in any image frame; if image region information is detected in any other image frame, it is determined that the lane line is temporarily obscured, and the image region information corresponding to the previous image frame of the image frame is used as the image region information of the image frame.

[0114] In one possible implementation, the detection module 802 is specifically used to determine the trajectory of the relative position change between the lane line and the center point based on the position information of the lane lines included in multiple image frames in the road video; if the trajectory of change determines that the center point crosses the lane line, then it is determined whether the target type of the crossed lane line is a preset type; if so, a first prompt message is output, wherein the preset type includes solid line and double solid line.

[0115] In one possible implementation, the detection module 802 is further configured to output a second prompt message if the center point is determined to have crossed the lane line N times consecutively within a preset time period based on the change trajectory, where N is a positive integer.

[0116] In one possible implementation, the acquisition module 801 is further configured to acquire target sample images in the sample set, as well as standard position information and standard type of lane lines corresponding to the target sample images; The training module 803 is used to input the target sample image into the feature extraction layer of the original lane detection model to extract features from the target sample image and obtain a second feature tensor; input the second feature tensor into the feature fusion layer of the original lane detection model to concatenate data of different dimensions in the second feature tensor to obtain a second fused feature vector; input the second fused feature vector into the first fully connected layer and the second fully connected layer of the original lane detection model respectively to obtain the predicted position information of the lanes included in the target sample image output by the first fully connected layer, and the predicted type of the lanes included in the target sample image output by the second fully connected layer; determine a first loss value based on the deviation between the predicted position information and the standard position information; determine a second loss value based on the deviation between the predicted type and the standard type; determine a target loss value based on the first loss value and the second loss value, and adjust the parameters of the original lane detection model based on the target loss value.

[0117] In one possible implementation, the acquisition module 801 is further configured to acquire an original image and a label corresponding to the original image, wherein the label includes standard location information and standard type of lane lines included in the original image; The construction module 804 is used to input the original image and the label into the augmented large model to obtain the augmented image corresponding to the original image; and to save the augmented image as a sample image corresponding to the label in the sample set.

[0118] In one possible implementation, the construction module 804 is further configured to preprocess the original image, the preprocessing including rotation, scaling, cropping, flipping, brightness adjustment, contrast adjustment, noise reduction, and sharpening.

[0119] In one possible implementation, the image acquisition device displays a crosshair in the acquisition screen, the crosshair coinciding with a calibration rod disposed on the vehicle's central axis.

[0120] Based on the above embodiments, this application also provides an electronic device. Figure 9 This application provides a schematic diagram of an electronic device structure, such as... Figure 9 As shown, it includes: processor 901, communication interface 902, memory 903 and communication bus 904, wherein processor 901, communication interface 902 and memory 903 communicate with each other through communication bus 904. The memory 903 stores a computer program. When the program is executed by the processor 901, the processor 901 performs the following steps: Acquire road video captured by the vehicle's image acquisition device; For multiple image frames in the road video, the image frame is input into the feature extraction layer of the lane detection model to extract features and obtain a first feature tensor. The first feature tensor is then input into the feature fusion layer of the lane detection model to concatenate data from different dimensions in the first feature tensor to obtain a first fused feature tensor. The first fused feature tensor is then input into the first fully connected layer and the second fully connected layer of the lane detection model to obtain the position information of the lane lines included in the image frame output by the first fully connected layer and the target type of the lane lines included in the image frame output by the second fully connected layer. Based on the lane line position information, target type, and center point of the image frames included in multiple image frames, it is determined whether the vehicle has engaged in illegal lane changing behavior.

[0121] In one possible implementation, after acquiring the road video captured by the image acquisition device of the vehicle, and before inputting the image frames from the road video into the feature extraction layer of the lane detection model, the method further includes: The image region information of lane lines included in multiple image frames in the road video is determined using an object detection model; For multiple image frames in the road video, the image frame is input into the feature extraction layer of the lane detection model to extract features from the image frame, resulting in a first feature tensor, including: For multiple image frames, the image frame and the image region information of the lane line corresponding to the image frame are input into the feature extraction layer of the lane line detection model. The feature extraction layer performs feature extraction on the image region corresponding to the image region information in the image frame to obtain the first feature tensor.

[0122] In one possible implementation, the method further includes: If no image region information is detected in any image frame, then determine whether image region information is detected in other image frames within a set time period after that image frame; If image region information is detected in any other image frame, it is determined that the lane line is temporarily obscured, and the image region information corresponding to the previous image frame is used as the image region information of this image frame.

[0123] In one possible implementation, determining whether the vehicle has engaged in illegal lane-changing behavior based on the lane line position information, target type, and center point of the image frames included in multiple image frames includes: Based on the positional information of the lane lines included in multiple image frames in the road video, determine the trajectory of the relative position change between the lane lines and the center point; If the center point is determined to cross the lane line based on the change trajectory, it is then determined whether the target type of the crossed lane line is a preset type. If so, a first prompt message is output, wherein the preset type includes solid line and double solid line.

[0124] In one possible implementation, the method further includes: If the center point is determined to have crossed the lane line N times consecutively within a preset time period based on the change trajectory, then a second prompt message is output, where N is a positive integer.

[0125] In one possible implementation, the training process of the lane detection model includes: Obtain the target sample image in the sample set, as well as the standard position information and standard type of the lane lines corresponding to the target sample image; The target sample image is input into the feature extraction layer of the original lane detection model to extract features from the target sample image, resulting in a second feature tensor. The second feature tensor is then input into the feature fusion layer of the original lane detection model to concatenate data from different dimensions in the second feature tensor, resulting in a second fused feature vector. The second fused feature vector is then input into the first fully connected layer and the second fully connected layer of the original lane detection model, respectively, to obtain the predicted position information of the lane lines included in the target sample image output by the first fully connected layer, and the predicted type of the lane lines included in the target sample image output by the second fully connected layer. A first loss value is determined based on the deviation between the predicted location information and the standard location information; A second loss value is determined based on the deviation between the predicted type and the standard type; Based on the first loss value and the second loss value, a target loss value is determined, and the parameters of the original lane detection model are adjusted based on the target loss value.

[0126] In one possible implementation, the process of constructing the sample set includes: Obtain the original image and the corresponding label of the original image, wherein the label includes the standard location information and standard type of the lane lines included in the original image; The original image and the label are input into the augmented model to obtain the augmented image corresponding to the original image; The enhanced image is stored in the sample set as a sample image corresponding to the label.

[0127] In one possible implementation, after acquiring the original image and before inputting the original image and the label into the augmented large model, the method further includes: The original image is preprocessed, including rotation, scaling, cropping, flipping, brightness adjustment, contrast adjustment, noise reduction, and sharpening.

[0128] In one possible implementation, the image acquisition device displays a crosshair in the acquisition screen, the crosshair coinciding with a calibration rod disposed on the vehicle's central axis.

[0129] The communication bus mentioned in the aforementioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 902 is used for communication between the aforementioned electronic device and other devices. The memory can include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0130] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0131] Based on the above embodiments, this invention also provides a computer-readable storage medium storing a computer program executable by a processor, such that when the processor executes the program, it performs the following steps: Acquire road video captured by the vehicle's image acquisition device; For multiple image frames in the road video, the image frame is input into the feature extraction layer of the lane detection model to extract features and obtain a first feature tensor. The first feature tensor is then input into the feature fusion layer of the lane detection model to concatenate data from different dimensions in the first feature tensor to obtain a first fused feature tensor. The first fused feature tensor is then input into the first fully connected layer and the second fully connected layer of the lane detection model to obtain the position information of the lane lines included in the image frame output by the first fully connected layer and the target type of the lane lines included in the image frame output by the second fully connected layer. Based on the lane line position information, target type, and center point of the image frames included in multiple image frames, it is determined whether the vehicle has engaged in illegal lane changing behavior.

[0132] In one possible implementation, after acquiring the road video captured by the image acquisition device of the vehicle, and before inputting the image frames from the road video into the feature extraction layer of the lane detection model, the method further includes: The image region information of lane lines included in multiple image frames in the road video is determined using an object detection model; For multiple image frames in the road video, the image frame is input into the feature extraction layer of the lane detection model to extract features from the image frame, resulting in a first feature tensor, including: For multiple image frames, the image frame and the image region information of the lane line corresponding to the image frame are input into the feature extraction layer of the lane line detection model. The feature extraction layer performs feature extraction on the image region corresponding to the image region information in the image frame to obtain the first feature tensor.

[0133] In one possible implementation, the method further includes: If no image region information is detected in any image frame, then determine whether image region information is detected in other image frames within a set time period after that image frame; If image region information is detected in any other image frame, it is determined that the lane line is temporarily obscured, and the image region information corresponding to the previous image frame is used as the image region information of this image frame.

[0134] In one possible implementation, determining whether the vehicle has engaged in illegal lane-changing behavior based on the lane line position information, target type, and center point of the image frames included in multiple image frames includes: Based on the positional information of the lane lines included in multiple image frames in the road video, determine the trajectory of the relative position change between the lane lines and the center point; If the center point is determined to cross the lane line based on the change trajectory, it is then determined whether the target type of the crossed lane line is a preset type. If so, a first prompt message is output, wherein the preset type includes solid line and double solid line.

[0135] In one possible implementation, the method further includes: If the center point is determined to have crossed the lane line N times consecutively within a preset time period based on the change trajectory, then a second prompt message is output, where N is a positive integer.

[0136] In one possible implementation, the training process of the lane detection model includes: Obtain the target sample image in the sample set, as well as the standard position information and standard type of the lane lines corresponding to the target sample image; The target sample image is input into the feature extraction layer of the original lane detection model to extract features from the target sample image, resulting in a second feature tensor. The second feature tensor is then input into the feature fusion layer of the original lane detection model to concatenate data from different dimensions in the second feature tensor, resulting in a second fused feature vector. The second fused feature vector is then input into the first fully connected layer and the second fully connected layer of the original lane detection model, respectively, to obtain the predicted position information of the lane lines included in the target sample image output by the first fully connected layer, and the predicted type of the lane lines included in the target sample image output by the second fully connected layer. A first loss value is determined based on the deviation between the predicted location information and the standard location information; A second loss value is determined based on the deviation between the predicted type and the standard type; Based on the first loss value and the second loss value, a target loss value is determined, and the parameters of the original lane detection model are adjusted based on the target loss value.

[0137] In one possible implementation, the process of constructing the sample set includes: Obtain the original image and the corresponding label of the original image, wherein the label includes the standard location information and standard type of the lane lines included in the original image; The original image and the label are input into the augmented model to obtain the augmented image corresponding to the original image; The enhanced image is stored in the sample set as a sample image corresponding to the label.

[0138] In one possible implementation, after acquiring the original image and before inputting the original image and the label into the augmented large model, the method further includes: The original image is preprocessed, including rotation, scaling, cropping, flipping, brightness adjustment, contrast adjustment, noise reduction, and sharpening.

[0139] In one possible implementation, the image acquisition device displays a crosshair in the acquisition screen, the crosshair coinciding with a calibration rod disposed on the vehicle's central axis.

[0140] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0143] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0144] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting vehicle lane changes, characterized in that, The method includes: Acquire road video captured by the vehicle's image acquisition device; For multiple image frames in the road video, the image frame is input into the feature extraction layer of the lane detection model to extract features and obtain a first feature tensor. The first feature tensor is then input into the feature fusion layer of the lane detection model to concatenate data from different dimensions in the first feature tensor to obtain a first fused feature tensor. The first fused feature tensor is then input into the first fully connected layer and the second fully connected layer of the lane detection model to obtain the position information of the lane lines included in the image frame output by the first fully connected layer and the target type of the lane lines included in the image frame output by the second fully connected layer. Based on the lane line position information, target type, and center point of the image frames included in multiple image frames, it is determined whether the vehicle has engaged in illegal lane changing behavior.

2. The method according to claim 1, characterized in that, After acquiring the road video captured by the vehicle's image acquisition device, and before inputting multiple image frames from the road video into the feature extraction layer of the lane detection model, the method further includes: The image region information of lane lines included in multiple image frames in the road video is determined using an object detection model; For multiple image frames in the road video, the image frame is input into the feature extraction layer of the lane detection model to extract features from the image frame, resulting in a first feature tensor, including: For multiple image frames, the image frame and the image region information of the lane line corresponding to the image frame are input into the feature extraction layer of the lane line detection model. The feature extraction layer performs feature extraction on the image region corresponding to the image region information in the image frame to obtain the first feature tensor.

3. The method according to claim 2, characterized in that, The method further includes: If no image region information is detected in any image frame, then determine whether image region information is detected in other image frames within a set time period after that image frame; If image region information is detected in any other image frame, it is determined that the lane line is temporarily obscured, and the image region information corresponding to the previous image frame is used as the image region information of this image frame.

4. The method according to claim 1, characterized in that, The step of determining whether the vehicle has engaged in illegal lane changing based on the lane line position information, target type, and center point of the image frames included in multiple image frames includes: Based on the positional information of lane lines included in multiple image frames in the road video, the trajectory of the relative position change between the lane lines and the center point is determined; If the center point is determined to cross the lane line based on the change trajectory, it is then determined whether the target type of the crossed lane line is a preset type. If so, a first prompt message is output, wherein the preset type includes solid line and double solid line.

5. The method according to claim 4, characterized in that, The method further includes: If the center point is determined to have crossed the lane line N times consecutively within a preset time period based on the change trajectory, then a second prompt message is output, where N is a positive integer.

6. The method according to claim 1, characterized in that, The training process of the lane detection model includes: Obtain the target sample image in the sample set, as well as the standard position information and standard type of the lane lines corresponding to the target sample image; The target sample image is input into the feature extraction layer of the original lane detection model to extract features from the target sample image, resulting in a second feature tensor. The second feature tensor is then input into the feature fusion layer of the original lane detection model to concatenate data from different dimensions in the second feature tensor, resulting in a second fused feature vector. The second fused feature vector is then input into the first fully connected layer and the second fully connected layer of the original lane detection model, respectively, to obtain the predicted position information of the lane lines included in the target sample image output by the first fully connected layer, and the predicted type of the lane lines included in the target sample image output by the second fully connected layer. A first loss value is determined based on the deviation between the predicted location information and the standard location information; A second loss value is determined based on the deviation between the predicted type and the standard type; Based on the first loss value and the second loss value, a target loss value is determined, and the parameters of the original lane detection model are adjusted based on the target loss value.

7. The method according to claim 6, characterized in that, The process of constructing the sample set includes: Obtain the original image and the corresponding label of the original image, wherein the label includes the standard location information and standard type of the lane lines included in the original image; The original image and the label are input into the augmented model to obtain the augmented image corresponding to the original image; The enhanced image is stored in the sample set as a sample image corresponding to the label.

8. The method according to claim 7, characterized in that, After acquiring the original image and before inputting the original image and the label into the augmented large model, the method further includes: The original image is preprocessed, including rotation, scaling, cropping, flipping, brightness adjustment, contrast adjustment, noise reduction, and sharpening.

9. The method according to claim 1, characterized in that, The image acquisition device displays a crosshair in the acquisition screen. The crosshair coincides with a calibration rod set on the vehicle. The calibration rod is set on the central axis of the vehicle.

10. A vehicle lane change detection device, characterized in that, The device includes: The acquisition module is used to acquire road videos captured by the vehicle's image acquisition device; The detection module is used to take multiple image frames from the road video, input the image frame into the feature extraction layer of the lane line detection model, extract features from the image frame to obtain a first feature tensor; input the first feature tensor into the feature fusion layer of the lane line detection model, concatenate the data of different dimensions in the first feature tensor to obtain a first fused feature tensor; input the first fused feature tensor into the first fully connected layer and the second fully connected layer of the lane line detection model respectively to obtain the position information of the lane lines included in the image frame output by the first fully connected layer, and the target type of the lane lines included in the image frame output by the second fully connected layer; and determine whether the vehicle has engaged in illegal lane changing behavior based on the position information of the lane lines included in multiple image frames, the target type, and the center point of the image frames.

Images