Vehicle lane-keeping behavior real-time monitoring method based on image semantic segmentation

Abstract

This invention discloses a real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation, comprising the following steps: acquiring traffic video data to obtain a road image sequence and performing preprocessing; inputting the image semantic segmentation model for semantic segmentation processing to obtain a semantic segmentation result image; performing morphological processing and connected component analysis to obtain the set of pixel boundaries of the vehicle region and the set of pixel boundaries of the lane lines; calculating the spatial position data of the lane lines and performing perspective correction processing to obtain perspective-corrected lane line spatial position data; generating the position region of the vehicle region in the image coordinate system; performing target association matching to generate vehicle motion trajectory data; establishing spatial relationship data and forming relative position changes; performing temporal consistency analysis and real-time determination, and outputting the vehicle lane-crossing behavior detection result, thereby achieving continuous, accurate, and real-time monitoring of vehicle lane-crossing behavior.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method for real-time monitoring of vehicle lane crossing behavior based on image semantic segmentation. Background Technology

[0002] With the continuous development of intelligent transportation systems and road video surveillance technology, the automatic identification and behavior analysis of motor vehicle operating status has become an important research direction in traffic management, evidence collection of violations, and road safety early warning. Among these, vehicle lane crossing behavior, a common traffic violation, typically manifests as vehicles contacting, straddling, or crossing lane lines during driving. This can easily lead to traffic disorder, traffic accidents, and reduced road efficiency. Therefore, rapid, accurate, and continuous automatic monitoring of vehicle lane crossing behavior has significant practical application value.

[0003] In existing technologies, vehicle lane crossing detection typically relies on road surveillance video, image processing algorithms, and object detection or tracking algorithms. Common methods include those based on traditional image processing, those based on object detection bounding boxes, and those based on multi-object tracking. Traditional image processing methods generally extract lane lines from the road using edge detection, Hough transform, thresholding, or morphological operations. These are then combined with foreground detection, background modeling, or moving object segmentation to identify vehicle targets. Finally, the positional relationship between the vehicle and the lane lines is used to determine whether lane crossing has occurred. While these methods can achieve a certain level of detection under simple scene conditions, stable lighting, and minimal background interference, in real-world road scenarios, they are often affected by factors such as shadow occlusion, worn road markings, vehicle appearance differences, heavy traffic, and weather changes. This leads to unstable lane line extraction and inaccurate vehicle region segmentation, thus affecting the lane crossing detection results. Summary of the Invention

[0004] One objective of this invention is to propose a real-time monitoring method for vehicle lane crossing behavior based on image semantic segmentation. This invention fully utilizes technologies such as image semantic segmentation, target association matching, lane line spatial location analysis, and temporal consistency analysis. It describes in detail the implementation method for extracting, tracking, constructing spatial relationships, and determining lane crossing behavior in road image sequences for vehicle regions and lane line regions. It has the advantages of high detection accuracy, strong real-time performance, good continuity, and strong adaptability to complex traffic scenarios.

[0005] The real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation according to an embodiment of the present invention includes the following steps: Traffic video data is collected to obtain road image sequences, which are then preprocessed. The preprocessed road image sequence is input into the image semantic segmentation model for semantic segmentation. The image semantic segmentation model adopts an encoder-decoder architecture. The encoder uses LDConv convolution for feature extraction, and the decoder uses DySample sampler for feature recovery to obtain the semantic segmentation result image. Morphological processing and connected component analysis are performed on the semantic segmentation result image to extract vehicle region and lane line region, and obtain the set of boundary pixels of vehicle region and lane line region. The spatial location data of the lane line is calculated based on the set of pixels at the lane line boundary, and perspective correction is performed to obtain perspective-corrected lane line spatial location data. Based on the set of pixels at the vehicle region boundary, the location region of the vehicle region in the image coordinate system is generated. Target association matching is performed on the location region to establish the correspondence between vehicle regions in consecutive image frames and generate vehicle motion trajectory data; Based on perspective-corrected lane line spatial location data and vehicle motion trajectory data, spatial relationship data between vehicle area and lane line area is established, and the relative positional changes between vehicle area boundary pixel set and lane line boundary pixel set are formed. Based on the spatial relationship data of the current frame and the relative position changes in consecutive image frames, a temporal consistency analysis is performed to determine the current vehicle's line-crossing behavior in real time and output the vehicle line-crossing behavior detection result.

[0006] Optionally, traffic scene video is continuously collected by road monitoring equipment set in the road monitoring area to obtain traffic video data, and the traffic video data is extracted frame by frame at preset time intervals to generate a road image sequence. The road image sequence is preprocessed, including resolution unification processing, noise suppression processing, and brightness equalization processing.

[0007] Optionally, obtaining the semantic segmentation result map specifically includes: The preprocessed road image sequence is input into the image semantic segmentation model. Tensor quantization is performed on each frame of the preprocessed road image sequence to obtain the input feature map. The input feature map is input into the encoder, and the LDConv convolution is used to extract features from the input feature map to obtain the LDConv convolution feature map; The LDConv convolutional feature map is subjected to multi-layer encoding and multi-scale downsampling to obtain encoded feature maps of different scales. The encoded feature maps of different scales are input into the decoder for layer-by-layer decoding. The DySample sampler is used for upsampling recovery to determine the dynamic sampling offset corresponding to each sampling point. The dynamic sampling offset in the DySample sampler is initialized, the dynamic sampling offset is updated according to the position of each sampling point after initialization, and feature sampling is completed according to the updated position of each sampling point to generate the upsampled and restored feature map. The upsampled and restored feature map is input into the classification layer of the image semantic segmentation model to calculate the predicted probability of each pixel for each category. The category label of each pixel is determined based on the predicted probability of each category, and the semantic segmentation result image is obtained.

[0008] Optionally, obtaining the vehicle region boundary pixel set and the lane line boundary pixel set specifically includes: Read the category labels of each pixel in the semantic segmentation result image, and construct the initial binary map of the vehicle region and the initial binary map of the lane line region based on the category labels of each pixel; Morphological processing is performed on the initial binary map of the vehicle region and the initial binary map of the lane line region to obtain the processed vehicle region map and the processed lane line region map. Connectivity analysis is performed on the vehicle area processing map and the lane line area processing map. The connectivity of each target area in the image is marked, the pixel set corresponding to each connected region is determined, and each connected region is numbered and labeled to obtain the labeled connected regions. The area of ​​the marked connected regions is calculated, and connected regions with an area smaller than the preset area threshold are removed according to the preset area threshold, while vehicle connected regions and lane line connected regions with an area greater than or equal to the preset area threshold are retained. The retained vehicle connectivity region is defined as the vehicle region, and the retained lane line connectivity region is defined as the lane line region. The boundary pixels of each vehicle region and each lane line region are extracted. When a pixel in the target region is not a pixel in the neighborhood of the current target region, the pixel is defined as a boundary pixel, thus obtaining the set of boundary pixels of the vehicle region and the set of boundary pixels of the lane line region.

[0009] Optionally, obtaining the perspective-corrected lane line spatial position data specifically includes: Extract the position coordinates of each pixel in the set of pixels at the lane boundary to obtain the set of pixel coordinates of each pixel at the lane boundary in the image coordinate system; Construct lane line spatial location data based on the set of pixel coordinates; Based on the installation location, shooting angle, and perspective imaging relationship of the road monitoring equipment, establish the perspective transformation relationship between the image coordinate system and the road plane coordinate system; Perspective correction is performed on the spatial position data of lane lines using perspective transformation. The position coordinates of the lane line boundary pixels in the image coordinate system are mapped to the perspective-corrected road plane coordinate system to obtain the position coordinates of the lane lines after perspective transformation. The position coordinates of the lane lines after perspective transformation are normalized to obtain the spatial position coordinates of the lane lines after perspective correction, and the spatial position data of the perspective-corrected lane lines are constructed.

[0010] Optionally, the generation of the location region specifically includes: The position coordinates of each pixel in the pixel set of the vehicle region boundary are extracted to obtain the set of pixel coordinates of each vehicle region boundary pixel in the image coordinate system. The boundary range of the vehicle region in the image coordinate system is determined based on the set of pixel coordinates; Construct the location region of the vehicle area in the image coordinate system based on the boundary range.

[0011] Optionally, the generation of the vehicle motion trajectory data specifically includes: Calculate the center coordinates of each vehicle area based on the location area; For vehicle regions in two adjacent frames, the degree of matching between vehicle regions is calculated based on the center coordinates of the vehicle regions and the spatial relationship between the regions. The target association between vehicle regions is determined based on the degree of matching. Under the condition that the degree of matching is greater than or equal to the preset matching threshold, the combination of vehicle regions with the highest degree of matching is selected as the association result, and the correspondence between vehicle regions in two adjacent frames of images is established. According to the temporal order of the image frames, based on the correspondence between vehicle regions in adjacent image frames, vehicle regions belonging to the same target in consecutive image frames are assigned the same target identifier, forming an associated sequence of vehicle regions in consecutive image frames. Based on the associated sequence, the center coordinates and location regions of the vehicle region corresponding to the same target identifier in each image frame are obtained, and arranged in the time order of the image frames to construct vehicle motion trajectory data.

[0012] Optionally, the establishment of the spatial relationship data and the formation of relative change positions specifically include: The spatial boundary position of each lane line region in the corresponding image frame is obtained based on the perspective-corrected lane line spatial position data. Based on vehicle motion trajectory data, target identifiers are read from vehicle regions in consecutive image frames to obtain vehicle regions belonging to the same target identifier in consecutive image frames; For each image frame, calculate the distance between each boundary pixel of the vehicle region and each boundary pixel of the lane line in the vehicle region boundary pixel set, and determine the minimum distance between the boundary pixels of the vehicle region boundary pixel and the lane line boundary pixel set, thus determining the minimum boundary distance between the vehicle region and the corresponding lane line region. Based on the center coordinates of the vehicle area, the location area, the spatial boundary positions of each lane line area in the corresponding image frame, and the minimum boundary distance between the vehicle area and the corresponding lane line area, spatial relationship data between the vehicle area and the lane line area is formed. For vehicle regions corresponding to the same target identifier in consecutive image frames, the difference between the minimum boundary distance in the current image frame and the minimum boundary distance in the previous image frame is used to obtain the change in the minimum boundary distance between adjacent image frames, forming the relative positional change between the set of vehicle region boundary pixels and the set of lane line boundary pixels.

[0013] Optionally, the output of the vehicle crossing behavior detection result specifically includes: Based on the spatial relationship data of the current image frame, the minimum boundary distance between the vehicle area and the corresponding lane line area in the current image frame is obtained in real time. Based on the comparison result between the minimum boundary distance in the current image frame and the preset contact distance threshold, the contact state between the vehicle area and the lane line area in the current image frame is determined. Based on the relative position changes in the current image frame, obtain the trend of the relative position change of the vehicle area relative to the lane line area; According to the time sequence of image frames, the contact state and relative position change trend in continuous image frames are updated in real time based on the input of the current image frame. Within a preset time window, the time consistency analysis of the updated contact state and relative position change trend is performed to obtain the continuous contact state and crossing state between the vehicle area and the lane line area. Based on the continuous contact state and the crossing state, the vehicle region in the current image frame is judged in real time for crossing the line, and the corresponding vehicle crossing the line behavior detection result is output.

[0014] The beneficial effects of this invention are: The proposed real-time monitoring method for vehicle lane crossing behavior based on image semantic segmentation first performs semantic segmentation on road image sequences, which can simultaneously extract vehicle regions and lane line regions at the pixel level. Compared with methods based solely on rectangular detection boxes or traditional edge extraction, it can more accurately represent the true contour boundaries of vehicles and the actual boundary positions of lane lines, thereby providing a more reliable data foundation for subsequent refined analysis of the spatial relationship between vehicles and lane lines and improving the accuracy of lane crossing behavior recognition.

[0015] This invention employs an encoder-decoder architecture in the image semantic segmentation stage. The encoder incorporates LDConv convolution for feature extraction, while the decoder uses a DySample sampler for feature recovery. This allows for more comprehensive feature representation and finer boundary restoration of vehicle regions, lane line regions, and road background regions in road images. Through these techniques, this invention enhances the representation of slender lane line structures and vehicle edge details in complex road scenes. Even when vehicles are close to lane lines, there is partial occlusion, lighting variations, or background interference, it can still consistently achieve high-quality semantic segmentation results, thereby improving the overall robustness and real-time monitoring capabilities of the system.

[0016] This invention extracts the pixel sets of vehicle region boundaries and lane line boundaries by performing morphological processing and connected component analysis on the semantic segmentation result image. It further constructs the spatial location data of the vehicle region and lane lines, enabling vehicle and lane line targets to move beyond simple category recognition and generate structured data results usable for spatial location calculation, boundary distance measurement, and continuous frame correlation analysis. This technical solution effectively overcomes the problems of coarse vehicle detection results, incomplete lane line position representation, and difficulty in supporting subsequent accurate judgment in existing technologies, providing a complete data link for continuous monitoring of vehicle lane crossing behavior. Furthermore, this invention performs perspective correction processing on the lane line spatial location data, mapping the positions of lane line boundary pixels in the image coordinate system to the perspective-corrected road plane coordinate system, thereby reducing the impact of perspective distortion in road monitoring images on spatial relationship analysis. Compared with existing technologies that directly determine the relationship between vehicles and lane lines from the original image perspective, this invention makes the spatial location description of lane lines more consistent with the actual road distribution characteristics, improving the consistency and reliability of distance and relative position relationship judgments between vehicle and lane line regions, thus enhancing the accuracy of lane crossing behavior detection results.

[0017] This invention performs target association matching on vehicle regions in consecutive image frames, establishing correspondences between adjacent image frames based on location regions, center coordinates, and the degree of region matching, and constructing vehicle motion trajectory data. By introducing a continuous frame association mechanism, this invention can continuously describe the temporal motion state of the same vehicle target, avoiding random errors and instantaneous misjudgments caused by judging line crossing solely based on a single image frame. This scheme extends vehicle line crossing behavior detection from a single static analysis to continuous dynamic analysis, which is beneficial for improving the continuity and stability of vehicle behavior recognition in real traffic scenarios.

[0018] In the core process of determining vehicle lane crossing behavior, this invention does not simply rely on the overall position of the vehicle or the relationship between the bounding box and the lane line. Instead, it establishes spatial relationship data and relative position change results between the vehicle region and the lane line region based on the minimum boundary distance between the pixel set of the vehicle region boundary and the pixel set of the lane line boundary, and the change of this minimum boundary distance between consecutive image frames. In this way, this invention can describe the dynamic process of a vehicle moving from approaching the lane line, to contacting the lane line, to crossing the lane line with finer granularity. This effectively overcomes the problem in existing technologies of difficulty in identifying behaviors such as local lane crossing, short-term contact, and boundary crossing, thereby significantly improving the precision of lane crossing behavior determination.

[0019] This invention comprehensively determines continuous contact and crossing states by combining contact state, relative position change trends, and time consistency analysis, and then outputs the vehicle lane crossing behavior detection result. Compared with existing technologies that rely solely on a single frame or local positional relationship for judgment, this invention can continuously verify the relationship between the vehicle and the lane line from a temporal dimension, thereby effectively reducing false positives and false negatives caused by image noise, short-term occlusion, segmentation fluctuations, or target jitter, and improving the stability, reliability, and engineering application value of the detection results. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of the real-time monitoring method for vehicle lane crossing behavior based on image semantic segmentation proposed in this invention; Figure 2 This is a schematic diagram illustrating the construction of the semantic segmentation result image of the real-time monitoring method for vehicle lane crossing behavior based on image semantic segmentation proposed in this invention. Figure 3 This is a schematic diagram illustrating the construction of perspective-corrected lane line spatial location data for the real-time monitoring method of vehicle lane crossing behavior based on image semantic segmentation proposed in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] refer to Figures 1-3 A real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation includes the following steps: Traffic video data is collected to obtain road image sequences, which are then preprocessed. The preprocessed road image sequence is input into the image semantic segmentation model for semantic segmentation. The image semantic segmentation model adopts an encoder-decoder architecture. The encoder uses LDConv convolution for feature extraction, and the decoder uses DySample sampler for feature recovery to obtain the semantic segmentation result image. Morphological processing and connected component analysis are performed on the semantic segmentation result image to extract vehicle region and lane line region, and obtain the set of boundary pixels of vehicle region and lane line region. The spatial location data of the lane line is calculated based on the set of pixels at the lane line boundary, and perspective correction is performed to obtain perspective-corrected lane line spatial location data. Based on the set of pixels at the vehicle region boundary, the location region of the vehicle region in the image coordinate system is generated. Target association matching is performed on the location region to establish the correspondence between vehicle regions in consecutive image frames and generate vehicle motion trajectory data; Based on perspective-corrected lane line spatial location data and vehicle motion trajectory data, spatial relationship data between vehicle area and lane line area is established, and the relative positional changes between vehicle area boundary pixel set and lane line boundary pixel set are formed. Based on the spatial relationship data of the current frame and the relative position changes in consecutive image frames, a temporal consistency analysis is performed to determine the current vehicle's line-crossing behavior in real time and output the vehicle line-crossing behavior detection result.

[0023] In this embodiment, traffic scene video is continuously collected by road monitoring equipment set in the road monitoring area to obtain traffic video data. The traffic video data is then extracted frame by frame at preset time intervals to generate a road image sequence. The road image sequence is then preprocessed, including resolution unification processing, noise suppression processing, and brightness equalization processing.

[0024] In this embodiment, obtaining the semantic segmentation result map specifically includes: The preprocessed road image sequence is input into the image semantic segmentation model. Tensor quantization is performed on each frame of the preprocessed road image sequence to obtain the input feature map. The specific process of Zhang quantization is as follows: read the pixel value information of each pixel in each frame of road image, and construct an image data matrix according to the row coordinates and column coordinates of each pixel in the image plane according to the preset spatial arrangement order. Organize the image data matrix by channel dimension, arrange the single-channel grayscale data or multi-channel color data according to the input channel format preset by the image semantic segmentation model, perform data type conversion on the image data after channel dimension organization, convert the pixel values ​​from the original storage format to the numerical representation format, perform numerical normalization on the converted image data, map the pixel values ​​of different value ranges to a unified numerical interval, and arrange the image data after the above processing according to the dimensions of image height, image width and feature channel number to obtain the input feature map. The input feature map is input into the encoder, and the LDConv convolution is used to extract features from the input feature map to obtain the LDConv convolution feature map; The feature extraction process involves determining the corresponding convolutional sampling region at each output position of the current convolutional layer of the encoder. For each output position, based on the preset base offset position of each sampling point in the convolutional kernel and combined with the learnable offset corresponding to each sampling point, the actual sampling position of each sampling point in the input feature map is determined. When the actual sampling position falls at a non-integer coordinate, bilinear interpolation is performed on the feature values ​​at adjacent integer grid points to obtain the input feature value at the actual sampling position. The input feature value at each actual sampling position is weighted and calculated with the convolution weight of the corresponding sampling point, and the weighted results of each sampling point are accumulated to obtain the convolutional output feature value at the current output position. The above process is repeated for each output position in the input feature map to obtain the LDConv convolutional feature map. The bilinear interpolation method involves multiplying the input feature values ​​at four adjacent integer grid points of the value location by their corresponding interpolation weights and then summing the results to obtain the input feature value at the value location. The interpolation weights are determined by the relative distance between the value location and the corresponding integer grid point. The LDConv convolutional feature map is processed by multi-layer encoding and multi-scale downsampling to obtain encoded feature maps of different scales. The encoded feature maps of different scales are input into the decoder for layer-by-layer decoding. The DySample sampler is used for upsampling recovery to determine the dynamic sampling offset corresponding to each sampling point. The multi-layer coding and multi-scale downsampling processes are as follows: After the LDConv convolutional feature map is input into the encoder, multi-layer coding is performed sequentially according to the encoder's hierarchical structure. In each coding layer, convolution, feature mapping, and channel transformation are performed on the feature map output from the previous layer to extract the semantic features of the current layer, thus obtaining the current layer's coded feature map. Between layers, downsampling operations are used to progressively reduce the spatial resolution of the current layer's coded feature map, while increasing the number of feature channels to expand the receptive field of the feature map and enhance its semantic representation ability at different scales. The above processing is repeated for each coding layer, so that the LDConv convolutional feature map gradually forms multi-scale feature representations corresponding to different spatial resolutions and different semantic levels, resulting in coded feature maps of different scales. The determination of dynamic sampling offset is as follows: In the current decoding layer, the encoded feature map is selected as the input of the DySample sampler, and the local feature response at each position in the encoded feature map is used as the basis for adjusting the sampling position; For the current position in the encoded feature map, an initial sampling region corresponding to the current position is established, and the position adjustment information of each sampling point relative to the current position is generated by combining the feature distribution state, edge response and spatial position relationship in the neighborhood of the current position, and the dynamic sampling offset corresponding to each sampling point is determined; The dynamic sampling offset in the DySample sampler is initialized, the dynamic sampling offset is updated according to the position of each sampling point after initialization, and feature sampling is completed according to the updated position of each sampling point to generate the upsampled and restored feature map. The generation process is as follows: The dynamic sampling offset in the DySample sampler is initialized. A regular sampling structure is established at the current position. The initial offset position of each sampling point is determined according to the preset sampling template, resulting in the initialized sampling point position. Using the initialized sampling point position as the base position, the corresponding dynamic sampling offset is calculated according to the offset prediction branch in the DySample sampler. The dynamic sampling offset is then superimposed on the initialized sampling point position to obtain the updated sampling point position. For each updated sampling point position, the corresponding sampling position is determined in the encoded feature map. When the sampling position is a non-integer coordinate, bilinear interpolation is performed on the feature values ​​at the adjacent integer grid points to obtain the feature value at the sampling position. The feature values ​​obtained at each sampling point are weighted with the corresponding sampling weights, and the weighted results of each sampling point are accumulated to obtain the upsampled output feature value at the current position. The above sampling and weighted calculation process is repeated for each position in the feature map to generate the upsampled and restored feature map. The upsampled and restored feature map is input into the classification layer of the image semantic segmentation model to calculate the predicted probability of each pixel for each category. The calculation process is as follows: Pixel-by-pixel classification mapping is performed on the upsampled and restored feature map. Each pixel position in the feature map corresponds to the classification response value of each category in the vehicle area, lane line area, and road background area. Normalization processing is performed on the classification response value of each category at each pixel position. First, an exponential operation is performed on the classification response value of each category. The sum of the exponential operation results corresponding to the classification response value of each category is used as the normalization benchmark. The exponential operation results corresponding to the classification response value of each category are divided by the normalization benchmark to obtain the probability value corresponding to each category. The category label of each pixel is determined based on the predicted probability of each category, and the semantic segmentation result image is obtained. The specific process is as follows: read the predicted probabilities of each pixel corresponding to the vehicle region, lane line region, and road background region; compare the predicted probabilities of each pixel to each category; determine the category with the highest predicted probability as the category label of that pixel; perform pixel-level category labeling on all pixels according to the category labels of each pixel to form a category labeling matrix; perform region mapping on each pixel in the category labeling matrix according to the correspondence between category labels and region identifiers to generate a semantic segmentation result map corresponding to each pixel position of the road image.

[0025] In this embodiment, obtaining the vehicle region boundary pixel set and the lane line boundary pixel set specifically includes: Read the category labels of each pixel in the semantic segmentation result image, and construct the initial binary map of the vehicle region and the initial binary map of the lane line region based on the category labels of each pixel; The construction process is as follows: Read the category label information corresponding to each pixel in the semantic segmentation result image; traverse all pixel positions in the semantic segmentation result image; and match the category label of each pixel position with the preset vehicle region category label and lane line region category label respectively. When the category label of a pixel matches the vehicle region category label, assign the pixel value as the target pixel value in the initial binary image of the vehicle region; when the category label of a pixel does not match the vehicle region category label, assign the pixel value as the background pixel value in the initial binary image of the vehicle region. Similarly, when the category label of a pixel matches the lane line region category label, assign the pixel value as the target pixel value in the initial binary image of the lane line region; when the category label of a pixel does not match the lane line region category label, assign the pixel value as the background pixel value in the initial binary image of the lane line region. Morphological processing is performed on the initial binary images of the vehicle region and the lane line region. The morphological processing includes erosion, dilation, opening and closing operations. Erosion removes isolated noise pixels, dilation restores the shape of the target region, opening removes small interference areas, and closing fills the holes inside the target region to obtain the processed images of the vehicle region and the lane line region. Connectivity analysis is performed on the vehicle area processing map and the lane line area processing map. The connectivity of each target area in the image is marked, the pixel set corresponding to each connected region is determined, and each connected region is numbered and labeled to obtain the labeled connected regions. The area of ​​the identified connected regions is calculated. The area of ​​the connected region is the number of all target pixels in the connected region. Connected regions with an area smaller than the preset area threshold are removed according to the preset area threshold, and vehicle connected regions and lane line connected regions with an area greater than or equal to the preset area threshold are retained. The retained vehicle connectivity region is defined as the vehicle region, and the retained lane line connectivity region is defined as the lane line region. The boundary pixels of each vehicle region and each lane line region are extracted. When a pixel in the target region is not a pixel in the neighborhood of the current target region, the pixel is defined as a boundary pixel, thus obtaining the set of boundary pixels of the vehicle region and the set of boundary pixels of the lane line region.

[0026] In this embodiment, obtaining the spatial location data of the perspective-corrected lane lines specifically includes: Extract the position coordinates of each pixel in the set of pixels at the lane boundary to obtain the set of pixel coordinates of each pixel at the lane boundary in the image coordinate system; The extraction process is as follows: traverse all pixels in the lane boundary pixel set, read the position index information of each pixel in the road image, and determine the coordinate values ​​of each lane boundary pixel based on the row and column positions of each pixel in the image coordinate system; combine the horizontal and vertical coordinates of each lane boundary pixel in a preset order to form pixel coordinates that correspond one-to-one with each lane boundary pixel; repeat the above process for all pixels in the lane boundary pixel set to obtain the pixel coordinate set of each lane boundary pixel in the image coordinate system. Construct lane line spatial location data based on the set of pixel coordinates; The construction process is as follows: Based on the pixel coordinates of each lane boundary pixel in the pixel coordinate set in the image coordinate system, arrange the lane boundary pixels according to the spatial distribution order of the pixel coordinates in the image; based on the coordinate distribution of each lane boundary pixel, aggregate the pixel coordinates belonging to the same lane region, and extract the positional distribution features of each lane boundary pixel in the image coordinate system; then, based on the aggregated pixel coordinate distribution results, describe the lane boundary direction and connect the positions of each lane boundary pixel to form lane spatial position data that characterizes the extension direction, boundary position, and spatial distribution state of the lane in the image coordinate system. Based on the installation location, shooting angle, and perspective imaging relationship of the road monitoring equipment, establish the perspective transformation relationship between the image coordinate system and the road plane coordinate system; The establishment process is as follows: First, read the installation height, orientation, pitch angle, and field-of-view parameters of the road monitoring equipment to determine its spatial installation status relative to the road plane. Second, establish the correspondence between pixel positions in the image coordinate system and the imaging rays of the monitoring equipment based on the imaging parameters of the road monitoring equipment. Third, combine this with the projection distribution of the road plane in the field of view of the monitoring equipment to determine the mapping points between the reference positions in the road plane coordinate system and the corresponding pixel positions in the image coordinate system. Finally, based on the coordinate mapping relationship between the image coordinate system and the road plane coordinate system using these mapping points, form a perspective transformation relationship that maps the lane line positions in the image coordinate system to the road plane coordinate system. Perspective correction is performed on the spatial position data of lane lines using perspective transformation. The position coordinates of the lane line boundary pixels in the image coordinate system are mapped to the perspective-corrected road plane coordinate system to obtain the position coordinates of the lane lines after perspective transformation. The position coordinates of the lane lines after perspective transformation are normalized to obtain the spatial position coordinates of the lane lines after perspective correction, and the spatial position data of the lane lines after perspective correction are constructed. The construction process is as follows: Based on the coordinates of each lane line after scale normalization, they are arranged according to the spatial distribution order of the coordinates of each lane line in the road plane coordinate system; based on the positional correspondence between the coordinates of each lane line, the coordinate points belonging to the same lane line boundary are associated and combined to form a continuous set of position points corresponding to the lane line boundary; each continuous set of position points is stored according to the lane line boundary, and the position range, extension direction and spatial distribution result of the corresponding lane line boundary are formed according to the coordinate distribution of each continuous set of position points, thus forming perspective-corrected lane line spatial position data.

[0027] In this embodiment, the generation of the location region specifically includes: The position coordinates of each pixel in the pixel set of the vehicle region boundary are extracted to obtain the set of pixel coordinates of each vehicle region boundary pixel in the image coordinate system. The extraction process is as follows: traverse all pixels in the vehicle area boundary pixel set, read the position index information of each pixel in the road image, and determine the horizontal and vertical coordinates of each vehicle area boundary pixel based on the row and column positions of each pixel in the image coordinate system; combine the horizontal and vertical coordinates of each vehicle area boundary pixel in a preset order to form pixel coordinates that correspond one-to-one with each vehicle area boundary pixel; repeat the above process for all pixels in the vehicle area boundary pixel set to obtain the pixel coordinate set of each vehicle area boundary pixel in the image coordinate system. The boundary range of the vehicle region in the image coordinate system is determined based on the set of pixel coordinates; The specific process is as follows: extract the horizontal and vertical coordinates of the boundary pixels of each vehicle area in the image coordinate system, and determine the minimum and maximum horizontal coordinate values ​​among all horizontal coordinates, as well as the minimum and maximum vertical coordinate values ​​among all vertical coordinates. Use the minimum, maximum, minimum, and maximum horizontal coordinate values ​​to determine the boundary range of the vehicle area in the image coordinate system. The location region of the vehicle area in the image coordinate system is constructed based on the boundary range. The location region is jointly constructed by the minimum horizontal coordinate value, the minimum vertical coordinate value, the maximum horizontal coordinate value, and the maximum vertical coordinate value.

[0028] In this embodiment, the generation of vehicle motion trajectory data specifically includes: Calculate the center coordinates of each vehicle area based on the location area; The calculation process is as follows: take the midpoint between the minimum and maximum lateral coordinates of each vehicle region as the lateral coordinate of the center position of the vehicle region, and take the midpoint between the minimum and maximum longitudinal coordinates as the longitudinal coordinate of the center position of the vehicle region, so as to obtain the center position coordinates of each vehicle region in the corresponding image frame. For vehicle regions in two adjacent frames, the matching degree between vehicle regions is calculated based on the center position coordinates of the vehicle regions and the spatial relationship between the regions. The matching degree is determined by the degree of region overlap and the center position distance. The degree of region overlap is the ratio between the area of ​​the overlapping region and the area of ​​the merged region between the vehicle regions in the current frame and the vehicle regions in the next frame. The center position distance is the spatial distance between the center position of the vehicle regions in the current frame and the center position of the vehicle regions in the next frame. The target association between vehicle regions is determined based on the degree of matching. Under the condition that the degree of matching is greater than or equal to the preset matching threshold, the combination of vehicle regions with the highest degree of matching is selected as the association result, and the correspondence between vehicle regions in two adjacent frames of images is established. According to the temporal order of the image frames, based on the correspondence between vehicle regions in adjacent image frames, vehicle regions belonging to the same target in consecutive image frames are assigned the same target identifier, forming an associated sequence of vehicle regions in consecutive image frames. Based on the associated sequence, the center coordinates and location regions of the vehicle region corresponding to the same target identifier in each image frame are obtained, and arranged in the time order of the image frames to construct vehicle motion trajectory data.

[0029] In this embodiment, the establishment of spatial relationship data and the formation of relative change positions specifically include: The spatial boundary position of each lane line region in the corresponding image frame is obtained based on the perspective-corrected lane line spatial position data. The spatial boundary position is obtained as follows: the spatial position points corresponding to each lane line region in the perspective correction lane line spatial position data are read, and the spatial position points are connected according to the distribution order of each spatial position point in the image coordinate system to form the boundary contour of each lane line region in the corresponding image frame; the boundary range of each lane line region in the corresponding image frame is determined according to the boundary contour, and the spatial boundary position of each lane line region in the corresponding image frame is obtained. Based on vehicle motion trajectory data, target identifiers are read from vehicle regions in consecutive image frames to obtain vehicle regions belonging to the same target identifier in consecutive image frames; For each image frame, calculate the distance between each boundary pixel of the vehicle region and each boundary pixel of the lane line in the vehicle region boundary pixel set, and determine the minimum distance between the boundary pixels of the vehicle region boundary pixel and the lane line boundary pixel set, thus determining the minimum boundary distance between the vehicle region and the corresponding lane line region. The calculation process is as follows: Based on the horizontal and vertical coordinates of each pixel in the vehicle area boundary pixel set and the horizontal and vertical coordinates of each pixel in the lane line boundary pixel set, calculate the distance between the vehicle area boundary pixel set and each boundary pixel in the lane line boundary pixel set. The distance is obtained by calculating the sum of the square of the difference between the horizontal coordinates and the square of the difference between the vertical coordinates of the two pixels and taking the square root. Based on the center coordinates of the vehicle area, the location area, the spatial boundary positions of each lane line area in the corresponding image frame, and the minimum boundary distance between the vehicle area and the corresponding lane line area, spatial relationship data between the vehicle area and the lane line area is formed. For vehicle regions corresponding to the same target identifier in consecutive image frames, the difference between the minimum boundary distance in the current image frame and the minimum boundary distance in the previous image frame is used to obtain the change in the minimum boundary distance between adjacent image frames, forming the relative positional change between the set of vehicle region boundary pixels and the set of lane line boundary pixels.

[0030] In this embodiment, the output of the vehicle crossing behavior detection result specifically includes: Based on the spatial relationship data of the current image frame, the minimum boundary distance between the vehicle area and the corresponding lane line area in the current image frame is obtained in real time. Based on the comparison result between the minimum boundary distance in the current image frame and the preset contact distance threshold, the contact state between the vehicle area and the lane line area in the current image frame is determined. The real-time acquisition process of minimum boundary distance is as follows: based on the continuous acquisition of traffic video data in the road monitoring area, the road image frame corresponding to the current moment is extracted according to a preset time interval, and the spatial relationship data corresponding to the current image frame is read synchronously; based on the spatial relationship data, the minimum boundary distance between the vehicle area and the corresponding lane line area is extracted from the current image frame. The contact state is determined as follows: when the minimum boundary distance in the current image frame is less than or equal to a preset contact distance threshold, the vehicle area and the lane line area are determined to be in contact; when the minimum boundary distance in the current image frame is greater than the preset contact distance threshold, the vehicle area and the lane line area are determined to be in non-contact. Based on the relative position changes in the current image frame, obtain the trend of the relative position change of the vehicle area relative to the lane line area; The acquisition of the relative position change trend is as follows: when the relative position change is less than zero, it is determined that the vehicle area is moving towards the lane line area; when the relative position change is equal to zero, it is determined that the relative position between the vehicle area and the lane line area remains unchanged; when the relative position change is greater than zero, it is determined that the vehicle area is moving away from the lane line area. According to the time sequence of image frames, the contact state and relative position change trend in continuous image frames are updated in real time based on the input of the current image frame. Within a preset time window, the time consistency analysis of the updated contact state and relative position change trend is performed to obtain the continuous contact state and crossing state between the vehicle area and the lane line area. The update process is as follows: Following the temporal order of image frames, the current image frame is input into the image semantic segmentation model to obtain the semantic segmentation result map corresponding to the current image frame. The vehicle region and lane line region in the current image frame are extracted, and the minimum boundary distance between the vehicle region and the corresponding lane line region in the current image frame is calculated. Based on the comparison between the minimum boundary distance in the current image frame and a preset contact distance threshold, the contact state corresponding to the current image frame is determined. Based on the change between the minimum boundary distance in the current image frame and the minimum boundary distance in the previous image frame, the relative position change trend corresponding to the current image frame is determined. The contact state and relative position change trend corresponding to the current image frame are appended to the contact state sequence and relative position change trend sequence corresponding to consecutive image frames, and the contact state sequence and relative position change trend sequence are synchronously updated according to a preset time window. When the number of image frames in the sequence exceeds the number of frames corresponding to the preset time window, the contact state and relative position change trend corresponding to the earliest input image frame are deleted, while the contact state and relative position change trend corresponding to each image frame within the preset time window are retained. The temporal consistency analysis is as follows: within a preset time window, the contact states corresponding to each image frame are continuously counted. When the number of image frames in the contact state is greater than or equal to a preset contact frame number threshold, it is determined that a continuous contact state has been formed between the vehicle area and the lane line area. When, within the preset time window, the relative position change trend shows that the vehicle area first moves towards the lane line area and then moves away from the lane line area, and there is an image frame in the corresponding contact state in the contact state, it is determined that a crossing state has been formed between the vehicle area and the lane line area. Based on the continuous contact state and the crossing state, the vehicle region in the current image frame is judged in real time for crossing the line, and the corresponding vehicle crossing the line detection result is output. The real-time lane crossing behavior determination process is as follows: when the vehicle area and the corresponding lane line area are in continuous contact, it is determined that the vehicle area has crossed the lane line area; when the vehicle area and the corresponding lane line area are in a crossing state, it is determined that the vehicle area has crossed the corresponding lane line area and crossed the lane line; when there is neither continuous contact nor crossing between the vehicle area and the corresponding lane line area, it is determined that the vehicle area has not crossed the lane line.

[0031] Example 1: Deploying the real-time vehicle lane-crossing behavior monitoring method based on image semantic segmentation described in this invention at a section where a main road and an auxiliary road merge in a city. This section includes three lanes for motor vehicles and one dedicated right-turn lane. Traffic volume is high during the day, with frequent lane changes, lane-changing, and short-term lane crossings occurring during morning and evening rush hours. The road monitoring equipment is installed on a pole approximately eight meters high in front of the intersection, with the camera providing overhead coverage of the road area within approximately sixty meters before entering the intersection. Due to the obvious perspective relationship in the monitoring viewpoint, distant lane lines tend to converge in the image. Furthermore, in the actual scenario, there are issues such as shadow occlusion, vehicle occlusion, localized lane line wear, and significant size differences between different vehicle models. While traditional methods based on target detection boxes can roughly identify vehicle targets, they cannot accurately describe the true outline boundary of a vehicle because vehicles are usually represented by bounded rectangles. This is especially true in situations where a vehicle is close to the lane line but has not yet crossed it, when a vehicle's tires partially cross the line, or when a vehicle is briefly close to the lane line. In these cases, normal approach is easily misjudged as lane crossing, or actual lane crossings are missed. At the same time, when relying solely on a single frame image for judgment, the detection results are easily affected by instantaneous noise, local occlusion, and changes in illumination, resulting in poor stability of the output results and failing to meet the requirements of continuity and accuracy for actual traffic monitoring systems.

[0032] When applying this invention in this scenario, road monitoring equipment first continuously collects traffic video data and extracts road image sequences at fixed time intervals. The extracted images undergo resolution unification, noise suppression, and brightness equalization to ensure consistent input quality across different time periods. The processed road images are then fed into an image semantic segmentation model. This model employs an encoder-decoder structure. During the encoding stage, LDConv convolution enhances the feature extraction capabilities for vehicle edges, lane line elongations, and complex road backgrounds. During the decoding stage, a DySample sampler performs fine-grained recovery of high-level semantic features, resulting in a semantic segmentation image for each frame. This segmentation image can distinguish vehicle regions, lane line regions, and road background regions at the pixel level. Therefore, compared to methods that only output rectangular bounding boxes for position detection, it is more suitable for subsequent boundary distance analysis and lane-crossing behavior determination.

[0033] After obtaining the semantic segmentation result image, morphological processing and connectivity analysis are performed on the vehicle region and lane line region, respectively. Scattered noise points and pseudo-target regions with too small an area are filtered out, while the real vehicle connectivity regions and lane line connectivity regions are retained. Furthermore, the boundary pixel sets of the vehicle region and lane line boundary pixel sets are extracted. For the lane line region, this invention further constructs lane line spatial location data and establishes perspective transformation relationships based on the installation height of the monitoring equipment, shooting angle, and imaging relationship, mapping the lane line boundaries in the original image to the perspective-corrected road plane coordinate system. The purpose of this processing is to reduce the interference of perspective effects on spatial location description, so that lane lines at different distances and near distances have better geometric consistency under a unified coordinate expression. For the vehicle region, the position region of the vehicle region in the image coordinate system is extracted based on the boundary pixel set, and the center position coordinates are calculated. Subsequently, target association matching is performed through the spatial relationship between the center position coordinates and position regions between adjacent frames, assigning the same target identifier to the same vehicle in consecutive image frames, forming the vehicle's motion trajectory data in the time dimension.

[0034] Building upon this, the present invention does not simply compare the relative positions of the vehicle's center point and the lane line. Instead, it calculates the minimum boundary distance by combining the set of pixels representing the vehicle's boundary region with the set of pixels representing the lane line boundary. For each vehicle in each frame of the image, the system reads the vehicle's boundary pixels and the lane line boundary pixels, calculates the spatial distance for each, and selects the minimum value as the minimum boundary distance between the current vehicle and the corresponding lane line. This minimum boundary distance more accurately characterizes the degree of proximity of the vehicle's outline to the lane line. Subsequently, by comparing the changes in the minimum boundary distance between adjacent frames in consecutive image frames, the system can determine whether the vehicle is gradually approaching, maintaining contact, or gradually moving away from the lane line. By combining the contact state and the change trend to perform temporal consistency analysis, it can distinguish between false contact caused by short-term noise and genuine continuous contact, and can also identify the crossing state of the vehicle moving from one side of the lane line to the other. Finally, the system outputs the vehicle's lane-crossing behavior detection results based on the continuous contact state and the crossing state.

[0035] To verify the beneficial effects of this invention, videos were continuously collected from peak and off-peak hours on the same road segment for three consecutive workdays, totaling six hours in length, with a frame rate of 25 frames per second. One thousand two hundred valid vehicle segments were selected from the videos, encompassing four scenarios: normal driving, driving close to the lane lines, brief contact with lane lines, and actual lane crossings. Of these, 328 were manually verified as actual lane crossing events. The method of this invention was compared with traditional lane crossing determination methods based on rectangular detection boxes and methods based solely on single-frame segmentation results. The same video source, monitoring perspective, and manual annotation standards were used for evaluation during the tests.

[0036] This invention can more stably identify vehicle lane-crossing behavior in complex traffic scenarios. For situations with localized lane line wear and vehicle partial occlusion, traditional detection box methods easily misjudge situations where the vehicle's frame edge overlaps with the lane line as lane-crossing, resulting in a high false alarm rate. While single-frame segmentation methods offer finer boundaries than detection box methods, the lack of continuous frame temporal consistency analysis still leads to numerous instantaneous false alarms due to shadow changes and segmentation edge jitter. This invention, by introducing minimum boundary distance and adjacent frame variation analysis on top of pixel-level boundaries, and combining contact and crossing states in continuous image frames for comprehensive judgment, significantly reduces false alarms while maintaining high recall. Especially in scenarios involving slow lane changes, partial tire lane-crossing, and complex single-frame boundary morphology changes due to the long bodies of large vehicles, this invention can still accurately distinguish between "approaching the lane line" and "actual lane-crossing" states. The test results are shown in Table 1 below.

[0037] Table 1. Performance Comparison Results of Different Vehicle Crossing Detection Methods in Complex Road Monitoring Scenarios

[0038] As shown in Table 1, under the same test data and the same manual annotation standards, the method of this invention significantly outperforms the comparative methods in all three key indicators: accuracy, recall, and precision. Compared with the method based on rectangular detection boxes, the accuracy of this invention is improved by 11.3 percentage points, the recall by 11.0 percentage points, and the precision by 14.2 percentage points. This indicates that the invention can more effectively reduce the misjudgment problem caused by coarse bounding box representation, especially when the vehicle partially contacts the lane line or the boundary changes are complex. Compared with the method based on single-frame segmentation, the accuracy of this invention is improved by 5.2 percentage points, the recall by 5.8 percentage points, and the precision by 6.3 percentage points. This shows that simply obtaining a finer segmentation boundary is not enough to stably complete the lane line determination. Combining the minimum boundary distance, relative position change, and time consistency analysis can further improve the reliability of the detection results.

[0039] The beneficial effects of this invention can also be directly reflected in the changes in the number of false positives and false negatives. The false negatives of this invention are only 16, significantly lower than the 52 of the rectangular detection box method and the 35 of the single-frame segmentation method, indicating that this invention has a stronger ability to capture real lane-crossing events. The false positives of this invention are 27, also significantly lower than the other two methods, indicating that this invention is more accurate in distinguishing between "close to the lane line" and "actual lane-crossing." In traffic management scenarios, excessively high false positives lead to frequent and ineffective alarms, while excessively high false negatives lead to missed violations. This invention achieves a better balance between the two.

[0040] From a real-time performance perspective, the average processing speed of this invention reaches 24.6 frames per second. Although slightly lower than methods based solely on rectangular detection boxes, it still meets the real-time monitoring requirements of single-channel traffic videos and significantly exceeds the processing speed of many offline analysis methods. This indicates that even after introducing pixel-level segmentation, perspective correction, target association, and temporal consistency analysis, this invention maintains good feasibility for engineering deployment. Therefore, based on the data in the table, it can be seen that this invention not only effectively solves the problems of insufficient accuracy, poor continuity, and weak robustness in complex scenarios in existing technologies for recognizing line-crossing behavior, but also achieves high detection performance and good real-time application results under real road monitoring conditions.

[0041] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation, characterized in that, Includes the following steps: Traffic video data is collected to obtain road image sequences, which are then preprocessed. The preprocessed road image sequence is input into the image semantic segmentation model for semantic segmentation. The image semantic segmentation model adopts an encoder-decoder architecture. The encoder uses LDConv convolution for feature extraction, and the decoder uses DySample sampler for feature recovery to obtain the semantic segmentation result image. Morphological processing and connected component analysis are performed on the semantic segmentation result image to extract vehicle region and lane line region, and obtain the set of boundary pixels of vehicle region and lane line region. The spatial location data of the lane line is calculated based on the set of pixels at the lane line boundary, and perspective correction is performed to obtain perspective-corrected lane line spatial location data. Based on the set of pixels at the vehicle region boundary, the location region of the vehicle region in the image coordinate system is generated. Target association matching is performed on the location region to establish the correspondence between vehicle regions in consecutive image frames and generate vehicle motion trajectory data. Based on perspective-corrected lane line spatial location data and vehicle motion trajectory data, spatial relationship data between vehicle area and lane line area is established, and the relative positional changes between vehicle area boundary pixel set and lane line boundary pixel set are formed. Based on the spatial relationship data of the current frame and the relative position changes in consecutive image frames, a temporal consistency analysis is performed to determine the current vehicle's line-crossing behavior in real time and output the vehicle line-crossing behavior detection result.

2. The real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation according to claim 1, characterized in that, Traffic video data is continuously collected by road monitoring equipment installed in the road monitoring area. The traffic video data is then extracted frame by frame at preset time intervals to generate a road image sequence. The road image sequence is then preprocessed, including resolution unification, noise suppression, and brightness equalization.

3. The real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation according to claim 1, characterized in that, The semantic segmentation result map is obtained specifically through: The preprocessed road image sequence is input into the image semantic segmentation model. Tensor quantization is performed on each frame of the preprocessed road image sequence to obtain the input feature map. The input feature map is input into the encoder, and the LDConv convolution is used to extract features from the input feature map to obtain the LDConv convolution feature map; The LDConv convolutional feature map is subjected to multi-layer encoding and multi-scale downsampling to obtain encoded feature maps of different scales. The encoded feature maps of different scales are input into the decoder for layer-by-layer decoding. The DySample sampler is used for upsampling recovery to determine the dynamic sampling offset corresponding to each sampling point. The dynamic sampling offset in the DySample sampler is initialized, the dynamic sampling offset is updated according to the position of each sampling point after initialization, and feature sampling is completed according to the updated position of each sampling point to generate the upsampled and restored feature map. The upsampled and restored feature map is input into the classification layer of the image semantic segmentation model to calculate the predicted probability of each pixel for each category. The category label of each pixel is determined based on the predicted probability of each category, and the semantic segmentation result image is obtained.

4. The real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation according to claim 1, characterized in that, The acquisition of the vehicle region boundary pixel set and the lane line boundary pixel set specifically includes: Read the category label of each pixel in the semantic segmentation result image, and construct the initial binary map of the vehicle region and the initial binary map of the lane line region based on the category label of each pixel; Morphological processing is performed on the initial binary map of the vehicle region and the initial binary map of the lane line region to obtain the processed vehicle region map and the processed lane line region map. Connectivity analysis is performed on the vehicle area processing map and the lane line area processing map. The connectivity of each target area in the image is marked, the pixel set corresponding to each connected region is determined, and each connected region is numbered and labeled to obtain the labeled connected regions. The area of ​​the marked connected regions is calculated, and connected regions with an area smaller than the preset area threshold are removed according to the preset area threshold, while vehicle connected regions and lane line connected regions with an area greater than or equal to the preset area threshold are retained. The retained vehicle connectivity region is defined as the vehicle region, and the retained lane line connectivity region is defined as the lane line region. The boundary pixels of each vehicle region and each lane line region are extracted. When a pixel in the target region is not a pixel in the neighborhood of the current target region, the pixel is defined as a boundary pixel, thus obtaining the set of boundary pixels of the vehicle region and the set of boundary pixels of the lane line region.

5. The real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation according to claim 1, characterized in that, The specific steps involved in obtaining the perspective-corrected lane line spatial position data are as follows: Extract the position coordinates of each pixel in the set of pixels at the lane boundary to obtain the set of pixel coordinates of each lane boundary pixel in the image coordinate system; Construct lane line spatial location data based on the set of pixel coordinates; Based on the installation location, shooting angle, and perspective imaging relationship of the road monitoring equipment, establish the perspective transformation relationship between the image coordinate system and the road plane coordinate system; Perspective correction is performed on the spatial position data of lane lines using perspective transformation. The position coordinates of the lane line boundary pixels in the image coordinate system are mapped to the perspective-corrected road plane coordinate system to obtain the position coordinates of the lane lines after perspective transformation. The position coordinates of the lane lines after perspective transformation are normalized to obtain the spatial position coordinates of the lane lines after perspective correction, and the spatial position data of the perspective-corrected lane lines are constructed.

6. The real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation according to claim 1, characterized in that, The generation of the location region specifically includes: The position coordinates of each pixel in the pixel set of the vehicle region boundary are extracted to obtain the set of pixel coordinates of each vehicle region boundary pixel in the image coordinate system. The boundary range of the vehicle region in the image coordinate system is determined based on the set of pixel coordinates; Construct the location region of the vehicle area in the image coordinate system based on the boundary range.

7. The real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation according to claim 1, characterized in that, The generation of the vehicle motion trajectory data specifically includes: Calculate the center coordinates of each vehicle area based on the location area; For vehicle regions in two adjacent frames, the degree of matching between vehicle regions is calculated based on the center coordinates of the vehicle regions and the spatial relationship between the regions. The target association between vehicle regions is determined based on the degree of matching. Under the condition that the degree of matching is greater than or equal to the preset matching threshold, the combination of vehicle regions with the highest degree of matching is selected as the association result, and the correspondence between vehicle regions in two adjacent frames of images is established. According to the temporal order of the image frames, based on the correspondence between vehicle regions in adjacent image frames, vehicle regions belonging to the same target in consecutive image frames are assigned the same target identifier, forming an associated sequence of vehicle regions in consecutive image frames. Based on the associated sequence, the center coordinates and location regions of the vehicle region corresponding to the same target identifier in each image frame are obtained, and arranged in the time order of the image frames to construct vehicle motion trajectory data.

8. The real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation according to claim 1, characterized in that, The establishment of spatial relationship data and the formation of relative change positions specifically include: The spatial boundary position of each lane line region in the corresponding image frame is obtained based on the perspective-corrected lane line spatial position data. Based on vehicle motion trajectory data, target identifiers are read from vehicle regions in consecutive image frames to obtain vehicle regions belonging to the same target identifier in consecutive image frames; For each image frame, calculate the distance between each boundary pixel of the vehicle region and each boundary pixel of the lane line in the vehicle region boundary pixel set, and determine the minimum distance between the boundary pixels of the vehicle region boundary pixel and the lane line boundary pixel set, thus determining the minimum boundary distance between the vehicle region and the corresponding lane line region. Based on the center coordinates of the vehicle area, the location area, the spatial boundary positions of each lane line area in the corresponding image frame, and the minimum boundary distance between the vehicle area and the corresponding lane line area, spatial relationship data between the vehicle area and the lane line area is formed. For vehicle regions corresponding to the same target identifier in consecutive image frames, the difference between the minimum boundary distance in the current image frame and the minimum boundary distance in the previous image frame is used to obtain the change in the minimum boundary distance between adjacent image frames, forming the relative positional change between the set of vehicle region boundary pixels and the set of lane line boundary pixels.

9. The real-time monitoring method for vehicle lane-crossing behavior based on image semantic segmentation according to claim 1, characterized in that, The output of the vehicle lane crossing behavior detection result specifically includes: Based on the spatial relationship data of the current image frame, the minimum boundary distance between the vehicle area and the corresponding lane line area in the current image frame is obtained in real time. Based on the comparison result between the minimum boundary distance in the current image frame and the preset contact distance threshold, the contact state between the vehicle area and the lane line area in the current image frame is determined. Based on the relative position changes in the current image frame, obtain the trend of the relative position change of the vehicle area relative to the lane line area; According to the time sequence of image frames, the contact state and relative position change trend in continuous image frames are updated in real time based on the input of the current image frame. Within a preset time window, the time consistency analysis of the updated contact state and relative position change trend is performed to obtain the continuous contact state and crossing state between the vehicle area and the lane line area. Based on the continuous contact state and the crossing state, the vehicle region in the current image frame is judged in real time for crossing the line, and the corresponding vehicle crossing behavior detection result is output.

Images