Partly-shielded vehicle detection method based on convolutional neural network

Abstract

The invention provides a partially-shielded vehicle detection method based on a convolutional neural network, and the method comprises the steps: extracting a lane center line and an intersection structure under a unified coordinate system through employing a road scene image and road function region reference data, constructing a lane, steering confluence and intersection conflict region partition map, and carrying out the recognition of the lane, steering confluence and intersection conflict region partition map. Generating a shielding sensitive belt along the edge of the lane and the boundary of the fixed shielding object; according to the partition map, cutting an image of each functional region, adaptively generating an anchor frame in combination with the type of the functional region, and obtaining an initial vehicle candidate frame through a convolutional neural network; meanwhile, the shielding sensitive sub-image and the shielding mask serve as input, and a shielding sensitive vehicle candidate frame is predicted only in the shielding sensitive band. And then under the constraint of a shielding sensitive band, matching and weighted fusion are carried out according to the area and position relation of the two types of candidate boxes, functional region vehicle target information is obtained, vehicle conflict attributes are judged according to an intersection conflict region and a steering convergence region, and a vehicle detection result is output.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation perception technology, and in particular to a method for detecting partially occluded vehicles based on convolutional neural networks. Background Technology

[0002] With the widespread deployment of intelligent transportation and urban road monitoring, video-based vehicle detection has become a crucial foundation for intersection operational status perception, traffic control, and safety early warning. In urban road scenarios, cameras are typically installed above or on the side of intersections to continuously capture complex traffic behaviors such as multi-lane straight-ahead, turning, and merging. During the process of entering an intersection, changing lanes, or merging, vehicles are often partially obscured by vehicles in front, adjacent lanes, roadside guardrails, traffic signal poles, and other fixed obstructions, resulting in incomplete vehicle outlines and shape features in the images. This poses a challenge to visual detection algorithms based on convolutional neural networks.

[0003] In existing technologies, vehicle detection in road scenes generally employs object detection models based on convolutional neural networks. These models take the entire road scene image or a simply cropped region as input, generate anchor boxes on multi-scale feature maps, and output vehicle candidate boxes for applications such as traffic volume statistics, violation detection, or trajectory tracking. Some solutions combine lane line detection results to perform coarse-grained lane region segmentation of the image to reduce background interference; others utilize road structure information or map data, applying different detection parameters to different lanes or regions to adapt to the size variations of vehicles at different distances. To address the occlusion problem, existing methods often rely on multi-scale feature fusion, data augmentation, or object tracking prediction to mitigate the risk of missed detections caused by partial vehicle occlusion. Based on the detection results, they also perform region judgment to identify potentially conflicting vehicle targets within intersections.

[0004] However, the aforementioned existing technologies generally lack fine integration with road functional area benchmark data in a unified coordinate system. They are unable to promptly utilize information such as lane centerlines and intersection topology to construct lane, turning and merging, and intersection conflict area partitions that closely correspond to vehicle driving behavior. They also fail to systematically model occlusion sensitive zones near lane edges and fixed obstructions, making it impossible to implement targeted enhanced detection and position constraints on partially obstructed vehicles during the candidate box generation and screening stages. Consequently, it is difficult to output vehicle detection results that combine spatial location and conflict attributes in a timely manner to meet the perception needs of complex intersections and merging scenarios. Summary of the Invention

[0005] One objective of this invention is to propose a partially occluded vehicle detection method based on convolutional neural networks. The core technical problem to be solved is: given road scene images and road functional area reference data, how to construct partitions and occlusion sensitive zones corresponding to lanes, turning merging and intersection conflict areas, and constrain and fuse the vehicle detection results of the convolutional neural network accordingly, so as to obtain vehicle target detection results with conflict attributes in partially occluded and complex intersection traffic scenarios.

[0006] A partially occluded vehicle detection method based on a convolutional neural network according to an embodiment of the present invention includes:

[0007] S1. Acquire road scene images and corresponding road functional area reference data, parse the road functional area reference data, and obtain lane centerline information and intersection structure information.

[0008] S2. Based on the road scene image, lane centerline information, and intersection structure information, determine the image boundaries of lanes, turning merging areas, and intersection conflict areas, generate a road functional zoning map, and offset the lane edge and fixed obstruction boundary along the lane edge and fixed obstruction boundary into the lane by a preset distance in the road functional zoning map to generate obstruction sensitive zone information that may partially obstruct the position of vehicles.

[0009] S3. Based on the road scene image and the road functional area zoning map, determine the image range of each functional area and crop the image of each functional area. Input the image of each functional area into the convolutional neural network vehicle detection model, perform target detection on the image of each functional area, and generate initial vehicle candidate box information. The initial vehicle candidate box information includes position data and size data.

[0010] S4. Based on the road scene image, occlusion sensitive zone information and road functional zoning map, determine the image range covered by the occlusion sensitive zone and crop the occlusion sensitive sub-image. Input the occlusion sensitive sub-image into the convolutional neural network vehicle detection model to generate occlusion sensitive vehicle candidate box information.

[0011] S5. Based on the initial vehicle candidate box information and the occlusion-sensitive vehicle candidate box information, determine the area and positional relationship of the candidate boxes under the constraint of the occlusion-sensitive zone information, merge the candidate boxes that meet the area threshold and position threshold, and generate the functional area vehicle target information.

[0012] S6. Based on the vehicle target information of the functional area and the road functional zoning map, determine the conflict attributes of vehicle targets located in the intersection conflict area and the turning merging area, and output the vehicle detection result data.

[0013] Optionally, S1 is as follows:

[0014] S11. Input the road scene image into the acquisition interface, take the road functional area reference data as input, establish a unified road reference coordinate system that is aligned with the input image coordinates of the convolutional neural network vehicle detection model based on the imaging parameters of the road scene image and the coordinate system of the road functional area reference data, and obtain the correspondence between pixels in the road scene image and the road functional area reference data.

[0015] S12. In the unified road reference coordinate system, the lane boundary vector segments in the road functional area reference data are paired according to the left-right correspondence. The candidate lane centerline segments are generated by connecting the midpoints of each pair of lane boundary vector segments. The candidate lane centerline segments that do not meet the direct connection condition are eliminated by length weight and included angle threshold to obtain the initial lane centerline.

[0016] S13. Based on the spatial orientation of the initial lane centerline in the unified road reference coordinate system, the initial lane centerline is sampled at fixed intervals to obtain lane centerline information arranged in the sampling order. At the same time, based on the guide arrows, stop lines and intersection boundaries in the road functional area reference data, the sampling points located in the intersection area are clustered into entrance nodes, exit nodes and conflict area nodes, and the connection relationship between the nodes constitutes the intersection structure information.

[0017] Optionally, S2 is as follows:

[0018] S21. Align the road scene image with the road functional area reference data in a unified road reference coordinate system, and project the lane centerline information onto the road scene image according to the pixel coordinates to obtain a sequence of lane centerline pixels arranged continuously along the vehicle driving direction.

[0019] S22. Using the lane centerline pixel sequence as input, based on the lane width value given for each lane in the road functional area reference data, generate lane edge polylines by offsetting along the normal direction on both sides of the lane centerline at the same interval, smooth the lane edge polylines and close them at the beginning and end to obtain the closed lane area boundary of the lane in the road scene image.

[0020] S23. Based on the connection relationship between the entrance node and the exit node in the intersection structure information, the boundary of the closed lane area is divided into a straight lane area and a turning lane area according to the corresponding relationship. In the unified road reference coordinate system, the turning lane area is extended according to the vehicle driving direction to obtain the image boundary of the turning lane area covering the turning trajectory.

[0021] S24. Taking the boundary of the turning lane area image as input, calculate the overlapping area between different turning lane areas in the unified road reference coordinate system based on the geometric topological relationship of the entrance node, exit node and conflict area node in the intersection structure information, and map the overlapping area back to the road scene image to obtain the boundary of the intersection conflict area image.

[0022] S25. The boundaries of the closed lane area, the turning lane area, and the intersection conflict area are uniformly encoded in the unified road reference coordinate system. The turning lane area image boundary is marked as the turning merging area image boundary. Different functional areas are distinguished by functional area type identifiers. The encoding results are mapped to the road scene image to generate a road functional area map containing the lane, turning merging, and intersection conflict area image boundaries.

[0023] S26. Based on the lane edge in the road functional zoning map, the lane edge is offset by a first preset distance in the normal direction away from the outer edge of the road and pointing towards the inside of the lane, forming a first occlusion sensitive zone extending along the lane in the road scene image. At the same time, based on the fixed occlusion boundary in the road functional area reference data, the fixed occlusion boundary is offset by a second preset distance in the normal direction away from the inside of the fixed occlusion and pointing towards the inside of the lane, forming a second occlusion sensitive zone around the fixed occlusion. The first occlusion sensitive zone and the second occlusion sensitive zone are merged in the unified road reference coordinate system to obtain continuous occlusion sensitive zone information.

[0024] S27. Map the occlusion-sensitive zone information onto the feature map coordinate system of the convolutional neural network vehicle detection model according to the downsampling step size of the model. Mark the grid cells falling within the occlusion-sensitive zone information using the feature map grid as the unit, and generate a road functional zone. Figure 1 A corresponding occlusion sensitive zone information is used to constrain the position selection and merging of vehicle candidate boxes output by the subsequent convolutional neural network vehicle detection model.

[0025] Optionally, S3 specifically refers to:

[0026] S31. Input the road functional zoning map into the functional area parsing module. Based on the image boundaries of lanes, turning merging areas and intersection conflict areas in the road functional zoning map, determine the minimum bounding rectangle of each functional area in the pixel coordinate system of the road scene image to obtain the functional area image range corresponding to each functional area.

[0027] S32. The road scene image and the functional area image range are cropped by coordinate. The functional area image is cropped with the pixels within the functional area image range as input. The functional area image is scaled to the input size range of the convolutional neural network vehicle detection model by adopting a proportional scaling strategy according to the aspect ratio of the functional area image. At the same time, the coordinate mapping relationship between the functional area image and the road scene image is recorded.

[0028] S33. The functional area image and the functional area type identifier of the corresponding functional area in the road functional area zoning map are jointly input into the feature extraction sub-network of the convolutional neural network vehicle detection model. In the shared convolutional layer, the functional area image is subjected to multi-layer convolution and downsampling to generate a multi-scale functional area feature map. In the multi-scale functional area feature map, the channel weight adjustment module with the functional area type identifier as input is applied to the channel weight of different scales.

[0029] S34. Using multi-scale functional region feature maps as input, based on the vertical position of the functional region image in the road scene image and the functional region type identifier, an anchor frame size and aspect ratio adaptive generation module is used to generate anchor frame sets for different distances and different functional regions on feature maps at each scale. The functional region image is divided into multiple height bands in the vertical direction, where the anchor frame size in the upper height band is smaller than the anchor frame size in the lower height band, and anchor frames that cross the boundary to the outside of the functional region are suppressed within the functional region.

[0030] S35. Input the multi-scale functional region feature map and its corresponding set of anchor boxes into the vehicle candidate box prediction subnetwork of the convolutional neural network vehicle detection model. Output the prediction result containing the target score, center point coordinate offset and width and height offset for each anchor box. Filter out anchor boxes with target scores lower than the target score threshold according to the preset target score threshold to obtain the initial set of candidate anchor boxes within the functional region.

[0031] S36. Based on the coordinate mapping relationship between the functional area image and the road scene image, the center point coordinates and width and height dimensions of the initial candidate anchor box set in the functional area image are converted into position data and size data in the road scene image. The initial candidate anchor boxes that fall in the same functional area and whose overlap exceeds the preset overlap threshold are retained by the non-maximum suppression module, and the initial candidate anchor boxes with the highest target score are retained.

[0032] S37. The initial candidate anchor frames processed by the non-maximum suppression module are classified into lane vehicle initial candidate frames, turning merging vehicle initial candidate frames, and intersection conflict area vehicle initial candidate frames according to the functional area type identifier in the road functional zoning map. The lane vehicle initial candidate frames, turning merging vehicle initial candidate frames, and intersection conflict area vehicle initial candidate frames are collectively recorded as initial vehicle candidate frame information. Each candidate frame in the initial vehicle candidate frame information carries position data and size data.

[0033] Optionally, S4 specifically refers to:

[0034] S41. Overlay the occlusion sensitive zone information and the road function zoning map on the road scene image. Calculate the minimum bounding rectangle of the connected regions in the occlusion sensitive zone information in the road scene image, and extend it along the lane direction on the side closest to the inside of the lane according to the expansion coefficient to obtain the range of the occlusion sensitive sub-image.

[0035] S42. Crop the road scene image according to the range of the occlusion sensitive sub-image to obtain the occlusion sensitive sub-image, and generate an occlusion mask image of the same size as the occlusion sensitive sub-image according to the coverage relationship of the occlusion sensitive sub-image in the occlusion sensitive band information. Align the occlusion sensitive sub-image and the occlusion mask image with the short side and scale them uniformly to the input size range of the convolutional neural network vehicle detection model.

[0036] S43. After concatenating the occlusion-sensitive sub-image and the occlusion mask image along the channel dimension, input them into the occlusion detection sub-network of the convolutional neural network vehicle detection model. Extract the occlusion feature map in the shared convolutional layer, and generate anchor boxes only at the occlusion-sensitive zone grid positions marked in the occlusion mask image. Use the vehicle candidate box prediction sub-network to output the center point coordinate offset and width and height offset. Based on the coordinate mapping relationship between the occlusion-sensitive sub-image and the road scene image, convert the prediction results into position and size data in the road scene image to obtain the occlusion-sensitive vehicle candidate box information.

[0037] Optional, S5 specifically includes:

[0038] S51. Match the initial vehicle candidate box information with the occlusion-sensitive vehicle candidate box information one by one on the road scene image. Calculate the overlap area with the occlusion-sensitive vehicle candidate box information based on the position data and size data in the initial vehicle candidate box information. Compare the ratio of the overlap area to the area of ​​each of the initial vehicle candidate box information and the occlusion-sensitive vehicle candidate box information with the area threshold. Filter out candidate box pairs whose area ratio is lower than the area threshold.

[0039] S52. Using the occlusion sensitive zone information as input, determine the candidate box center point position of each candidate box pair based on the pixel range of the occlusion sensitive zone information in the road scene image. Eliminate candidate box pairs that are located outside the coverage area of ​​the occlusion sensitive zone information or whose center point deviates from the boundary of the occlusion sensitive zone information by more than the position threshold, and retain only candidate box pairs that meet the position threshold.

[0040] S53. The center point position data and size data of the retained candidate boxes are weighted and fused according to the preset area weight and preset position weight to obtain fused candidate boxes. Based on the road functional zoning map, the fused candidate boxes are assigned to functional areas according to lane, turning merging and intersection conflict areas, and summarized into functional area vehicle target information.

[0041] Optional, S6 specifically includes:

[0042] S61. Spatially align the vehicle target information of the functional area with the road functional zoning map on the road scene image. Based on the pixel coordinates of the center point of each vehicle target in the functional area vehicle target information, determine whether the center point falls into the intersection conflict area or turning merging area marked by the road functional zoning map. Mark the vehicle targets that meet the conditions as conflict candidate vehicle targets.

[0043] S62. Combine the candidate conflicting vehicle targets in pairs, calculate the overlap and center point distance between the vehicle targets in the road scene image based on the position data and size data in the vehicle target information of the functional area, and mark the vehicle target combinations with an overlap greater than a preset overlap threshold or a center point distance less than a preset distance threshold as vehicle target combinations with conflicting relationships.

[0044] S63. Taking the combination of vehicle targets with conflicting relationships as input, assign conflict attributes to each vehicle target according to the type of intersection conflict area or turning merging area where the vehicle target is located, and summarize the vehicle targets with conflict attributes and the vehicle targets without conflict attributes into vehicle detection result data.

[0045] The beneficial effects of this invention are:

[0046] 1. This proposal presents an improved vehicle detection method for road scenes based on convolutional neural networks. By aligning the road scene image with road functional area reference data under a unified road reference coordinate system, lane centerline information and intersection structure information are extracted. Furthermore, image boundaries for lanes, turning merging, and intersection conflict areas are constructed. Based on this structured information, the entire frame image is divided and cropped into functional regions. Functional region type identifiers and functional region images are input into a shared convolutional layer. Functional region type and vertical position constraints are introduced into the channel weight adjustment module and the anchor frame size and aspect ratio adaptive generation module. This ensures that the anchor frame size, aspect ratio, and density within different functional regions and at different distances better match the actual lane width and vehicle imaging size, while suppressing anchor frames that extend beyond the partition boundaries within the functional regions. Compared with existing detection algorithms that uniformly generate anchor boxes across the entire frame and lack a fine correspondence with road functional areas, this improvement helps reduce redundant candidate boxes in non-lane background areas, making the initial vehicle candidate boxes output by the convolutional neural network more consistent with the lane and intersection functional zones in terms of spatial distribution, providing accurate and structured prior constraints for subsequent occlusion processing and conflict attribute determination.

[0047] 2. This proposal presents a novel method for partially occluded vehicle detection and candidate box fusion based on occlusion-sensitive zones. It utilizes the lane edge and the boundary of a fixed occluder to offset along a normal direction away from the road's outer edge and towards the lane's interior in a unified road reference coordinate system, generating a first occlusion-sensitive zone extending along the lane and a second occlusion-sensitive zone surrounding the fixed occluder. These two zones are then merged to form continuous occlusion-sensitive zone information. This information is then mapped onto a feature map grid using a downsampling stride of the convolutional neural network feature map, and the grids falling within the occlusion-sensitive zone are explicitly marked. Based on this, a sub-image covering the occlusion-sensitive zone is cropped and an occlusion mask of the same size is generated. The two are then concatenated along the channel dimension and input into the occlusion detection sub-network. Anchor boxes are generated only at the occlusion-sensitive zone grid positions marked by the occlusion mask, resulting in occlusion-sensitive vehicle candidate boxes for the occluded area. Meanwhile, under the constraint of occlusion-sensitive zone information, this proposal uses the ratio of overlapping areas of candidate boxes and the position of the center point relative to the boundary of the occlusion-sensitive zone as conditions to perform one-to-one matching and weighted fusion of the initial vehicle candidate boxes and occlusion-sensitive vehicle candidate boxes. This makes the center position and width and height of the final fused candidate boxes closer to the spatial range of the actual visible part of the vehicle near the occlusion edge. Compared with traditional algorithms that rely on only a single detection result, this is more conducive to stably representing partially occluded vehicle targets near lane edges and fixed occlusions.

[0048] 3. This proposal suggests a vehicle conflict attribute recognition method based on road functional zoning maps and vehicle target information within functional areas. The method spatially aligns the fused vehicle candidate boxes on the road scene image and the road functional zoning map. Vehicle targets are categorized as conflict candidate vehicles based on whether their center points fall within marked intersection conflict areas or turning / merging areas. Overlap and center point distances are calculated based on the vehicle target's position and size data in the image to filter out vehicle combinations with potential conflict relationships. These combinations are then assigned corresponding conflict attributes based on their functional area type. This method completes road functional area modeling, occlusion-sensitive zone construction, intra-zone convolutional neural network detection, candidate box fusion, and conflict attribute determination within the same technical framework. The output not only includes precise position and size information of vehicles in lanes, turning / merging areas, and intersection conflict areas, but also directly carries conflict attribute information relevant to actual intersection operation scenarios. This effectively addresses the technical problem of obtaining vehicle detection results with road functional semantics and applicable to partially occluded situations in complex intersection and turning / merging scenarios. Attached Figure Description

[0049] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0050] Figure 1 This is a flowchart of a partially occluded vehicle detection method based on a convolutional neural network proposed in this invention. Detailed Implementation

[0051] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0052] refer to Figure 1 A method for detecting partially occluded vehicles based on convolutional neural networks, characterized by comprising:

[0053] S1. Acquire road scene images and corresponding road functional area reference data, parse the road functional area reference data, and obtain lane centerline information and intersection structure information.

[0054] S2. Based on the road scene image, lane centerline information, and intersection structure information, determine the image boundaries of lanes, turning merging areas, and intersection conflict areas, generate a road functional zoning map, and offset the lane edge and fixed obstruction boundary along the lane edge and fixed obstruction boundary into the lane by a preset distance in the road functional zoning map to generate obstruction sensitive zone information that may partially obstruct the position of vehicles.

[0055] S3. Based on the road scene image and the road functional area zoning map, determine the image range of each functional area and crop the image of each functional area. Input the image of each functional area into the convolutional neural network vehicle detection model, perform target detection on the image of each functional area, and generate initial vehicle candidate box information. The initial vehicle candidate box information includes position data and size data.

[0056] S4. Based on the road scene image, occlusion sensitive zone information and road functional zoning map, determine the image range covered by the occlusion sensitive zone and crop the occlusion sensitive sub-image. Input the occlusion sensitive sub-image into the convolutional neural network vehicle detection model to generate occlusion sensitive vehicle candidate box information.

[0057] S5. Based on the initial vehicle candidate box information and the occlusion-sensitive vehicle candidate box information, determine the area and positional relationship of the candidate boxes under the constraint of the occlusion-sensitive zone information, merge the candidate boxes that meet the area threshold and position threshold, and generate the functional area vehicle target information.

[0058] S6. Based on the vehicle target information of the functional area and the road functional zoning map, determine the conflict attributes of vehicle targets located in the intersection conflict area and the turning merging area, and output the vehicle detection result data.

[0059] In this embodiment, step S1 specifically includes:

[0060] In this embodiment, road scene images are first acquired using a camera device, and road functional area reference data are read from the road network database. The road scene image is a single-frame image captured by a camera device fixed above or to the side of the road. The road functional area reference data is road geometry data pre-constructed for the same road segment. The road functional area reference data includes at least lane boundary vector segments, guide arrow positions, stop line positions, and intersection boundary polygons. The road scene image is input into the acquisition interface, and the road functional area reference data is simultaneously input into the parsing module. The two types of data are managed in a unified manner to provide a data source for the subsequent establishment of a unified road reference coordinate system.

[0061] Subsequently, the imaging parameters of the road scene image are used as input. The imaging parameters include the internal and external parameters of the camera device. The internal parameters are used to describe the focal length and principal point position of the imaging plane, while the external parameters are used to describe the position and orientation of the camera device in the world coordinate system. The original coordinate system of the road functional area reference data is used as the road geometric reference coordinate system. The spatial transformation relationship from the road geometric reference coordinate system to the camera device coordinate system is obtained based on the external parameters of the camera device. Combined with the internal parameters, the coordinates in the road geometric reference coordinate system are mapped to the pixel coordinate plane of the road scene image. Thus, a unified road reference coordinate system is established that is aligned with the coordinates of the input image of the convolutional neural network vehicle detection model. Through this unified road reference coordinate system, a one-to-one correspondence is established between any pixel position in the road scene image and the geometric entity in the road functional area reference data. A sampling point sequence in the pixel coordinate system is assigned to each lane boundary vector line segment.

[0062] In the unified road reference coordinate system, lane boundary vector segments marked as the left and right edges of the same lane in the road functional area reference data are paired. For each pair of lane boundary vector segments, the two lane boundary vector segments are sampled synchronously along the vehicle driving direction with a fixed longitudinal step size to obtain corresponding sampling point pairs on the left and right lane boundaries. The midpoint of the line connecting each pair of corresponding sampling points is taken as the lane center sampling point. All lane center sampling points are connected in the sampling order to generate a lane centerline candidate segment. The above operation is repeated for each lane boundary vector segment to obtain a set of multiple lane centerline candidate segments as the lane centerline candidate set. In order to avoid the influence of noise boundaries or duplicate lane boundaries in the road functional area reference data on the reliability of the lane centerline, the total length of each candidate segment in the lane centerline candidate set is calculated, and the turning angle between two adjacent segments in the candidate segment is calculated.

[0063] The total length of each candidate lane centerline segment is compared with a preset length weight parameter. Candidate lane centerline segments with a total length less than the length weight parameter are deemed invalid and removed. At the same time, the turning angles of each candidate lane centerline segment are compared with a preset angle threshold. Candidate lane centerline segments with turning angles greater than the angle threshold are deemed not to meet the direct connection condition and removed. Only candidate lane centerline segments with a total length not less than the length weight parameter and all turning angles not greater than the angle threshold are retained. The set of retained candidate lane centerline segments is defined as the initial lane centerline. Through the above filtering, the initial lane centerline presents a coherent, smooth spatial orientation that is aligned with the actual vehicle trajectory in the unified road reference coordinate system, providing a stable geometric basis for subsequent road functional zoning.

[0064] After obtaining the initial lane centerlines, each initial lane centerline is sampled at fixed intervals according to the vehicle's direction of travel. A series of lane centerline sampling points are extracted along the line using a uniform sampling interval, and the lane centerline sampling points are sorted according to the sampling order to form lane centerline information. Each sampling point in the lane centerline information includes its planar coordinates in a unified road reference coordinate system and the lane identifier to which it belongs. Subsequently, the position of the guide arrow in the road functional area reference data is mapped to the unified road reference coordinate system, and the minimum distance between the position of the guide arrow and each initial lane centerline is calculated. Lane centerline sampling points with a distance less than a preset guide matching threshold are marked as candidate nodes with guide attributes. Similarly, the position of the stop line is mapped to the unified road reference coordinate system, and the intersection of the stop line and each initial lane centerline is calculated. Sampling points near the intersection are marked as candidate nodes with stop line attributes.

[0065] Furthermore, the intersection boundary polygons in the road functional area benchmark data are mapped to a unified road reference coordinate system. It is determined whether each lane centerline sampling point falls inside or near the intersection boundary polygon. The set of sampling points that fall inside the intersection boundary polygon is taken as the intersection area sampling point set. The sampling points in the intersection area sampling point set are divided according to their sequential position on the initial lane centerline and their relative orientation to the intersection boundary. The sampling points located outside the intersection boundary and close to the intersection boundary along the vehicle driving direction are divided into entrance nodes. The sampling points located outside the intersection boundary and far from the intersection boundary along the vehicle driving direction are divided into exit nodes. The sampling points located inside the intersection boundary polygon and in the overlapping area of ​​multiple initial lane centerlines are divided into conflict area nodes.

[0066] To organize entrance nodes, exit nodes, and conflict zone nodes into structured intersection structure information, entrance nodes and their corresponding exit nodes on the same initial lane centerline are connected in the sampling order to form a directed path. Conflict zone nodes are associated with adjacent lane centerlines according to their positions in a unified road reference coordinate system, forming multiple directed paths that intersect near the conflict zone nodes. Based on the starting entrance node, ending exit node, and path sequence crossing conflict zone nodes of each directed path, intersection structure information containing node sets and directed connection relationships is constructed. In the intersection structure information, each node records its coordinate position in the unified road reference coordinate system, node type, and lane identifier. Each directed connection records the starting node, ending node, and the sequence of conflict zone nodes it passes through. The lane centerline information and intersection structure information obtained in the above manner can be consistent with the input image coordinates of the convolutional neural network vehicle detection model. This ensures that subsequent road functional zoning map generation, occlusion sensitive zone information construction, and vehicle candidate box position constraints are all completed based on the unified road reference coordinate system, thereby guaranteeing the consistency and traceability of geometric relationships in the overall processing flow.

[0067] In this embodiment, step S2 specifically includes:

[0068] In this embodiment, after establishing a unified road reference coordinate system and obtaining lane centerline information and intersection structure information, the road scene image and the road functional area reference data are re-aligned under the unified road reference coordinate system. Specifically, the unified road reference coordinates of each lane centerline sampling point in the lane centerline information are mapped to the pixel coordinate plane of the road scene image through imaging parameters. These projection points are connected in the sampling order on the pixel coordinate plane to form a sequence of lane centerline pixels arranged continuously along the vehicle driving direction. For each lane centerline pixel sequence, its pixel coordinate position and the lane identifier are recorded to provide basic data for the construction of lane edges.

[0069] Subsequently, using the lane centerline pixel sequence as input, and based on the lane width values ​​given for each lane in the road functional area reference data, the lane width is evenly distributed to both sides of the lane centerline along the normal direction. For each pixel in the lane centerline pixel sequence, its normal direction vector is calculated in the pixel coordinate plane, and offset is performed on both sides of the normal direction at the same pixel interval. The offset distance is calculated from the lane width value of the corresponding lane. The offset discrete points are connected to obtain the left lane edge polyline and the right lane edge polyline. To avoid sawtooth fluctuations caused by projection errors and sampling errors, a smoothing strategy is adopted for the lane edge polylines. The positions of the inflection points whose slope changes more than the preset slope change threshold along the polyline direction are adjusted to keep the curvature change of the lane edge polylines within a controllable range. The beginning and end of the polylines are closed to form the closed lane area boundary. The closed lane area boundary is used to accurately define the coverage area of ​​each lane in the road scene image.

[0070] After the boundary of the closed lane area is constructed, the boundary of each closed lane area is divided into a straight lane area and a turning lane area according to the connection relationship between the entrance node and the exit node in the intersection structure information. Specifically, each directed connection path in the intersection structure information is matched with the corresponding initial lane centerline. For paths that extend along a straight line only outside the intersection area and directly connect the entrance node and the exit node, their corresponding closed lane area boundary is marked as a straight lane area. For paths that turn inside the intersection boundary polygon and pass through multiple conflict area nodes, their corresponding closed lane area boundary is marked as a turning lane area. For each turning lane area, the lane centerline is extended along the vehicle's direction of travel in the unified road reference coordinate system. The lane centerline is extended from the entrance node to the exit node and covers all conflict area nodes. The turning lane area image boundary that envelops the turning trajectory is regenerated based on the extended turning trajectory, so that the turning lane area image boundary accurately covers the position that the vehicle may occupy when turning at the intersection.

[0071] Taking the boundary of the turning lane area image as input, and based on the geometric topological relationship of the entrance node, exit node, and conflict area node in the intersection structure information, the overlapping area between different turning lane areas is calculated in a unified road reference coordinate system. Specifically, the polygonal areas corresponding to each turning lane area image boundary are superimposed pairwise to obtain the polygon intersection. The part containing the conflict area node in the intersection area is extracted, and the coordinates of these overlapping areas in the unified road reference coordinate system are projected onto the pixel coordinate plane of the road scene image to form the intersection conflict area image boundary. The intersection conflict area image boundary is expressed in polygonal form, covering the spatial location where multiple turning lane areas intersect inside the intersection, and is used to mark the possibility of driving conflicts in the area.

[0072] After obtaining the image boundaries of the closed lane area, the turning lane area, and the intersection conflict area, these areas are uniformly encoded in a unified road reference coordinate system. Specifically, a functional area type identifier is assigned to each functional area. The boundary of the closed lane area is encoded as the lane functional area, the image boundary of the turning lane area is encoded as the turning merging area image boundary, and the image boundary of the intersection conflict area is encoded as the conflict functional area. A functional area type identifier is assigned to each pixel position in the unified road reference coordinate system, and the encoding result is mapped to the actual image plane according to the pixel coordinates of the road scene image to generate a road functional area zoning map. In the road functional area zoning map, each pixel position has a clear functional area type identifier, realizing the integrated expression of the image boundaries of lanes, turning merging areas, and intersection conflict areas.

[0073] After generating the road functional zoning map, based on the lane edges in the road functional zoning map, the lane edges are offset by a first preset distance along the normal direction away from the outer edge of the road and pointing towards the inside of the lane, forming a first occlusion sensitive zone extending along the lane in the road scene image. The first occlusion sensitive zone is used to describe the location where only part of the vehicle is exposed in the field of view due to vehicles parked near roadside guardrails, curb lines, or road exit contraction. At the same time, based on the fixed occlusion boundary in the road functional zone reference data, the fixed occlusion boundary is extracted in the unified road reference coordinate system, and the fixed occlusion boundary is calculated along the normal direction away from the inside of the fixed occlusion and pointing towards the inside of the lane, and offset by a second preset distance. The offset area forms a second occlusion sensitive zone around the fixed occlusion in the road scene image. The second occlusion sensitive zone mainly covers the area near road facilities and buildings that may cause partial occlusion by vehicles.

[0074] To obtain continuous occlusion sensitive zone information, the first and second occlusion sensitive zones are merged in a unified road reference coordinate system. Specifically, the two types of occlusion sensitive zones are converted into binary region representations in the unified road reference coordinate system. Pixels belonging to any occlusion sensitive zone are marked as occlusion sensitive pixels. Connectivity analysis is performed on the set of occlusion sensitive pixels. Small fragment regions with a length shorter than a preset length threshold are deleted. Occlusion sensitive zones that are within a preset spacing threshold but have gaps are connected to form continuous occlusion sensitive zone information along the lane direction. The occlusion sensitive zone information exists in the form of a raster map with the same resolution as the road scene image. Each raster position records whether it belongs to an occlusion sensitive zone.

[0075] Finally, the occlusion-sensitive zone information is mapped to the feature map coordinate system of the convolutional neural network (CNN) vehicle detection model according to the downsampling step size. Specifically, based on the structure of the backbone feature extraction network and multi-scale detection head of the CNN vehicle detection model, the downsampling step size of each scale feature map relative to the road scene image is determined. The occlusion-sensitive zone grids in the road scene image are mapped to each feature map grid unit according to the corresponding downsampling ratio. For each feature map grid unit, the receptive field range of the feature map grid unit in the road scene image is calculated, and overlap judgment is performed with the region of the occlusion-sensitive zone information in the road scene image. Feature map grid units whose receptive field range overlaps with the occlusion-sensitive zone information and whose overlap ratio is greater than a preset overlap threshold are marked as occlusion-sensitive grids, and the remaining feature map grid units are marked as non-occlusion-sensitive grids. Through the above mapping, a functional zone corresponding to the road is generated in the feature map coordinate system of the CNN vehicle detection model. Figure 1 The corresponding occlusion-sensitive zone information provides a structured input for generating vehicle candidate boxes only within the occlusion-sensitive grid during the subsequent target detection process, and for using the occlusion-sensitive zone information for constraints during the candidate box filtering and merging stages. This allows for the fusion of road functional zoning and geometric priors under partially occluded scenarios at the network architecture level.

[0076] In this embodiment, step S3 specifically includes:

[0077] In this embodiment, after obtaining the road scene image and the road functional area zoning map, the road functional area zoning map is input into the functional area parsing module. The functional area parsing module first reads the functional area type identifier of each pixel in the road functional area zoning map, and divides the set of pixels with the same functional area type identifier and connected into three categories: lane functional area, turning merging area, and intersection conflict area. For each type of functional area, the minimum bounding rectangle containing all pixels of the functional area is calculated in the pixel coordinate system of the road scene image. The pixel coordinates of the upper left and lower right corners of the minimum bounding rectangle are used as the functional area image range, thereby obtaining a set of functional area image ranges that correspond one-to-one with each functional area.

[0078] After obtaining the functional region image range, the road scene image and the functional region image range are cropped by coordinates. Specifically, for each functional region image range, a corresponding rectangular image region is extracted from the road scene image, starting from its upper left pixel coordinates and using its width and height as the side lengths, to form the functional region image. To ensure that the functional region image can adapt to the input size of the convolutional neural network vehicle detection model, the aspect ratio of the functional region image is used as the scaling input. A proportional scaling strategy is adopted to scale the functional region image to a range not exceeding the preset input size without changing the aspect ratio. The insufficient scaling is filled by padding. During the scaling and filling process, the scaling ratio of each functional region image and the starting offset in the road scene image are recorded to establish a coordinate mapping relationship between the coordinates of the functional region image and the coordinates of the road scene image. This coordinate mapping relationship is used when writing back the coordinates of the vehicle candidate boxes in the future.

[0079] After the functional area image and coordinate mapping relationship are determined, the functional area image and the functional area type identifier of the corresponding functional area in the road functional area zoning map are jointly input into the feature extraction subnetwork of the convolutional neural network vehicle detection model. The feature extraction subnetwork adopts a multi-layer convolutional structure and a downsampling structure to perform layer-by-layer convolution and pooling operations on the functional area image to obtain intermediate feature maps at multiple scales. These intermediate feature maps are arranged in descending order of spatial resolution to form a multi-scale functional area feature map. For each functional area type identifier, the channel weight adjustment module encodes the functional area type identifier into a set of type feature vectors of a preset dimension, and then maps the type feature vectors to channel weight coefficients that are consistent with the number of channels in the multi-scale functional area feature map. Subsequently, at each scale, each channel of the multi-scale functional area feature map is multiplied by the corresponding channel weight coefficient. Channels with high correlation with the current functional area type are enhanced, and channels with low correlation are suppressed, thereby introducing functional area priors in the feature extraction stage.

[0080] After obtaining the multi-scale functional region feature map with channel weight adjustment, an anchor box set is generated using an adaptive generation module for anchor box size and aspect ratio, based on the vertical position of the functional region image in the road scene image and the functional region type identifier. Specifically, the vertical start and end positions of the functional region image in the road scene image are normalized, and the functional region image is divided into multiple height bands in the vertical direction according to a preset number of height bands. Each height band corresponds to a set of basic anchor box size and aspect ratio parameters. The height band located near the top of the image corresponds to a smaller basic anchor box size to cover distant vehicle targets, while the height band located near the bottom of the image corresponds to a larger basic anchor box size. The anchor frame size is used to cover nearby vehicle targets. For different functional area types, the anchor frame size and aspect ratio adaptive generation module also adjusts the aspect ratio of the basic anchor frame based on the vehicle scale statistics of lane functional areas, turning merging areas, and intersection conflict areas. For example, it increases the vertical anchor frame ratio in turning merging areas and increases the horizontal anchor frame ratio in intersection conflict areas. On each scale feature map, the center of the feature map grid is used as the center position of the candidate anchor frame. A set of anchor frames is generated based on the basic anchor frame size and aspect ratio of the corresponding height band. Anchor frames that exceed the functional area image range are cropped or directly removed to prevent anchor frames from going beyond the functional area boundary.

[0081] After the anchor box set is generated, the multi-scale functional region feature map and its corresponding anchor box set are input into the vehicle candidate box prediction sub-network of the convolutional neural network vehicle detection model. The vehicle candidate box prediction sub-network adopts a multi-layer convolutional structure, taking the feature vector corresponding to each anchor box as input, and outputting the prediction result containing the target score, center point coordinate offset, and width and height offset. The target score is used to indicate whether there is a vehicle target in the current anchor box, the center point coordinate offset is used to refine the center position of the anchor box, and the width and height offset is used to scale and correct the width and height of the anchor box. Among all the prediction results of the anchor boxes, the anchor boxes with target scores lower than the preset target score threshold are discarded as invalid anchor boxes, and the anchor boxes with target scores not lower than the preset target score threshold are retained to form the initial candidate anchor box set in the functional region.

[0082] To transform the initial candidate anchor box set from functional region image coordinates to road scene image coordinates, the center point coordinates and width and height dimensions of each initial candidate anchor box are reverse-transformed according to the coordinate mapping relationship between the functional region image and the road scene image. Specifically, the horizontal and vertical coordinates of the center point in the functional region image coordinate system are divided by the scaling ratio of the functional region image, and the starting offset of the functional region image in the road scene image is added. The width and height are also restored according to the scaling ratio to obtain the position and size data in the road scene image. For initial candidate anchor boxes in the same functional region, the non-maximum suppression module is used to screen their overlap. The non-maximum suppression module calculates the overlap between any two initial candidate anchor boxes based on a preset overlap threshold. When the overlap is greater than the preset overlap threshold, only the initial candidate anchor box with the higher target score is retained, and the initial candidate anchor box with the lower target score is removed to avoid generating multiple duplicate candidate boxes for the same vehicle target.

[0083] After the non-maximum suppression module is completed, the retained initial candidate anchor boxes are classified according to the functional area type identifier in the road functional zoning map. For initial candidate anchor boxes whose center point is located in the lane functional area, they are marked as lane vehicle initial candidate boxes. For initial candidate anchor boxes whose center point is located in the turning and merging area, they are marked as turning and merging vehicle initial candidate boxes. For initial candidate anchor boxes whose center point is located in the intersection conflict area, they are marked as intersection conflict area vehicle initial candidate boxes. Finally, the lane vehicle initial candidate boxes, turning and merging vehicle initial candidate boxes, and intersection conflict area vehicle initial candidate boxes are uniformly recorded as initial vehicle candidate box information. The position data and size data of each candidate box in the initial vehicle candidate box information in the road scene image, as well as the corresponding functional area type identifier, are recorded. This allows subsequent steps to use functional area semantic information to differentiate the candidate boxes when constraining occlusion sensitive zone information and determining conflict attributes.

[0084] In this embodiment, step S4 specifically includes:

[0085] In this embodiment, after constructing the occlusion sensitive zone information and the road function zoning map, the occlusion sensitive zone information and the road function zoning map are superimposed on the road scene image. Specifically, the positions marked as occlusion sensitive zones in the occlusion sensitive zone information are aligned pixel by pixel with the pixel coordinates of the road scene image. The minimum bounding rectangle in the road scene image is calculated for each connected occlusion sensitive zone region. This minimum bounding rectangle is used as the initial occlusion sensitive region box. At the same time, based on the direction information of the lane function regions in the road function zoning map, the side boundary closest to the inside of the lane in each initial occlusion sensitive region box is determined. This side boundary is extended along the lane direction by a preset expansion coefficient, and a certain length is added in the extension direction to cover the positions where vehicles may partially obstruct the view in front of or behind the obstruction. Thus, the occlusion sensitive sub-image range is obtained.

[0086] After obtaining the range of the occlusion-sensitive sub-images, the ranges of multiple occlusion-sensitive sub-images are traversed according to their pixel coordinates in the road scene image. For each occlusion-sensitive sub-image range, the corresponding rectangular area in the road scene image is cropped to form a functionally independent occlusion-sensitive sub-image. The top-left pixel coordinates and width and height of each occlusion-sensitive sub-image in the road scene image are recorded. Then, based on the spatial coverage relationship of the occlusion-sensitive sub-images in the occlusion-sensitive zone information, the occlusion-sensitive zone information is cropped with the same coordinate range. The cropped binary region is used to generate an occlusion mask image according to the size of the occlusion-sensitive sub-image. Pixels at the original occlusion-sensitive zone position are assigned foreground markers, and pixels at the non-occlusion-sensitive zone position are assigned background markers, so that the occlusion mask image and the occlusion-sensitive sub-image maintain a one-to-one correspondence at the pixel level.

[0087] To adapt the occlusion-sensitive sub-images and occlusion mask images to the input of the convolutional neural network vehicle detection model, each pair of occlusion-sensitive sub-images and occlusion mask images is simultaneously input into the size adjustment module. The size adjustment module compares the long and short sides of the occlusion-sensitive sub-images, calculates the scaling ratio based on the short side, and scales the occlusion-sensitive sub-images and occlusion mask images according to the same scaling ratio. After scaling, edge padding is used to fill the two to the fixed input size required by the convolutional neural network vehicle detection model. During the size adjustment process, the scaling ratio and the padding offset information in the input tensor are recorded for each pair of occlusion-sensitive sub-images and occlusion mask images, which are used to restore the coordinates output by the convolutional neural network vehicle detection model to the road scene image coordinate system.

[0088] After obtaining uniformly sized occlusion-sensitive sub-images and occlusion mask images, the occlusion-sensitive sub-images and occlusion mask images are concatenated along the channel dimension to form a multi-channel input. This multi-channel input is fed into the occlusion detection sub-network of the convolutional neural network vehicle detection model. The occlusion detection sub-network shares part of the convolutional layer structure with the feature extraction sub-network for functional region images. In the shared convolutional layers, multi-layer convolution operations and downsampling operations are performed on the multi-channel input to obtain an occlusion feature map that characterizes the details of the occluded region. The occlusion feature map corresponds spatially to the grid position of the downsampled input image and encodes texture information, edge information, and geometric prior information introduced by the occlusion mask image along the channels.

[0089] After the occlusion feature map is generated, the occlusion mask image is mapped to the raster coordinate system of the occlusion feature map using the same downsampling ratio to obtain the occlusion feature map. Figure 1 For each grid position in the occlusion feature map, the foreground or background identifier of the corresponding mask grid mark is read. If the identifier is foreground, the grid position is marked as the occlusion sensitive zone grid position, and an anchor frame set is generated at the grid position according to the preset anchor frame size combination and aspect ratio combination. If the identifier is background, no anchor frame is generated at the grid position. In this way, the anchor frames are only distributed in the occlusion sensitive zone grid positions marked by the occlusion mask image, avoiding the generation of redundant anchor frames in non-occlusion areas, while keeping the anchor frame density consistent with the occlusion sensitive zone information.

[0090] Subsequently, the feature vectors corresponding to the grid positions of the occlusion sensitive zone in the occlusion feature map and their anchor box sets are input into the vehicle candidate box prediction sub-network in the convolutional neural network vehicle detection model. The vehicle candidate box prediction sub-network outputs the target score, center point coordinate offset, and width and height offset for each anchor box. Anchor boxes with target scores lower than the preset target score threshold are removed. For the remaining anchor boxes, the center position of the anchor box is corrected according to the corresponding center point coordinate offset, and the width and height of the anchor box are corrected according to the width and height offset to obtain the candidate box coordinates in the occlusion feature map coordinate system. Then, according to the downsampling ratio between the occlusion feature map and the occlusion sensitive sub-image, the candidate box coordinates are mapped back to the pixel coordinate system of the occlusion sensitive sub-image to obtain the candidate box position and size in the occlusion sensitive sub-image.

[0091] Finally, based on the scaling ratio and fill offset information recorded by the aforementioned size adjustment module, the position and size of the candidate boxes in the occlusion-sensitive sub-image are scaled up proportionally and the fill offset is subtracted to restore their position and size data in the original road scene image. At the same time, the center point coordinates of the candidate boxes are translated using the upper left pixel coordinates of the occlusion-sensitive sub-image in the road scene image, so that the center point coordinates and width and height dimensions of each candidate box are completely aligned with the coordinate system of the road scene image. The candidate boxes obtained from all the occlusion-sensitive sub-images are summarized, and these candidate boxes generated under the constraint of the occlusion-sensitive zone information are uniformly recorded as occlusion-sensitive vehicle candidate box information. The position and size data of each occlusion-sensitive vehicle candidate box in the road scene image are recorded, providing a special candidate set for partially occluded vehicles for subsequent merging and filtering with the initial vehicle candidate box information.

[0092] In this embodiment, step S5 specifically includes:

[0093] In this embodiment, after obtaining the initial vehicle candidate box information and the occlusion-sensitive vehicle candidate box information, the two types of candidate boxes are uniformly mapped to the pixel coordinate system of the road scene image. Each candidate box is represented by a rectangle with its center point pixel coordinates and its width and height in the horizontal and vertical directions. First, based on the initial vehicle candidate box information, each initial vehicle candidate box is matched one by one with all occlusion-sensitive vehicle candidate boxes on the road scene image. The intersection area of ​​the initial vehicle candidate box and the occlusion-sensitive vehicle candidate box in the pixel coordinate system is calculated, and the area of ​​the intersection area is taken as the overlap area. At the same time, the area of ​​the initial vehicle candidate box itself and the area of ​​the occlusion-sensitive vehicle candidate box themselves are calculated respectively. The overlap area is divided by the area of ​​the initial vehicle candidate box and the area of ​​the occlusion-sensitive vehicle candidate box respectively to obtain two area ratios.

[0094] The two area ratios mentioned above are compared with a preset area threshold. Only when both area ratios are not less than the preset area threshold are the initial vehicle candidate box and the corresponding occlusion-sensitive vehicle candidate box formed into a candidate box pair. For the same initial vehicle candidate box, if it meets the area threshold condition with multiple occlusion-sensitive vehicle candidate boxes, the pair with the largest area ratio is selected as the unique candidate box pair to ensure a one-to-one matching relationship between the initial vehicle candidate box information and the occlusion-sensitive vehicle candidate box information. For initial vehicle candidate boxes whose area ratio with any occlusion-sensitive vehicle candidate box is less than the preset area threshold, they are retained as unmatched initial vehicle candidate boxes for subsequent vehicle target information directly entering the functional area. Through the above area ratio filtering, invalid combinations with excessively small overlapping areas due to positional deviations are eliminated to ensure that subsequent fusion is only performed between candidate boxes given by different detection branches for the same target vehicle.

[0095] After obtaining candidate bounding box pairs filtered by area ratio, the occlusion sensitive zone information is used as input to further constrain the position of the candidate bounding box pairs by utilizing the pixel coverage range of the occlusion sensitive zone information in the road scene image. Specifically, for each candidate bounding box pair, the center point pixel coordinates are read from the initial vehicle candidate box and the occlusion sensitive vehicle candidate box, respectively. The inclusion relationship between the center point pixel coordinates and the set of pixels marked as occlusion sensitive zones in the occlusion sensitive zone information is determined. If neither of the two center points falls within the pixel range covered by the occlusion sensitive zone information, the candidate bounding box pair is considered irrelevant to the occlusion sensitive zone information, and the candidate bounding box pair is removed from the list. For the elimination, for candidate box pairs where at least one center point falls within the coverage area of ​​the occlusion sensitive zone information, the minimum distance from the center point to the boundary of the occlusion sensitive zone information is further calculated. This minimum distance is compared with a preset position threshold. If the minimum distance is greater than the preset position threshold, it is considered that the center position of the candidate box pair deviates too far from the occlusion sensitive zone and does not conform to the spatial distribution pattern of partially occluded vehicles. The candidate box pair is then eliminated. Only candidate box pairs whose minimum distance is not greater than the preset position threshold are retained. Through the above position constraint steps, fusion only occurs between candidate boxes that are highly consistent with the occlusion sensitive zone information space.

[0096] For the candidate box pairs retained after area ratio filtering and position threshold filtering, the center point position data and size data of the initial vehicle candidate box and the occlusion-sensitive vehicle candidate box are weighted and fused. Specifically, two parameters, area weight and position weight, are preset. The area weight is used to reflect the influence of the rectangle area on the fusion result, and the position weight is used to reflect the influence of the center point position on the fusion result. For each candidate box pair, the horizontal and vertical coordinates of the center point of the initial vehicle candidate box and the horizontal and vertical coordinates of the center point of the occlusion-sensitive vehicle candidate box are weighted and summed respectively. The center point coordinates of the fused candidate box are obtained by linear combination of area weight and position weight. The width and height are also weighted and summed according to area weight and position weight to obtain the width and height of the fused candidate box. Thus, the fused candidate box in the road scene image coordinate system is constructed. In this way, the fused candidate box takes into account the information of the initial vehicle candidate box and the occlusion-sensitive vehicle candidate box in terms of position. On the one hand, it maintains the coverage of the overall outline of the vehicle, and on the other hand, it fits the partially occluded area constrained by the occlusion-sensitive zone information.

[0097] After obtaining the fusion candidate boxes, the pixel coordinates of the center point of the fusion candidate box in the road scene image are aligned with the road functional zoning map. The functional area type identifier of the pixel position where the center point of the fusion candidate box is located is read in the road functional zoning map. If the functional area type identifier corresponds to the lane functional area, the fusion candidate box is classified as a lane vehicle target. If the functional area type identifier corresponds to the turning merging area, the fusion candidate box is classified as a turning merging vehicle target. If the functional area type identifier corresponds to the intersection conflict area, the fusion candidate box is classified as an intersection conflict area vehicle target. For the aforementioned unmatched initial vehicle candidate boxes, the functional area is also assigned according to the functional area type identifier of their center point in the road functional zoning map to ensure that all valid detection results have a clear functional area category.

[0098] Finally, the fusion candidate boxes belonging to lane vehicle targets and the unmatched initial vehicle candidate boxes are uniformly organized into vehicle targets in the lane functional area. The fusion candidate boxes belonging to turning and merging vehicle targets are uniformly organized into vehicle targets in the turning and merging functional area. The fusion candidate boxes belonging to intersection conflict area vehicle targets are uniformly organized into vehicle targets in the intersection conflict functional area. The three types of vehicle target sets are unified into functional area vehicle target information. In the functional area vehicle target information, for each vehicle target, the center point position data, width and height dimension data and functional area type identifier in the road scene image are recorded. This provides complete input for further determining the conflict attributes of vehicle targets located in intersection conflict area and turning and merging area based on the road functional zoning map and outputting vehicle detection result data.

[0099] In this embodiment, step S6 specifically includes:

[0100] In this embodiment, after generating the functional area vehicle target information, the functional area vehicle target information and the road functional zoning map are spatially aligned in the pixel coordinate system of the same road scene image. Specifically, for each vehicle target in the functional area vehicle target information, its center point pixel coordinates, width, and height are read, and the center point pixel coordinates are mapped to the corresponding position in the road functional zoning map. The functional area type identifier of the pixel position is queried from the road functional zoning map. If the functional area type identifier corresponds to the lane functional area, the vehicle target is marked as a lane vehicle target. If the functional area type identifier corresponds to the turning merging area or the intersection conflict area, the two are not distinguished for the time being. The vehicle target is added to the conflict candidate set and its original position data, size data, and functional area type identifier are retained. In the above way, vehicle targets located in the turning merging area and the intersection conflict area are selected from all functional area vehicle target information to form the conflict candidate vehicle target set for subsequent conflict relationship determination.

[0101] After obtaining the set of candidate vehicle targets for conflict, the vehicle targets in the set are combined in pairs. Specifically, one vehicle target in the set is selected as the first vehicle target, and then the vehicle target following it is selected as the second vehicle target. The first vehicle target and the second vehicle target are combined to form a vehicle target combination. For each vehicle target combination, the rectangular boundaries of the two vehicle targets in the road scene image are compared according to the position data and size data in the vehicle target information of the functional area. The maximum left boundary and minimum right boundary of the two rectangles in the horizontal direction, and the maximum upper boundary and minimum lower boundary in the vertical direction are determined respectively. If the minimum right boundary is greater than the maximum left boundary and the minimum lower boundary is greater than the maximum upper boundary, the area of ​​the intersection region is obtained by multiplying the horizontal overlap width and the vertical overlap height. Then, the area of ​​the two vehicle target rectangles is calculated separately. The overlap degree is the ratio of the area of ​​the intersection region to the sum of the areas of the two rectangles minus the area of ​​the intersection region. If the minimum right boundary is not greater than the maximum left boundary or the minimum lower boundary is not greater than the maximum upper boundary, the overlap degree is recorded as zero.

[0102] While calculating the overlap, for two vehicle targets in a vehicle target combination, the lateral and longitudinal coordinates of the two center points are read from the position data in the vehicle target information of the functional area. The difference between the lateral and longitudinal coordinates is used as input to calculate the distance between the center points of the two points in the road scene image. Then, the overlap is compared with a preset overlap threshold, and the center point distance is compared with a preset distance threshold. For vehicle target combinations with an overlap greater than the preset overlap threshold, they are directly marked as vehicle target combinations with conflicting relationships. For vehicle target combinations with an overlap not greater than the preset overlap threshold but a center point distance less than the preset distance threshold, the overlap is not considered a conflicting relationship. Vehicle target combinations that meet the threshold are also marked as vehicle target combinations with conflicting relationships. Vehicle target combinations that do not meet either the overlap threshold condition or the distance threshold condition are not marked as conflicting combinations. In the above comparison process, the spatial range of the intersection conflict area and the turning merging area in the road functional zoning map can also be introduced as an additional constraint. Only vehicle target combinations whose center points both fall within the same intersection conflict area or both fall within the same turning merging area are calculated for overlap and center point distance, thereby ensuring that vehicle target combinations with conflicting relationships are limited to the same functional area.

[0103] After obtaining the vehicle target combinations with conflicting relationships, each vehicle target combination with conflicting relationships is used as input. A conflict attribute is assigned to the vehicle target based on the type of intersection conflict area or turning / merging area where the vehicle target is located. Specifically, for a vehicle target located in an intersection conflict area, the intersection conflict area identifier where the center point of the vehicle target is located is read from the road functional zoning map, its conflict attribute is set to intersection conflict attribute, and a conflict attribute field is added to the vehicle target information in the functional area to record its status of having an intersection conflict attribute. For a vehicle target located in a turning / merging area, the turning / merging area identifier where the center point of the vehicle target is located is read from the road functional zoning map, its conflict attribute is set to turning / merging conflict attribute, and a conflict attribute field is added to the vehicle target information in the functional area to record its status of having a turning / merging conflict attribute. For vehicle targets appearing in multiple conflicting combinations simultaneously, if the functional area type identifiers of the multiple combinations are the same, a single conflict attribute is retained. If the functional area type identifiers of the combinations are different, the intersection conflict attribute can be assigned priority over the turning / merging conflict attribute according to a preset priority, or multiple conflict attributes can be recorded for the vehicle target simultaneously.

[0104] For vehicle targets not marked as having conflict relationships, their conflict attribute field is set to non-conflict attribute to distinguish them from vehicle targets with conflict attributes. Finally, vehicle targets with intersection conflict attributes or turning-in conflict attributes, as well as vehicle targets without conflict attributes, are uniformly organized to form vehicle detection result data. In the vehicle detection result data, each vehicle target must contain at least the location data, size data, functional area type identifier, and conflict attribute information in the road scene image. The vehicle detection result data can be directly provided to the upper-level traffic state analysis module to identify intersection conflict scenarios and turning-in scenarios. It can also be used as a supervisory signal to backact on the training process of the convolutional neural network vehicle detection model and occlusion detection sub-network during the offline training stage. This allows the network to explicitly learn the spatial relationship patterns related to intersection conflict areas and turning-in areas while learning vehicle appearance features. As a result, in subsequent online inference, it can combine functional area priors and conflict attributes to more reliably identify and judge vehicle targets.

[0105] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1.A method for partially occluded vehicle detection based on convolutional neural network, characterized in that, The method comprises the following steps: S1, acquiring a road scene image and road function area reference data corresponding to the road scene image, analyzing the road function area reference data to obtain lane center line information and intersection structure information; S2, determining the image boundaries of lanes, turning merging areas and intersection conflict areas according to the road scene image and the lane center line information and the intersection structure information, generating a road function area zoning map, and offsetting a preset distance along the lane edge and the fixed barrier boundary to the inside of the lane in the road function area zoning map to generate occlusion sensitive band information representing the positions of the partially occluded vehicles that may appear; S3, determining the image ranges of the functional areas according to the road scene image and the road function area zoning map, and cutting the functional area images, inputting the functional area images into a convolutional neural network vehicle detection model, performing target detection on the functional area images, and generating initial vehicle candidate box information, the initial vehicle candidate box information including position data and size data; S4, determining the image ranges covered by the occlusion sensitive bands and cutting the occlusion sensitive sub-images according to the road scene image, the occlusion sensitive band information and the road function area zoning map, inputting the occlusion sensitive sub-images into the convolutional neural network vehicle detection model, and generating occlusion sensitive vehicle candidate box information; S5, determining the area and position relationship of the candidate boxes under the constraint of the occlusion sensitive band information according to the initial vehicle candidate box information and the occlusion sensitive vehicle candidate box information, merging the candidate boxes that meet the area threshold and the position threshold, and generating functional area vehicle target information; S6, determining the conflict attributes of the vehicle targets located in the intersection conflict areas and the turning merging areas according to the functional area vehicle target information and the road function area zoning map, and outputting vehicle detection result data. 2.The partially occluded vehicle detection method based on convolutional neural network according to claim 1, wherein, S1 specifically comprises: S11, inputting the road scene image into a collection interface, taking the road function area reference data as input, establishing a unified road reference coordinate system aligned with the input image coordinates of the convolutional neural network vehicle detection model according to the imaging parameters of the road scene image and the coordinate system of the road function area reference data, and obtaining the corresponding relationship between the pixels in the road scene image and the road function area reference data; S12, in the unified road reference coordinate system, pairing the lane boundary vector line segments in the road function area reference data according to the left-right correspondence, generating lane center line candidate line segments according to the midline of each pair of lane boundary vector line segments, and removing the lane center line candidate line segments that do not meet the straight connection condition through length weight and angle threshold, to obtain initial lane center lines; S13, according to the spatial trend of the initial lane center lines in the unified road reference coordinate system, sampling the initial lane center lines at a fixed interval to obtain lane center line information arranged in a sampling order, and clustering the sampling points located in the intersection area into entrance nodes, exit nodes and conflict area nodes according to the guide arrows, stop lines and intersection boundaries in the road function area reference data, and constructing the intersection structure information according to the connection relationship between the nodes. 3.The partially occluded vehicle detection method based on convolutional neural network according to claim 1, wherein, S2 specifically comprises: S21, align the road scene image and the road functional area reference data in a unified road reference coordinate system, project the lane center line information to the road scene image according to the pixel coordinates, and obtain a lane center line pixel point sequence arranged continuously along the driving direction of the vehicle; S22, taking the lane center line pixel point sequence as input, generating lane edge polylines on both sides of the lane center line in the normal direction according to the lane width values given in the road functional area reference data, smoothing the lane edge polylines and closing the beginning and end, and obtaining the closed lane area boundary in the road scene image; S23, according to the connection relationship between the entry node and the exit node in the intersection structure information, the closed lane area boundary is divided into straight lane area and turning lane area according to the corresponding relationship, and the turning lane area is extended according to the driving direction of the vehicle in the unified road reference coordinate system, and the turning lane area image boundary covering the turning track is obtained; S24, taking the turning lane area image boundary as input, according to the geometric topological relationship among the entry node, the exit node and the conflict area node in the intersection structure information, the overlapping area between different turning lane areas in the unified road reference coordinate system is calculated, and the overlapping area is mapped back to the road scene image to obtain the intersection conflict area image boundary; S25, the closed lane area boundary, the turning lane area image boundary and the intersection conflict area image boundary are uniformly encoded in the unified road reference coordinate system, the turning lane area image boundary is marked as the turning merging area image boundary, the functional area type identifier is used to distinguish different functional areas, and the encoding result is mapped to the road scene image to generate the road functional area partition map containing the lane, the turning merging and the intersection conflict area image boundary; S26, according to the lane edge in the road functional area partition map, the lane edge is offset by a first preset distance in the normal direction away from the outer edge of the road and pointing to the inside of the lane, forming a first occlusion sensitive band extending along the lane in the road scene image, and according to the fixed occlusion boundary in the road functional area reference data, the fixed occlusion boundary is offset by a second preset distance in the normal direction away from the inside of the fixed occlusion and pointing to the inside of the lane, forming a second occlusion sensitive band around the fixed occlusion, and the first occlusion sensitive band and the second occlusion sensitive band are merged in the unified road reference coordinate system to obtain continuous occlusion sensitive band information; S27, the occlusion sensitive band information is mapped to the feature map coordinate system of the convolutional neural network vehicle detection model according to the downsampling step of the convolutional neural network vehicle detection model, and the grid units falling into the occlusion sensitive band information are marked in units of feature map grids to generate occlusion sensitive band information corresponding to the road functional area partition map, which is used to constrain the position selection and merging of the vehicle candidate boxes output by the subsequent convolutional neural network vehicle detection model. 4.The partially occluded vehicle detection method based on convolutional neural network according to claim 1, wherein, S3 is specifically: S31, input the road function area zoning map into a function area analysis module, determine the minimum circumscribed rectangle of each function area in the pixel coordinate system of the road scene image according to the image boundary of the lane, turning-in area and intersection conflict area in the road function area zoning map, and obtain the function area image range corresponding to each function area; S32, coordinate crop the road scene image and the function area image range, take the pixels in the function area image range as the input to crop the function area image, and according to the length-width ratio of the function area image, adopt the scaling strategy to scale the function area image to the input size range of the convolutional neural network vehicle detection model, and record the coordinate mapping relationship between the function area image and the road scene image; S33, input the function area image and the function area type identifier of the corresponding function area in the road function area zoning map into the feature extraction subnetwork of the convolutional neural network vehicle detection model, perform multi-layer convolution and down-sampling on the function area image in the shared convolutional layer, generate multi-scale function area feature maps, and apply weights to different scale channels in the multi-scale function area feature maps through the channel weight adjustment module with the function area type identifier as the input; S34, input the multi-scale function area feature map, according to the vertical position of the function area image in the road scene image and the function area type identifier, adopt the anchor box size and aspect ratio adaptive generation module to generate an anchor box set for different distances and different function areas on each scale feature map, divide the function area image into multiple height bands in the vertical direction, wherein the anchor box size in the upper height band is smaller than the anchor box size in the lower height band, and suppress the anchor boxes that cross the boundary of the function area to the outside of the function area within the function area; S35, input the multi-scale function area feature map and its corresponding anchor box set into the vehicle candidate box prediction subnetwork of the convolutional neural network vehicle detection model, output the prediction result containing the target score, center point coordinate offset and width-height offset for each anchor box, and exclude the anchor boxes with target scores lower than the preset target score threshold according to the preset target score threshold, to obtain an initial candidate anchor box set in the function area; S36, according to the coordinate mapping relationship between the function area image and the road scene image, convert the center point coordinates and width-height size of the initial candidate anchor box set in the function area image into position data and size data in the road scene image, and retain the initial candidate anchor box with the largest target score through the non-maximum suppression module for the initial candidate anchor boxes falling in the same function area and having an overlap degree exceeding the preset overlap degree threshold; S37, attribute the initial candidate anchor boxes processed by the non-maximum suppression module to lane vehicle initial candidate boxes, turning-in vehicle initial candidate boxes and intersection conflict area vehicle initial candidate boxes according to the function area type identifier in the road function area zoning map, and uniformly record the lane vehicle initial candidate boxes, turning-in vehicle initial candidate boxes and intersection conflict area vehicle initial candidate boxes as initial vehicle candidate box information, wherein each candidate box in the initial vehicle candidate box information carries position data and size data. 5.The partially occluded vehicle detection method based on convolutional neural network according to claim 1, wherein, S4 is specifically: S41, superimpose the occlusion-sensitive strip information and the road function zoning map on the road scene image, calculate the minimum circumscribed rectangle of the connected region in the occlusion-sensitive strip information in the road scene image, and expand along the lane direction on the side close to the inside of the lane according to an expansion coefficient to obtain an occlusion-sensitive sub-image range; S42, crop the road scene image according to the occlusion-sensitive sub-image range to obtain an occlusion-sensitive sub-image, and generate an occlusion mask image with the same size as the occlusion-sensitive sub-image according to the coverage relationship of the occlusion-sensitive sub-image in the occlusion-sensitive strip information, align and uniformly scale the occlusion-sensitive sub-image and the occlusion mask image to the input size range of the convolutional neural network vehicle detection model according to the short side; S43, input the occlusion-sensitive sub-image and the occlusion mask image after channel dimension splicing into the occlusion detection subnetwork of the convolutional neural network vehicle detection model, extract occlusion feature maps in the shared convolutional layer, and only generate anchor boxes at the occlusion-sensitive strip grid positions marked by the occlusion mask image, output the center point coordinate offset and the width and height offset by the vehicle candidate box prediction subnetwork, convert the prediction results into position and size data in the road scene image according to the coordinate mapping relationship between the occlusion-sensitive sub-image and the road scene image, and obtain occlusion-sensitive vehicle candidate box information. 6.The partially occluded vehicle detection method based on convolutional neural network according to claim 1, wherein, S5 is specifically: S51, one-to-one match the initial vehicle candidate box information and the occlusion-sensitive vehicle candidate box information on the road scene image, calculate the overlapping area of the occlusion-sensitive vehicle candidate box information with the position data and size data in the initial vehicle candidate box information as the reference, compare the area ratio of the overlapping area to the area of the initial vehicle candidate box information and the occlusion-sensitive vehicle candidate box information with the area threshold, and screen out the candidate box pairs with an area ratio below the area threshold; S52, input the occlusion-sensitive strip information, determine the center point position of each candidate box pair according to the pixel range of the occlusion-sensitive strip information in the road scene image, remove the candidate box pairs located outside the coverage range of the occlusion-sensitive strip information or with a center point deviating from the boundary of the occlusion-sensitive strip information by a distance exceeding the position threshold, and only keep the candidate box pairs meeting the position threshold; S53, weight and fuse the center point position data and size data of the initial vehicle candidate box information and the occlusion-sensitive vehicle candidate box information in the retained candidate box pairs according to a preset area weight and a preset position weight to obtain a fused candidate box, attribute the fused candidate box to a functional area according to the road function zoning map, and summarize the functional area vehicle target information according to the lane, the turning and merging area, and the conflict area of the intersection. 7.The partially occluded vehicle detection method based on convolutional neural network according to claim 1, wherein, S6 is specifically: S61, spatially align the functional area vehicle target information and the road function zoning map on the road scene image, determine whether the center point falls into the conflict area of the intersection or the turning and merging area according to the center point pixel coordinates of each vehicle target in the functional area vehicle target information, and mark the vehicle targets meeting the conditions as conflict candidate vehicle targets. S62, combine the conflict candidate vehicle targets two by two, calculate the overlap degree and center point distance between the vehicle targets in the road scene image according to the position data and size data in the functional area vehicle target information, and mark the vehicle target combination with the overlap degree greater than a preset overlap degree threshold or the center point distance less than a preset distance threshold as the vehicle target combination with a conflict relationship; S63, take the vehicle target combination with the conflict relationship as input, respectively assign a conflict attribute to each vehicle target according to the intersection conflict area or the type of the turning merging area where the vehicle target is located, and combine the vehicle targets with the conflict attribute and the vehicle targets without the conflict attribute into vehicle detection result data.

Images