A parking scene detection method based on multi-view and time series fusion
By introducing multi-view angle and timing fusion technology in parking scene detection, combined with lidar point cloud annotation, the problems of low detection accuracy and poor robustness in the existing technology are solved, and high-precision and efficient parking scene detection are achieved. Improved obstacle detection accuracy and calculation efficiency.
Patent Information
- Application Number
- CN202311063538.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-22
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2043-08-22
AI Technical Summary
The existing parking scene detection methods have problems such as lack of timing information, loss of information caused by inverse perspective transformation, poor ultrasonic detection accuracy, low calculation efficiency and low distance accuracy of obstacles, resulting in low detection accuracy and poor robustness, which cannot be done in the High-precision detection is achieved in complex scenarios.
Using detection methods based on multi-view angle and timing fusion, multi-way fisheye views are extracted through convolutional neural networks, and static splicing is used to transform bird's eye view into an inverse perspective, and a spatial cross-attention mechanism and timing attention mechanism are introduced to fuse each Fisheye view features and historical frame information, parking spaces and travelable areas are detected, combined with lidar point cloud labeling to improve obstacle detection accuracy, and ultrasonic radar is abandoned.
It improves the continuity and accuracy of detection, retains obstacle height information, improves calculation efficiency, realizes high-precision obstacle detection and parking space detection, and enhances robustness and adaptability.
Smart Images

Figure CN117274036B_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to the field of computer vision technology, and in particular to a parking scene detection method based on multi-view and time series fusion. [Background technology]
[0002] Traditional parking space detection is usually performed on a single fisheye camera view or a bird's eye view (IPM) of multiple fisheye cameras. The obstacle detection results of ultrasonic radar and multiple fisheye cameras are then post-fused to determine whether the parking space is occupied and to form a drivable area for parking. The current parking scene detection methods have the following main problems:
[0003] 1. Lack of timing information: During the parking process, parking spaces and obstacles are often blocked. Especially during the parking process, the target parking space is easily blocked by the vehicle. At the same time, the detection of parking spaces is very dependent on the clarity of corner points and parking space lines, so the detection of a single frame will be very unstable.
[0004] 2. Defects of the Inverse Perspective Mapping (IPM) bird's-eye view: Feature extraction on a statically spliced bird's-eye view cannot fuse the features of each fisheye view, resulting in information loss, especially in areas where multiple fisheye views overlap. At the same time, the bird's-eye view also lacks height information, and defaults to a unified plane with a height of 0. As a result, some parking spaces with uneven ground, such as grass brick parking spaces, or parking spaces with slopes on the roadside, will appear as trapezoids on the bird's-eye view, so their true coordinate positions cannot be obtained. Inverse Perspective Mapping (IPM) is a method that uses the geometric constraints of the camera's internal and external matrices to map pixels to the BEV (bird's eye view) plane. Although it is inaccurate due to the flat ground assumption, it provides the possibility that images and point clouds can be unified in BEV.
[0005] 3. Disadvantages of ultrasonic radar: The detection range of ultrasonic sensors is usually limited, and they can only provide approximate distance and existence judgments, but cannot give precise locations. In complex environments, echoes may also be affected by multipath interference, resulting in inaccurate measurement results. Another important point is that ultrasonic waves cannot give the type of obstacles, which is not conducive to parking path planning.
[0006] 4. Poor obstacle distance accuracy: Due to the inherent defects of ultrasound, it can only guarantee the accuracy of short-range distances, so obstacle detection usually adopts the method of fusing the detection results of multiple fisheye cameras and ultrasonic results. Fisheye camera obstacle detection usually uses ground point detection and inverse perspective transformation to obtain the real coordinate position, but the detection of ground points on the fisheye view is difficult to do well. In addition, the complexity of post-fusion of multiple fisheye cameras usually makes it difficult to guarantee the accuracy of obstacle distance.
[0007] 5. Low computational efficiency: Parking space detection is usually performed on an inverse perspective transform (IPM) bird's-eye view, which requires multiple fisheye cameras to be stitched before entering the neural network, while obstacle detection is performed on a single fisheye view and can directly enter the neural network. The inputs of the two neural network models are inconsistent, so parameter sharing is impossible, resulting in low overall computational efficiency.
[0008] BEV (bird's eye view) perception is a method of converting a three-dimensional scene into a horizontal two-dimensional plane. It has been widely used in two-dimensional and three-dimensional object detection, perception map generation, and detection of walkable areas.
[0009] After searching (keywords: parking, parking space detection, drivable area detection), the following related patents were retrieved.
[0010] Related search result 1: Chinese patent announcement number, CN108875911B, name, a vision-based parking space detection method and device, the detection method, including: constructing a large-scale, annotated, surround image dataset for parking space perception problems; using the YOLOv2 framework based on deep learning to improve the target detection framework to complete the parking space control point detection; pairing the detected control points to form a point pair combination, judging their distance constraints for preliminary screening, and designing a custom model based on the shallow model AlexNet of the classification model based on the deep convolutional neural network to classify the parking space control point pairs that meet the distance constraints, and complete the parking space inference by judging the type of parking space and the direction of the entrance line. Not only is the detection accuracy high, but the response speed is fast, stable and reliable.
[0011] Technical disadvantages of patent CN108875911B: 1. Preprocess the environmental images on both sides of the vehicle and perform detection on the obtained inverse perspective transform (IPM) bird's-eye view, which will lose the characteristics of the multi-way surround view image itself and the height information; 2. Use YOLOV2 to detect the corner points of the parking space on the inverse perspective transform (IPM) bird's-eye view map, then match them, and finally use the classification task of the deep convolutional neural network to determine the type and orientation of the parking space. The accuracy is not high, and determining the parking space orientation through the classification model has certain limitations.
[0012] Related search result 2: Chinese patent announcement number, CN110775052B, name, an automatic parking method based on the fusion of vision and ultrasonic perception, including the following steps: visual parking space detection, ultrasonic radar parking space detection, fusion of the two parking space detection results, parking space tracking, parking space display; determine the target parking space, combine the current scene map, plan the parking speed and parking trajectory; based on visual detection and ultrasonic radar detection, detect obstacles in the drivable area around the vehicle and the edge of the drivable area, and estimate the motion of the obstacles; calculate the possibility of collision between the vehicle and the obstacle, and trigger the obstacle avoidance function if the collision possibility is greater than the preset threshold; track the parking trajectory through the low-speed controller, realize the gear control, vehicle speed control, steering wheel steering control and obstacle avoidance control along the parking trajectory, and complete automatic parking. It can improve the detection effect of parking spaces and obstacles around the vehicle, improve robustness, add a moving obstacle trajectory estimation module, and improve the reliability of obstacle avoidance.
[0013] Technical disadvantages of patent CN110775052B: 1. Parking space corner point detection is performed under the input of the left and right surround original fisheye scene. After matching, it is transformed to the ground coordinate system through inverse perspective. This method can only handle the situation where the parking space corner points are on the same surround image, and cannot handle various complex scenes; 2. Ultrasonic wave is used to detect obstacles to determine whether the parking space can be parked and plan the parking path. The detection distance of ultrasonic wave is short, and the obstacle position accuracy is not high. It cannot detect a farther range, which is not conducive to parking trajectory planning.
[0014] The present invention aims to solve the technical problems that the existing parking technology has low detection accuracy, poor robustness and cannot achieve high-precision detection in various complex parking scenarios, and makes technical improvements to the parking scene detection method. [Summary of the invention]
[0015] The purpose of the present invention is to provide a parking scene detection method with high detection accuracy and strong robustness for parking spaces and drivable areas based on multi-view and time series fusion.
[0016] To achieve the above object, the technical solution adopted by the present invention is a parking scene detection method based on multi-view and time series fusion, comprising the following steps:
[0017] S1. Extract features from multiple fisheye views through a convolutional neural network to generate corresponding feature maps;
[0018] S2, statically stitching the multiple fisheye views into an inverse perspective transformed bird's-eye view, constructing a top-down perspective rectangular area of a size that can accommodate parking spaces and vehicle drivable areas based on the inverse perspective transformed bird's-eye view, and dividing it into a number of grids;
[0019] S3, introduce the spatial cross attention mechanism, associate the top-view rectangular area feature map with the multi-path fisheye view feature map according to the internal and external parameters of the fisheye camera, and obtain the top-view rectangular area feature map of the current frame;
[0020] S4. Introduce the temporal information, use the vehicle posture and the temporal fusion feature map of the rectangular area of the previous frame to align the features in time sequence, and obtain the temporal fusion feature map of the rectangular area of the current frame, so that the historical frame information can affect the detection result of the current frame.
[0021] S5. Perform parking space detection and drivable area detection based on the temporal fusion feature map of the rectangular area of the top-down perspective of the current frame.
[0022] Preferably, step S4 further includes: introducing extensibility information to fuse the rectangular area feature map of the top-down perspective of the current frame with the rectangular area feature maps of the top-down perspective of other sensors.
[0023] Preferably, the other sensor is a lidar or other surrounding camera.
[0024] Preferably, the multiple fisheyes include a front surround fisheye, a rear surround fisheye, a left surround fisheye, and a right surround fisheye.
[0025] Preferably, in step S2, the top-view viewing angle rectangular area is a square, which is divided into a number of square grids of equal size.
[0026] Preferably, the specific implementation method of step S3 association is to start from each grid center point of the top-view rectangular area, and after sampling in height, project the internal and external parameters of the multi-channel fisheye camera onto the corresponding multi-channel fisheye view feature map as a grid reference point, and then sample the fisheye view feature map near each grid reference point, and use the attention mechanism to construct the current frame top-view rectangular area feature map.
[0027] Preferably, the specific implementation of step S4 timing fusion is to change the position of the sampling point, use the bilinear interpolation method to sample on the timing fusion feature map of the rectangular area of the top-down perspective of the previous frame, and generate the timing fusion feature map of the rectangular area of the top-down perspective of the current frame after timing alignment.
[0028] Preferably, in the parking scene detection method based on multi-view and time series fusion, step S5 specifically includes the following sub-steps:
[0029] S51, performing parking space detection using a parking space detection head;
[0030] S52, performing drivable area detection;
[0031] S53. By integrating the drivable area detection result with the parking space detection result, it is determined whether there is an obstacle inside the parking space and whether there is an obstacle in the parking path. If so, the parking space is determined to be an occupied parking space and cannot be parked in. Otherwise, the parking space is determined to be an unoccupied parking space and parking is initiated.
[0032] Preferably, step S51 specifically includes the following sub-steps:
[0033] The parking space detection head initializes a specific query, uses a cross-attention mechanism to associate the target query with the current frame top-down perspective rectangular area temporal fusion feature map, generates a feature representation corresponding to each query, and finally uses a regression model to predict the center point coordinates, existence probability, parking space orientation, vehicle arrangement category, and the offset of the four corner points relative to the center point of the parking space;
[0034] The parking spaces are formed through the regression results of the parking space detection head, and the parking spaces whose center point existence probability is greater than the threshold are retained. The four corner points of the regression are arranged clockwise by default, and the first two corner points are the entrance corner points of the parking spaces.
[0035] The parking scene detection method based on multi-view and time series fusion of the present invention has the following beneficial effects:
[0036] 1. Temporal consistency: The introduction of the temporal attention mechanism can fuse the features of historical frames and achieve temporal alignment, thereby improving the continuity and accuracy of detection. By considering the vehicle's motion trajectory and historical information, the algorithm can better understand the state changes during parking and provide more reliable detection results.
[0037] 2. Strong robustness: By introducing the bird's-eye view (BEV) and cross-attention mechanism, the image features of each fisheye view are better integrated, avoiding the loss of each fisheye view feature caused by directly extracting the features of the static splicing inverse perspective transform (IPM) bird's-eye view. At the same time, when calculating the bird's-eye view (BEV) and the spatial cross-attention mechanism of multiple fisheye views, sampling will be performed at the height of the bird's-eye view (BEV) grid and projected to each fisheye view, without losing the height information of the detected parking space and obstacles;
[0038] 3. High-precision obstacle detection: Use high-precision Lidar point cloud annotation as true value supervision, rely on spatiotemporal fusion feature maps under the bird's-eye view (BEV) to detect obstacle segmentation around the vehicle, improve distance and category detection accuracy, do not require post-fusion, and abandon ultrasonic radar;
[0039] 4. Improve computing efficiency: Since the neural network inputs of the two tasks are multiple fisheye views and detection is performed at the top-down perspective (BEV), the parameters of the neural network backbone can be shared. Only two different detection heads are required, which can reduce the consumption of computing resources, improve computing efficiency, and improve real-time performance.
[0040] 5. Scalability: Since the detection is performed under the bird's-eye view (BEV), information fusion with other sensors such as Lidar can be performed, which makes the algorithm highly adaptable and scalable under different vehicle and environmental settings.
Brief Description of the Drawings
[0041] Figure 1 It is a flow chart of a parking scene detection method based on multi-view and time series fusion. [Specific implementation method]
[0042] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the present invention. But it is obvious to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention. The present invention is by no means limited to any specific configuration and algorithm proposed below, but covers any modification, replacement and improvement of elements, parts and algorithms without departing from the concept of the present invention. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid unnecessary ambiguity to the present invention.
[0043] Example
[0044] This embodiment implements a parking scene detection method based on multi-view and time series fusion.
[0045] The core innovation of this embodiment lies in the parking scene detection based on multi-perspective and time-series fusion. It only uses multiple fisheye cameras. Under the bird's-eye view (BEV), it fuses the features of different perspectives and times through a variety of attention mechanisms, thereby improving the accuracy and robustness of parking space and drivable area detection, and achieving high-precision detection in various complex parking scenarios.
[0046] This embodiment provides a parking scene detection method based on multi-view and time series fusion, and its core technical features are the following four points.
[0047] 1. The spatial cross-attention mechanism is introduced to fuse the features of multiple fisheye cameras in the bird's-eye view (BEV) space, avoiding the technical problem of loss of features of each fisheye view caused by direct extraction of static splicing inverse perspective transform (IPM) bird's-eye view features in traditional parking scene detection, while also retaining the height information of the features.
[0048] 2. The temporal attention mechanism is introduced to fuse the features of historical frames and achieve temporal alignment, thus avoiding the unstable detection of occluded obstacles and parking spaces in traditional single-frame detection. By considering the vehicle's motion trajectory and historical information, the algorithm can better understand the state changes during parking and provide more reliable detection results.
[0049] 3. The parking space detection head initializes a specific query, uses the cross-attention mechanism to associate the target query with the fusion features of the bird's-eye view (BEV), generates a feature representation corresponding to each query, and finally uses a regression model to predict the center point coordinates of the parking space, the parking space orientation, the vehicle category (vertical, horizontal, diagonal), and the offset of the four corner points relative to the center point. This is different from the traditional parking space detection method that detects corner points, matches them through prior parking space width parameters, and uses template matching to determine the parking space type and orientation. It has better generalization.
[0050] 4. The drivable area detection decodes the bird's-eye view (BEV) features, regresses the semantic map of the surrounding obstacles on the one hand, and regresses the height map of the obstacles on the other hand, and finally obtains high-precision obstacle information with categories and heights. Different from the traditional obstacle detection method of using ultrasonic radar and post-fusion of single fisheye detection results, it avoids tedious post-processing work and uses high-precision laser radar (Lidar) point cloud annotation as true value supervision, so that obstacles have higher distance accuracy under the bird's-eye view (BEV).
[0051] The algorithm input of this embodiment is:
[0052] 1. The input is multiple fisheye cameras, usually 4 fisheye cameras, namely, front surround view, rear surround view, left surround view, and right surround view;
[0053] 2. The top-down view (BEV) feature map after fusion of the previous frame;
[0054] 3. Internal and external parameters of the camera;
[0055] 4. The position and posture of the vehicle.
[0056] The algorithm output of this embodiment is:
[0057] 1. Parking space detection results of the current frame, including whether it is occupied;
[0058] 2. The drivable area result of the current frame;
[0059] 3. The top-down view angle (BEV) feature map of the current frame.
[0060] Figure 1 This is a flow chart of a parking scene detection method based on multi-view and time series fusion. Figure 1 As shown, the main steps of the method in this embodiment are as follows:
[0061] 1. Use convolutional neural network (CNN) to extract features from multiple fisheye views and generate corresponding feature maps.
[0062] 2. Construct a 16m*16m bird’s-eye view (BEV) area and divide it into 320*320 grid points, each with a size of 0.05m*0.05m.
[0063] 3. By introducing the spatial cross attention mechanism, the feature map under the bird's-eye view (BEV) is associated with the features on the fisheye image according to the camera's internal and external parameters. In this way, a more accurate feature representation can be obtained under the bird's-eye view. At the same time, the attention mechanism enables features at different positions to pay attention to and cross-influence each other, thereby improving the detection performance of parking spaces and drivable areas. The specific implementation method is to start from the bird's-eye view (BEV) grid point, sample it at height, and then project it onto the corresponding image feature map through the camera's internal and external parameters as reference points. Then, the image features near these reference points are sampled, and the attention mechanism is used to construct the feature map under the bird's-eye view.
[0064] 4. By introducing temporal information, the top-down perspective (BEV) features of the current frame are fused with the features of the historical frames. The temporal alignment of features is achieved through the temporal self-attention mechanism by using the vehicle's posture and the fused feature map of the previous frame, so that the information of the historical frame can affect the detection results of the current frame. The specific implementation is to transform the position of the sampling points, use the bilinear interpolation method, perform sampling on the historical feature map, and generate a feature map after temporal alignment. This method of temporal fusion can improve the continuity and accuracy of parking scene detection.
[0065] 5. As an option, this embodiment fuses the feature map of the bird's-eye view (BEV) generated by multiple fisheye sensors with the feature maps of the bird's-eye view (BEV) generated by other sensors, such as Lidar or other panoramic cameras, which reflects the scalability of the solution of this embodiment. The advantage of doing so is that the robustness of the system can be further improved. When a sensor is interfered with or cannot provide accurate information, other sensors can provide redundant data to maintain the reliability of the system. At the same time, the information of multiple sensors can be integrated to provide a more comprehensive and accurate environmental perception capability. In particular, for the detection of drivable areas, dense and multimodal features can greatly improve the perception accuracy of this embodiment.
[0066] 6. After obtaining the fused bird's-eye view (BEV) feature map, all parking detection tasks will be performed based on this feature map, including parking space detection and drivable area detection. For these two tasks, all model parameters before this can be shared, so compared with the traditional inverse perspective transformation (IPM) bird's-eye view detection of parking spaces and fisheye view detection of obstacles, it can greatly reduce the consumption of computing resources and improve computing efficiency and real-time performance.
[0067] 7. The main task of the parking space detection head is to initialize a specific query, use the cross-attention mechanism to associate the target query with the fusion features of the bird's-eye view (BEV), generate the feature representation corresponding to each query, and finally use the regression model to predict the center point coordinates, existence probability, parking space orientation, vehicle arrangement category (vertical, horizontal, diagonal), and the offset of the four corner points relative to the center point.
[0068] 8. It is necessary to form a parking space through the regression result of the parking space detection head. In this embodiment, the parking spaces whose center points are greater than a certain threshold (usually 0.5) will be retained. The four regressed corner points are arranged clockwise by default, and the first two corner points are the entrance corner points of the parking space, which is consistent with the standard when making the true value. In this embodiment, it is required that the entrance corner points of the reserved parking spaces must be within the range of the bird's-eye view (BEV), because usually in the process of parking vehicles, the accuracy requirements for the entrance corner points are high. For the other two corner points, if they are both within the range of the bird's-eye view (BEV), then this embodiment can directly obtain the position of the parking space; if one of them is within the range of the bird's-eye view (BEV), this embodiment can infer the position of the other corner point based on the parallelogram criterion; but if neither of them is within the range of the bird's-eye view (BEV), this embodiment needs to calculate the positions of the two corner points through the regressed parking space orientation and the standard size of the parking space category. Finally, this embodiment will use the corner points of the parking space to calculate the orientation of a parking space, and make a judgment based on the regressed parking space type. If the two do not match, for example, the regression type is a vertical parking space, but the calculated angle is greater than 100 degrees, this embodiment will consider the confidence level of this parking space to be low and discard it.
[0069] 9. The method for detecting the drivable area is to perform semantic segmentation on the one hand. The size of the regression result is 320*320*(N+1). The length and width are consistent with the bird's-eye view (BEV) grid. N is the number of obstacle categories defined, and 1 is the background category. The implementation method is to first upsample the bird's-eye view (BEV) feature map, and then obtain the semantic map through the convolutional neural network (CNN). On the other hand, the height of the obstacle is estimated. The regression result is the highest obstacle height on each grid. The size of the regression result is 320*320*1. The implementation method is similar to semantic segmentation. The feature map is upsampled and then regressed using the convolutional neural network (CNN). In this way, the obstacle category and height information on each grid are obtained.
[0070] 10. Through the detection results of the drivable area and the integration of the previous parking space detection results, it is determined whether there are obstacles inside the parking space and whether there are obstacles in the parking path. If so, the parking space is determined to be occupied and cannot be parked.
[0071] Compared with traditional parking space and drivable area detection, the parking scene detection method based on multi-view and time series fusion in this embodiment has the following five advantages.
[0072] 1. Temporal consistency: The introduction of the temporal attention mechanism can fuse the features of historical frames and achieve temporal alignment, thereby improving the continuity and accuracy of detection. By considering the vehicle's motion trajectory and historical information, the algorithm can better understand the state changes during parking and provide more reliable detection results.
[0073] 2. Strong robustness: By introducing the bird's-eye view (BEV) and cross-attention mechanism, the image features of each fisheye view are better integrated, avoiding the loss of each fisheye view feature caused by directly extracting the static splicing inverse perspective transform (IPM) bird's-eye view features. At the same time, when calculating the bird's-eye view (BEV) and the spatial cross-attention mechanism of multiple fisheye views, sampling will be performed at the height of the bird's-eye view (BEV) grid and projected to each fisheye view, without losing the height information of the detected parking space and obstacles.
[0074] 3. High-precision obstacle detection: Use high-precision Lidar point cloud annotation as true value supervision, rely on spatiotemporal fusion feature maps under the bird's-eye view (BEV) to detect and segment obstacles around the vehicle, improve the accuracy of distance and category detection, do not need post-fusion, and abandon ultrasonic radar.
[0075] 4. Improve computing efficiency: Since the neural network inputs of the two tasks are multiple fisheye views and detection is performed on the bird's-eye view (BEV), the parameters of the backbone of the neural network can be shared. Only two different detection heads are required, which can reduce the consumption of computing resources, improve computing efficiency, and improve real-time performance.
[0076] 5. Scalability: Since the detection is performed under the bird's-eye view (BEV), information fusion can be performed with other sensors such as Lidar. This makes the algorithm highly adaptable and scalable under different vehicle and environment settings.
[0077] Those skilled in the art will appreciate that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0078] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and supplements without departing from the principle of the present invention. These improvements and supplements should also be regarded as the scope of protection of the present invention.
Claims
1. A parking scene detection method based on multi-view and time series fusion, characterized in that The following steps are involved: S1. Extract features from multiple fisheye views through a convolutional neural network to generate corresponding feature maps; S2, statically stitching the multiple fisheye views into an inverse perspective transformed bird's-eye view, constructing a top-down perspective rectangular area of a size that can accommodate parking spaces and vehicle drivable areas based on the inverse perspective transformed bird's-eye view, and dividing it into a number of grids; S3. Introduce the spatial cross attention mechanism, associate the top-view rectangular area feature map with the multi-channel fisheye view feature map according to the internal and external parameters of the fisheye camera, and obtain the top-view rectangular area feature map of the current frame; the specific implementation method of the association is to start from each grid center point of the top-view rectangular area, and after sampling in height, project the internal and external parameters of the multi-channel fisheye camera onto the corresponding multi-channel fisheye view feature map as a grid reference point, and then sample the fisheye view feature map near each grid reference point, and use the attention mechanism to construct the top-view rectangular area feature map of the current frame; S4. Introduce the timing information, use the vehicle posture and the previous frame's top-view rectangular area temporal fusion feature map to perform temporal fusion on the feature map of the top-view rectangular area of the current frame through the temporal attention mechanism to achieve temporal alignment of the features, and obtain the current frame's top-view rectangular area temporal fusion feature map, so that the historical frame information can affect the current frame detection result; the specific implementation of temporal fusion is to change the position of the sampling point, use the bilinear interpolation method to sample on the previous frame's top-view rectangular area temporal fusion feature map, and generate the current frame's top-view rectangular area temporal fusion feature map after temporal alignment; S5. Perform parking space detection and drivable area detection based on the temporal fusion feature map of the rectangular area of the top-down perspective of the current frame.
2. The parking scene detection method based on multi-view and time series fusion according to claim 1, characterized in that: The multi-path fisheye includes a front surround fisheye, a rear surround fisheye, a left surround fisheye, and a right surround fisheye.
3. The parking scene detection method based on multi-view and time series fusion according to claim 1, characterized in that: Step S2: The top-down viewing angle rectangular area is a square, which is divided into a number of square grids of equal size.
4. The parking scene detection method based on multi-view and time series fusion according to claim 1, characterized in that: Step S5 specifically includes the following sub-steps: S51, performing parking space detection using a parking space detection head; S52, performing drivable area detection; S53. By integrating the drivable area detection result with the parking space detection result, it is determined whether there is an obstacle inside the parking space and whether there is an obstacle in the parking path. If so, the parking space is determined to be an occupied parking space and cannot be parked in. Otherwise, the parking space is determined to be an unoccupied parking space and parking is initiated.
5. The parking scene detection method based on multi-view and time series fusion according to claim 4, characterized in that: Step S51 specifically includes the following sub-steps: The parking space detection head initializes a specific query, uses a cross-attention mechanism to associate the target query with the current frame top-down perspective rectangular area temporal fusion feature map, generates a feature representation corresponding to each query, and finally uses a regression model to predict the center point coordinates, existence probability, parking space orientation, vehicle arrangement category, and the offset of the four corner points relative to the center point of the parking space; The parking spaces are formed through the regression results of the parking space detection head, and the parking spaces whose center point existence probability is greater than the threshold are retained. The four corner points of the regression are arranged clockwise by default, and the first two corner points are the entrance corner points of the parking spaces.
Citation Information
Patent Citations
A parking space detection method
CN108875911B
An Automated Parking Method Based on Visual and Ultrasonic Perception Fusion
CN110775052B
Parking space detection method and related device
CN114511836A
Semantic map building and positioning method and device for indoor parking lot
CN114863096A
