A parking space information determination method, device, apparatus and vehicle

Abstract

This application provides a method and apparatus for determining parking space information; including: obtaining a bird's-eye view, ultrasonic radar data, inter-frame pose change data, and a parking space time-series observation queue; when the existence of the same physical parking space is determined, determining the parking space occupancy rate of the current frame based on the ultrasonic radar data; obtaining the projection position of the parking space center in the previous frame in the bird's-eye view based on the inter-frame pose change data; determining a target local area centered on the projection position, and performing parking space detection on the bird's-eye view and the target local area respectively to obtain first and second parking space information; when the matching condition of the same physical parking space is met, adding the second parking space information and the parking space occupancy rate of the current frame as the parking space observation information of the current frame to the parking space time-series observation queue; performing multi-dimensional analysis on the physical parking spaces of the current frame to obtain the comprehensive confidence level of the current frame, updating the parking space observation information of the current frame based on the comprehensive confidence level of the current frame and the parking space occupancy rate of the current frame, and outputting the parking space observation information of the current frame based on the comprehensive confidence level of the current frame.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology for vehicles, and in particular to a method, device, equipment, and vehicle for determining parking space information. Background Technology

[0002] In Automatic Parking Assist (APA) and Low-Speed ​​Driving Automation (LDL) systems, parking space perception is the prerequisite and foundation for realizing parking path planning and vehicle control, and its accuracy directly affects the system's availability and user experience.

[0003] Parking space detection technology in related fields mainly relies on an in-vehicle surround-view monitor (SVM) and an ultrasonic sensor (USS), which typically operate independently or undergo preliminary data fusion. Specifically, the SVM acquires images of the vehicle's surrounding environment using a fisheye camera, and combines image distortion correction, perspective transformation, and deep learning models to detect parking space markings, boundary points, and potential parking space areas. The ultrasonic sensor analyzes echo time and intensity to determine the distance and presence of obstacles, assisting in identifying the boundaries of vertical or lateral parking spaces. However, this approach uses single-frame data for detection and output, meaning each frame of sensor data is processed independently to directly generate parking space information such as vertex coordinates, width, depth, and occupancy status, which is then uploaded to the decision module in real time. Consequently, the parking spaces identified using this method often suffer from insufficient accuracy. Summary of the Invention

[0004] One objective of this application is to provide a method for determining parking space information to solve the problem of inaccurate parking space detection in related technologies; another objective is to provide a device for determining parking space information; a third objective is to provide a device for determining parking space information; a fourth objective is to provide a vehicle; a fifth objective is to provide a computer-readable storage medium; and a sixth objective is to provide a computer program product.

[0005] To achieve the above objectives, the technical solution of this application embodiment is implemented as follows: This application provides a method for determining parking space information, the method including: The system obtains a bird's-eye view of the vehicle scene image, ultrasonic radar data, inter-frame pose change data, and a parking space time-series observation queue. The parking space time-series observation queue includes multiple consecutive frames of parking space observation information for the same physical parking space. Each frame of parking space observation information includes the parking space corner coordinates, parking space occupancy rate, and comprehensive confidence level. The inter-frame pose change data is calculated from the vehicle motion information. When the existence of the same physical parking space is determined based on ultrasonic radar data and bird's-eye view, the occupancy rate of the physical parking space in the current frame is determined based on the ultrasonic radar data. Based on the inter-frame pose change data, the coordinates of the center of the physical parking space in the previous frame in the parking space time sequence observation queue are projected onto the current frame's vehicle coordinate system to obtain the projected position of the parking space center in the bird's-eye view. Centered on the projection position, a target local area of ​​the bird's-eye view is determined according to a preset window size. Parking space detection is performed on the bird's-eye view and the target local area respectively to obtain the first parking space information and the second parking space information. Among them, the parking space information includes: the coordinates of the corner point of the parking space. When the first parking space information and the second parking space information meet the same physical parking space matching condition, the second parking space information and the current frame parking space occupancy rate are used as the current frame parking space observation information and added to the parking space time sequence observation queue. Based on the parking space time-series observation queue, the physical parking spaces of the current frame are analyzed in multiple dimensions to obtain the comprehensive confidence of the current frame. Based on the comprehensive confidence of the current frame and the parking space occupancy rate of the current frame, the parking space observation information of the current frame in the parking space time-series observation queue is updated, and the parking space observation information of the current frame is output based on the comprehensive confidence of the current frame.

[0006] Based on the aforementioned technical means, robust perception and dynamic updating of physical parking spaces are achieved by fusing bird's-eye view of vehicle scene images, ultrasonic radar data, inter-frame pose changes, and parking space temporal observation queues. When ultrasonic radar and bird's-eye view jointly confirm the same physical parking space, the radar data is used to determine the parking space occupancy rate of the current frame, and the historical parking space center is projected onto the current frame using inter-frame pose changes to guide local area re-detection to obtain second parking space information. This second parking space information is then matched with the first parking space information detected globally, thereby adding reliable parking space observation information to the temporal queue. On this basis, a multi-dimensional analysis of parking space geometry, temporal stability, and state consistency is performed based on the multi-frame temporal queue to obtain a comprehensive confidence level, which is used to update the current frame observation information and output decisions. In this way, the advantages of ultrasonic radar in detecting non-metallic obstacles and the ability of visual bird's-eye view to accurately identify parking space structures are fully integrated. By using temporal projection and local re-detection, the missed detections and false detections in global detection are effectively reduced. The introduction of matching conditions and confidence evaluation mechanisms avoids erroneous associations caused by single-frame noise or temporary occlusion, thus improving the accuracy of parking space association. By using comprehensive confidence to dynamically update the temporal queue, the consistency and stability of the output parking space information are ensured, significantly enhancing the perception robustness, detection reliability and user experience of the automatic parking system in complex dynamic environments.

[0007] This application provides a parking space information determination device, the device comprising: The acquisition module is used to acquire the bird's-eye view, ultrasonic radar data, inter-frame pose change data and parking space time-series observation queue corresponding to the scene image of the vehicle; wherein, the parking space time-series observation queue includes multiple consecutive frames of parking space observation information for the same physical parking space, and each frame of parking space observation information includes parking space corner coordinates, parking space occupancy rate and comprehensive confidence level, and the inter-frame pose change data is calculated from the vehicle motion information. The determination module is used to determine the current frame parking space occupancy rate of a physical parking space based on ultrasonic radar data when the existence of the same physical parking space is determined based on ultrasonic radar data and bird's-eye view. The processing module is used to project the coordinates of the center of the physical parking space in the previous frame to the current frame's vehicle coordinate system based on the inter-frame pose change data, so as to obtain the projection position of the center of the parking space in the previous frame in the bird's-eye view. The processing module is also used to determine the target local area of ​​the bird's-eye view with the projection position as the center and according to the preset window size, and to perform parking space detection on the bird's-eye view and the target local area respectively to obtain the first parking space information and the second parking space information; wherein, the parking space information includes: the coordinates of the corner point of the parking space; The processing module is also used to add the second parking space information and the current frame parking space occupancy rate as the current frame parking space observation information and add them to the parking space time sequence observation queue when the first parking space information and the second parking space information meet the same physical parking space matching condition. The processing module is also used to perform multi-dimensional analysis of the physical parking spaces in the current frame based on the parking space time-series observation queue, obtain the comprehensive confidence level of the current frame, and update the parking space observation information in the current frame in the parking space time-series observation queue according to the comprehensive confidence level of the current frame and the parking space occupancy rate of the current frame. The output module is used to output the parking space observation information of the current frame based on the comprehensive confidence level of the current frame.

[0008] This application provides a parking space information determination device, including: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the parking space information determination method provided in the embodiments of this application.

[0009] This application provides a vehicle that includes the parking space information determination device described above.

[0010] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the parking space information determination method provided in this application.

[0011] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the parking space information determination method provided in this application. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the parking space information determination method provided in the embodiments of this application. Figure 1 ; Figure 2 This is a flowchart illustrating the parking space information determination method provided in the embodiments of this application. Figure 2 ; Figure 3 This is a flowchart illustrating the parking space information determination method provided in the embodiments of this application. Figure 3 ; Figure 4 This is a flowchart illustrating the parking space identification and closed-loop correction method based on time-series re-sensing provided in the embodiments of this application; Figure 5 This is a block diagram of multi-source sensing data preprocessing and initial parking space detection provided in an embodiment of this application; Figure 6 This is a schematic diagram of timing re-sensing and cross-frame association of parking spaces provided in an embodiment of this application; Figure 7 This is a schematic diagram of the multimodal consistency measurement and confidence calculation process provided in the embodiments of this application; Figure 8 This is a block diagram illustrating the principle of perception closed-loop correction and iterative update provided in the embodiments of this application; Figure 9 This is a schematic diagram of the parking space information determination device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the parking space information determination device provided in the embodiments of this application; Figure 11 This is a schematic diagram of the vehicle structure provided in the embodiments of this application.

[0013] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] This application provides a method for determining parking space information, applied to a parking space information determining device. This method enables accurate detection of parking spaces in a parking scenario. The parking space information determining device may include mobile phones, tablets, laptops, handheld computers, in-vehicle electronic devices, wearable devices, and ultra-mobile personal computers (Ultra). Mobile personal computers (UMPCs), netbooks or personal digital assistants (PDAs), servers, network attached storage (NAS), and personal computers (PCs), etc. (See reference) Figure 1 As shown, Figure 1 A schematic diagram of the implementation process of a parking space information determination method provided in this application embodiment. Figure 1 The method may include the following steps: Step 101: Obtain the bird's-eye view, ultrasonic radar data, inter-frame pose change data, and parking space time sequence observation queue corresponding to the scene image of the vehicle.

[0016] The parking space time-series observation queue includes multiple consecutive frames of parking space observation information for the same physical parking space. Each frame of parking space observation information includes the coordinates of the parking space corner point, the parking space occupancy rate, and the overall confidence level. The inter-frame pose change data is calculated from the vehicle motion information.

[0017] In practical applications, vehicles are equipped with multi-channel surround-view fisheye cameras, multi-channel ultrasonic radars, and six-axis inertial measurement units (IMUs).

[0018] In this embodiment, the scene image can be multiple fisheye images obtained by taking pictures of the environment around the vehicle using a multi-channel surround-view fisheye camera mounted on the vehicle; the bird's-eye view can be a panoramic top-down view image centered on the vehicle obtained by using the camera parameters of the surround-view fisheye camera to perform distortion correction and inverse perspective transformation on multiple fisheye images. The size of the bird's-eye view can be 100 pixels (px) × 100px, and the bird's-eye view can cover the actual scene of a preset area around the vehicle, such as an area of ​​10 meters (m) × 10m.

[0019] In this embodiment of the application, ultrasonic radar data can be obtained in the following way: multiple ultrasonic radars mounted on the vehicle emit ultrasonic waves, and convert the echo signals into distance, thereby generating raw point cloud data. Furthermore, the raw point cloud data is processed and analyzed to identify obstacle distance and obstacle attributes.

[0020] In this embodiment, the ultrasonic radar data can be obtained by performing a series of data processing steps on the raw point cloud data detected by lidar, including but not limited to point cloud denoising, filtering, clustering, grid division, feature extraction, obstacle recognition and tracking, etc. Through these processing steps, useful information is extracted from the complex point cloud data to obtain the ultrasonic radar data.

[0021] In this embodiment, the inter-frame pose change data is calculated from the vehicle's motion information. This motion information includes, but is not limited to, one or more of the vehicle's speed, yaw rate, and sideslip angle, which can be measured or estimated by wheel speed sensors, IMU, and integrated inertial navigation systems, respectively. Dead reckoning (DR) calculates the pose change of each new frame relative to the starting position based on the vehicle's initial position and motion information. This includes, but is not limited to, changes in the vehicle's ordinate, abscissa, and yaw angle relative to the starting point (the starting point being the vehicle's position in the first frame of the stored N historical frames). Finally, the pose change relationship between frames is obtained.

[0022] In this embodiment, the scene image, the raw point cloud data acquired by the ultrasonic radar, and the vehicle's motion information can also be timestamped. This alignment can be understood as controlling the time synchronization error within a target time, such as 20 milliseconds (ms), thus ensuring the time consistency of multi-source data. It should be noted that the scene image, the raw point cloud data acquired by the ultrasonic radar, and the vehicle's motion information are all data from the current frame.

[0023] In this embodiment, the parking space time-series observation queue stores continuous multi-frame parking space observation information for each of multiple physical parking spaces. Each frame of parking space observation information includes the corner coordinates of the parking space, the parking space occupancy rate, and the overall confidence level for that physical parking space in the corresponding frame. The overall confidence level can be determined by the geometric consistency, temporal stability, and state consistency of the physical parking space across multiple consecutive frames. It should be noted that the corner coordinates of the parking space can be the physical coordinates of the parking space corner in the vehicle coordinate system; where the vehicle coordinate system is a coordinate system formed with the rear axle center of the vehicle as the origin, the longitudinal direction of the vehicle body as the X-axis, the front direction of the vehicle as the positive X-axis direction, the lateral direction of the vehicle body as the Y-axis, and the leftward direction as the positive Y-axis direction.

[0024] In this embodiment of the application, the parking space time-series observation queue can store parking space observation information for each physical parking space for a fixed number of N frames, such as the parking space time-series observation queue. For example, N=8, where each queue element It is a data structure containing multi-dimensional information; of course, the parking space time-series observation queue can store parking space observation information for all consecutive frames of each physical parking space, and this application does not impose specific restrictions on this.

[0025] Step 102: When it is determined that there is a physical parking space based on ultrasonic radar data and bird's-eye view, determine the current frame parking space occupancy rate of the physical parking space based on ultrasonic radar data.

[0026] In this embodiment of the application, determining the existence of the same physical parking space based on ultrasonic radar data and a bird's-eye view can be achieved through the following process: performing connected region detection on the ultrasonic radar data to obtain the radar parking space area; determining the visual parking space area based on the parking space corner coordinates in the first parking space information; calculating the overlap rate and overlapping area by using an overlap formula on the visual parking space area and the radar parking space area; if the overlap rate is greater than the overlap rate threshold and the overlapping area satisfies the area constraint condition, it is determined that the radar parking space area and the visual parking space area exist in the same physical parking space.

[0027] In this embodiment, after obtaining ultrasonic radar data and a bird's-eye view, parking space detection is performed on the bird's-eye view to obtain the first parking space information. Specifically, this includes: performing semantic segmentation of parking space lines on the bird's-eye view; extracting parking space line pixels using a deep learning-based segmentation network; detecting straight line segments in the parking space line pixels using Hough transform; locating four parking space corner points on the straight line segment pixels using features such as the intersection of two parking space lines; and converting the pixel coordinates of the four parking space corner points in the image coordinate system into physical coordinates in the vehicle coordinate system based on the bird's-eye view and its corresponding coverage area. The first parking space information is obtained by outputting the information. Then, the quadrilateral of the parking space is fitted based on the physical coordinates of the four corner points of the parking space to obtain the visual parking space area.

[0028] In this embodiment of the application, connected component detection is performed on ultrasonic radar data to obtain radar parking space areas, including: extracting idle areas from ultrasonic radar data, determining and outputting radar parking space areas through connected component analysis and clustering.

[0029] In this embodiment of the application, the regional constraints include the width of the overlapping region being in the width range such as 2.0-3.0m, and the depth of the overlapping region being in the depth range such as 4.5-6.0m.

[0030] In this embodiment, after obtaining the visual parking space area and the radar parking space area, the overlapping area and overlap rate between the visual parking space area and the radar parking space area are calculated. If the overlap rate is greater than the overlap rate threshold, and the width of the overlapping area is within the width range and the depth is within the depth range, it is determined that the overlapping area meets the area constraint conditions. At this time, it can be determined that the radar parking space area and the visual parking space area are the same physical parking space, and the initial data fusion is completed. In this way, by fusing the connected region detection of ultrasonic radar and the parking space corner information of visual bird's-eye view, and based on the dual judgment of overlap rate calculation and area constraint conditions, the accuracy and robustness of parking space detection can be effectively improved, and the false alarm rate caused by single sensor noise or environmental interference can be significantly reduced, thereby providing more reliable consistency information of the same parking space for automatic parking or low-speed autonomous driving systems.

[0031] In this embodiment, determining the current frame occupancy rate of a physical parking space based on ultrasonic radar data includes: extracting idle areas from the ultrasonic radar data, and determining the current frame occupancy rate of the physical parking space based on the height and / or position attributes of obstacles in the idle areas. Specifically, if an obstacle has a low attribute (e.g., obstacle height is less than a height threshold, such as 5cm), the current frame occupancy rate of the physical parking space is determined to be 0, and it needs to be retained; if an obstacle has a high attribute (e.g., obstacle height is greater than or equal to a height threshold), and the obstacle is located in the first three-quarters of the physical parking space, the current frame occupancy rate of the physical parking space is determined to be 1, and it needs to be discarded. If an obstacle has a high attribute, and the obstacle is located in the last quarter of the physical parking space, the obstacle height and obstacle position are weighted using a height correction coefficient and a position correction coefficient to obtain the current frame occupancy rate of the physical parking space. Thus, by determining the parking space occupancy rate of the current frame based on the height and / or position attributes of obstacles in the vacant area, a refined and continuous assessment of the parking space occupancy status can be achieved. This not only overcomes the limitation of the traditional binary judgment of vacancy / occupancy in failing to reflect partial occupancy (such as ground locks and small obstacles), but also effectively eliminates misjudgments of non-interfering objects such as low curbs by using height attributes, and distinguishes between edge clutter and core obstruction areas by combining position attributes, significantly improving the accuracy and robustness of parking space occupancy detection.

[0032] Step 103: Based on the inter-frame pose change data, project the coordinates of the center of the physical parking space in the previous frame in the parking space time sequence observation queue onto the current frame's vehicle coordinate system to obtain the projection position of the parking space center in the bird's-eye view in the previous frame.

[0033] In this embodiment of the application, the center coordinates of the parking space in the previous frame can be determined based on the coordinates of the four corner points of the parking space in the previous frame of the parking space observation information before the current frame of the physical parking space in the parking space time sequence observation queue.

[0034] In this embodiment of the application, during the vehicle's driving process, the vehicle's position moves in the world coordinate system. By using the inter-frame pose change relationship between consecutive frames, the coordinates of the parking space center in the previous frame are projected onto the vehicle's coordinate system in the current frame to obtain the projection coordinates of the parking space center in the actual scene corresponding to the bird's-eye view. Based on the correspondence between the bird's-eye view and the corresponding actual scene, the projection position of the parking space center in the bird's-eye view in the current frame is obtained.

[0035] Here, the projection position can be obtained by the following formula (1).

[0036]

[0037] in, This represents the projected coordinates of the parking space center in the vehicle coordinate system of the previous frame (i.e., time t-1) in the current frame (i.e., time t). This contains the vehicle's pose transformation data from time t-1 to time t. The coordinates of the center of the physical parking space in the actual scene in the vehicle coordinate system in the previous frame (i.e., time t-1).

[0038] Step 104: Using the projection position as the center, determine the target local area of ​​the bird's-eye view according to the preset window size, and perform parking space detection on the bird's-eye view and the target local area respectively to obtain the first parking space information and the second parking space information.

[0039] The parking space information includes the coordinates of the parking space corner points.

[0040] In this embodiment, after determining the projection position, the bird's-eye view is divided into target local areas centered on the projection position and according to a preset window size. Parking space detection is then performed on both the bird's-eye view and the target local area to obtain first parking space information and second parking space information. It should be noted that the corner coordinates of the parking spaces in the second parking space information are physical coordinates in the vehicle coordinate system. Furthermore, the method for detecting parking spaces in the target local area of ​​the bird's-eye view is similar to the method for detecting parking spaces in the bird's-eye view itself, and therefore will not be repeated here.

[0041] Step 105: When the first parking space information and the second parking space information meet the same physical parking space matching condition, the second parking space information and the current frame parking space occupancy rate are used as the current frame parking space observation information and added to the parking space time sequence observation queue.

[0042] In this embodiment, the matching condition of the first parking space information and the second parking space information being the same physical parking space can be achieved through the following process: connecting the corner points of the entrance parking spaces in the first parking space information to obtain a first direction vector, and connecting the corner points of the entrance parking spaces in the second parking space information to obtain a second direction vector; determining the first parking space angle formed between the first direction vector and the second direction vector; determining the first parking space distance based on the coordinates of each corner point of the parking spaces in the first parking space information and the coordinates of the corresponding corner points of the parking spaces in the second parking space information, thereby obtaining multiple first parking space distances; if the multiple first parking space distances are less than a parking space distance threshold, and the first parking space angle is less than a parking space angle threshold, it is determined that the first parking space information and the second parking space information are matching the same physical parking space.

[0043] In this embodiment, based on the coordinates of each parking space corner point in the first parking space information, two first entrance parking space corner points of the physical parking space are determined; and based on the coordinates of each parking space corner point in the second parking space information, two second entrance parking space corner points of the physical parking space are determined. A first direction vector is obtained by connecting the two first entrance parking space corner points, and a second direction vector is obtained by connecting the two second entrance parking space corner points in a corresponding manner. A first parking space angle is determined between the first and second direction vectors. The coordinates of each parking space corner point in the first parking space information and the corresponding parking space corner point in the second parking space information are calculated to determine the first parking space distance, thus obtaining four first parking space distances. If all four first parking space distances are less than a parking space distance threshold, and the first parking space angle is less than a parking space angle threshold, it is determined that the first and second parking space information meet the same physical parking space matching condition. At this time, the second parking space information and the current frame parking space occupancy rate can be used as the current frame parking space observation information and added to the parking space time sequence observation queue. In this way, the orientation vector of the entrance corner point accurately represents the posture consistency of the parking space, while the distance constraints of multiple corresponding corner points are used to comprehensively measure the degree of overlap of the geometric structure of the parking space, thereby achieving highly reliable matching of the same physical parking space while avoiding misjudgment of a single feature.

[0044] Step 106: Based on the parking space time-series observation queue, perform multi-dimensional analysis on the physical parking spaces of the current frame to obtain the comprehensive confidence level of the current frame. Then, based on the comprehensive confidence level of the current frame and the parking space occupancy rate of the current frame, update the parking space observation information of the current frame in the parking space time-series observation queue. Finally, based on the comprehensive confidence level of the current frame, output the parking space observation information of the current frame.

[0045] In this embodiment, after adding the second parking space information and the current frame parking space occupancy rate as the current frame parking space observation information to the parking space temporal observation queue, the physical parking spaces of the current frame are analyzed from multiple dimensions, including geometry, temporal sequence, and state, based on the parking space temporal observation queue to calculate the current frame comprehensive confidence score. Then, this comprehensive confidence score and the current frame parking space occupancy rate are used to jointly update the current frame observation information in the parking space temporal observation queue. Based on the relationship between the comprehensive confidence score and the confidence score threshold, the current frame parking space observation information is selectively output. Thus, by introducing a temporal correlation and confidence score evaluation mechanism, false detections caused by single-frame sensor noise or temporary occlusion are effectively suppressed, improving the reliability and output stability of parking space detection. Simultaneously, by combining the occupancy rate update queue, historical and current frame information are smoothly correlated, avoiding parking space jumps or loss, and enhancing the system's robustness in complex scenarios. Finally, the high-confidence parking space information output can provide more accurate and continuous decision-making basis for subsequent path planning and control, significantly improving the success rate and user experience of the automatic parking system.

[0046] In some embodiments, each frame of parking space observation information further includes: a valid frame flag; and before performing multi-dimensional analysis of the physical parking spaces in the current frame based on the parking space time-series observation queue to obtain the comprehensive confidence level of the current frame, the following steps may also be performed: Based on the valid frame flag, valid parking space observation information is filtered out from the parking space time-series observation queue. This allows for multi-dimensional analysis of the physical parking spaces in the current frame, thereby improving the accuracy of parking space information.

[0047] In some embodiments, step 106 involves performing multi-dimensional analysis of the physical parking spaces in the current frame based on the parking space time-series observation queue to obtain the comprehensive confidence level of the current frame. Figure 2 illustrate.

[0048] Step 161: Based on the parking space corner coordinates in the parking space time-series observation queue, determine the multi-frame parking space center coordinates, multi-frame parking space center offset, and multi-frame parking space angle for the same physical parking space.

[0049] In this embodiment of the application, the same physical parking space can have multiple frames of parking space angles. Each frame of parking space angle can be based on the parking space angle formed by the line connecting the entrance parking space corner points of the previous frame and the line connecting the entrance parking space corner points of the current frame.

[0050] In this embodiment of the application, the same physical parking space can have multiple frames of parking space center coordinates. The center of each parking space can be obtained based on the corner coordinates of the parking space in the current frame. That is, the center coordinates of each parking space can be obtained by the following formula (2).

[0051]

[0052] in, and Let K be the coordinates of the parking space center in the kth frame. , , and These are the coordinates of the four parking space corner points in the k-th frame of parking space observation information, which can be used as follows: This represents the set of parking space corner coordinates in the parking space observation information of the k-th frame.

[0053] In this embodiment of the application, the same physical parking space can have multiple frames of parking space center offset. The parking space center offset of each frame can be obtained based on the parking space center coordinates of the previous frame and the current frame, i.e., adjacent frames. That is, the parking space center offset of each frame can be obtained by the following formula (3).

[0054]

[0055] in, This represents the offset of the parking space center in the k-th frame. and All coordinates are the center coordinates of the parking space in the (k-1)th frame.

[0056] Step 162: Based on the center coordinates of parking spaces in multiple frames, analyze the geometric consistency of physical parking spaces to obtain the confidence level of geometric consistency of physical parking spaces.

[0057] In this embodiment, the geometric consistency confidence score is used to characterize the stability of the parking space's geometric position in the spatial dimension. It should be noted that a higher geometric consistency confidence score indicates that the observed parking space center positions in each frame are more concentrated and fluctuate less, and the repeatability and consistency of the parking space's geometric structure are better; conversely, a lower score indicates that there are large positional jumps or noise interference.

[0058] In this embodiment, the geometric consistency of physical parking spaces is analyzed based on the center coordinates of multiple parking spaces to obtain the geometric consistency confidence level of the physical parking spaces. This can be achieved through the following process: obtaining the mean center coordinates of the physical parking spaces from the center coordinates of multiple parking spaces; calculating the distance deviation between the center coordinates of each parking space and the mean center coordinates of the parking spaces using the Euclidean distance formula to obtain multiple distance deviations; calculating the mean square deviation of the multiple distance deviations and normalizing the mean square deviations to obtain the geometric consistency confidence level.

[0059] In this embodiment of the application, the average center coordinate of the physical parking space can be obtained by the following formula (4).

[0060]

[0061] in, and All represent the average center coordinates of the parking spaces, and K is the number of valid frames.

[0062] In this embodiment of the application, the distance deviation between the center coordinates of the parking space in each frame and the average center coordinates of the parking space can be obtained by the following formula (5).

[0063]

[0064] in, denoted as the distance deviation in the k-th frame.

[0065] In this embodiment of the application, the mean square deviation of multiple distance deviations can be calculated by the following formula (6).

[0066]

[0067] in, This represents the mean square deviation.

[0068] In this embodiment of the application, the geometric consistency confidence level can be obtained by the following formula (7).

[0069]

[0070] in, For geometric consistency confidence, This is the geometric deviation threshold, used for normalization; generally... The value is 0.15m because when the root mean square of the deviation of the parking space center in each town reaches 15 centimeters, The value is approximately 0.368, falling into the low confidence interval. The closer Cg is to 1, the better the geometric consistency.

[0071] As described above, this embodiment of the application effectively quantifies the stability of parking space positions across multiple frames by statistically modeling the mean and deviation of parking space center coordinates in time-series observations. Using the mean squared deviation and normalization, an intuitive and continuous consistency index is obtained, reflecting the geometric reliability of the current parking space detection result. This method has a strong suppression effect on abnormal jumps or noise interference in single frames, helping to screen out parking space observations with small positional fluctuations and geometric stability. This provides a reliable basis for subsequent parking space association, fusion confidence assessment, and time-series output, improving the robustness and output accuracy of the automatic parking system in dynamic or complex environments.

[0072] Step 163: Based on the multi-frame parking space center offset and multi-frame parking space angle, analyze the temporal stability of the physical parking space and obtain the confidence level of the temporal stability of the physical parking space.

[0073] In this embodiment, the temporal stability confidence score is used to characterize the smoothness of parking space parameters over time. It should be noted that a higher temporal stability confidence score indicates that the parking space moves less and its angle swings less over a continuous period of time, the parking space status changes more smoothly, and the stability is stronger; conversely, a lower score indicates that there is significant temporal jitter or abnormal jumps, which may be caused by sensor noise, vehicle movement, or temporary obstruction.

[0074] In this embodiment, the temporal stability of physical parking spaces is analyzed based on multi-frame parking space center offset and multi-frame parking space angle to obtain the temporal stability confidence level of physical parking spaces. This can be achieved through the following process: averaging the multi-frame parking space center offset to obtain the average parking space center offset; averaging the multi-frame parking space angle to obtain the average parking space angle; calculating the angle deviation between the parking space angle of each frame and the average parking space angle to obtain multiple angle deviations; determining the target angle deviation based on the multiple angle deviations; and determining the temporal stability confidence level based on the target angle deviation and the average parking space center offset.

[0075] In this embodiment of the application, the average parking space center offset can be obtained by the following formula (8).

[0076]

[0077] in, This represents the average offset of the parking space center.

[0078] In this embodiment of the application, the average parking space angle can be obtained by the following formula (9).

[0079]

[0080] in, This represents the average parking space angle. The angle of the parking space is shown in the k-th frame.

[0081] In this embodiment of the application, the angle deviation can be obtained by the following formula (10).

[0082]

[0083] in, denoted as the angle deviation in the k-th frame.

[0084] In this embodiment of the application, the target angle deviation can be the maximum deviation among multiple angle deviations, which can be obtained by formula (11), or it can be the average of multiple angle deviations, which can be obtained by formula (12).

[0085]

[0086]

[0087] in, This refers to the target angle deviation.

[0088] In this embodiment of the application, the timing stability confidence can be obtained by the following formula (13).

[0089]

[0090] in, The confidence level for time series stability. The angle threshold can be 5°. The displacement threshold can be 0.1m. The closer it is to 1, the better the timing stability.

[0091] As described above, the embodiments of this application calculate the average center offset and the average parking space angle, and determine the target deviation (such as the maximum deviation or the average deviation) based on the angle deviation. The average center offset is used to jointly construct the temporal stability confidence, which can comprehensively quantify the temporal fluctuation of the parking space in terms of both position and attitude. This confidence reflects the stability and consistency of the parking space observation results over a continuous period of time, and can effectively identify and suppress abnormal jumps caused by sensor noise, vehicle vibration or temporary obstruction.

[0092] Step 164: Based on the current frame parking space occupancy rate in the parking space time-series observation queue, analyze the state consistency of physical parking spaces and obtain the state consistency confidence of physical parking spaces.

[0093] In this embodiment of the application, the state consistency confidence level is used to characterize the degree of certainty of the parking space occupancy status in the current frame.

[0094] In this embodiment of the application, the state consistency of physical parking spaces is analyzed based on the current frame parking space occupancy rate in the parking space time-series observation queue to obtain the state consistency confidence of physical parking spaces. This can be achieved through the following process: calculate the information entropy of physical parking spaces based on the current frame parking space occupancy rate in the parking space time-series observation queue; and determine the state consistency confidence based on the information entropy.

[0095] In this embodiment of the application, the information entropy of the physical parking space can be obtained by the following formula (14).

[0096]

[0097] in, Let be the information entropy of the physical parking spaces in the current frame, i.e., at time t. This represents the parking space occupancy rate at time t in the current frame.

[0098] In this embodiment of the application, the state consistency confidence can be obtained by the following formula (15).

[0099]

[0100] in, The confidence level for state consistency. The maximum entropy of a binary distribution (when (When), used to normalize to the [0,1] interval. Of course, if Having been normalized to the [0,1] interval, the above formula (15) can be simplified to the following formula (16). The closer it is to 1, the more certain the state judgment is (St is close to 0 or 1), and the better the consistency.

[0101]

[0102] As described above, in this embodiment, by calculating the information entropy and determining the state consistency confidence level based on the parking space occupancy rate of the current frame, the inherent determinism of the parking space occupancy status can be quantified in real time and on a single frame. When the occupancy rate is close to 0 or 1 (i.e., clearly vacant or clearly occupied), the entropy value approaches 0, the state consistency confidence level is high, and the current observation result is clear and reliable. When the occupancy rate is at an intermediate value (e.g., 0.5, indicating partial occupancy or an ambiguous state), the entropy value is the largest, the confidence level is low, and the system can identify that there is significant ambiguity in the parking space status of that frame. Thus, there is no need for accumulation across multiple frames, the response speed is fast, and it is suitable for immediate evaluation of the reliability of instantaneous states. It can effectively assist the automatic parking system in determining whether to adopt the occupancy result, whether to trigger re-detection, or to fuse other sensor information in a single frame decision, thereby improving the system's flexibility and robustness in dynamic scenarios.

[0103] Step 165: Weight the geometric consistency confidence, temporal stability confidence, and state consistency confidence to obtain the overall confidence of the current frame.

[0104] In this embodiment, weights are assigned to the confidence scores of each dimension based on their importance. Specifically, geometric accuracy, the core indicator for parking space detection, has the highest weight (e.g., 0.4); temporal stability directly affects the parking experience, with a slightly higher weight (e.g., 0.35 or 0.3); and state judgment directly affects parking decisions, with a moderate weight (e.g., 0.25 or 0.3). Furthermore, the geometric consistency confidence score, temporal stability confidence score, and state consistency confidence score are weighted using the weights corresponding to the confidence scores of each dimension to obtain the overall confidence score of the current frame.

[0105] As described above, in this embodiment, based on the multi-frame parking space center coordinates, center offset, parking space angle, and parking space occupancy rate in the parking space time-series observation queue, geometric consistency confidence, temporal stability confidence, and state consistency confidence are extracted respectively, and then weighted and fused to obtain the comprehensive confidence of the current frame. In this way, the reliability of parking space observation is comprehensively quantified from three dimensions: geometric position, temporal change, and state predictability, overcoming the one-sidedness of single-frame or single-index evaluation. Geometric consistency confidence suppresses position jump noise, temporal stability confidence reflects the stability of parking space posture and offset, and state consistency confidence characterizes the regularity of occupancy status. The weighted fusion of these three can adaptively highlight high-reliability dimensions and suppress low-reliability dimensions, thus obtaining a comprehensive and robust confidence index. This comprehensive confidence can effectively guide subsequent parking space information updates and output decisions, significantly improving the detection accuracy, anti-interference ability, and overall robustness of the automatic parking system in complex dynamic environments, providing users with a safer and more reliable parking experience.

[0106] In some embodiments, the process of updating the current frame parking space observation information in the parking space time-series observation queue based on the current frame comprehensive confidence level and the current frame parking space occupancy rate in step 106 is combined with... Figure 3 Please provide an explanation.

[0107] Step 166: Obtain the first N frames of parking space observation information and the filtering coefficients.

[0108] In this embodiment, the filtering coefficient is used to control the weight of the current frame parking space observation information and the historical frame parking space observation information. The filtering coefficient can be determined based on the degree of trust in the current frame parking space observation information. If the current frame parking space observation information is highly trusted, the filtering coefficient can be 0.95, which results in a fast response but is easily affected by noise. If the current frame parking space observation information is trusted to a certain extent, while taking into account the historical frame parking space observation information, the filtering coefficient can be 0.7, which balances response speed and stability. If the current frame parking space observation information is not trusted and the historical frame parking space observation information is highly trusted, the filtering coefficient can be 0, which results in extreme stability but a sluggish response. It should be noted that in parking scenarios, the change in parking space occupancy status is usually gradual, that is, it takes time for a vehicle to enter / exit. Therefore, the filtering coefficient can be 0.7, which allows the filter to respond to the real state change within the target frame, such as 12 frames, while having sufficient suppression capability for instantaneous interference in a single frame (such as a pedestrian passing by for 12 seconds).

[0109] Step 167: Based on the comprehensive confidence level of the current frame and the parking space observation information of the previous N frames, update the coordinates of the parking space corner points in the parking space observation information of the current frame.

[0110] In this embodiment of the application, the coordinates of the corner points of the parking spaces in the current frame parking space observation information can be updated by the following formula (17).

[0111]

[0112] in, This is the set of parking space corner coordinates updated at time t in the current frame. Let k be the overall confidence level of the k-th frame. Let N be the set of parking space corner coordinates from the parking space observation information in the k-th frame, and N be the sliding window length. It should be noted that there are N+1 frames from tN to t (including the current frame). It should also be noted that the observation results of high-confidence frames have a larger weight in the weighted average, effectively suppressing noise interference from low-confidence frames.

[0113] Step 168: Update the parking space occupancy rate of the current frame based on the filtering coefficients and the parking space occupancy rate of the previous frame.

[0114] In this embodiment of the application, the current frame parking space occupancy rate can be updated by the following formula (18).

[0115]

[0116] in, This represents the parking space occupancy rate updated in the current frame, i.e., at time t. These are the filter coefficients. This represents the parking space occupancy rate at time t-1 of the previous frame. This filter can effectively suppress state jitter and improve the stability of state judgment.

[0117] As described above, in this embodiment, the parking space corner coordinates are updated based on the parking space observation information and filtering coefficients of the previous N frames, combined with the comprehensive confidence of the current frame. Simultaneously, the occupancy rate of the current frame is smoothly updated using the filtering coefficients and the parking space occupancy rate of the previous frame. Thus, by introducing a temporal filtering mechanism, the jumps in parking space geometric parameters caused by single-frame detection noise or temporary occlusion can be effectively suppressed, allowing the parking space corner coordinates to transition smoothly between consecutive frames and avoiding drastic fluctuations in the output results. Furthermore, the recursive update of the current frame occupancy rate by combining the occupancy rate of the previous frame with the filtering coefficients can eliminate abnormal fluctuations in instantaneous occupancy status, making the changes in parking space occupancy rate more consistent with physical laws and temporal continuity. This improves the stability and robustness of the parking space observation information, providing a smooth and reliable parking space state sequence for the automatic parking system, improving the performance of subsequent path planning and control, and enhancing the user's parking experience in dynamic scenarios.

[0118] In some embodiments, step 106 outputs parking space observation information for the current frame based on the comprehensive confidence level of the current frame, including: If the overall confidence level of the current frame is greater than or equal to the confidence threshold, output the parking space observation information of the current frame; if the overall confidence level of the current frame is less than the confidence threshold, determine the center coordinates and size of the parking space based on the corner coordinates of the parking space in the current frame observation information, and project the center coordinates of the parking space in the current frame onto the vehicle coordinate system of the next frame according to the inter-frame pose change data obtained in the next frame to obtain the projection position of the center of the parking space in the next frame; using the projection position of the next frame as the center, adjust the preset window size according to the parking space size and perform iterative projection to recalculate the overall confidence level of the next frame, until the obtained overall confidence level is greater than or equal to the confidence threshold or the number of iterations is reached.

[0119] In this embodiment, the confidence threshold is used to ensure that the output parking space observation information has sufficiently high reliability; for example, the confidence threshold can be 0.9. If the confidence threshold is too high, such as 0.95, it will be difficult to converge in complex scenarios, resulting in output delay.

[0120] In this embodiment, the comparison between the overall confidence level of the current frame and the confidence threshold determines whether to directly output or trigger an iterative projection update mechanism. Specifically, when the confidence level meets the threshold, the parking space observation information of the current frame is directly output to ensure real-time performance. When the overall confidence level of the frame is lower than the threshold, unreliable information is not directly output. Instead, iterative projection is performed using the parking space center coordinates and parking space size of the current frame, combined with the inter-frame pose change data of the next frame: the parking space center of the current frame is projected onto the vehicle coordinate system of the next frame, and the window size is dynamically adjusted according to the parking space size to perform local re-detection, thereby iteratively calculating the overall confidence level of the next frame until the threshold is met or the number of iterations is reached. In this way, by introducing an iterative projection mechanism based on historical parking space information and motion models, the geometric constraints of the parking space and the vehicle pose changes are effectively utilized. When the confidence level is insufficient, it actively guides the targeted detection of subsequent frames, avoiding redundant re-detection of the entire image and reducing computational overhead. At the same time, the search window is adaptively adjusted according to the parking space size, improving the targeting and success rate of local detection. This mechanism can significantly improve the continuity and reliability of parking space tracking in low-confidence scenarios, prevent parking space loss due to poor observation quality in a single or multiple frames, and ensure that the automatic parking system can still obtain stable and accurate parking space information in complex environments (such as temporary occlusion and changes in lighting), ultimately enhancing the robustness of the system and the user experience.

[0121] The process of the method provided in this application will be described below by way of a specific embodiment.

[0122] In automated parking and low-speed autonomous driving systems, parking space perception serves as the prerequisite and foundation for parking path planning and vehicle control, directly impacting system availability and user experience. Current mainstream parking space detection technologies primarily rely on independent or preliminary fusion perception judgments from onboard surround-view vision systems and ultrasonic radar. Specifically, the vision system acquires images of the vehicle's surrounding environment using a fisheye camera, combining image distortion correction, perspective transformation, and deep learning models to detect parking space markings, boundary points, and potential parking space areas. Meanwhile, ultrasonic radar uses echo time and intensity analysis to determine the distance and presence of obstacles, assisting in identifying the boundaries of vertical or lateral parking spaces. Most mass-produced solutions currently employ a "single-frame detection + instantaneous output" model, where each frame of sensor data is processed independently, directly generating parking space geometric parameters (such as vertex coordinates, width, and depth) and occupancy status, which are then uploaded to the decision module in real time. This method demonstrates effectiveness under ideal lighting, clear markings, and open environments and has been widely applied in Level 2 parking assistance systems. However, this type of method has at least the following problems: First, ground markings may become incomplete or even disappear due to long-term wear, rain, or construction obstruction; environmental factors such as sudden changes in lighting (e.g., entering or exiting underground parking garages), ground reflection (slippery surfaces), and shadow occlusion significantly affect the detection confidence of visual algorithms; or dynamic obstacles (e.g., pedestrians, moving vehicles) may temporarily obscure parking space areas, leading to misjudgments of "occupied." Second, due to the lack of temporal state continuity modeling, single-frame detection results are prone to jumps, causing key geometric parameters such as parking space vertex coordinates, width, and depth to oscillate violently between consecutive frames, making it difficult to support high-precision path planning requirements. Third, frequent switching between occupied and vacant parking space states can cause system erroneous triggering or function exit, seriously affecting users' trust in and experience of the automatic parking function. Fourth, although some systems introduce a simple fusion of vision and ultrasound, they do not deeply explore the complementary characteristics of the two in the spatial and temporal dimensions and lack a quantitative evaluation mechanism for the consistency of multi-source observations.

[0123] To address the aforementioned issues, this application provides a parking space identification and closed-loop correction method (corresponding to the parking space information determination method described above) and system (corresponding to the parking space information determination device described above) based on time-series re-sensing. Through multi-frame re-observation, multi-modal consistency evaluation, and confidence-weighted iterative correction, it achieves accurate and stable output of parking space geometry and state.

[0124] One example is a parking space recognition and closed-loop correction system based on temporal re-perception, comprising: a multi-source perception acquisition module for acquiring surround view images (corresponding to the scene images mentioned above), radar data, IMU pose increments, and completing synchronous calibration; a parking space initial detection module for visual parking space extraction, radar idle area clustering, and multi-sensor initial fusion; a temporal re-perception module for parking space projection, local re-detection, and cross-frame parking space correlation tracking; a consistency verification module for calculating geometric, temporal, and state consistency and outputting a comprehensive confidence score; a perception closed-loop correction module for confidence score-weighted iterative refinement of parking space geometry and state; and a parking space output module for outputting stable and high-precision final parking space information.

[0125] In practical applications, the embodiments of this application employ the following hardware platform: a 4-channel surround-view fisheye camera, a 12-channel radar, and a 6-axis IMU. The fisheye camera has a resolution of 1280×720 and a frame rate of 30 frames per second (fps). The radar includes ultrasonic radar, but millimeter-wave radar can also be used. The ultrasonic radar has a detection range of 0.2-5m and an accuracy of ±2cm. The IMU has an accelerometer range of ±16 grams (g) and a gyroscope range of ±2000 degrees per second (° / s).

[0126] like Figure 4 As shown, the overall process of the parking space recognition and closed-loop correction method based on time-series re-perception provided in this application embodiment includes steps S1 to S6, thereby forming a complete internal closed loop of perception, specifically.

[0127] Step S1: Multi-source sensing data acquisition and preprocessing.

[0128] Here, as Figure 5 As shown, data from the three types of sensors are collected and preprocessed simultaneously.

[0129] First, raw fisheye images 501 are obtained from four surround-view fisheye cameras. Intrinsic and extrinsic parameter calibrations 502 are then performed on the four surround-view fisheye cameras to obtain calibration parameters. The calibration accuracy requires a reprojection error of less than 0.5 pixels. Intrinsic parameters include, but are not limited to, focal length, principal point, and distortion coefficients; extrinsic parameters include the camera's pose relative to the vehicle's coordinate system. Distortion correction 502 is then performed on the raw fisheye images using the calibration parameters, and a bird's-eye view is generated through inverse perspective mapping (IPM). The resolution of the bird's-eye view can be set to 10 centimeters per pixel, covering a 10m × 10m actual scene area around the vehicle. It should be noted that the bird's-eye view can be a representation of the actual scene area in the vehicle's coordinate system. The two-dimensional projection below already possesses physical scale.

[0130] Secondly, radar data 503 from 12 radar acquisitions is obtained. This radar data can include only ultrasonic radar data, or it can include both ultrasonic and millimeter-wave radar data. Further, the radar data is preprocessed, including: filtering out echo signals with amplitudes less than an amplitude threshold to remove clutter 504; using the DBSCAN algorithm to perform distance clustering on points within a neighborhood radius of, for example, 0.3m 504, with a minimum cluster size of 3 points; then, rasterization processing 504 is performed to construct an occupancy raster map with 10cm accuracy.

[0131] Finally, IMU data is acquired using an IMU, and integral calculations are performed on the IMU data to obtain the inter-frame pose increment 505. This inter-frame pose increment is used to record the relative motion of the parking space between two frames. The IMU data, image data, and radar data are then timestamped 506, with the time synchronization error controlled within 20ms to ensure the time consistency of multi-source data. The pose transformation matrix of the current frame is constructed using the inter-frame pose increment. The pose transformation matrix is ​​used to project the center coordinates of the parking space in the previous frame onto the current frame.

[0132] Step S2: Initial detection and geometric modeling of parking spaces.

[0133] Continue to refer to Figure 5 In this embodiment, visual data and radar data are processed respectively to complete the initial parking space detection and multi-sensor fusion.

[0134] For visual parking space detection, semantic segmentation of parking lines is performed on the bird's-eye view 507, and a deep learning-based segmentation network is used to extract parking line pixels; straight line segments in the parking line pixels are detected by Hough transform; four corner points on the bird's-eye view are located using the corner point features (intersections of two parking lines) 507; and boundary lines are fitted based on the corner point coordinates 507 to obtain a quadrilateral model of the parking space. (Also known as visual parking space), output visual parking space In the vehicle coordinate system The coordinates of the four corner points (also known as vertices) are 508. .

[0135] For radar parking space detection, vacant regions 509 are extracted from the grid map. Connectivity analysis is used to cluster these 509 candidate parking space regions, and the radar parking space candidate regions are output. (Also known as radar parking space) and rasterize the candidate area 510.

[0136] For multi-sensor fusion, computer vision parking spaces With radar parking space The overlap ratio (IOU) is calculated. When the IOU > 0.6 and the parking space size meets the size constraints (width 2.0-3.0m, depth 4.5-6.0m), they are determined to be the same physical parking space, completing the initial fusion and thus obtaining the initial parking space set 511. Simultaneously, based on the height attributes of obstacles in the radar echo characteristics and the position of obstacles within the radar parking spaces, the initial parking space occupancy probability S0511 is estimated. It should be noted that the higher the proportion of vacant area, the closer S0 is to 0; conversely, it is closer to 1.

[0137] Step S3: Temporal re-perception based on pose increment and association with parking space.

[0138] like Figure 6 As shown, this embodiment of the application utilizes an inter-frame pose transformation matrix to realize the temporal projection and re-sensing of parking spaces.

[0139] For pose transformation projection, the pose transformation matrix is ​​used. The parking space model from the previous frame Projected onto the vehicle coordinate system of the current frame This gives the projected position of the parking space in the current frame. The projection formula can be expressed by the following formula (19).

[0140]

[0141] It should be noted that the meaning of formula (19) is that at time t-1, The detected parking space coordinates in the coordinate system are transformed to the current time t through vehicle pose transformation. In a coordinate system, this allows for re-detection within a local window of the current frame.

[0142] For local window re-detection, a 50×50 pixel local search window (corresponding to an actual 5m×5m area) is defined on the current frame's bird's-eye view, centered on the projection position. Fast parking space re-detection is performed within this window to obtain the coordinates of the four parking space vertices, avoiding the computational overhead of full-image search.

[0143] For cross-frame ID association, the Euclidean distances are calculated using the coordinates of the four parking space vertices detected in step S1 and the corresponding parking space vertices detected in step S3, resulting in four Euclidean distances. The angle formed by the line connecting the coordinates of the two entrance parking space vertices detected in step S1 and the line connecting the coordinates of the two entrance parking space vertices detected in step S3 is also obtained. If all four Euclidean distances are less than a distance threshold (e.g., 0.3m) and the angle is less than an angle threshold (e.g., 10 degrees), then the visual parking space detected in step S1 is considered to match the visual parking space detected in step S3. This achieves stable ID association of the same physical parking space across different frames. After successful association, the detection result Pt of the current frame is added to the corresponding temporal observation queue Q.

[0144] The temporal observation queue Q stores parking space observation data for the same physical parking space (same ID) across multiple consecutive frames. If a physical parking space loses more than 3 consecutive frames, the parking space ID is deemed invalid, and the queue is deregistered. Each queue element It is a data structure that contains multi-dimensional information. ,in, Let k be the set of quadrilateral vertex coordinates of the parking space geometry detected in the k-th frame. The estimated parking space occupancy probability for the k-th frame. This is the timestamp for the acquisition of the k-th frame of data. This is a valid flag for the k-th frame of data; it is valid. If true, then true; otherwise, false. Here, the time-series observation queue can maintain the parking space observation data of the most recent N=8 frames, forming a time-series observation queue to provide a data foundation for subsequent consistency assessment.

[0145] Step S4: Calculation of multimodal consistency measurement and overall confidence level.

[0146] like Figure 7 As shown, the embodiments of this application calculate the consistency index from three dimensions: geometry, time series, and state, and then weighted and fused to obtain the comprehensive confidence level.

[0147] For geometric consistency, firstly, select from the time-series observation queue Q... The set of parking space vertex coordinates of valid frames Then, using the above formula (2), the center coordinates of the parking spaces in each frame are calculated; next, using the above formula (4), the mean of the center coordinates of the parking spaces in multiple frames is calculated; then, using the above formula (5), the Euclidean distance deviation between the center of the parking spaces in each frame and the mean is calculated; then, using the above formula (6), the mean square deviation is calculated; finally, using the above formula (7), the mean square deviation is normalized and mapped to the [0,1] interval, thereby obtaining the geometric consistency confidence.

[0148] Here, temporal stability measures the smoothness of changes in parking space parameters over time, i.e., the central mean of parking spaces is consistent across multiple frames. (High), but if the parking space angle or position changes drastically between adjacent frames, it still indicates that the detection result is unstable.

[0149] For temporal stability, it is based on angular and displacement fluctuations between adjacent frames. Regarding angular fluctuations, firstly, it is based on the set of parking space vertex coordinates of the valid frames. Determine the parking space angle of the corresponding frame. Then, the mean value of parking space angles in multiple frames is calculated using the above formula (9); the absolute deviation between the parking space angle in the kth frame and the mean value is calculated using the above formula (10); the angle fluctuation is determined based on multiple absolute deviations using the above formula (11) or formula (12). For displacement fluctuation, firstly, the displacement of adjacent frames is calculated using the above formula (3), and the displacement of adjacent frames is averaged using the above formula (8) to obtain the average displacement of adjacent frames; finally, based on the angle fluctuation and the average displacement of adjacent frames, the time series stability confidence is calculated by normalization using the above formula (13).

[0150] Here, state consistency measures the degree of certainty in determining the parking space occupancy status. When the parking space occupancy rate... When the parking space occupancy rate is close to 0, it is considered vacant; when the occupancy rate is close to 1, it is considered occupied. The status judgment is clear and consistent. When the value approaches 0.5 (indecisiveness), the status judgment becomes ambiguous and consistency is low. Therefore, it is necessary to determine the consistency of the parking space status.

[0151] Regarding state consistency, firstly, the information entropy of the parking space occupancy rate is calculated using the above formula (14); based on the information entropy, the state consistency confidence is calculated by normalization using the above formula (15).

[0152] Weights are set for each dimension of the overall confidence level. ， Satisfied, satisfied Based on the weights, the three types of consistency indicators are weighted and fused using the following formula (20) to obtain the comprehensive confidence score. The comprehensive confidence score C∈[0,1], the closer it is to 1, the more reliable the parking space detection result is.

[0153]

[0154] It should be noted that, regarding the weight design, Geometric accuracy is the core indicator for parking space inspection and has the highest weight. Timing stability directly affects the parking experience, and has the second highest weight. Status assessment influences parking decisions, with a moderate weighting.

[0155] Step S5: Iterative correction of the closed loop.

[0156] Here, as Figure 8 As shown, in this embodiment of the application, the geometric parameters 802 of the parking space and the occupancy status, i.e., the parking space occupancy rate 803, are iteratively corrected using the comprehensive confidence level 801 as the weight, and the correction results are fed back to form a closed loop.

[0157] It should be noted that the input to this module may include: a time-series observation queue (containing the confidence score for each frame). ), Current frame overall confidence Correction results from the previous frame , The output may include: a corrected parking space geometry model. and the corrected parking space occupancy rate .

[0158] Here, for the correction of parking space geometric parameters, the confidence-weighted moving average method is adopted, and the coordinates of the parking space vertices in the time series observation queue are smoothed by using the above formula (17) to obtain the corrected parking space model, including the corrected parking space vertices, width and depth 804.

[0159] Here, for the correction of parking space occupancy status, a first-order time-series probability filtering method is adopted, and the parking space occupancy rate is smoothed by the above formula (18) to obtain the corrected parking space occupancy rate 805.

[0160] For closed-loop feedback, the corrected parking space model and parking space occupancy probability are fed back to step S3 as prior information for the next frame's re-perception. Step S3 uses the corrected parking space model to perform projection prediction to guide the estimation of the projection position and size of the local window, forming a continuously iterative perception closed loop 806. Specifically, the corrected parking space model serves as the reference point for the next frame's projection, improving projection accuracy; the corrected parking space size is used to adjust the local search window; and the corrected occupancy status is used to predict the parking space occupancy situation in the next frame, thereby improving association accuracy and queue data accuracy.

[0161] Step S6: Output stable parking space information.

[0162] It should be noted that the inputs to this module can include: the overall confidence level of the current frame, the corrected parking space model, the corrected parking space occupancy rate, and the current iteration number. The output can determine whether to output stable parking space information based on the overall confidence level. A confidence threshold Cth = 0.9 is set. When the overall confidence level C ≥ Cth, the parking space detection results are considered to have converged, and the corrected parking space model and the corrected parking space occupancy rate are output, thus achieving stable parking space information output. When C < Cth, the process returns to step S3 to continue temporal re-sensing and iterative correction until the confidence level reaches the threshold or the maximum iteration number (set to 20 frames) is reached. This forms a positive feedback loop, gradually converging to a stable value.

[0163] This application provides a parking space information determination device, referring to... Figure 9 As shown, Figure 9 This is a schematic diagram of a parking space information determination device provided in an embodiment of this application. The parking space information determination device 9 includes: The acquisition module 901 is used to acquire the bird's-eye view, ultrasonic radar data, inter-frame pose change data and parking space time-series observation queue corresponding to the scene image of the vehicle; wherein, the parking space time-series observation queue includes multiple consecutive frames of parking space observation information for the same physical parking space, and each frame of parking space observation information includes the parking space corner coordinates, parking space occupancy rate and comprehensive confidence level, and the inter-frame pose change data is calculated from the vehicle motion information; The determination module 902 is used to determine the current frame parking space occupancy rate of the physical parking space based on the ultrasonic radar data when it is determined that there is the same physical parking space based on the ultrasonic radar data and the bird's-eye view. The processing module 903 is used to project the coordinates of the center of the physical parking space in the previous frame in the parking space time sequence observation queue onto the current frame's vehicle coordinate system based on the inter-frame pose change data, so as to obtain the projection position of the center of the parking space in the previous frame in the bird's-eye view. The processing module 903 is also used to determine the target local area of ​​the bird's-eye view with the projection position as the center and according to the preset window size, and to perform parking space detection on the bird's-eye view and the target local area respectively to obtain the first parking space information and the second parking space information; wherein, the parking space information includes: the coordinates of the corner point of the parking space; The processing module 903 is also used to add the second parking space information and the current frame parking space occupancy rate as the current frame parking space observation information and add them to the parking space time sequence observation queue when the first parking space information and the second parking space information meet the same physical parking space matching condition. The processing module 903 is also used to perform multi-dimensional analysis of the physical parking spaces in the current frame based on the parking space time-series observation queue, obtain the comprehensive confidence of the current frame, and update the parking space observation information in the current frame in the parking space time-series observation queue according to the comprehensive confidence of the current frame and the parking space occupancy rate of the current frame. Output module 904 is used to output parking space observation information for the current frame based on the comprehensive confidence level of the current frame.

[0164] This application provides a hardware entity diagram of a parking space information determination device, such as... Figure 10 As shown, the hardware entity of the parking space information determination device 10 includes: a processor 1001 and a memory 1002. The memory 1002 stores a computer program that can run on the processor 1001. When the processor 1001 executes the computer program, it implements some or all of the steps in the parking space information determination method as described in the above embodiments.

[0165] The memory 1002 stores computer programs that can run on the processor. The memory 1002 is configured to store instructions and applications that can be executed by the processor 1001. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) of the processor 1001 and the various modules in the parking space information determination device 10. It can be implemented by flash memory or random access memory (RAM).

[0166] The processor 1001 executes the program to implement the steps of the parking space information determination method described above. The processor 1001 typically controls the overall operation of the parking space information determination device 10.

[0167] This application provides a vehicle, referring to... Figure 11 As shown, Figure 11 This is a schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle 11 includes the parking space information determination device 10 described above.

[0168] This application provides a computer-readable storage medium storing one or more computer programs, which can be executed by one or more processors to implement some or all of the steps in the above-described method. The storage medium can be transient or non-transient.

[0169] This application provides a computer program including computer-readable code, wherein when the computer-readable code is run in a vehicle, a processor in the vehicle executes some or all of the steps in the above-described method.

[0170] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium; in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0171] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0172] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0173] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0174] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0175] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0176] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0177] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0178] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0179] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0180] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an in-vehicle terminal (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0181] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for determining parking space information, characterized in that, The method includes: The system obtains a bird's-eye view of the vehicle scene image, ultrasonic radar data, inter-frame pose change data, and a parking space time-series observation queue. The parking space time-series observation queue includes multiple consecutive frames of parking space observation information for the same physical parking space. Each frame of parking space observation information includes the parking space corner coordinates, parking space occupancy rate, and comprehensive confidence level. The inter-frame pose change data is calculated from the vehicle motion information. When it is determined that there is a physical parking space based on the ultrasonic radar data and the bird's-eye view, the current frame parking space occupancy rate of the physical parking space is determined based on the ultrasonic radar data. Based on the inter-frame pose change data, the center coordinates of the physical parking space in the previous frame in the parking space time sequence observation queue are projected onto the current frame's vehicle coordinate system to obtain the projected position of the parking space center in the bird's-eye view. Centered on the projection position, a target local area of ​​the bird's-eye view is determined according to a preset window size. Parking space detection is performed on the bird's-eye view and the target local area respectively to obtain first parking space information and second parking space information; wherein, the parking space information includes: parking space corner coordinates; When the first parking space information and the second parking space information meet the same physical parking space matching condition, the second parking space information and the current frame parking space occupancy rate are used as the current frame parking space observation information and added to the parking space time sequence observation queue. Based on the parking space time-series observation queue, the physical parking spaces of the current frame are analyzed in multiple dimensions to obtain the comprehensive confidence level of the current frame. Based on the comprehensive confidence level of the current frame and the parking space occupancy rate of the current frame, the parking space observation information of the current frame in the parking space time-series observation queue is updated, and the parking space observation information of the current frame is output based on the comprehensive confidence level of the current frame.

2. The method according to claim 1, characterized in that, The first parking space information and the second parking space information meet the matching conditions of the same physical parking space, including: Connect the corner points of the entrance parking spaces in the first parking space information to obtain a first direction vector, and connect the corner points of the entrance parking spaces in the second parking space information to obtain a second direction vector; Determine the first parking space angle formed between the first direction vector and the second direction vector; Based on the coordinates of each corner point of the first parking space information and the coordinates of the corresponding corner point of the second parking space information, the distance of the first parking space is determined, thereby obtaining multiple distances of the first parking space; If the distance between the plurality of first parking spaces is less than a parking space distance threshold, and the angle of the first parking space is less than a parking space angle threshold, it is determined that the first parking space information and the second parking space information meet the same physical parking space matching condition.

3. The method according to claim 1, characterized in that, The process involves performing a multi-dimensional analysis of the physical parking spaces in the current frame based on the parking space time-series observation queue to obtain the comprehensive confidence level of the current frame, including: Based on the parking space corner coordinates in the parking space time sequence observation queue, determine the parking space center coordinates, parking space center offset, and parking space angle of the same physical parking space in multiple frames; Based on the multi-frame parking space center coordinates, the geometric consistency of the physical parking space is analyzed to obtain the geometric consistency confidence level of the physical parking space. Based on the multi-frame parking space center offset and the multi-frame parking space angle, the temporal stability of the physical parking space is analyzed to obtain the temporal stability confidence level of the physical parking space. Based on the current frame parking space occupancy rate in the parking space time-series observation queue, the state consistency of the physical parking space is analyzed to obtain the state consistency confidence of the physical parking space. The combined confidence of the current frame is obtained by weighting the geometric consistency confidence, the temporal stability confidence, and the state consistency confidence.

4. The method according to any one of claims 1 to 3, characterized in that, The step of analyzing the geometric consistency of the physical parking spaces based on the multi-frame parking space center coordinates to obtain the geometric consistency confidence level of the physical parking spaces includes: For the center coordinates of the parking spaces in the multiple frames, the average center coordinates of the physical parking spaces are obtained; Using the Euclidean distance formula, the distance deviation between the center coordinates of the parking space in each frame and the mean center coordinates of the parking space is calculated, resulting in multiple distance deviations; Calculate the mean square deviation of the multiple distance deviations, and normalize the mean square deviation to obtain the geometric consistency confidence level.

5. The method according to any one of claims 1 to 3, characterized in that, The analysis of the temporal stability of the physical parking space based on the multi-frame parking space center offset and the multi-frame parking space angle, to obtain the temporal stability confidence level of the physical parking space, includes: The average parking space center offset is obtained by averaging the parking space center offset of the multiple frames. The average parking space angle is obtained by averaging the parking space angles of the multiple frames. Calculate the angle deviation between the parking space angle of each frame and the average parking space angle to obtain multiple angle deviations; The target angle deviation is determined based on the multiple angle deviations; The time-series stability confidence level is determined based on the target angle deviation and the average parking space center offset.

6. The method according to any one of claims 1 to 3, characterized in that, The step of analyzing the state consistency of the physical parking spaces based on the current frame parking space occupancy rate in the parking space time-series observation queue to obtain the state consistency confidence of the physical parking spaces includes: Based on the current frame parking space occupancy rate in the parking space time-series observation queue, calculate the information entropy of the physical parking space; The state consistency confidence level is determined based on the information entropy.

7. The method according to any one of claims 1 to 3, characterized in that, The step of updating the parking space observation information in the parking space time-series observation queue based on the comprehensive confidence level of the current frame and the parking space occupancy rate of the current frame includes: Obtain the first N frames of parking space observation information and filtering coefficients for the physical parking space; Based on the comprehensive confidence level of the current frame and the parking space observation information of the previous N frames, the coordinates of the parking space corner points in the parking space observation information of the current frame are updated; The parking space occupancy rate of the current frame is updated based on the filtering coefficients and the parking space occupancy rate of the previous frame.

8. The method according to any one of claims 1 to 3, characterized in that, The step of outputting the parking space observation information for the current frame based on the comprehensive confidence level of the current frame includes: If the overall confidence level of the current frame is greater than or equal to the confidence threshold, output the parking space observation information of the current frame; If the overall confidence score of the current frame is less than the confidence score threshold, the center coordinates and size of the parking space in the current frame are determined based on the corner coordinates of the parking space in the parking space observation information. According to the inter-frame pose change data obtained in the next frame, the center coordinates of the parking space in the current frame are projected onto the vehicle coordinate system of the next frame to obtain the projection position of the center of the parking space in the next frame. Taking the projection position of the next frame as the center, the preset window size is adjusted according to the parking space size to perform iterative projection to recalculate the overall confidence score of the next frame until the obtained overall confidence score is greater than or equal to the confidence score threshold or the number of iterations is reached.

9. The method according to any one of claims 1 to 3, characterized in that, The step of determining the existence of the same physical parking space based on the ultrasonic radar data and the bird's-eye view includes: Connectivity detection is performed on the ultrasonic radar data to obtain the radar parking space area; The visual parking space area is determined based on the corner coordinates of the parking space in the first parking space information; The overlapping area and the overlapping rate are obtained by calculating the visual parking space area and the radar parking space area using an overlap formula. If the overlap rate is greater than the overlap rate threshold and the overlapping area satisfies the area constraint condition, it is determined that the radar parking space area and the visual parking space area have the same physical parking space.

10. A parking space information determination device, characterized in that, include: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the parking space information determination method according to any one of claims 1 to 9.

Images