Parking space fusion method, device, equipment and medium

Abstract

The invention provides a parking space fusion method and device, equipment and a medium, and relates to the technical field of automatic driving. The method comprises the following steps: acquiring parking space detection information in a panoramic spliced image, wherein the information comprises position information for sensing a parking space and observation noise; based on the position information of the sensing parking space, determining a target sensing parking space in an effective range, a target distance from the target sensing parking space and whether the target sensing parking space contains a part located in an image splicing area of the panoramic spliced image; carrying out observation noise management on the target sensing parking space according to the target distance and whether the target sensing parking space contains a part located in the image splicing area; and performing parking space fusion based on the information after observation noise management to obtain parking space information after parking space fusion. According to the invention, the participation degree of the target sensing parking space during parking space fusion is dynamically adjusted, so that the influence of long-distance parking space observation or splicing area parking space observation on the fusion result is reduced, and the precision of the fused parking space information is improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a parking space fusion method, device, equipment and medium. Background Art

[0002] With the rapid development of autonomous driving technology, parking space fusion technology has become one of the key technologies for systems such as Automated Parking Assist (APA) and Automated Valet Parking (AVP). This technology uses the vehicle's multi-camera system to capture environmental images from multiple perspectives. Based on the camera intrinsics and extrinsic parameters obtained through camera calibration, the environmental images from multiple perspectives are stitched together to obtain a panoramic bird's-eye view, also known as a panoramic stitched image. Furthermore, parking space detection information is obtained based on the panoramic stitched image, and parking space fusion is performed based on the parking space detection information to obtain the fused parking space information. However, the obtained parking space information has the problem of low accuracy. Summary of the Invention

[0003] The present application provides a parking space fusion method, device, equipment and medium to improve the problem of low accuracy of parking space information after parking space fusion.

[0004] In a first aspect, the present application provides a parking space fusion method, comprising:

[0005] Obtain parking space detection information from the panoramic stitching image, where the parking space detection information includes the position information of the perceived parking space and the observed noise;

[0006] Based on the position information of the sensed parking space, determine the target sensed parking space within the effective range, where the effective range is determined based on the sensing accuracy;

[0007] determining, based on the position information of the sensed parking space, a target distance to the target sensed parking space and whether the target sensed parking space includes a portion located in an image stitching area of ​​the panoramic stitching image;

[0008] Based on the target distance and whether the target perception parking space includes a portion within the image stitching area, observation noise management is performed on the target perception parking space. Observation noise management is used to reduce the participation of the target perception parking space in parking space fusion. The greater the target distance or the larger the portion of the target perception parking space within the image stitching area, the lower the participation.

[0009] Parking space fusion is performed based on the information after observation noise management to obtain the parking space information after fusion.

[0010] In a possible implementation, observation noise management is performed on the target perception parking space based on the target distance and whether the target perception parking space includes a portion located in the image stitching area, including: if the target distance is less than or equal to the first distance, and the target perception parking space includes a portion located in the image stitching area, a first noise processing strategy is executed on the observation noise corresponding to the target perception parking space, wherein the range determined by the first distance is within the valid range; if the target distance is less than or equal to the first distance, and the target perception parking space does not include a portion located in the image stitching area, a second noise processing strategy is executed on the observation noise corresponding to the target perception parking space; if the target distance is greater than the first distance, and the target perception parking space does not include a portion located in the image stitching area, a second noise processing strategy is executed on the observation noise corresponding to the target perception parking space; wherein the first noise processing strategy refers to amplifying the observation noise by a set value; the first noise processing strategy refers to performing a step-by-step linear amplification on the observation noise based on the shortest distance from the target perception parking space to the vehicle.

[0011] In a possible implementation, the parking space fusion method further includes: if the target distance is greater than the first distance, and the target perceived parking space includes a part located in the image stitching area, executing a prohibition fusion strategy, which means not performing parking space fusion on the target perceived parking space.

[0012] In one possible embodiment, the image stitching area is determined as follows: based on the calibrated extrinsic parameters of each image acquisition component of the vehicle, the first boundary coordinates of the field of view boundary corresponding to each image acquisition component in the vehicle coordinate system are determined, and the vehicle coordinate system is a coordinate system with the center of the rear axle of the vehicle as the origin; each first boundary coordinate is mapped to the bird's-eye view coordinate system through inverse perspective mapping (IPM) to obtain each second boundary coordinate in the bird's-eye view coordinate system; based on each first boundary coordinate and the corresponding second boundary coordinate, a center line segment model of adjacent image stitching areas in the panoramic stitching image is established, and the center line segment model includes four center line segments, and the center line segment refers to the line segment between each first boundary coordinate and the corresponding second boundary coordinate; for each center line segment in the center line segment model, the center line segment is translated in a first direction by a first offset to obtain a first boundary line segment, and the center line segment is translated in a second direction opposite to the first direction by a second offset to obtain a second boundary line segment; and the area between the first boundary line segment and the second boundary segment corresponding to each center line segment in the center line segment model is determined as the image stitching area.

[0013] In a possible embodiment, whether the target perceived parking space includes a portion located in the image stitching area of ​​the panoramic stitching image is determined as follows: based on the position information of the target perceived parking space, the four parking space edges corresponding to the target perceived parking space are obtained; the intersection of the four parking space edges corresponding to the target perceived parking space and the boundary line segments of the image stitching area is judged; if any parking space edge of the target perceived parking space intersects with any boundary line segment in the boundary line segments of the image stitching area, it is determined that the target perceived parking space includes a portion located in the image stitching area of ​​the panoramic stitching image; if none of the four parking space edges of the target perceived parking space intersects with any boundary line segment in the boundary line segments of the image stitching area, it is determined that the target perceived parking space does not include a portion located in the image stitching area of ​​the panoramic stitching image.

[0014] In a possible implementation, an intersection judgment is performed on the four parking space edges corresponding to the target perceived parking space and the boundary line segments of the image stitching area, including: performing a rapid exclusion experiment on each of the four parking space edges and each of the boundary line segments of the image stitching area; for the target parking space edge and the target boundary line segment detected by the rapid exclusion experiment, determining the vector cross product corresponding to the line segment pair composed of the target parking space edge and the target boundary line segment according to a straddle experiment; if the vector cross product is less than or equal to 0, it is determined that the target parking space edge and the target boundary line segment intersect.

[0015] In one possible implementation, parking space fusion is performed based on the information after observation noise management to obtain parking space information after fusion, including: filtering the information after observation noise management to obtain an estimated parking space coordinate at the current moment; determining the absolute value difference between the estimated parking space coordinate and the parking space coordinate at the previous moment; if the absolute difference is greater than a set threshold, limiting the update amount of the parking space coordinate at the current moment to determine that the parking space coordinate at the current moment is the sum of the parking space coordinate at the previous moment and the updated amount after limiting; if the absolute difference is less than or equal to the set threshold, determining the estimated parking space coordinate at the current moment to be the parking space coordinate at the current moment; and obtaining the parking space information after fusion based on the parking space coordinate at the current moment.

[0016] In a second aspect, the present application provides a parking space fusion device, comprising:

[0017] An acquisition module is used to obtain parking space detection information in the panoramic stitching image, where the parking space detection information includes position information of the sensed parking space and observed noise;

[0018] An observation filtering module is used to determine a target sensing parking space within a valid range based on the position information of the sensing parking space. The valid range is determined based on the sensing accuracy.

[0019] The observation filtering module is also used to determine the target distance from the target perception parking space based on the position information of the perception parking space, and whether the target perception parking space includes a portion located in the image stitching area of ​​the panoramic stitching image.

[0020] A filtering module is used to perform observation noise management on the target-perceived parking space based on the target distance and whether the target-perceived parking space includes a portion located in the image stitching area. Observation noise management is used to reduce the participation of the target-perceived parking space in parking space fusion. The greater the target distance or the larger the portion of the target-perceived parking space located in the image stitching area, the lower the participation.

[0021] The fusion module is used to perform parking space fusion based on the information after the observed noise management to obtain the parking space information after fusion.

[0022] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0023] Memory for storing computer-executable instructions;

[0024] A processor is configured to execute computer-executable instructions stored in a memory to implement the method described in any one of the first aspects.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed, they are used to implement any of the methods described in the first aspect.

[0026] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which implements the method described in any one of the first aspects when executed.

[0027] The parking space fusion method, device, equipment and medium provided in the present application obtain parking space detection information in a panoramic stitching image, where the parking space detection information includes position information of the sensed parking space and observation noise; based on the position information of the sensed parking space, determine the target sensed parking space within an effective range, where the effective range is determined based on the perception accuracy; based on the position information of the sensed parking space, determine the target distance from the target sensed parking space, and whether the target sensed parking space includes a portion located in an image stitching area of ​​the panoramic stitching image; and based on the target distance and whether the target sensed parking space includes a portion located in an image stitching area, perform observation noise management on the target sensed parking space, where the observation noise management is used to reduce the participation of the target sensed parking space in parking space fusion, where the greater the target distance or the larger the portion of the target sensed parking space included in the image stitching area, the lower the participation; further, perform parking space fusion based on the information after observation noise management to obtain parking space information after parking space fusion. In this process, by utilizing the position information of the perceived parking space, the target perceived parking space within the effective range is accurately identified, and the target distance between the vehicle and the target perceived parking space is calculated. At the same time, it is evaluated whether the target perceived parking space includes the part of the image stitching area located in the panoramic stitching image, and noise management is performed according to the target distance and the proportion of the image stitching area included in the target perceived parking space, so as to dynamically adjust the participation of the target perceived parking space in parking space fusion, effectively reducing the negative impact of long-distance parking space observation or stitching area parking space observation on the fusion result, significantly reducing the error caused by distortion, reducing the parking space overlap or jump problem caused by uneven ground or changes in external parameters, making the fusion result smoother, thereby significantly improving the accuracy of the parking space information after parking space fusion, which provides more reliable and smoother parking space information for the autonomous driving parking system, and effectively promotes the practical application of APA and AVP technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0029] Figure 1 A schematic flow chart of a parking space fusion method provided in an exemplary embodiment of the present application;

[0030] Figure 2 A schematic diagram of region division in a panoramic stitched image provided by an exemplary embodiment of the present application;

[0031] Figure 3 A schematic diagram of the shortest distance from a parking space to a vehicle provided by an exemplary embodiment of the present application;

[0032] Figure 4 Schematic diagram of a rapid repulsion experiment and a straddle experiment provided for exemplary embodiments of the present application;

[0033] Figure 5 Another flowchart of the parking space fusion method provided by an exemplary embodiment of the present application;

[0034] Figure 6 A schematic structural diagram of a parking space fusion device provided by an exemplary embodiment of the present application;

[0035] Figure 7 A schematic structural diagram of an electronic device provided as an exemplary embodiment of the present application.

[0036] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0037] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0038] The terms "first", "second" etc. in the specification and claims of the present application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable in appropriate circumstances, so that the embodiments of the present application described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, system, product or equipment comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, products or equipment.

[0039] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0040] In the prior art, panoramic stitching images are stitched together using camera intrinsic and extrinsic parameters obtained through camera calibration, but this has certain limitations in practical applications. Specifically, when obtaining parking space detection information and performing parking space fusion based on panoramic stitching images, more accurate parking space information can only be obtained when the ground is relatively flat; if the ground is uneven, the accuracy of parking space detection and fusion will be seriously affected, especially in the image stitching area. Due to the changes in the extrinsic parameters of each camera, the misalignment and distortion of the image stitching area will become particularly obvious. In this case, the accuracy of the corner points of the perceived parking space is greatly reduced, resulting in the introduction of noise during parking space fusion, which reduces the accuracy and smoothness of the parking space information and even causes overlap and other problems. In addition, when the ground is uneven, the farther the perceived parking space is from the camera, the lower the accuracy after fusion, and the closer the perceived parking space is to the image stitching area, the further the accuracy will decrease. These factors combined have led to the performance and reliability of parking space fusion technology being limited under complex terrain conditions, resulting in the problem of low accuracy of the obtained parking space information.

[0041] In order to solve the above problems, an embodiment of the present application provides a parking space fusion solution, which sets an effective detection range based on perception accuracy, determines the target perception parking spaces within the effective range, and excludes low-confidence long-distance parking spaces; and combines the target distance and the proportion of the image stitching area included in the panoramic stitching image to quantify the observation noise and adjust the fusion weight, so that the parking spaces with higher observation noise, such as long-distance parking spaces or parking spaces located in the stitching area, have lower corresponding fusion participation, so as to reduce the negative impact of poor observations on the fused parking space information, thereby improving the accuracy of the fused parking space information.

[0042] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0043] Figure 1 A flow chart of a parking space fusion method provided by an exemplary embodiment of the present application. Figure 1 As shown, the parking space fusion method includes the following steps:

[0044] S101 : Acquire parking space detection information in the panoramic stitched image, where the parking space detection information includes position information of the sensed parking space and observed noise.

[0045] For example, a vehicle-mounted multi-camera system deployed around the vehicle, such as a four-way fisheye surround view camera, collects environmental images. Based on the camera intrinsic parameters (such as focal length or distortion coefficient, etc.) and camera extrinsic parameters (such as the position relationship between cameras, etc.) obtained by camera calibration, the environmental images from multiple perspectives are stitched together to obtain a panoramic stitched image (i.e., an IPM image). A deep learning model or traditional computer vision method is used to detect parking spaces on the IPM image and extract parking space detection information. The parking space detection information includes the position information of the perceived parking space and observation noise. Among them, the deep learning model includes but is not limited to the target detection model (You Only Look Once, abbreviated as YOLO) or the convolutional neural network (Convolutional Neural Network, abbreviated as CNN); the traditional computer vision method includes but is not limited to edge detection and geometric fitting; the position information of the perceived parking space is, for example, the coordinates of the four corner points of the parking space or the coordinates of the center point of the parking space; the observation noise can be a preset initial noise, the size of which can be related to the position information of the perceived parking space, for example, the farther away from the vehicle, the larger the corresponding initial noise, etc., or the observation noise can also be determined based on the confidence of the detection model or the sensor error estimate.

[0046] S102. Determine a target sensing parking space within an effective range based on the position information of the sensing parking space. The effective range is determined based on sensing accuracy.

[0047] For example, based on the perception accuracy, the area within a value of 1 from the center of each fisheye surround view camera of the vehicle is determined to be area 1, and area 1 does not include the area where the vehicle is located. For example, the value 1 is 4.2 meters. Based on the perception accuracy, the area within a value of 2 from the center of the fisheye surround view camera is determined to be area 2. The value 2 is greater than the value 1 and area 2 does not include area 1. For example, the value 2 is 7.6 meters. Accordingly, the area where area 1 and area 2 are located is determined to be the valid range, and the area beyond area 2 is determined to be the invalid range. For example, Figure 2 This is a schematic diagram of the region division in the panoramic stitching image provided by the exemplary embodiment of the present application. Figure 2 As shown, area 1 is, for example, an area composed of four semicircles, and area 2 is, for example, an area between the circle and the four semicircles. The area where area 1 and area 2 are located is the valid range, and the area outside area 2 is the invalid range.

[0048] Correspondingly, the sensed parking spaces that completely fall into the invalid range are determined to be unreliable parking spaces, and no observation update is performed on such parking spaces; correspondingly, the sensed parking spaces that completely or partially fall into the valid range are determined to be target sensed parking spaces within the valid range.

[0049] S103: Determine a target distance from the target perceived parking space based on the position information of the perceived parking space, and whether the target perceived parking space includes a portion located in an image stitching area of ​​the panoramic stitching image.

[0050] For example, Figure 3 This is a schematic diagram of the shortest distance from a parking space to a vehicle provided by an exemplary embodiment of this application. Figure 3 As shown, the shortest distance from the midpoint of the four parking space edges of the target perception parking space to the vehicle is calculated, and the shortest distance is determined to be the target distance between the vehicle and the target perception parking space; accordingly, based on the overlapping relationship between the field of view angles of multiple cameras, a dynamically adjustable stitching area mask is constructed in the IPM image, and the spatial intersection area ratio of the four parking space edges of the target perception parking space and the stitching area mask is calculated. Further, according to the spatial intersection area ratio, it is determined whether the target perception parking space includes a portion located in the image stitching area of ​​the panoramic stitching image. For example, if the spatial intersection area ratio is greater than 0, it is determined that the target perception parking space includes a portion located in the image stitching area of ​​the panoramic stitching image; if the spatial intersection area ratio is 0, it is determined that the target perception parking space does not include a portion located in the image stitching area of ​​the panoramic stitching image.

[0051] S104. Perform observation noise management on the target-perceived parking space based on the target distance and whether the target-perceived parking space includes a portion located in the image stitching area. The observation noise management is used to reduce the participation of the target-perceived parking space in parking space fusion. The greater the target distance or the greater the portion of the target-perceived parking space located in the image stitching area, the lower the participation.

[0052] For example, according to different target distances and whether the target perception parking space includes a part located in the image stitching area or the target perception parking space does not include a part located in the image stitching area, different observation noise management is performed on the target perception parking space. For example, a nonlinear mapping relationship is used to make the participation of the target perception parking space in parking space fusion negatively correlated with the target distance, or a progressive attenuation interval is set, etc., and a noise compensation mechanism is implemented for the target perception parking space including the part located in the image stitching area, and graded compensation is implemented according to the degree of area overlap, so that the greater the target distance or the larger the part of the target perception parking space included in the image stitching area, the lower the participation of the target perception parking space in parking space fusion, so as to reduce the negative impact of bad observations on the fused parking space information.

[0053] S105: Perform parking space fusion based on the observed noise management information to obtain fused parking space information.

[0054] For example, after noise management is completed, the weighted parking space observation information is fused with historical tracking data through a multi-source data fusion framework to output stable, high-precision parking space information. Specifically, the noise management information is converted into a standardized fusion input format and combined with historical parking space status (such as the previous frame fusion result and motion estimation) to obtain the fused parking space information.

[0055] The parking space fusion method provided in the embodiment of the present application utilizes the position information of the perceived parking space to accurately identify the target perceived parking space within the effective range, calculates the target distance between the vehicle and the target perceived parking space, and evaluates whether the target perceived parking space includes a part of the image stitching area located in the panoramic stitching image. Noise management is performed according to the target distance and the proportion of the image stitching area included in the target perceived parking space, so as to dynamically adjust the participation of the target perceived parking space in the parking space fusion, effectively reducing the negative impact of long-distance parking space observation or stitching area parking space observation on the fusion result, significantly reducing the error caused by distortion, reducing the parking space overlap or jump problem caused by uneven ground or changes in external parameters, making the fusion result smoother, thereby significantly improving the accuracy of the parking space information after parking space fusion, which provides more reliable and smoother parking space information for the autonomous driving parking system, and effectively promotes the practical application of APA and AVP technologies.

[0056] In some embodiments, the image stitching area is determined as follows: based on the calibrated external parameters of each image acquisition component of the vehicle, the first boundary coordinates of the field of view boundary corresponding to each image acquisition component in the vehicle coordinate system are determined, and the vehicle coordinate system is a coordinate system with the center of the rear axle of the vehicle as the origin; each first boundary coordinate is mapped to the bird's-eye view coordinate system through IPM to obtain each second boundary coordinate in the bird's-eye view coordinate system; based on each first boundary coordinate and the corresponding second boundary coordinate, a center line segment model of adjacent image stitching areas in the panoramic stitching image is established, and the center line segment model includes four center line segments, and the center line segment refers to the line segment between each first boundary coordinate and the corresponding second boundary coordinate; for each center line segment in the center line segment model, the center line segment is translated in a first direction by a first offset to obtain a first boundary line segment, and the center line segment is translated in a second direction opposite to the first direction by a second offset to obtain a second boundary line segment; the area between the first boundary line segment and the second boundary line segment corresponding to each center line segment in the center line segment model is determined as the image stitching area.

[0057] For example, the image stitching area in the IPM image is modeled. Specifically, based on the calibration external parameters of the four fisheye surround view cameras, the first boundary coordinates of the camera field of view boundary corresponding to each fisheye surround view camera in the vehicle coordinate system are determined, such as Figure 2The coordinates of E, F, G and H shown in the figure are the coordinates of the vehicle coordinate system with the center of the rear axle of the vehicle as the origin. The coordinates of E, F, G and H are mapped to the bird's-eye view coordinate system through IPM to obtain the coordinates of I, J, K and L in the bird's-eye view coordinate system. Figure 2 The EL, FJ, GK, and HL shown are center line segments of the adjacent image stitching regions, i.e., stitching seams. A center line segment model of the adjacent image stitching regions in the IPM image is established based on each center line segment. Furthermore, for each center line segment in the center line segment model, for example, EL, EL is translated to the left by a first offset to obtain a first boundary line segment, and EL is translated to the right by a second offset to obtain a second boundary line segment. The area between the first boundary line segment and the second boundary segment corresponding to each center line segment in the center line segment model is determined as the image stitching region, for example, Figure 2 The area formed by the eight dashed line segments on both sides of the center line segments EL, FJ, GK and HL shown.

[0058] It should be noted that the first offset and the second offset can be the same, for example, both are 0.5m, or the first offset and the second offset can be different, for example, the first offset is 0.5m and the second offset is 1m. The specific values ​​of the first offset and the second offset are not limited here, and the specific values ​​of the first offset and the second offset can also be flexibly adjusted based on the external environment in which the vehicle is located.

[0059] In the embodiment of the present application, dual mapping of the vehicle coordinate system and the bird's-eye view coordinate system is used to ensure that the geometric representation of the image stitching area is strictly consistent with the real physical space, significantly reduce the alignment error of the field of view boundary, and help improve the accuracy of the image stitching area; based on the calibrated external parameters, the boundary line segment model is dynamically generated, which can automatically adapt to different camera configurations and help improve compatibility with different vehicle models; in addition, only the translation operation of the four center line segments needs to be processed to determine the image stitching area, which significantly improves the calculation efficiency, and by adjusting the translation distance, the width of the stitching area can be flexibly controlled, which can better adapt to the noise management needs under different lighting and distortion conditions, thereby further improving the accuracy of the fused parking space information.

[0060] In some embodiments, observation noise management is performed on the target perception parking space based on the target distance and whether the target perception parking space includes a portion located in the image stitching area, including: if the target distance is less than or equal to the first distance, and the target perception parking space includes a portion located in the image stitching area, a first noise processing strategy is executed on the observation noise corresponding to the target perception parking space, wherein the range determined by the first distance is within the valid range; if the target distance is less than or equal to the first distance, and the target perception parking space does not include a portion located in the image stitching area, a second noise processing strategy is executed on the observation noise corresponding to the target perception parking space; if the target distance is greater than the first distance, and the target perception parking space does not include a portion located in the image stitching area, a second noise processing strategy is executed on the observation noise corresponding to the target perception parking space; wherein the first noise processing strategy refers to amplifying the observation noise by a set value; the first noise processing strategy refers to performing a step-by-step linear amplification on the observation noise based on the shortest distance from the target perception parking space to the vehicle.

[0061] For example, still refer to Figure 2 If the target distance is less than or equal to the value 1, and the target perception parking space includes the part located in the image stitching area, it indicates that the target perception parking space is located in area 1 and includes the part located in the image stitching area, then the observation noise corresponding to the target perception parking space is amplified by a set value, such as 80 times; if the target distance is less than or equal to the value 1 and the target perception parking space does not include the part located in the image stitching area, it indicates that the target perception parking space is located in area 1 and does not include the part located in the image stitching area, or if the target distance is greater than the value 1 and the target perception parking space does not include the part located in the image stitching area, it indicates that the target perception parking space For the portion located in area 2 that does not include the portion located in the image stitching area, a table is used to look up the observation noise amplification weight for the observation noise corresponding to the target perception parking space based on the shortest distance from the target perception parking space to the vehicle. For example, if the shortest distance is less than or equal to 1.5 meters, the observation noise is amplified by 1.5 times; if the shortest distance is greater than 1.5 meters and less than or equal to 5 meters, the weight of the observation noise is linearly increased based on the shortest distance, such as 2.5 + shortest distance * 1.5; if the shortest distance is greater than 5 meters, the weight of the observation noise is linearly increased based on the shortest distance, such as 2.5 + shortest distance * 5.0;

[0062] In the embodiments of the present application, by selecting different noise processing strategies based on the target distance and whether the parking space contains part of the image stitching area, the observation noise can be managed more accurately. This accuracy helps to improve the robustness under different conditions and can adapt to different distances and scenarios. In addition, by performing targeted amplification processing on the observation noise, the interference of noise on system decision-making can be effectively reduced, thereby improving the accuracy of the fused parking space information, which is of positive significance for enhancing system performance and reliability.

[0063] Based on the above embodiments, in some embodiments, the parking space fusion method further includes: if the target distance is greater than the first distance, and the target perceived parking space includes a part located in the image stitching area, a fusion prohibition strategy is executed, and the fusion prohibition strategy means that parking space fusion is not performed on the target perceived parking space.

[0064] For example, if the target distance is greater than the value 1 and the target perceived parking space includes a part located in the image stitching area, it indicates that the target perceived parking space is located in area 2 and includes a part located in the image stitching area. For this type of parking space, the fusion prohibition strategy is executed, that is, the target perceived parking space is not observed and updated.

[0065] In the embodiment of the present application, a dual judgment mechanism of distance and image stitching area is used to pre-screen low-reliability observations for fusion qualification, thereby eliminating the propagation path of spatial distortion and long-range noise from the source, thereby further improving the accuracy of fused parking space information.

[0066] In some embodiments, whether the target perceived parking space includes a portion located in the image stitching area of ​​the panoramic stitching image is determined by: based on the position information of the target perceived parking space, obtaining the four parking space edges corresponding to the target perceived parking space; performing an intersection judgment on the four parking space edges corresponding to the target perceived parking space and the boundary line segments of the image stitching area; if any parking space edge of the target perceived parking space intersects with any boundary line segment in the boundary line segments of the image stitching area, then determining that the target perceived parking space includes a portion located in the image stitching area of ​​the panoramic stitching image; if none of the four parking space edges of the target perceived parking space intersects with any boundary line segment in the boundary line segments of the image stitching area, then determining that the target perceived parking space does not include a portion located in the image stitching area of ​​the panoramic stitching image.

[0067] For example, still refer to Figure 2 , the four parking space edges corresponding to the target perception parking space and the eight boundary line segments of the image splicing area (i.e. Figure 2 If any parking space edge of the target perception parking space intersects with any of the 8 boundary line segments, it is determined that the target perception parking space includes a portion located in the image stitching area of ​​the panoramic stitching image; if none of the four parking space edges of the target perception parking space intersects with any of the 8 boundary line segments, it is determined that the target perception parking space does not include a portion located in the image stitching area of ​​the panoramic stitching image.

[0068] The embodiment of the present application accurately determines whether the parking space contains part of the image stitching area by judging the intersection of the parking space edge and the boundary line segment of the image stitching area. It can adapt to different parking space shapes and sizes, ensure that the relationship between the parking space and the image stitching area can be accurately judged in various situations, and improve the overall robustness. In addition, by accurately judging the relationship between the parking space and the image stitching area, it helps to more effectively manage observation noise and other interference factors during the parking space fusion process, thereby further improving the accuracy of the fused parking space information.

[0069] In some embodiments, an intersection judgment is performed on the four parking space edges corresponding to the target perceived parking space and the boundary line segments of the image stitching area, including: performing a rapid exclusion experiment on each of the four parking space edges and each boundary line segment of the image stitching area; for the target parking space edge and the target boundary line segment detected by the rapid exclusion experiment, determining the vector cross product corresponding to the line segment pair composed of the target parking space edge and the target boundary line segment according to the straddle experiment; if the vector cross product is less than or equal to 0, it is determined that the target parking space edge and the target boundary line segment intersect.

[0070] Among them, the rapid exclusion test is to quickly exclude the impossible intersection by checking whether the projections of two line segments on the coordinate axis overlap. When intersecting, the minimum value of the coordinates of the endpoints of the line segment must be less than the maximum value of the coordinates of the endpoints of the other line segment; the straddle test is to further determine whether the two line segments actually intersect based on the vector cross product. For example, Figure 4 Schematic diagram of the rapid repulsion experiment and straddle experiment provided by the exemplary embodiment of the present application. Figure 4 As shown, assuming that line segment AB is one of the four parking space edges, and line segment CD is a boundary line segment of the image stitching area, the intersection of line segment AB and line segment CD requires that the minimum value of the X coordinates of endpoints A and B of line segment AB is less than the maximum value of the X coordinates of endpoints C and D of line segment CD; swap the two lines. Similarly, the minimum value of the X coordinates of endpoints C and D of line segment CD is less than the maximum value of the X coordinates of endpoints A and B of line segment AB. At the same time, the Y coordinate also needs to meet the above conditions. When these conditions are met, it is considered that line segment AB and line segment CD may intersect, and the target perception parking space may include a part located in the image stitching area of ​​the panoramic stitching image.

[0071] Further, perform the cross product operation by straddling experiment, still referring to Figure 4, vector BA×vector BC, vector BA×vector BD, if the two vectors (i.e., vector BC and vector BD) are arranged on both sides of vector BA, then the corresponding operation results of vector BA×vector BC and vector BA×vector BD point to the outside and inside of the paper respectively, and the corresponding cross product results are subjected to dot product operation, and a negative value is obtained, then it is determined that line segment AB and line segment CD intersect, and it is determined that the target perception parking space includes the part located in the image stitching area of ​​the panoramic stitching image; when the dot product is equal to 0, it means that line segment AB and line segment CD coincide, and since the first step of the rapid exclusion experiment has been passed, it also meets the judgment condition that line segment AB and line segment CD intersect; when the positions of the two line segments are exchanged, the operation process is the same; if the dot product operation result is greater than 0, it means that line segment AB and line segment CD do not intersect, and it is determined that the target perception parking space does not include the part located in the image stitching area of ​​the panoramic stitching image.

[0072] The embodiment of the present application, through the dual verification mechanism of rapid exclusion experiment and straddle experiment, can ensure the accuracy of intersection judgment, reduce the misjudgment problem caused by floating-point error or approximate calculation, provide a precise standard for parking space screening in the image stitching area, and further improve the accuracy of parking space fusion.

[0073] In some embodiments, parking space fusion is performed based on the information after observation noise management to obtain parking space information after parking space fusion, including: filtering the information after observation noise management to obtain an estimated parking space coordinate at the current moment; determining the absolute value difference between the estimated parking space coordinate and the parking space coordinate at the previous moment; if the absolute difference is greater than a set threshold, limiting the update amount of the parking space coordinate at the current moment to determine that the parking space coordinate at the current moment is the sum of the parking space coordinate at the previous moment and the update amount after limiting processing; if the absolute difference is less than or equal to the set threshold, determining the estimated parking space coordinate at the current moment to be the parking space coordinate at the current moment; based on the parking space coordinate at the current moment, obtaining parking space information after parking space fusion.

[0074] For example, the information after observation noise management is filtered by a filtering algorithm such as Kalman filtering to obtain an estimated parking space coordinate at the current moment; the absolute value difference between the estimated parking space coordinate at the current moment and the parking space coordinate at the previous moment is calculated; if the absolute value difference is greater than a set threshold, such as 0.05 meters, it is considered that the change in the parking space coordinate is too drastic and may be affected by noise or abnormal data, so the parking space coordinate update amount at the current moment is limited, for example, it is limited to within 0.05 meters, and the parking space coordinate at the current moment is determined to be the sum of the parking space coordinate at the previous moment and 0.05 meters, that is, the parking space coordinate at the previous moment is updated by 0.05 meters; if the absolute difference is less than or equal to the set threshold, such as 0.05 meters, it is considered that the parking space coordinate change is within a reasonable range, and the estimated parking space coordinate at the current moment is directly used as the parking space coordinate at the current moment; based on the parking space coordinate at the current moment and other relevant information (such as historical data and environmental factors of the parking space), the parking space information after parking space fusion is obtained.

[0075] It should be noted that setting the threshold value to 0.05 meters is only an example. In actual applications, the threshold value is set according to application requirements. The specific value of the threshold value is not limited here.

[0076] In the embodiments of the present application, by filtering the observed data, the influence of noise on the parking space coordinate estimation is reduced, thereby improving the accuracy of the parking space coordinate estimation; in addition, through limiting processing, the drastic changes in the parking space coordinates caused by noise or abnormal data are reduced, ensuring that the changes in the parking space coordinates are within a reasonable range, making the update of the parking space information smoother, facilitating the automatic parking process of the vehicle, making the parking operation more smooth and reliable, thereby improving the user's parking management experience.

[0077] For example, Figure 5 Another flow chart of the parking space fusion method provided by the exemplary embodiment of this application. Figure 5 As shown, the parking space fusion method includes the following steps:

[0078] S501: Acquire parking space detection information in the panoramic stitched image, where the parking space detection information includes position information of the sensed parking space and observed noise.

[0079] S502: Based on the position information of the sensed parking space, determine whether the sensed parking space is within a valid range.

[0080] Among them, the effective range is determined based on the perception accuracy;

[0081] For example, Figure 2 As shown, area 1 is, for example, an area composed of four semicircles, and area 2 is, for example, an area between the circle and the four semicircles. The area where area 1 and area 2 are located is the valid range, and the area outside area 2 is the invalid range.

[0082] If yes, execute S503;

[0083] If not, execute S512.

[0084] S503: Determine a sensing parking space within the effective range as a target sensing parking space.

[0085] S504: Determine a target distance between the vehicle and the target perceived parking space based on the position information of the perceived parking space, and whether the target perceived parking space includes a portion located in an image stitching area of ​​the panoramic stitching image.

[0086] For example, refer to Figure 3 , calculate the shortest distance from the midpoint of the four parking space edges of the target perception parking space to the vehicle, and determine the shortest distance as the target distance between the vehicle and the target perception parking space; accordingly, based on the overlapping relationship of the field of view angles of multiple cameras, construct a dynamically adjustable stitching area mask in the IPM image, and calculate the spatial intersection area ratio of the four parking space edges of the target perception parking space and the stitching area mask. Further, determine whether the target perception parking space includes a part located in the image stitching area of ​​the panoramic stitching image according to the spatial intersection area ratio. For example, if the spatial intersection area ratio is greater than 0, determine whether the target perception parking space includes a part located in the image stitching area of ​​the panoramic stitching image; if the spatial intersection area ratio is 0, determine that the target perception parking space does not include a part located in the image stitching area of ​​the panoramic stitching image.

[0087] S505. Perform observation noise management on the target-perceived parking space based on the target distance and whether the target-perceived parking space includes a portion located in the image stitching area. The observation noise management is used to reduce the participation of the target-perceived parking space in parking space fusion. The greater the target distance or the greater the portion of the target-perceived parking space located in the image stitching area, the lower the participation.

[0088] For example, if the target distance is less than or equal to the first distance, and the target perception parking space includes a part located in the image stitching area, the first noise processing strategy is executed on the observation noise corresponding to the target perception parking space, wherein the range determined by the first distance is within the effective range; if the target distance is less than or equal to the first distance, and the target perception parking space does not include a part located in the image stitching area, the second noise processing strategy is executed on the observation noise corresponding to the target perception parking space; if the target distance is greater than the first distance, and the target perception parking space does not include a part located in the image stitching area, the second noise processing strategy is executed on the observation noise corresponding to the target perception parking space; wherein, the first noise processing strategy refers to amplifying the observation noise by a set value; the first noise processing strategy refers to performing a step-by-step linear amplification processing on the observation noise according to the shortest distance from the target perception parking space to the vehicle.

[0089] S506: Filter the information managed based on the observed noise to obtain an estimated value of the parking space coordinates at the current moment.

[0090] For example, the information after observation noise management is filtered by a filtering algorithm such as Kalman filtering to obtain an estimated value of the parking space coordinate at the current moment.

[0091] S507: Determine the absolute value difference between the estimated parking space coordinate and the parking space coordinate at the previous moment.

[0092] S508: Determine whether the absolute difference is greater than a set threshold.

[0093] For example, set the threshold to 0.05 meters.

[0094] If yes, execute S509;

[0095] If not, execute S510.

[0096] S509: performing a clipping process on the update amount of the parking space coordinates at the current moment, and determining that the parking space coordinates at the current moment are the sum of the parking space coordinates at the previous moment and the update amount after the clipping process.

[0097] S510: Determine the estimated value of the parking space coordinate at the current moment as the parking space coordinate at the current moment.

[0098] S511. Obtain parking space information after fusion based on the parking space coordinates at the current moment.

[0099] S512: Determine that the sensed parking spaces that completely fall within the invalid range are unreliable parking spaces, and do not perform observation updates on such parking spaces.

[0100] In summary, this application has at least the following advantages:

[0101] 1. By utilizing the position information of the perceived parking space, the target perceived parking space within the effective range is accurately identified, and the target distance between the vehicle and the target perceived parking space is calculated. At the same time, it is evaluated whether the target perceived parking space includes the part located in the image stitching area of ​​the panoramic stitching image, and noise management is performed according to the target distance and the proportion of the image stitching area included in the target perceived parking space, so as to dynamically adjust the participation of the target perceived parking space in parking space fusion, effectively reducing the negative impact of long-distance parking space observation or stitching area parking space observation on the fusion result, significantly reducing the error caused by distortion, reducing the problem of parking space overlap or jump caused by uneven ground or changes in external parameters, making the fusion result smoother, thereby significantly improving the accuracy of the parking space information after parking space fusion, which provides more reliable and smoother parking space information for the autonomous driving parking system, and effectively promotes the practical application of APA and AVP technologies.

[0102] Second, through dual mapping between the vehicle coordinate system and the bird's-eye view coordinate system, the geometric representation of the image stitching area is ensured to be strictly consistent with the real physical space, significantly reducing the alignment error of the field of view boundary, which helps to improve the accuracy of the image stitching area; based on the calibration of external parameters, the boundary line segment model is dynamically generated, which can automatically adapt to different camera configurations and help improve compatibility with different vehicle models; in addition, only the translation operation of the four center line segments needs to be processed to determine the image stitching area, which significantly improves the computational efficiency. By adjusting the translation distance, the width of the stitching area can be flexibly controlled, which can better adapt to the noise management needs under different lighting and distortion conditions, thereby further improving the accuracy of the fused parking space information.

[0103] Third, by selecting different noise processing strategies based on the target distance and whether the parking space is part of the image stitching area, the observation noise can be managed more accurately. This accuracy helps improve the robustness under different conditions and can adapt to different distances and scenarios. In addition, by performing targeted amplification processing on the observation noise, the interference of noise on system decision-making can be effectively reduced, thereby improving the accuracy of the fused parking space information, which has positive significance for enhancing system performance and reliability.

[0104] Fourth, by filtering the observed data, the impact of noise on the parking space coordinate estimation is reduced, and the accuracy of the parking space coordinate estimation is improved. In addition, through limiting processing, the drastic changes in the parking space coordinates caused by noise or abnormal data are reduced, ensuring that the changes in the parking space coordinates are within a reasonable range, making the update of parking space information smoother, which helps the vehicle's automatic parking process, making parking operations more smooth and reliable, thereby improving the user's parking management experience.

[0105] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0106] Figure 6 A schematic diagram of a parking space fusion device provided by an exemplary embodiment of the present application. Figure 6 As shown, the parking space fusion device 60 includes an acquisition module 61, an observation and filtering module 62, a filtering module 63 and a fusion module 64, wherein:

[0107] An acquisition module 61 is configured to acquire parking space detection information from the panoramic stitched image, where the parking space detection information includes position information of the sensed parking space and observed noise;

[0108] An observation filtering module 62 is configured to determine a target sensing parking space within a valid range based on the position information of the sensing parking space, where the valid range is determined based on the sensing accuracy;

[0109] The observation filtering module 62 is further configured to determine the target distance from the target perception parking space based on the position information of the perception parking space, and whether the target perception parking space includes a portion located in the image stitching area of ​​the panoramic stitching image.

[0110] A filtering module 63 is configured to perform observation noise management on the target-perceived parking space based on the target distance and whether the target-perceived parking space includes a portion located in the image stitching area. The observation noise management is used to reduce the participation of the target-perceived parking space in parking space fusion. The greater the target distance or the greater the portion of the target-perceived parking space located in the image stitching area, the lower the participation.

[0111] The fusion module 64 is used to perform parking space fusion based on the observed noise management information to obtain fused parking space information.

[0112] In one possible implementation, the filtering module 63 may be specifically configured to: when the target distance is less than or equal to the first distance and the target perception parking space includes a portion located in the image stitching area, execute a first noise processing strategy on the observation noise corresponding to the target perception parking space, wherein the range determined by the first distance is within the effective range; when the target distance is less than or equal to the first distance and the target perception parking space does not include a portion located in the image stitching area, execute a second noise processing strategy on the observation noise corresponding to the target perception parking space; when the target distance is greater than the first distance and the target perception parking space does not include a portion located in the image stitching area, execute a second noise processing strategy on the observation noise corresponding to the target perception parking space; wherein the first noise processing strategy refers to amplifying the observation noise by a set value; the first noise processing strategy refers to performing a step-by-step linear amplification on the observation noise according to the shortest distance from the target perception parking space to the vehicle.

[0113] In a possible implementation, the filtering module 63 may also be used to: when the target distance is greater than the first distance and the target perceived parking space includes a part located in the image stitching area, execute a prohibiting fusion strategy, wherein the prohibiting fusion strategy means not performing parking space fusion on the target perceived parking space.

[0114] In one possible embodiment, the image stitching area is determined as follows: based on the calibrated extrinsic parameters of each image acquisition component of the vehicle, the first boundary coordinates of the field of view boundary corresponding to each image acquisition component in the vehicle coordinate system are determined, and the vehicle coordinate system is a coordinate system with the center of the rear axle of the vehicle as the origin; each first boundary coordinate is mapped to the bird's-eye view coordinate system through inverse perspective mapping (IPM) to obtain each second boundary coordinate in the bird's-eye view coordinate system; based on each first boundary coordinate and the corresponding second boundary coordinate, a center line segment model of adjacent image stitching areas in the panoramic stitching image is established, and the center line segment model includes four center line segments, and the center line segment refers to the line segment between each first boundary coordinate and the corresponding second boundary coordinate; for each center line segment in the center line segment model, the center line segment is translated in a first direction by a first offset to obtain a first boundary line segment, and the center line segment is translated in a second direction opposite to the first direction by a second offset to obtain a second boundary line segment; and the area between the first boundary line segment and the second boundary segment corresponding to each center line segment in the center line segment model is determined as the image stitching area.

[0115] In one possible implementation, the observation filtering module 62 may also be used to: obtain four parking space edges corresponding to the target perception parking space based on the position information of the target perception parking space; perform intersection judgment on the four parking space edges corresponding to the target perception parking space and the boundary line segments of the image stitching area; if any parking space edge of the target perception parking space intersects with any boundary line segment in the boundary line segments of the image stitching area, it is determined that the target perception parking space includes a portion located in the image stitching area of ​​the panoramic stitching image; if none of the four parking space edges of the target perception parking space intersects with any boundary line segment in the boundary line segments of the image stitching area, it is determined that the target perception parking space does not include a portion located in the image stitching area of ​​the panoramic stitching image.

[0116] In one possible implementation, the observation filtering module 62 may also be used to: perform a rapid exclusion test on each of the four parking space edges and each boundary line segment in the image stitching area; for the target parking space edge and target boundary line segment detected through the rapid exclusion test, determine the vector cross product corresponding to the pair of line segments composed of the target parking space edge and the target boundary line segment according to a straddle test; if the vector cross product is less than or equal to 0, it is determined that the target parking space edge and the target boundary line segment intersect.

[0117] In one possible implementation, the fusion module 64 can also be used to: filter the information based on the observed noise management to obtain the estimated parking space coordinates at the current moment; determine the absolute value difference between the estimated parking space coordinates and the parking space coordinates at the previous moment; when the absolute difference is greater than a set threshold, limit the update amount of the parking space coordinates at the current moment to determine that the parking space coordinates at the current moment are the sum of the parking space coordinates at the previous moment and the update amount after limiting; when the absolute difference is less than or equal to the set threshold, determine that the estimated parking space coordinates at the current moment are the parking space coordinates at the current moment; and obtain the parking space information after parking space fusion based on the parking space coordinates at the current moment.

[0118] The parking space fusion device provided in the embodiment of the present application can execute the technical solution shown in the above-mentioned parking space fusion method embodiment. Its implementation principle and beneficial effects are similar and will not be repeated here.

[0119] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.

[0120] It should be further noted that, although the various steps in the flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0121] It should be noted that the above-described device embodiments are merely illustrative, and the device of the present application may also be implemented in other ways. Furthermore, it should be understood that the division of the various modules of the above-described device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a single physical entity, or physically separated. Furthermore, these modules may be implemented entirely in the form of software invoked by a processing element, or entirely in the form of hardware. Alternatively, some modules may be implemented in the form of software invoked by a processing element, and some modules may be implemented in the form of hardware. For example, the fusion module may be a separate processing element, or it may be integrated into a chip of the above-described device. Furthermore, it may be stored in the form of program code in the memory of the above-described device, and invoked and executed by a processing element of the above-described device. The implementation of other modules is similar. Furthermore, these modules may be fully or partially integrated together, or implemented independently. The processing element here may be an integrated circuit with signal processing capabilities. During implementation, each step of the above-described method or each of the above modules may be performed by hardware integrated logic circuits in the processor element, or by software instructions.

[0122] For example, the above modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0123] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. Available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, Digital Video Discs (DVDs)), or semiconductor media (eg, solid state disks (SSDs)).

[0124] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. Figure 7 As shown, the electronic device 70 of this embodiment includes:

[0125] At least one processor 71; and a memory 72 communicatively connected to the at least one processor;

[0126] The memory 72 stores instructions that can be executed by the at least one processor 71 , and the instructions are executed by the at least one processor 71 to enable the electronic device to execute the method as described in any of the above embodiments.

[0127] Optionally, the memory 72 can be independent or integrated with the processor 71.

[0128] The memory 72 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0129] The processor 71 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. Specifically, when implementing the parking space fusion method described in the aforementioned method embodiment, the electronic device may be, for example, an electronic device with processing capabilities, such as a server.

[0130] Optionally, the electronic device may further include a communication interface 73. In a specific implementation, if the communication interface 73, the memory 72, and the processor 71 are implemented independently, the communication interface 73, the memory 72, and the processor 71 may be interconnected via a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be divided into address buses, data buses, control buses, etc., but this does not mean that there is only one bus or one type of bus.

[0131] Optionally, in a specific implementation, if the communication interface 73, the memory 72 and the processor 71 are integrated on a chip, the communication interface 73, the memory 72 and the processor 71 can complete communication through an internal interface.

[0132] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the aforementioned embodiments and will not be described in detail here.

[0133] An embodiment of the present application also provides a computer-readable storage medium, which stores computer execution instructions. When the computer execution instructions are executed, they are used to implement the method steps in the above method embodiment. The specific implementation method and technical effects are similar and will not be repeated here.

[0134] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0135] An exemplary readable storage medium is coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium may also be an integral part of the processor. The processor and the readable storage medium may be located in an application-specific integrated circuit. Of course, the processor and the readable storage medium may also exist as discrete components in the parking space fusion device.

[0136] An embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed, the method steps in the above method embodiment are implemented. The specific implementation method and technical effects are similar and will not be repeated here.

[0137] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0139] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A parking space fusion method, characterized in that: include: Acquire parking space detection information in the panoramic stitched image, where the parking space detection information includes position information of the sensed parking space and observed noise; Determining a target sensing parking space within a valid range based on the position information of the sensing parking space, wherein the valid range is determined according to sensing accuracy; determining, based on the position information of the sensed parking space, a target distance to the target sensed parking space, and whether the target sensed parking space includes a portion located in an image stitching area of ​​the panoramic stitching image; performing observation noise management on the target perception parking space based on the target distance and whether the target perception parking space includes a portion located in the image stitching area. The observation noise management is used to reduce the participation of the target perception parking space in parking space fusion, wherein the greater the target distance or the greater the portion of the target perception parking space located in the image stitching area, the lower the participation; Parking space fusion is performed based on the information after observation noise management to obtain the parking space information after fusion.

2. The parking space fusion method according to claim 1, characterized in that: The performing observation noise management on the target perception parking space according to the target distance and whether the target perception parking space includes a portion located in the image stitching area includes: If the target distance is less than or equal to a first distance, and the target sensing parking space includes a portion located in the image stitching area, performing a first noise processing strategy on the observed noise corresponding to the target sensing parking space, wherein the range determined by the first distance is within the effective range; If the target distance is less than or equal to the first distance, and the target perception parking space does not include a portion located in the image stitching area, performing a second noise processing strategy on the observation noise corresponding to the target perception parking space; If the target distance is greater than the first distance, and the target perception parking space does not include a portion located in the image stitching area, performing a second noise processing strategy on the observation noise corresponding to the target perception parking space; Among them, the first noise processing strategy refers to amplifying the observed noise by a set value; the first noise processing strategy refers to performing a step-by-step linear amplification process on the observed noise based on the shortest distance from the target perception parking space to the vehicle.

3. The parking space fusion method according to claim 2, characterized in that: Also includes: If the target distance is greater than the first distance, and the target perceived parking space includes a portion located in the image stitching area, a fusion prohibition strategy is executed, where the fusion prohibition strategy means not performing parking space fusion on the target perceived parking space.

4. The parking space fusion method according to any one of claims 1 to 3, characterized in that: The image stitching area is determined in the following manner: Determining, based on calibrated extrinsic parameters of each image acquisition component of the vehicle, first boundary coordinates of the field of view boundary corresponding to each image acquisition component in a vehicle coordinate system, wherein the vehicle coordinate system is a coordinate system with a rear axle center of the vehicle as an origin; Mapping each of the first boundary coordinates to a bird's-eye view coordinate system through an inverse perspective transformation to obtain each of the second boundary coordinates in the bird's-eye view coordinate system; Establishing a center line segment model of a stitching area of ​​adjacent images in the panoramic stitching image according to each first boundary coordinate and the corresponding second boundary coordinate, wherein the center line segment model includes four center line segments, and the center line segments are line segments between each first boundary coordinate and the corresponding second boundary coordinate; For each center line segment in the center line segment model, translate the center line segment in a first direction by a first offset to obtain a first boundary line segment, and translate the center line segment in a second direction opposite to the first direction by a second offset to obtain a second boundary line segment; An area between a first boundary line segment and a second boundary line segment respectively corresponding to each of the center line segments in the center line segment model is determined as the image stitching area.

5. The parking space fusion method according to any one of claims 1 to 3, characterized in that: Whether the target perceived parking space includes a portion located in the image stitching area of ​​the panoramic stitching image is determined in the following manner: Based on the position information of the target sensing parking space, four parking space edges corresponding to the target sensing parking space are obtained; Performing an intersection determination on the four parking space edges corresponding to the target perceived parking space and the boundary line segments of the image stitching area; If any parking space edge of the target perception parking space intersects with any boundary line segment of the image stitching area, it is determined that the target perception parking space includes a portion located in the image stitching area of ​​the panoramic stitching image; If none of the four parking space edges of the target perception parking space intersects with any boundary line segments in the image stitching area, it is determined that the target perception parking space does not include a portion located in the image stitching area of ​​the panoramic stitching image.

6. The parking space fusion method according to claim 5, characterized in that: The performing intersection determination on the four parking space edges corresponding to the target perception parking space and the boundary line segments of the image stitching area includes: Performing a rapid repulsion test on each of the four parking space edges and each of the boundary line segments of the image stitching area; For the target parking space edge and the target boundary line segment detected by the rapid repulsion experiment, determining the vector cross product corresponding to the line segment pair consisting of the target parking space edge and the target boundary line segment according to the straddling experiment; If the vector cross product is less than or equal to 0, it is determined that the target parking space edge intersects the target boundary line segment.

7. The parking space fusion method according to any one of claims 1 to 3, characterized in that: The parking space fusion is performed based on the observed noise management information to obtain the fused parking space information, including: Filtering the information after the observed noise management to obtain an estimated value of the parking space coordinates at the current moment; Determine the absolute value difference between the estimated parking space coordinates and the parking space coordinates at the previous moment; If the absolute difference is greater than a set threshold, the updated amount of the parking space coordinate at the current moment is clipped, and the parking space coordinate at the current moment is determined to be the sum of the parking space coordinate at the previous moment and the updated amount after clipping; If the absolute difference is less than or equal to the set threshold, determining the estimated value of the parking space coordinate at the current moment as the parking space coordinate at the current moment; Based on the parking space coordinates at the current moment, the parking space information after parking space fusion is obtained.

8. A parking space fusion device, characterized in that: include: an acquisition module, configured to acquire parking space detection information from the panoramic stitching image, wherein the parking space detection information includes position information of the sensed parking space and observed noise; an observation filtering module, configured to determine, based on the position information of the sensed parking space, a target sensed parking space within a valid range, wherein the valid range is determined according to the sensing accuracy; The observation filtering module is further configured to determine the target distance from the target perception parking space based on the position information of the perception parking space, and whether the target perception parking space includes a portion located in the image stitching area of ​​the panoramic stitching image. a filtering module, configured to perform observation noise management on the target perception parking space based on the target distance and whether the target perception parking space includes a portion located in the image stitching area, wherein the observation noise management is used to reduce the participation of the target perception parking space in parking space fusion, wherein the greater the target distance or the greater the portion of the target perception parking space located in the image stitching area, the lower the participation; The fusion module is used to perform parking space fusion based on the information after the observed noise management to obtain the parking space information after fusion.

9. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory is used to store computer-executable instructions; The processor is configured to execute the computer-executable instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed.

Images