Obstacle fusion method and system in parking scene, computing device and vehicle
Obtain obstacle information through ultrasonic radar and surround view cameras, generate and fuse grid maps, solving the problem of insufficient efficiency and accuracy of obstacle fusion in parking scenes, and achieving a more efficient and safe parking process.
Patent Information
- Application Number
- CN202310505213.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-06
- Publication Date
- 2024-11-08
AI Technical Summary
In parking scenarios, the obstacle fusion process is inefficient, costly and inadequate.
Obstacle point information is obtained through vehicle-mounted ultrasonic radar and multi-way fisheye cameras, ultrasonic grid maps and circumferential grid maps, and fusion processing is carried out to form a global grid map, combining parking space and pedestrian information to optimize obstacle classification and fusion .
Improve the efficiency and accuracy of obstacle fusion, correct ultrasonic false alarm areas near parking spaces, enhance the accuracy of pedestrian areas, and improve safety during parking.
Smart Images

Figure CN118918561A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field related to intelligent driving environment perception, and specifically to an obstacle fusion method, system, computing device and vehicle in a parking scenario. Background Art
[0002] In the intelligent driving environment, many parking assistance functions have gradually emerged, such as: Auto Parking Assist (APA), Remote Parking Assist (Assist), and Home-zone Parking Assist (HPA). In the field of intelligent driving, the perception devices in these car assistance functions provide high-precision and real-time perception information to the back-end planning and decision-making modules, especially obstacle information, such as the location of pedestrians and vehicles in the surrounding environment.
[0003] In the prior art, obstacle modeling usually adopts two forms: one is the obstacle list form, and the other is the grid map form. In different scenarios and planning modules, the required obstacle modeling forms are different. For example, in HPA, after the environment map is established through the Simultaneous Localization and Mapping (SLAM) technology, the obstacle list form is usually used to describe the surrounding obstacles, and the obstacle tracking and fusion are performed based on the Kalman filter framework to adapt to functions such as automatic obstacle avoidance. In the APA function, the grid map form is often used, based on the Bayesian filtering framework, through a multi-frame filtering model to model the environment and obstacles.
[0004] In addition, different sensor combinations will have different perception algorithms at the front end and fusion frameworks at the back end. However, in the obstacle fusion process of parking scenes, most of them have the disadvantages of poor operating efficiency, high cost, and lack of fusion accuracy. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide an obstacle fusion method, system, computing device and vehicle in a parking scenario, and to improve the efficiency and accuracy of the fusion process by optimizing the obstacle fusion strategy in the parking scenario.
[0006] To solve the above technical problems, as one aspect of the present invention, a method for obstacle fusion in a parking scenario is provided, which at least includes the following steps:
[0007] Obtaining ultrasonic obstacle point information around the vehicle body according to the vehicle-mounted ultrasonic radar, and generating an ultrasonic grid map based on the ultrasonic obstacle points, wherein the ultrasonic grid map includes a plurality of grids, each grid corresponding to ultrasonic classification information, and the ultrasonic classification information includes: idle, occupied, and unknown;
[0008] Obtaining surround view obstacle point information according to the multi-channel fisheye camera on the vehicle, generating a surround view grid map based on the surround view obstacle points, wherein the surround view grid map includes a plurality of grids, each grid corresponds to surround view classification information, and the surround view classification information includes: idle and preset obstacle types;
[0009] The ultrasonic grid map and the surround view grid map are fused to form a global grid map, and obstacle classification information of each grid in the global grid map is determined according to the corresponding ultrasonic classification information and the surround view classification information, wherein the obstacle classification information includes: idle, occupied, and various obstacle types;
[0010] After selecting a target parking space in the global grid map, pedestrian information obtained based on the surround obstacle point information and input parking space information are added to the global grid map, and the parking space information at least includes: parking space area information and available status information.
[0011] Wherein, generating an ultrasonic grid map based on the ultrasonic obstacle points further comprises:
[0012] Establishing local and global ultrasonic grid maps, setting the occupied probability values of all grids to be uniformly set to the initial first probability value, and the category is unknown;
[0013] According to the preset inverse sensor model, the obstacle point information measured by the sensor at the current moment is mapped into the local ultrasonic grid map of the vehicle coordinate system, and the current classification and update probability of each grid of the local ultrasonic grid map are determined;
[0014] According to the current position information of the vehicle, the local ultrasonic grid map is projected into the global ultrasonic grid map;
[0015] For the area where the local grid map and the global map overlap, the global grid map is iteratively updated according to the current classification and update probability of each grid in the local ultrasound grid map and the historical cumulative probability of the corresponding grid in the global ultrasound grid map, so as to determine the latest cumulative probability of the corresponding grid in the global ultrasound grid map;
[0016] The latest cumulative probability of each grid is compared with the decision threshold corresponding to each ultrasonic classification information, and the ultrasonic classification information of each grid in the global ultrasonic grid map is updated.
[0017] Among them, according to the preset inverse sensor model, the ultrasonic obstacle point information measured by the sensor at the current moment is mapped to the local ultrasonic grid map with the vehicle position as the origin, and the current classification and update probability of each grid of the local ultrasonic grid map are determined, specifically:
[0018] In the local ultrasonic grid map, the current classification and update probability of each grid of the local ultrasonic grid map are determined according to the distance value between the ultrasonic obstacle point and the origin of the vehicle coordinate system, the distance value at the same angle between the current grid and the obstacle point, and the size of the obstacle point, wherein each classification corresponds to a preset update probability.
[0019] Wherein, for the area where the local grid map and the global map overlap, the global grid map is iteratively updated according to the current classification and update probability of each grid in the local ultrasound grid map and the historical cumulative probability of the corresponding grid in the global ultrasound grid map, so as to determine the latest cumulative probability of the corresponding grid in the global ultrasound grid map; specifically:
[0020] The corresponding grid m in the global ultrasonic grid map is obtained according to the following formula i The logarithm of the latest cumulative probability of :
[0021] L(m i |Z 1:t ,x 1:t )=L(m i |Z t ,x t )+L(m i |Z 1:t-1 ,x 1:t-1 )-L(m i )
[0022] Among them, L(m i |Z 1:t ,x 1:t ) is the grid m i The logarithm of the latest cumulative probability; L(m i |Z t ,x t ) is the grid m i The logarithm of the probability of updating in the local grid map at the current time t; L(m i |Z 1:t-1 ,x 1:t-1 ) is the grid m i The logarithm of the historical cumulative probability from the beginning to time t-1; L(m i ) is the grid m i The logarithm of the initial first probability value of ;
[0023] The latest cumulative probability of each grid is obtained according to the formal calculation:
[0024]
[0025] Wherein, generating a surround grid map based on the surround obstacle points further includes:
[0026] Establish local and global look-around grid maps, set the occupied probability values of all grids to be uniformly set to the initial second probability value, and the category is idle;
[0027] Determine an area of interest in a local surround grid map, and obtain surround obstacle point information in the area of interest, where each surround obstacle point includes: a distance value from the origin of the vehicle coordinate system and an obstacle type at the current point;
[0028] According to a preset inverse sensor model, the obstacle point information in the region of interest at the current moment is mapped to a local surround grid map of the vehicle coordinate system, and the current classification and update probability of each grid of the local surround grid map are updated;
[0029] According to the current position information of the vehicle, the local surround grid map is projected onto the global surround grid map;
[0030] For the area where the local grid map and the global map overlap, the global grid map is iteratively updated according to the current classification and update probability of each grid in the local look-around grid map and the historical cumulative probability of the corresponding grid in the global look-around grid map, and the latest cumulative probability of the corresponding grid in the global look-around grid map is determined;
[0031] The latest cumulative probability of each grid is compared with the decision threshold corresponding to each surround view classification information, and the surround view classification information of each grid in the global surround view grid map is updated.
[0032] Among them, according to the preset inverse sensor model, the obstacle point information in the current region of interest is mapped to the local surround grid map of the vehicle coordinate system, and the current classification and update probability of each grid of the local surround grid map are updated, specifically:
[0033] In the local look-around grid map, the current classification and update probability of the corresponding grid in the local look-around grid map are updated according to the obstacle type of the look-around obstacle point, wherein each classification corresponds to a preset update probability.
[0034] The ultrasonic grid map and the surround view grid map are fused to form a global grid map, and obstacle classification information of each grid in the global grid map is determined according to the corresponding ultrasonic classification information and surround view classification information, further comprising:
[0035] Create a global grid map;
[0036] Obstacle classification information of each grid in the global grid map is determined according to the ultrasound classification information in the ultrasound global grid map and the surround classification information in the surround global grid map in the following manner:
[0037] When the ultrasonic classification information corresponding to a grid is occupied, if the corresponding surround view classification information is idle, then its obstacle classification information is determined to be occupied; if the corresponding surround view classification information is an obstacle type, then its obstacle classification information is determined to be the obstacle type;
[0038] When the ultrasonic classification information corresponding to a grid is idle, its obstacle classification information is directly determined to be idle;
[0039] When the ultrasonic classification information corresponding to a grid is unknown, its obstacle classification information is directly determined as the current surround view classification information.
[0040] Among them, further include:
[0041] In the local surround grid map, obstacle points of pedestrian categories are clustered, and based on Kalman filtering, they are continuously tracked in the vehicle coordinate system to obtain a dynamic pedestrian target list.
[0042] Wherein, after selecting the target parking space in the global grid map, adding the pedestrian information obtained according to the obstacle point information and the input parking space information into the global grid map, further comprising:
[0043] For available parking spaces, set the type attribute of the corresponding area in the global grid map to free; if the parking area contains a wheel stopper, set the area from the wheel stopper position to the end of the parking space to occupied according to the position of the wheel stopper;
[0044] The dynamic pedestrian target list is traversed, the position of the pedestrian at the time of selecting the parking space is projected into the global grid map, and the category attribute is set to pedestrian; the historical trajectory of each pedestrian is calculated to determine whether it overlaps with obstacles in the current global grid map, and if so, the obstacle area in the overlapping area is removed.
[0045] Among them, further include:
[0046] The obstacles on both sides of the target parking space are respectively fitted into a parallelogram obstacle area, and the angle of the parallelogram is adapted to the angle of the parking space;
[0047] When it is determined that the parking space type of the target parking space is a regular-sized parking space, the sizes of obstacles on the left and right sides of the target parking space are expanded at a first ratio;
[0048] When the obstacle detected in front of the vehicle is a wall, the road in front of the vehicle is set as an impassable area; when the obstacle detected in front of the vehicle is other obstacles, the size of the obstacle in front of the vehicle is expanded at a second ratio, and the second ratio is greater than the first ratio;
[0049] During the parking process, the current global grid map is updated based on the real-time ultrasonic obstacle point information and surround obstacle point information.
[0050] Accordingly, another aspect of the present invention further provides an obstacle fusion system in a parking scenario, which is used to implement the aforementioned method, and at least includes:
[0051] An ultrasonic obstacle point information acquisition unit, used to obtain ultrasonic obstacle point information around the vehicle body according to the vehicle-mounted ultrasonic radar;
[0052] An ultrasonic grid map generating unit, configured to generate an ultrasonic grid map based on the ultrasonic obstacle point, wherein the ultrasonic grid map includes a plurality of grids, each grid corresponding to ultrasonic classification information, and the ultrasonic classification information includes: idle, occupied, and unknown;
[0053] A surround view obstacle point information acquisition unit, used to acquire surround view obstacle point information based on the multi-channel fisheye cameras on the vehicle;
[0054] A surround view grid map generating unit, used to generate a surround view grid map based on the surround view obstacle point, wherein the surround view grid map includes a plurality of grids, each grid corresponds to surround view classification information, and the surround view classification information includes: idle and preset obstacle types;
[0055] A global grid map fusion unit is used to fuse the ultrasonic grid map and the surround view grid map to form a global grid map, and determine the obstacle classification information of each grid in the global grid map according to the corresponding ultrasonic classification information and the surround view classification information, wherein the obstacle classification information includes: idle, occupied and various obstacle types;
[0056] The updating processing unit is used to add pedestrian information obtained according to the surrounding obstacle point information and input parking space information to the global grid map after selecting the target parking space in the global grid map, wherein the parking space information at least includes: parking space area information and available status information.
[0057] Among them, it further includes: an analysis and processing unit, which is used to cluster obstacle points of pedestrian categories in the local surround grid map, and continuously track them in the vehicle coordinate system based on Kalman filtering to obtain a dynamic pedestrian target list.
[0058] Among them, further include:
[0059] A parking restriction processing unit, used for selecting a target parking space to be parked in the global grid map, restricting the size of the target parking space area, forming a global map for parking, and updating the global map in real time during the parking process;
[0060] The parking restriction processing unit further comprises:
[0061] A fitting processing unit is used to fit the obstacles on both sides of the target parking space into a parallelogram obstacle area, where the angle of the parallelogram is adapted to the angle of the parking space;
[0062] The obstacle limiting unit on both sides is used to expand the obstacle sizes on the left and right sides of the target parking space at a first ratio when it is determined that the parking space type of the target parking space is a regular parking space;
[0063] A vehicle front obstacle limiting unit, used to set the road in front of the vehicle as an impassable area when the obstacle detected in front of the vehicle is a wall; when the obstacle detected in front of the vehicle is other obstacles, expand the size of the obstacle in front of the vehicle at a second ratio, wherein the second ratio is greater than the first ratio;
[0064] The update processing unit is used to update the current global grid map according to the ultrasonic obstacle point information and the surround obstacle point information obtained in real time during the parking process.
[0065] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned method are implemented.
[0066] According to another aspect of the present invention, a computing device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned method when executing the computer program.
[0067] Another aspect of the present invention further provides a vehicle, on which at least an automatic parking planning device is provided, and the obstacle fusion system in the parking scenario as described above is deployed in the automatic parking planning device.
[0068] The implementation of the embodiments of the present invention has the following beneficial effects:
[0069] The present invention provides an obstacle fusion method, system, computing device and vehicle in a parking scenario, which constructs an ultrasonic grid map based on obstacle points of ultrasonic radar and a surround view grid map based on semantically segmented obstacle points provided by a surround view camera. The surround view grid map and the ultrasonic grid map are fused according to a preset obstacle attribute fusion table. At the same time, the parking space information and dynamic pedestrian information are fused into the final obstacle map, providing a reliable environment map for the planning module, thereby improving the efficiency and accuracy of obstacle fusion.
[0070] In the process of constructing the surround grid map of the present invention, applying specific obstacle type information to the Bayesian filter mapping process can improve the accuracy of the edge and avoid the deficiency in the prior art that some obstacle edge information is filtered out due to fluctuations in the distance value of the surround obstacle point;
[0071] By adding the fused parking space information to the final global map, the ultrasonic false alarm areas near the parking spaces can be corrected.
[0072] Pedestrians are tracked individually and map obstacle information is corrected based on their trajectories, improving the accuracy of pedestrian areas in the map.
[0073] During the parking phase, after selecting the target parking space, the safety of parking can be improved by imposing targeted restrictions on the left and right areas of the parking space and optimizing special scenarios (such as narrow parking spaces and situations where the front of the vehicle is blocked).
[0074] The present invention targets medium and low computing power platforms. Based on the sensor information of ultrasonic radar and surround-view camera, the automatic parking function optimizes the perception requirements, thereby improving the practicality and operating efficiency of the algorithm, and can be applied to most parking scenarios in China. BRIEF DESCRIPTION OF THE DRAWINGS
[0075] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, without paying creative labor, other drawings obtained based on these drawings still belong to the scope of the present invention.
[0076] Figure 1 A schematic diagram of the main process of an embodiment of an obstacle fusion method in a parking scenario provided by the present invention;
[0077] Figure 2 A more detailed schematic diagram of the application principle of the present invention;
[0078] Figure 3It is a partial schematic diagram of the global ultrasonic grid map involved in the present invention;
[0079] Figure 4 It is a partial schematic diagram of the global look-around grid map involved in the present invention;
[0080] Figure 5 A schematic diagram showing the comparison between the present method and the existing method for obstacle recognition;
[0081] Figure 6 This is a schematic diagram of using this method to remove the ultrasonic false alarm area;
[0082] Figure 7 A schematic diagram of the structure of an embodiment of an obstacle fusion system in a parking scenario provided by the present invention;
[0083] Figure 8 for Figure 7 Schematic diagram of the structure of the parking restriction processing unit. DETAILED DESCRIPTION
[0084] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.
[0085] like Figure 1 FIG. 1 is a schematic diagram showing the main process of an embodiment of an obstacle fusion method in a parking scene provided by the present invention. Figures 2 to 6 As shown, in this embodiment, the method at least includes the following steps:
[0086] Step S10, obtaining ultrasonic obstacle point information around the vehicle body according to the vehicle-mounted ultrasonic radar, and generating an ultrasonic grid map based on the ultrasonic obstacle points, wherein the ultrasonic grid map includes a plurality of grids, each grid corresponding to ultrasonic classification information, and the ultrasonic classification information includes: idle, occupied, and unknown;
[0087] First, the vehicle-mounted ultrasonic radar is required to obtain the ultrasonic obstacle point information around the vehicle body. Specifically, the vehicle-mounted ultrasonic module obtains the output obstacle point list based on triangulation positioning and other methods. Each ultrasonic obstacle point information at least includes the coordinate position (x, y) of the obstacle point in the vehicle coordinate system.
[0088] For the above obstacle point information, an ultrasonic grid map can be constructed based on the Bayesian mapping framework, wherein each grid in the final grid map needs to contain an occupancy probability value and ultrasonic classification information, and the ultrasonic classification information can be free, unknown, and occupied.
[0089] In a specific example, in the step S10, the step of generating an ultrasonic grid map based on the ultrasonic obstacle points further includes:
[0090] Step S100, establish a local and a global ultrasonic grid map, set the occupied probability values of all its grids to a unified initial first probability value (such as 0.5), and set the classification to unknown;
[0091] Step S101, according to the preset inverse sensor model, map the information of the surrounding obstacle points measured by the sensor at the current moment to the local ultrasonic grid map in the vehicle coordinate system, and determine the current classification and updated probability of each grid in the local ultrasonic grid map;
[0092] In the local ultrasonic grid map, according to the relationship between the distance value of the ultrasonic obstacle point from the origin of the vehicle coordinate system, the distance value at the same angle as the current grid and the obstacle point, and the size of the obstacle point, determine the current classification and updated probability of each grid in the local ultrasonic grid map, where each classification corresponds to a preset updated probability.
[0093] In a specific example, according to the inverse sensor model, map the sensor measurement data z at the current moment (such as the t moment) t to the local grid map with the position x of the vehicle itself t as the origin, and determine the state of the local grid map.
[0094] The model is usually as follows. R is the distance value calculated according to the position of the obstacle from the origin of the vehicle coordinate system, r is the distance value at the same angle as the obstacle point. d is the correction amount of the size of the obstacle point. In an application environment, Table 1 below is used to determine the current updated probability of each grid.
[0095] Table 1 Ultrasonic Grid Update Probability Comparison Table
[0096]
[0097] Specifically, in the local ultrasonic grid map, if the position information corresponding to a certain grid satisfies r < R - d, the category of this grid is determined to be free, and its current updated probability is determined to be -0.2, and other categories are judged and processed according to the conditions in the table.
[0098] Step S102, project the local ultrasonic grid map into the global ultrasonic grid map according to the current position information of the vehicle;
[0099] Step S103, the area where the local grid map and the global map overlap is updated in the form of a logarithmic odds ratio, according to the current classification and update probability of each grid in the local ultrasound grid map and the historical cumulative probability of the corresponding grid in the global ultrasound grid map, and the global grid map is iteratively updated to determine the latest cumulative probability of the corresponding grid in the global ultrasound grid map;
[0100] Specifically, the corresponding grid m in the global ultrasound grid map is obtained according to the following formula: i The logarithm of the latest cumulative probability of :
[0101] L(m i |Z 1:t ,x 1:t )=L(m i |Z t ,x t )+L(m i |Z 1:t-1 ,x 1:t-1 )-L(m i )
[0102] Among them, L(m i |Z 1:t ,x 1:t ) is the grid m i The logarithm of the latest cumulative probability; L(m i |Z t ,x t ) is the grid m i The logarithm of the probability of updating in the local grid map at the current time t; L(m i |Z 1:t-1 ,x 1:t-1 ) is the grid m i The logarithm of the historical cumulative probability from the beginning to time t-1; L(m i ) is the grid m i The logarithm of the initial first probability value of ;
[0103] The latest cumulative probability of each grid is obtained according to the formal calculation:
[0104]
[0105] Step S104, compare the latest cumulative probability of each grid with the decision threshold corresponding to each ultrasonic classification information, update the ultrasonic classification information of each grid in the global ultrasonic grid map, and modify the maximum and minimum values of the grid probability according to the probability value range.
[0106] like Figure 3As shown, a partial schematic diagram of a global ultrasonic grid map involved in the present invention is shown, wherein dark grids are occupied, white grids are idle, and light grids are unknown; in other examples, different colors may also be used to represent different ultrasonic classification information.
[0107] Step S11, obtaining surround view obstacle point information according to the multi-channel fisheye camera on the vehicle, and generating a surround view grid map based on the surround view obstacle points, wherein the surround view grid map includes a plurality of grids, each grid corresponds to surround view classification information, and the surround view classification information includes: idle and preset obstacle types;
[0108] First, it is necessary to obtain the surround view obstacle point information based on the multi-channel fisheye cameras on the vehicle.
[0109] For example, in a specific example, four fisheye cameras are set on the front bumper, trunk, and two rearview mirrors of the vehicle. The original images are captured by the four fisheye cameras, and after being processed by the calibration file preset by the vehicle (such as de-distortion, joint calibration, projection transformation, etc.), they are spliced and fused to form a spliced image of multiple images. It can be understood that different calibration algorithms can be used to process different types of vehicles to form a calibration file with a unified format. In this calibration file, it is necessary to record the internal and external parameters of each of the four fisheye cameras, as well as the corresponding relationship between the pixels on the spliced image and the coordinates in the actual vehicle coordinate system. And through semantic segmentation, the drivable area (free space) information can be obtained, and the obstacle type of each obstacle point can be obtained.
[0110] It can be understood that the obstacle point list is output by the front-end surround view module based on the stitching and calibration results. The obstacle points surround the vehicle in a circle and are arranged at a fixed angle. The number of points is fixed to N (N>100), and the angle value of adjacent points is 360° / N. The attributes of each point include: the distance value to the center point of the rear axle of the vehicle (vehicle coordinate system), and the obstacle type of the current point (such as pedestrians, pillars, walls, etc.).
[0111] After obtaining the surround obstacle point information, in the present invention, an improved Bayesian mapping method can be used to remove unknown types, replace the occupancy type with multiple obstacle types, and simultaneously track dynamic pedestrians separately.
[0112] In a specific example, in the step S11, generating a surround grid map based on the surround obstacle points further includes:
[0113] Step S110, establishing local and global look-around grid maps, setting the occupied probability values of all grids to be uniformly set to an initial second probability value (such as 0.2), and the category is free;
[0114] Step S111, determining an area of interest in the local surround grid map (such as the R+d area of each obstacle), and obtaining surround obstacle point information in the area of interest, where each surround obstacle point includes: a distance value from the origin of the vehicle coordinate system and an obstacle type at the current point;
[0115] Step S112, mapping the surround obstacle point information in the region of interest at the current moment into the local surround grid map of the vehicle coordinate system according to the preset inverse sensor model, and updating the current classification and update probability of each grid of the local surround grid map;
[0116] In the local look-around grid map, the current classification and update probability of the corresponding grid in the local look-around grid map are updated according to the obstacle type of the look-around obstacle point, wherein each classification corresponds to a preset update probability.
[0117] Specifically, in one example, the sensor measurement data zt at time t is mapped to a local grid map with the vehicle position xt as the origin, the state of the local grid map is determined, and then the grid points within the range of R+d are updated according to the distance value R and category of each obstacle point. In an application environment, the following Table 2 is used to determine the current update probability of each grid.
[0118] Table 2. Comparison table of the probability of updating the surround grid
[0119] Point Status Pedestrian category Non-pedestrian obstacle categories Update Probability +0 +0.2
[0120] Step S113, projecting the local surround grid map onto the global surround grid map according to the current position information of the vehicle;
[0121] Step S114, for the area where the local grid map and the global map overlap, the global grid map is iteratively updated according to the current classification and update probability of each grid in the local look-around grid map and the historical cumulative probability of the corresponding grid in the global look-around grid map, to determine the latest cumulative probability of the corresponding grid in the global look-around grid map;
[0122] The specific calculation formula may refer to and be combined with the two formulas in the above step S103.
[0123] Step S115, compare the latest cumulative probability of each grid with the determination threshold corresponding to each surround view classification information, update the surround view classification information of each grid in the global surround view grid map, and modify the maximum and minimum values of the grid probability according to the probability value range.
[0124] like Figure 4As shown, a partial schematic diagram of a global surround view grid map involved in the present invention is shown, wherein a dark grid indicates that the obstacle type is a pillar, a white grid is in an idle state, and a light grid indicates that the obstacle type is a wall; in other examples, different colors can also be used to represent different surround view classification information.
[0125] It is understandable that in the present invention, in the above steps S110 to S115, the unknown type is removed because in practical applications, when looking around at obstacle points generated based on semantic segmentation, the distance value is prone to jump when passing over certain obstacles (such as pillars in the basement). If the map is built based on traditional Bayesian filtering, the obstacle edge information will be filtered out due to the fluctuation of the distance value. Figure 5 As shown, a comparison diagram of the local view grid map obtained by using the traditional Bayesian filter mapping and the method of the present invention in an application scenario is shown; the left figure is a photo, the middle figure is the view grid map obtained by the traditional method, and the right figure is the view grid map obtained by the method of the present invention. It can be seen that the right figure can more accurately reflect the column part (shown by the dotted circle).
[0126] Step S12, fusing the ultrasonic grid map and the surround view grid map to form a global grid map, and determining obstacle classification information of each grid in the global grid map according to the corresponding ultrasonic classification information and surround view classification information, wherein the obstacle classification information includes: idle, occupied, and various obstacle types;
[0127] In a specific example, the step S12 further includes:
[0128] Create a global grid map;
[0129] Obstacle classification information of each grid in the global grid map is determined according to the ultrasound classification information in the ultrasound global grid map and the surround classification information in the surround global grid map in the following manner:
[0130] When the ultrasonic classification information corresponding to a grid is occupied, if the corresponding surround view classification information is idle, then its obstacle classification information is determined to be occupied; if the corresponding surround view classification information is an obstacle type, then its obstacle classification information is determined to be the obstacle type;
[0131] When the ultrasonic classification information corresponding to a grid is idle, its obstacle classification information is directly determined to be idle;
[0132] When the ultrasonic classification information corresponding to a grid is unknown, its obstacle classification information is directly determined as the current surround view classification information.
[0133] Specifically, the above fusion rules can be referred to as shown in Table 3 below:
[0134] Table 3, obstacle attribute fusion table
[0135]
[0136] Step S13, after selecting the target parking space in the global grid map, adding the pedestrian information obtained based on the obstacle point information and the input parking space information to the global grid map, and performing parking area restriction.
[0137] It can be understood that before step S13, the following steps are further included:
[0138] Step S15, obtaining fused parking space information, wherein the fused parking space information at least includes: parking space area information and available status information;
[0139] More specifically, the visual parking space and the ultrasonic parking space obtain fused parking space information through IOU matching and other methods; the fused parking space information may include the parking space ID number, the coordinates of the four corner points (in the vehicle coordinate system), the parking space direction, the available status, the confidence level, etc.
[0140] And step S16, clustering the obstacle points of pedestrian categories in the local surround grid map, and continuously tracking them in the vehicle coordinate system based on Kalman filtering to obtain a dynamic pedestrian target list.
[0141] In a specific example, the step S13 further includes:
[0142] For available parking spaces, the type attribute of the corresponding area in the global grid map is set to free; if the parking area contains a wheel stopper, the area from the wheel stopper position to the end of the parking space is set to occupied according to the position of the wheel stopper to avoid collision with the wheel stopper during planning; for unavailable parking spaces, the corresponding area of the grid map is not processed. Figure 6 As shown, it shows a schematic diagram before and after adding the fused parking space information, wherein the dotted circle in the left figure is the ultrasonic false alarm area, and after adding the fused parking space information, the said area prone to false alarm can be removed.
[0143] The dynamic pedestrian target list is traversed, the position of the pedestrian at the time of selecting the parking space is projected into the global grid map, and the category attribute is set to pedestrian; the historical trajectory of each pedestrian is calculated to determine whether it overlaps with obstacles in the current global grid map, and if so, the obstacle area in the overlapping area is removed.
[0144] Furthermore, the method of the present invention further includes a process of restricting the target parking space and updating the global grid map in real time during parking, specifically including:
[0145] The obstacles on both sides of the target parking space are respectively fitted into a parallelogram obstacle area, and the angle of the parallelogram is adapted to the angle of the parking space, for example, the axial angle of the parallelogram is the same as the angle of the parking space;
[0146] When it is determined that the parking space type of the target parking space is a regular-sized parking space, the sizes of obstacles on the left and right sides of the target parking space are expanded by a first ratio (such as 1.1 times);
[0147] When the obstacle detected in front of the vehicle is a wall, the road in front of the vehicle is set as an impassable area; when the obstacle detected in front of the vehicle is other obstacles, the size of the obstacle in front of the vehicle is expanded by a second ratio (such as 1.2 times), and the second ratio is greater than the first ratio;
[0148] It is understandable that the above-mentioned global map with targeted restrictions will be provided to the parking planning device of the vehicle for use in subsequent parking operations;
[0149] During the parking process, the current global grid map is updated based on the real-time ultrasonic obstacle point information and surround obstacle point information.
[0150] like Figure 7 FIG. 1 is a schematic diagram showing a structure of an embodiment of an obstacle fusion system in a parking scenario provided by the present invention. Figure 8 As shown, in this embodiment, the system 1 is used to implement the aforementioned Figures 1 to 4 The method described, the system 1 at least comprises:
[0151] The ultrasonic obstacle point information acquisition unit 10 is used to obtain ultrasonic obstacle point information around the vehicle body according to the vehicle-mounted ultrasonic radar;
[0152] An ultrasonic grid map generating unit 11 is used to generate an ultrasonic grid map based on the ultrasonic obstacle point, wherein the ultrasonic grid map includes a plurality of grids, each grid corresponds to ultrasonic classification information, and the ultrasonic classification information includes: idle, occupied, and unknown;
[0153] A surround view obstacle point information acquisition unit 12, used to acquire surround view obstacle point information based on the multi-channel fisheye camera on the vehicle;
[0154] A surround view grid map generating unit 13 is used to generate a surround view grid map based on the surround view obstacle point, wherein the surround view grid map includes a plurality of grids, each grid corresponds to surround view classification information, and the surround view classification information includes: idle and preset obstacle types;
[0155] A global grid map fusion unit 14 is used to fuse the ultrasonic grid map and the surround view grid map to form a global grid map, and determine obstacle classification information of each grid in the global grid map according to the corresponding ultrasonic classification information and surround view classification information, wherein the obstacle classification information includes: idle, occupied and various obstacle types;
[0156] The updating processing unit 5 is used to add pedestrian information obtained according to the surrounding obstacle point information and input parking space information to the global grid map after selecting the target parking space in the global grid map, and the parking space information at least includes: parking space area information and available status information.
[0157] Among them, further include:
[0158] A parking restriction processing unit 16, used for selecting a target parking space to be parked in the global grid map, restricting the size of the target parking space area, forming a global map for parking, and updating the global map in real time during the parking process;
[0159] like Figure 8 As shown, the parking restriction processing unit 16 further includes:
[0160] A fitting processing unit 160 is used to fit the obstacles on both sides of the target parking space into a parallelogram obstacle area, where the angle of the parallelogram is adapted to the angle of the parking space;
[0161] The obstacle limiting unit 161 on both sides is used to expand the size of obstacles on the left and right sides of the target parking space at a first ratio when it is determined that the parking space type of the target parking space is a regular parking space;
[0162] The vehicle front obstacle limiting unit 162 is used to set the road in front of the vehicle as an impassable area when the obstacle detected in front of the vehicle is a wall; when the obstacle detected in front of the vehicle is other obstacles, the size of the obstacle in front of the vehicle is expanded by a second ratio, and the second ratio is greater than the first ratio;
[0163] The update processing unit 163 is used to update the current global grid map according to the ultrasonic obstacle point information and the surround obstacle point information obtained in real time during the parking process.
[0164] It is understandable that the system 1 further includes: an analysis and processing unit 17, which is used to cluster obstacle points of pedestrian categories in the local surround grid map, and continuously track them in the vehicle coordinate system based on Kalman filtering to obtain a dynamic pedestrian target list.
[0165] For more details, please refer to and combine the above Figures 1 to 4 The description is not repeated here.
[0166] In another aspect of the present invention, there is also provided a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the above-mentioned Figures 1 to 4 For more details, please refer to and combine the above Figures 1 to 4 The description is not repeated here.
[0167] In another aspect of the present invention, a computing device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the aforementioned Figures 1 to 4 For more details, please refer to and combine the above Figures 1 to 4 The description is not repeated here.
[0168] Another aspect of the present invention further provides a vehicle, on which at least an automatic parking planning device is provided, wherein the automatic parking planning device is deployed with the aforementioned Figures 7 and 8 For more details, please refer to and combine the above Figures 7 and 8 The description is not repeated here.
[0169] The implementation of the embodiments of the present invention has the following beneficial effects:
[0170] The present invention provides an obstacle fusion method, system, computing device and vehicle in a parking scenario, which constructs an ultrasonic grid map based on obstacle points of ultrasonic radar and a surround view grid map based on semantically segmented obstacle points provided by a surround view camera. The surround view grid map and the ultrasonic grid map are fused according to a preset obstacle attribute fusion table. At the same time, the parking space information and dynamic pedestrian information are fused into the final obstacle map, providing a reliable environment map for the planning module, thereby improving the efficiency and accuracy of obstacle fusion.
[0171] In the process of constructing the surround grid map of the present invention, applying specific obstacle type information to the Bayesian filter mapping process can improve the accuracy of the edge and avoid the deficiency in the prior art that some obstacle edge information is filtered out due to fluctuations in the distance value of the surround obstacle point;
[0172] By adding the fused parking space information to the final global map, the ultrasonic false alarm areas near the parking spaces can be corrected.
[0173] Pedestrians are tracked individually and map obstacle information is corrected based on their trajectories, improving the accuracy of pedestrian areas in the map.
[0174] During the parking phase, after selecting the target parking space, the safety of parking can be improved by imposing targeted restrictions on the left and right areas of the parking space and optimizing special scenarios (such as narrow parking spaces and blocked areas in front of the vehicle).
[0175] The present invention targets medium and low computing power platforms. Based on the sensor information of ultrasonic radar and surround-view camera, the automatic parking function optimizes the perception requirements, thereby improving the practicality and operating efficiency of the algorithm, and can be applied to most parking scenarios in China.
[0176] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0177] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0178] The above disclosure is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.
Claims
1. An obstacle fusion method in a parking scene, characterized in that: At least the following steps are included: Obtaining ultrasonic obstacle point information around the vehicle body according to the vehicle-mounted ultrasonic radar, and generating an ultrasonic grid map based on the ultrasonic obstacle point information, wherein the ultrasonic grid map includes a plurality of grids, each grid corresponding to ultrasonic classification information, and the ultrasonic classification information includes: idle, occupied, and unknown; Obtaining surround view obstacle point information according to the vehicle-mounted camera, and generating a surround view grid map based on the surround view obstacle point information, wherein the surround view grid map includes a plurality of grids, each grid corresponds to surround view classification information, and the surround view classification information includes: idle and preset obstacle types; The ultrasonic grid map and the surround view grid map are fused to form a global grid map, and obstacle classification information of each grid in the global grid map is determined according to the corresponding ultrasonic classification information and the surround view classification information, wherein the obstacle classification information includes: idle, occupied, and various obstacle types; After selecting the target parking space in the global grid map, pedestrian information obtained based on the surround obstacle point information and input parking space information are added to the global grid map, and parking area restrictions are performed; the parking space information at least includes: parking space area information and available status information.
2. The method according to claim 1, characterized in that The generating of the ultrasonic grid map based on the ultrasonic obstacle point information further comprises: Establishing local and global ultrasound grid maps, setting the occupied probability values of all grids to be uniformly set to an initial first probability value, and the category is unknown; According to the preset inverse sensor model, the ultrasonic obstacle point information measured by the sensor at the current moment is mapped into the local ultrasonic grid map of the vehicle coordinate system, and the current classification and update probability of each grid of the local ultrasonic grid map are determined; According to the current position information of the vehicle, the local ultrasonic grid map is projected into the global ultrasonic grid map; For the area where the local grid map and the global map overlap, the global grid map is iteratively updated according to the current classification and update probability of each grid in the local ultrasound grid map and the historical cumulative probability of the corresponding grid in the global ultrasound grid map, so as to determine the latest cumulative probability of the corresponding grid in the global ultrasound grid map; The latest cumulative probability of each grid is compared with the decision threshold corresponding to each ultrasonic classification information, and the ultrasonic classification information of each grid in the global ultrasonic grid map is updated.
3. The method according to claim 2, characterized in that According to the preset inverse sensor model, the ultrasonic obstacle point information measured by the sensor at the current moment is mapped to the local ultrasonic grid map with the vehicle position as the origin, and the current classification and update probability of each grid of the local ultrasonic grid map are determined, specifically: In the local ultrasonic grid map, the current classification and update probability of each grid of the local ultrasonic grid map are determined according to the distance value between the ultrasonic obstacle point and the origin of the vehicle coordinate system, the distance value at the same angle between the current grid and the obstacle point, and the size of the obstacle point, wherein each classification corresponds to a preset update probability.
4. The method according to claim 3, characterized in that The method of iteratively updating the global grid map according to the current classification and update probability of each grid in the local ultrasonic grid map and the historical cumulative probability of the corresponding grid in the global ultrasonic grid map in the area where the local grid map and the global map overlap, and determining the latest cumulative probability of the corresponding grid in the global ultrasonic grid map, further comprises: The corresponding grid m in the global ultrasonic grid map is obtained according to the following formula i The logarithm of the latest cumulative probability of : L(m i |Z 1:t ,x 1:t )=L(m i |Z t ,x t )+L(m i |Z 1:t-1 ,x 1:t-1 )-L(m i ) Among them, L(m i |Z 1:t ,x 1:t ) is the grid m i The logarithm of the latest cumulative probability; L(m i |Z t ,x t ) is the grid m i The logarithm of the probability of updating in the local grid map at the current time t; L(m i |Z 1:t-1 ,x 1:t-1 ) is the grid m i The logarithm of the historical cumulative probability from the beginning to time t-1; L(m i ) is the grid m i The logarithm of the initial first probability value of ; The latest cumulative probability of each grid is obtained according to the formal calculation:
5. The method according to any one of claims 1 to 4, characterized in that: Based on the obstacle point information, a surround grid map is generated, further comprising: Establish local and global look-around grid maps, set the occupied probability values of all grids to be uniformly set to the initial second probability value, and the category is idle; Determine an area of interest in a local surround grid map, and obtain surround obstacle point information in the area of interest, where each surround obstacle point includes: a distance value from the origin of the vehicle coordinate system and an obstacle type at the current point; According to a preset inverse sensor model, the obstacle point information in the region of interest at the current moment is mapped to a local surround grid map of the vehicle coordinate system, and the current classification and update probability of each grid of the local surround grid map are updated; According to the current position information of the vehicle, the local surround grid map is projected onto the global surround grid map; For the area where the local grid map and the global map overlap, the global grid map is iteratively updated according to the current classification and update probability of each grid in the local look-around grid map and the historical cumulative probability of the corresponding grid in the global look-around grid map, and the latest cumulative probability of the corresponding grid in the global look-around grid map is determined; The latest cumulative probability of each grid is compared with the decision threshold corresponding to each surround view classification information, and the surround view classification information of each grid in the global surround view grid map is updated.
6. The method according to claim 5, characterized in that According to the preset inverse sensor model, the obstacle point information in the current region of interest is mapped to the local surround grid map of the vehicle coordinate system, and the current classification and update probability of each grid of the local surround grid map are updated, specifically: In the local surround grid map, the current classification and update probability of the corresponding grid in the local ultrasonic grid map are updated according to the obstacle type of the surround obstacle point, wherein each classification corresponds to a preset update probability.
7. The method according to claim 6, characterized in that The ultrasonic grid map and the surround view grid map are fused to form a global grid map, and obstacle classification information of each grid in the global grid map is determined according to the corresponding ultrasonic classification information and surround view classification information, further comprising: Create a global grid map; Obstacle classification information of each grid in the global grid map is determined according to the ultrasound classification information in the ultrasound global grid map and the surround classification information in the surround global grid map in the following manner: When the ultrasonic classification information corresponding to a grid is occupied, if the corresponding surround view classification information is idle, then its obstacle classification information is determined to be occupied; if the corresponding surround view classification information is an obstacle type, then its obstacle classification information is determined to be the obstacle type; When the ultrasonic classification information corresponding to a grid is idle, its obstacle classification information is directly determined to be idle; When the ultrasonic classification information corresponding to a grid is unknown, its obstacle classification information is directly determined as the current surround view classification information.
8. The method according to claim 7, characterized in that Further including: In the local surround grid map, obstacle points of pedestrian categories are clustered, and based on Kalman filtering, they are continuously tracked in the vehicle coordinate system to obtain a dynamic pedestrian target list.
9. The method according to claim 8, characterized in that After selecting a target parking space in the global grid map, pedestrian information obtained according to the obstacle point information and input parking space information are added to the global grid map, further comprising: For available parking spaces, set the type attribute of the corresponding area in the global grid map to free; if the parking area contains a wheel stopper, set the area from the wheel stopper position to the end of the parking space to occupied according to the position of the wheel stopper; The dynamic pedestrian target list is traversed, the position of the pedestrian at the time of selecting the parking space is projected into the global grid map, and the category attribute is set to pedestrian; the historical trajectory of each pedestrian is calculated to determine whether it overlaps with obstacles in the current global grid map, and if so, the obstacle area in the overlapping area is removed.
10. The method according to claim 9, characterized in that The parking area restriction further includes: The obstacles on both sides of the target parking space are respectively fitted into a parallelogram obstacle area, and the angle of the parallelogram is adapted to the angle of the parking space; When it is determined that the parking space type of the target parking space is a regular-sized parking space, the sizes of obstacles on the left and right sides of the target parking space are expanded at a first ratio; When the obstacle detected in front of the vehicle is a wall, the road in front of the vehicle is set as an impassable area; when the obstacle detected in front of the vehicle is other obstacles, the size of the obstacle in front of the vehicle is expanded at a second ratio, and the second ratio is greater than the first ratio.
11. An obstacle fusion system in a parking scenario, used to implement the method according to any one of claims 1 to 10, characterized in that: At least: An ultrasonic obstacle point information acquisition unit, used to obtain ultrasonic obstacle point information around the vehicle body according to the vehicle-mounted ultrasonic radar; An ultrasonic grid map generating unit, configured to generate an ultrasonic grid map based on the ultrasonic obstacle point, wherein the ultrasonic grid map includes a plurality of grids, each grid corresponding to ultrasonic classification information, and the ultrasonic classification information includes: idle, occupied, and unknown; A surround obstacle point information acquisition unit, used to acquire surround obstacle point information according to the vehicle-mounted camera; A surround view grid map generating unit, used to generate a surround view grid map based on the surround view obstacle point, wherein the surround view grid map includes a plurality of grids, each grid corresponds to surround view classification information, and the surround view classification information includes: idle and preset obstacle types; A global grid map fusion unit is used to fuse the ultrasonic grid map and the surround view grid map to form a global grid map, and determine the obstacle classification information of each grid in the global grid map according to the corresponding ultrasonic classification information and the surround view classification information, wherein the obstacle classification information includes: idle, occupied and various obstacle types; The update processing unit is used to add pedestrian information obtained based on the surround obstacle point information and input parking space information to the global grid map after selecting the target parking space in the global grid map, and perform parking area restrictions, wherein the parking space information at least includes: parking space area information and available status information.
12. The system according to claim 11, characterized in that Further including: The analysis and processing unit is used to cluster obstacle points of pedestrian categories in the local surround grid map, and continuously track them in the vehicle coordinate system based on Kalman filtering to obtain a dynamic pedestrian target list.
13. The system according to claim 11 or 12, characterized in that Further including: A parking restriction processing unit, used for selecting a target parking space to be parked in the global grid map, restricting the size of the target parking space area, forming a global map for parking, and updating the global map in real time during the parking process; The parking restriction processing unit further comprises: A fitting processing unit is used to fit the obstacles on both sides of the target parking space into a parallelogram obstacle area, where the angle of the parallelogram is adapted to the angle of the parking space; The obstacle limiting unit on both sides is used to expand the obstacle sizes on the left and right sides of the target parking space at a first ratio when it is determined that the parking space type of the target parking space is a regular parking space; A vehicle front obstacle limiting unit, used to set the road in front of the vehicle as an impassable area when the obstacle detected in front of the vehicle is a wall; when the obstacle detected in front of the vehicle is other obstacles, expand the size of the obstacle in front of the vehicle at a second ratio, wherein the second ratio is greater than the first ratio; The update processing unit is used to update the current global grid map according to the ultrasonic obstacle point information and the surround obstacle point information obtained in real time during the parking process.
14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.
15. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.
16. A vehicle, at least provided with an automatic parking planning device, wherein the obstacle fusion system for parking scenarios according to any one of claims 11 to 13 is deployed in the automatic parking planning device.
Citation Information
Patent Citations
Obstacle information fusion method and system for intelligent parking system
CN114802207A
Smart parking lot semantic map construction method and system based on global perception
CN115273028A
Parking map construction method and device, electronic equipment and storage medium
CN115690733A
