A drivable space fusion method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a drivable space fusion method and device, electronic equipment and storage medium, relating to the technical field of visual fusion. The method comprises: acquiring a drivable boundary point image perceived by multiple cameras; performing single-frame fusion on the boundary points by the multiple cameras to generate a single-frame grid map; and performing time-series multi-frame fusion on the single-frame grid map to obtain a fusion result. The method uses drivable boundary points for tracking fusion to obtain a drivable area, and uses a grid map for probability fusion, thereby improving the accuracy of drivable space edge calculation and solving the problem that existing methods are easily affected by environmental factors and the results are inaccurate.

Description

Technical Field

[0001] This application relates to the field of visual fusion technology, and more specifically, to a drivable space fusion method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the field of autonomous driving, perception and tracking fusion, as the upstream of the driver assistance system, needs to obtain the situation around the vehicle from the raw data of sensors and output the information required by the planning and control part. Among them, the drivable space submodule is responsible for obtaining the drivable area in the surrounding space, that is, the area that can be planned and controlled. It needs to extract environmental elements such as curbs, lane lines, and obstacles from the scene around the vehicle, and provide a backup for the target detection task to prevent the target detection task from having semantic incomplete coverage or missed detection.

[0003] Existing methods for fusion of drivable space, such as analyzing point clouds generated by lidar to obtain environmental elements around the vehicle and then fusing them to obtain the drivable area, are costly and the detection range is easily affected by the environment. Using millimeter-wave radar to detect surrounding obstacles and then fusing them into a drivable area is easily affected by weather and is prone to false alarms when detecting objects with small reflective surfaces, such as traffic cones, leading to inaccurate results. Summary of the Invention

[0004] The purpose of this application is to provide a drivable space fusion method, apparatus, electronic device and storage medium, which uses drivable boundary points for tracking and fusion to obtain drivable areas, and uses grid maps for probabilistic fusion, thereby improving the accuracy of drivable space edge calculation and solving the problem that existing methods are easily affected by environmental factors and the results are inaccurate.

[0005] This application provides a method for merging drivable spaces, the method comprising: Acquire images of drivable boundary points perceived by multiple cameras; Multi-camera single-frame fusion is performed on the boundary points to generate a single-frame raster map; The single-frame raster map is fused over time to obtain the fusion result.

[0006] In the above implementation process, the drivable area is obtained by tracking and fusing images of drivable boundary points and then converted into a raster map. The multi-frame fusion probability under vehicle motion is obtained by using a single-frame raster map, which improves the accuracy of drivable space edge calculation and solves the problem that existing methods are easily affected by environmental factors and the results are inaccurate.

[0007] Further, the step of performing multi-camera single-frame fusion on the boundary points to generate a single-frame raster map includes: The image of the drivable boundary point is converted using IPM based on the camera's intrinsic and extrinsic parameters. Based on the preset sensing domain, the boundary points outside the sensing domain are projected into the sensing domain, and the boundary points with IPM conversion abnormalities are deleted. Remove redundant points within a single camera and at camera boundaries; Connect the boundary points in a clockwise direction to form the envelope of the drivable space, and generate a grid of boundary points within the envelope.

[0008] In the above implementation process, abnormal boundary point processing and redundant point deletion are performed on the boundary points in the drivable boundary point image, so that the envelope of the formed drivable space is closer to the real drivable space.

[0009] Furthermore, the deletion of redundant points within a single camera and at camera boundaries includes: Get the line connecting any boundary point to the farthest point within a specified range; Delete redundant points outside the lines within the defined range; Obtain the boundary point where the current camera intersects with the adjacent cameras; Delete the boundary points of the adjacent cameras that are outside the boundary point.

[0010] In the above implementation process, unnecessary redundant points are removed, making the output raster map smoother and reducing the number of boundary points that need to be processed in subsequent operations.

[0011] Further, the step of performing temporal multi-frame fusion on the single-frame raster map to obtain the fusion result includes: Motion compensation is performed on the boundary points based on the vehicle body motion information to determine the position of the boundary points in the current frame raster map; Probabilistic fusion of multiple raster maps is performed to obtain the fusion result.

[0012] In the above implementation process, motion compensation is used to eliminate the influence of vehicle motion, making the position of the boundary point more accurate.

[0013] Furthermore, the probabilistic fusion of multiple raster maps to obtain the fusion result includes: The probabilities of any grid in the current frame's raster map being occupied by obstacles and empty are expressed as follows: and and , and further expressed as: ; After motion compensation, the next frame of the raster map is determined based on the probability that the grid is occupied. The formula determines that the probability of the grid being occupied is a constant; The probability value of an object being occupied is set based on the accuracy of the sensor and the actual operating environment, and the probability value of any grid in each frame of the raster map being occupied by an obstacle is updated based on the probability value.

[0014] In the above implementation process, by setting the probability value of grid occupancy, the probability value of any grid in each frame of the grid map being occupied by an obstacle can be obtained, thus accurately obtaining the drivable area.

[0015] This application provides a drivable space fusion device, the device comprising: The image acquisition module is used to acquire images of drivable boundary points perceived by multiple cameras; A multi-camera single-frame fusion module is used to perform multi-camera single-frame fusion on the boundary points to generate a single-frame raster map. The temporal multi-frame fusion module is used to perform temporal multi-frame fusion on the single-frame raster map to obtain the fusion result.

[0016] In the above implementation process, the drivable area is obtained by tracking and fusing images of drivable boundary points and then converted into a raster map. The multi-frame fusion probability under vehicle motion is obtained by using a single-frame raster map, which improves the accuracy of drivable space edge calculation and solves the problem that existing methods are easily affected by environmental factors and the results are inaccurate.

[0017] Furthermore, the multi-camera single-frame fusion module includes: The conversion module is used to perform IPM conversion on the drivable boundary point image based on the camera's intrinsic and extrinsic parameters; The abnormal boundary point deletion module is used to project boundary points outside the sensing domain into the sensing domain according to the preset sensing domain, and delete boundary points with IPM conversion abnormalities. The redundant point removal module is used to remove redundant points within a single camera and at camera boundaries; The grid generation module is used to connect the remaining boundary points in a clockwise direction to form the envelope of the drivable space and generate a grid from the boundary points within the envelope.

[0018] In the above implementation process, abnormal boundary point processing and redundant point deletion are performed on the boundary points in the drivable boundary point image, so that the envelope of the formed drivable space is closer to the real drivable space.

[0019] Furthermore, the temporal multi-frame fusion module includes: The motion compensation module is used to perform motion compensation on the boundary points based on the vehicle body motion information in order to determine the position of the boundary points in the current frame grid map. The probabilistic fusion module is used to perform probabilistic fusion of multiple frames of raster maps to obtain the fusion result.

[0020] In the above implementation process, motion compensation is used to eliminate the influence of vehicle motion, making the position of the boundary point more accurate.

[0021] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the drivable space fusion method described in any one of the above-described methods.

[0022] This application also provides a readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the drivable space fusion method described in any of the above embodiments. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a drivable space fusion method provided in this application embodiment; Figure 2 This is a schematic diagram of a drivable boundary point image provided in an embodiment of this application; Figure 3 A flowchart illustrating the specific implementation of drivable space tracking fusion provided in this application embodiment; Figure 4 A flowchart illustrating the specific generation process of a single-frame raster map provided in this application embodiment; Figure 5 A detailed flowchart of the redundancy point deletion provided in the embodiments of this application; Figure 6 This is a schematic diagram of the envelope of the drivable area provided in an embodiment of this application; Figure 7 A detailed flowchart of temporal multi-frame fusion provided in the embodiments of this application; Figure 8 A schematic diagram illustrating the probability of grid occupancy in the frame raster map obtained in the first frame of the vehicle according to an embodiment of this application; Figure 9 This is a schematic diagram illustrating the probability of a grid being occupied in a frame raster map after the location of a certain boundary point has been updated, as provided in an embodiment of this application. Figure 10 A structural block diagram of a drivable space tracking fusion device provided in an embodiment of this application; Figure 11 This is a structural block diagram of another drivable space tracking fusion device provided in an embodiment of this application.

[0025] icon: 100 - Image acquisition module; 200 - Multi-camera single-frame fusion module; 201 - Conversion module; 202 - Abnormal boundary point deletion module; 203 - Redundant point deletion module; 204 - Raster generation module; 300 - Temporal multi-frame fusion module; 301 - Motion compensation module; 302 - Probabilistic fusion module. Detailed Implementation

[0026] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0027] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] Example 1 Please refer to Figure 1 , Figure 1 A flowchart illustrating a drivable space fusion method provided in this application embodiment. The method specifically includes the following steps: Step S100: Acquire images of drivable boundary points perceived by multiple cameras; The drivable boundary point image is obtained from the sensing end. The sensing end obtains the drivable area image from various cameras placed on the vehicle body (e.g., there are 6 cameras, specifically distributed as front-view, front right, rear right, rear-view, rear left, and front left), and uses the drivable area image to obtain the boundary points of the drivable space, such as 100 points. The acquisition of boundary points is existing technology and will not be described in detail here.

[0029] like Figure 2 The image shown is a schematic diagram of drivable boundary points. The space below the boundary points is the drivable area, and the space above the boundary points is the non-drivable area. However, it is clearly visible that some boundary points in the image, such as the 5th point and points 65 to 70 (numbered sequentially from left to right), have shifted. The tracking fusion method described in this application can correct the shift of the boundary points.

[0030] Therefore, this application utilizes the drivable boundary points obtained in the visual drivable space perception task (perception end), performs tracking and fusion to obtain the drivable area, and converts it into a grid map for easy processing in subsequent planning and control tasks. The fusion based on perception corrects the occasional false detections and missed detections in the perception task, improving the robustness of the system. On the other hand, it improves performance and the accuracy of the drivable space edge. Moreover, the algorithm of this application consumes less time and resources, making it highly practical.

[0031] Step S200: Perform multi-camera single-frame fusion on the boundary points to generate a single-frame raster map; Step S300: Perform temporal multi-frame fusion on the single-frame raster map to obtain the fusion result.

[0032] Among them, such as Figure 3 The diagram shows the specific implementation flowchart of drivable space tracking fusion. Multi-camera single-frame fusion is performed on the drivable boundary point images corresponding to the multi-camera perceived boundary points, combining camera intrinsic and extrinsic parameters to obtain a single-frame raster map. Multiple single-frame raster maps are then combined with vehicle motion information for temporal multi-frame fusion to obtain the fusion result. Specifically, as shown... Figure 4 The diagram shows the specific flowchart for generating a single-frame raster map. Step S200 specifically includes the following steps: Step S210: Perform IPM conversion on the drivable boundary point image based on the camera's intrinsic and extrinsic parameters; IPM (Inverse Perspective Transformation) is the process of transforming boundary points from the camera coordinate system to the vehicle's body coordinate system based on the camera's intrinsic and extrinsic parameters.

[0033] Step S220: Based on the preset sensing domain, project the boundary points outside the sensing domain into the sensing domain, and delete the boundary points with IPM conversion abnormalities. The sensing field can be set by adjusting its length and width according to the camera height and angle, and the desired sensing range.

[0034] The handling of abnormal boundary points involves first projecting boundary points outside the perception domain into the perception domain, and then deleting abnormal boundary points whose IPM conversion results are inconsistent based on the camera's intrinsic and extrinsic parameters and the camera's placement angle. For example, boundary points obtained from the forward-looking camera are displayed at the rear of the vehicle after IPM conversion.

[0035] Step S230: Delete redundant points within a single camera and at camera boundaries; Among them, such as Figure 4 The diagram shows the specific flowchart for redundancy point deletion. Step S230 may include the following steps: Step S231: Obtain the line connecting any boundary point to the farthest point within the set range; Step S232: Delete redundant points outside the lines within the set range; Specifically, redundant points within a single camera are removed. For example, a line is drawn between the 5th point and the 10th point, the farthest point within a set range such as 0.5 meters. It is then determined whether points 6 through 9 are outside the line (further from the vehicle body). If so, the redundant points located outside the line are deleted. The purpose of deleting redundant points is to remove unnecessary spikes in the drivable boundary point image, making the output raster image smoother and more even.

[0036] Step S233: Obtain the boundary point where the current camera intersects with the adjacent camera; Step S234: Delete the boundary points of the adjacent cameras that are outside the boundary points.

[0037] For example, deduplication at camera boundaries involves determining whether to delete boundary points at camera boundaries after setting the range of camera boundaries, starting from the "forward-looking" camera and proceeding clockwise.

[0038] For example, first iterate through the boundary points obtained by the "forward-looking" camera, finding the boundary points at the camera intersection (i.e., the intersection of the "forward-looking" camera and the "front right" camera). Then, determine whether the boundary point obtained by the "front right" camera is near a point on the "forward-looking" camera and outside the line connecting that point. If so, delete the boundary point obtained by the "front right" camera. Continue in this manner until all the boundary points of the six cameras have been processed. The purpose of this step is to reduce the number of boundary points that need to be processed in subsequent operations.

[0039] Step S240: Connect the remaining boundary points in a clockwise direction to form the envelope of the drivable space and generate a grid of boundary points within the envelope.

[0040] A single-frame raster map is constructed, and the boundary points processed as described above are connected in a clockwise direction to form an envelope. The space within this envelope is the drivable space. Figure 5 The diagram shows the envelope of the drivable area; then, the points within the envelope are output as a raster, as shown below. Figure 6 The image shown is a schematic diagram of the output single-frame raster image.

[0041] like Figure 7 The diagram shown is a flowchart of the specific process for temporal multi-frame fusion. Step S300 specifically includes the following steps: Step S310: Perform motion compensation on the boundary points based on the vehicle body motion information to determine the position of the boundary points in the current frame grid map; After obtaining a single-frame raster image, motion compensation is required. Specifically, the vehicle movement speed obtained by the vehicle IMU and RTK is used to perform motion compensation on the boundary points of images with different timestamps obtained by different cameras, so as to clarify the actual position of the current frame boundary point in the vehicle coordinate system.

[0042] For example, starting with the front-view camera, the perception results from the front-view, front-right, rear-right, rear-view, rear-left, and front-left cameras are fused sequentially, ending with the front-left camera, to obtain a single frame of drivable space grid map.

[0043] When performing the next round of fusion, the vehicle may have already moved. Therefore, it is necessary to move the origin of the vehicle coordinate system in this round relative to the origin of the vehicle coordinate system in the previous round. For example, when fusing the next frame of drivable grid image, if the vehicle has moved forward by 5 meters relative to the start of the previous frame of fusion, then the origin of the next frame of grid image needs to be moved forward by 5 meters to achieve the purpose of moving the boundary point forward by the corresponding distance and realizing motion compensation.

[0044] Step S320: Perform probabilistic fusion on the multi-frame raster map to obtain the fusion result.

[0045] The dynamic and static properties of boundary points are utilized, and the probability of boundary points being occupied is updated using the raster probability formula.

[0046] Specifically: For a certain grid in the raster image m There are only two possibilities: either it is occupied or it is empty. Therefore, the probability of any grid in the current frame's raster map being occupied by an obstacle and empty can be expressed as follows: and and To reduce the number of variables, it can be further expressed as: ; After motion compensation, the next frame of the raster map is determined based on the probability that the grid is occupied. The probability that the grid is occupied is determined to be a constant value; After the next frame of the raster image is input, and after motion compensation, the grid... m New observations were obtained. A value of 0 indicates the grid is empty; a value of 1 indicates it is occupied. According to Bayes' theorem, the probability ratio of the grid being empty or occupied at a new time is: ; To reduce the computational load, taking the logarithm of both sides yields: ; And in Since the grid will only have two states—empty or occupied—at the new moment corresponding to the next frame of the raster image, the value can be obtained as follows: or The probability that a grid is occupied is a fixed value, which can be set according to the sensor accuracy and the actual operating environment. For example, in an underground parking garage, where there are many obstacles, the probability value can be set to a relatively large value.

[0047] The probability value of an object being occupied is set based on the sensor accuracy and the actual operating environment, and the probability value of any grid in each frame of the raster map being occupied by an obstacle is updated based on the probability value.

[0048] For example, in an outdoor parking environment, the two probability values ​​mentioned above can be set to -0.5 and 0.8, representing the probability that the grid point is occupied in two states: whether the sensor outputs an empty or occupied grid point (the negative sign is for ease of subsequent calculation). The probability that the sensor outputs an occupied grid point is actually occupied is 0.8, and the probability that the sensor outputs an empty grid point is actually empty is 0.5. Therefore, they are set to 0.8 and -0.5 to achieve addition and subtraction.

[0049] Assuming the vehicle is stationary, in the vehicle coordinate system, the probability that a grid cell in the first frame of the raster map obtained for the vehicle is occupied is as follows: Figure 8 As shown, at a later point in the fusion process, the position of a certain boundary point was updated. It was discovered that the coordinates of this boundary point had changed, and it was farther from the vehicle. Therefore, the position of this boundary point in the vehicle coordinate system needs to be updated, and the positions of all grid points on the line connecting this boundary point to the origin of the vehicle coordinate system also need to be updated. After the position update, points that were originally empty (occupancy probability of -0.5) remain empty after perception (subtracting 0.5), so the probability becomes -1. Meanwhile, grid points that were previously displayed as occupied (occupancy probability of 0.8) are now displayed as empty in the perception output (subtracting 0.5), so the probability becomes 0.3. Figure 9 The figure shows a schematic diagram illustrating the probability of a grid being occupied in a frame raster map after the boundary point has been updated.

[0050] This method uses drivable boundary points for tracking and fusion to obtain drivable areas, and uses grid maps for probabilistic fusion, which improves the accuracy of drivable space edge calculation and solves the problem that existing methods are easily affected by environmental factors and have inaccurate results.

[0051] Example 2 This application provides a drivable space tracking fusion device, applied to the drivable space tracking fusion method described in Embodiment 1, such as... Figure 10 The diagram shown is a structural block diagram of a drivable space tracking fusion device, which includes, but is not limited to: Image acquisition module 100 is used to acquire images of drivable boundary points perceived by multiple cameras; The multi-camera single-frame fusion module 200 is used to perform multi-camera single-frame fusion on the boundary points to generate a single-frame raster map. The temporal multi-frame fusion module 300 is used to perform temporal multi-frame fusion on the single-frame raster map to obtain the fusion result.

[0052] like Figure 11 The diagram shown is a structural block diagram of another drivable space tracking fusion device, wherein the multi-camera single-frame fusion module 200 includes: The conversion module 201 is used to perform IPM conversion on the drivable boundary point image based on the camera's intrinsic and extrinsic parameters; The abnormal boundary point deletion module 202 is used to project boundary points outside the sensing domain into the sensing domain according to the preset sensing domain, and delete boundary points with IPM conversion abnormalities. Redundant point deletion module 203 is used to delete redundant points within a single camera and at camera boundaries; Specifically: Get the line connecting any boundary point to the farthest point within a specified range; Delete redundant points outside the lines within the defined range; Obtain the boundary point where the current camera intersects with the adjacent cameras; Delete the boundary points of the adjacent cameras that are outside the boundary point.

[0053] The grid generation module 204 is used to connect the remaining boundary points in a clockwise direction to form the envelope of the drivable space and generate a grid from the boundary points within the envelope.

[0054] The time-series multi-frame fusion module 300 includes: Motion compensation module 301 is used to perform motion compensation on the boundary points based on vehicle body motion information in order to determine the position of the boundary points in the current frame grid map; The probability fusion module 302 is used to perform probability fusion on multiple frames of raster maps to obtain the fusion result.

[0055] Specifically, the probabilities of any grid in the current frame's raster map being occupied by obstacles and empty are represented as follows: and and And can be further expressed as: ; After motion compensation, the next frame of the raster map is determined based on the probability that the grid is occupied. Add(m) The formula determines that the probability of the grid being occupied is a constant; The probability value of an object being occupied is set based on the accuracy of the sensor and the actual operating environment, and the probability value of any grid in each frame of the raster map being occupied by an obstacle is updated based on the probability value.

[0056] This device uses drivable boundary points for tracking and fusion to obtain drivable areas, and uses grid maps for probabilistic fusion, which improves the accuracy of drivable space edge calculation and solves the problem that existing methods are easily affected by environmental factors and have inaccurate results.

[0057] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the drivable space fusion method described in Embodiment 1.

[0058] This application also provides a readable storage medium storing computer program instructions, which are read and executed by a processor to perform the drivable space fusion method described in Embodiment 1.

[0059] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0060] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0061] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0062] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0063] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0064] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for merging drivable spaces, characterized in that, The method includes: Acquire images of drivable boundary points perceived by multiple cameras; Multi-camera single-frame fusion is performed on the boundary points to generate a single-frame raster map, including: The image of the drivable boundary points is subjected to IPM conversion based on the camera's internal and external parameters; boundary points outside the perception domain are projected into the perception domain according to a preset perception domain, and boundary points with abnormal IPM conversion are deleted; redundant points within a single camera and at camera boundaries are deleted; the remaining boundary points are connected in a clockwise direction to form an envelope of the drivable space, and the boundary points within the envelope are generated into a grid; wherein, deleting redundant points within a single camera and at camera boundaries includes: obtaining the connection between any boundary point and the farthest point within a set range; deleting redundant points outside the connection within the set range; obtaining the boundary points at the boundary between the current camera and adjacent cameras; and deleting the boundary points of the adjacent cameras located outside the boundary points. The single-frame raster map is fused temporally across multiple frames to obtain a fusion result. Specifically, this includes: performing motion compensation on the boundary points based on vehicle movement information to determine the position of the boundary points in the current frame raster map; specifically, using the vehicle's movement speed to perform motion compensation on the boundary points of images with different timestamps obtained from different cameras to clarify the actual position of the current frame boundary points in the vehicle coordinate system. Specifically, when fusing the next drivable raster map, if the vehicle has moved forward a certain distance relative to the start time of the previous frame's fusion, the origin of the next frame raster map needs to be moved forward by the same distance to achieve the purpose of moving the boundary points forward by the corresponding distance; and performing probabilistic fusion on the multiple frames raster maps to obtain a fusion result. Specifically, the probability of any grid in the current frame raster map being occupied or empty by an obstacle is expressed as: and and And further expressed as: ; The next frame of the raster map, after motion compensation, is based on the observations. The probability that the grid is occupied is determined to be a constant, wherein, The value can be 0 or 1, where 0 indicates that the grid is empty and 1 indicates that it is occupied; The probability value of an object being occupied is set based on the sensor accuracy and the actual operating environment, and the probability value of any grid in each frame of the raster map being occupied by an obstacle is updated based on the probability value.

2. A drivable space fusion device, characterized in that, The device includes: The image acquisition module is used to acquire images of drivable boundary points perceived by multiple cameras; A multi-camera single-frame fusion module is used to perform multi-camera single-frame fusion on the boundary points to generate a single-frame raster map. It includes: a conversion module for performing IPM conversion on the drivable boundary point image based on internal and external camera parameters; an abnormal boundary point deletion module for projecting boundary points outside the perception domain into the perception domain according to a preset perception domain, and deleting boundary points with abnormal IPM conversion; a redundant point deletion module for deleting redundant points within a single camera and at camera boundaries; and a raster generation module for connecting the remaining boundary points in a clockwise direction to form an envelope of the drivable space and generating a raster from the boundary points within the envelope. The deletion of redundant points within a single camera and at camera boundaries includes: obtaining a connection between any boundary point and the farthest point within a set range; deleting redundant points outside the connection within the set range; obtaining the boundary points at the boundary between the current camera and adjacent cameras; and deleting the boundary points of the adjacent cameras located outside the boundary points. A temporal multi-frame fusion module is used to perform temporal multi-frame fusion on the single-frame raster map to obtain a fusion result. The temporal multi-frame fusion module includes: The motion compensation module is used to perform motion compensation on the boundary points based on the vehicle body motion information to determine the position of the boundary points in the current frame raster map. Specifically, it includes: using the vehicle body movement speed to perform motion compensation on the boundary points of images with different timestamps obtained by different cameras to clarify the actual position of the boundary points in the vehicle body coordinate system in the current frame. Specifically, when fusing the next frame of drivable raster map, if the vehicle body has moved forward a certain distance relative to the start time of the previous frame fusion, the origin of the next frame raster map needs to be moved forward by the same distance to achieve the purpose of moving the boundary points forward by the corresponding distance. The probabilistic fusion module is used to perform probabilistic fusion on multi-frame raster maps to obtain the fusion result. Specifically, the probability of any grid in the current frame's raster map being occupied by an obstacle and empty is expressed as follows: and and And further expressed as: ; The next frame of the raster map, after motion compensation, is based on the observations. The probability that the grid is occupied is determined to be a constant, wherein, The value can be 0 or 1, where 0 indicates that the grid is empty and 1 indicates that it is occupied; The probability value of an object being occupied is set based on the sensor accuracy and the actual operating environment, and the probability value of any grid in each frame of the raster map being occupied by an obstacle is updated based on the probability value.

3. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the drivable space fusion method according to claim 1.

4. A readable storage medium, characterized in that, The readable storage medium stores computer program instructions, which are read and executed by a processor to perform the drivable space fusion method of claim 1.

Images