Around view monitor image generation method

Abstract

The present application relates to the technical field of vehicles, and discloses an around view monitor image generation method and apparatus and a vehicle. The around view monitor image generation method comprises: on the basis of a plurality of environment images of a target vehicle under a plurality of viewing angles, determining obstacle projection information and a fusion region between target adjacent environment images among the plurality of environment images; on the basis of the fusion region, generating an initial stitching path; on the basis of the initial stitching path and the obstacle projection information, determining a target stitching path; and, on the basis of the target stitching path, processing the target adjacent environment images to obtain an around view monitor image.

Description

Panoramic surround view image generation method

[0001] Cross-reference to Related Applications

[0002] This application is based on the Chinese patent application No. 2024110644367, filed on August 5, 2024, and claims priority to the Chinese patent application No. 2024110644367, and the entire contents of the Chinese patent application No. 2024110644367 are hereby incorporated by reference into the present application. TECHNICAL FIELD

[0003] The present application relates to the technical field of vehicles, in particular to a panoramic surround view image generation method. BACKGROUND

[0004] The panoramic surround view image (AVM) belongs to a part of the automatic parking system, and is a function with very high practicability and can greatly improve the user experience and intelligent driving safety. In the related art, the panoramic image around the vehicle body is mainly obtained by splicing four-way surround fisheye lens images; however, the effect of reducing the blind area is limited, thereby affecting the user experience and driving safety. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides a panoramic surround view image generation method, which can more flexibly avoid multi-obstacle projection to obtain a splicing path, has better blind area elimination effect, thereby helping to improve the user experience and ensuring driving safety; and is widely applicable to various scenes.

[0006] In a first aspect, the present application provides a panoramic surround view image generation method, which comprises:

[0007] Based on a plurality of environment images of a target vehicle at a plurality of viewing angles, determining obstacle projection information and a fusion area between target adjacent environment images in the plurality of environment images;

[0008] Based on the fusion area, generating an initial splicing path;

[0009] Based on the initial splicing path and the obstacle projection information, determining a target splicing path;

[0010] Based on the target splicing path, processing the target adjacent environment images to obtain a panoramic surround view image.

[0011] According to the panoramic surround view image generation method provided in the application, the target adjacent environment image corresponding target splicing path is determined by combining the initial splicing path generated based on the fusion region and the obstacle projection information, so that the splicing path can be more flexibly avoided from the multiple obstacle projections, and the blind area elimination effect is better, thereby helping to improve the user experience and ensure the driving safety; and the application scenarios are wide.

[0012] According to an embodiment of the application, the target splicing path is determined based on the initial splicing path and the obstacle projection information, including:

[0013] In a case where it is determined based on the obstacle projection information that at least one obstacle projection intersects with the initial splicing path, the target splicing path is determined based on the driving parameter of the target vehicle.

[0014] According to an embodiment of the application, the target splicing path is determined based on the driving parameter of the target vehicle, including:

[0015] The predicted driving path corresponding to the target vehicle is determined based on the driving parameter;

[0016] The target splicing path is determined based on the intersection of the predicted driving path and the fusion region.

[0017] According to an embodiment of the application, the target splicing path is determined based on the intersection of the predicted driving path and the fusion region, including:

[0018] In a case where the predicted driving path and the fusion region do not intersect, the target splicing path is determined based on at least one of the priority corresponding to each obstacle projection and the distance between each obstacle projection and the target vehicle.

[0019] According to an embodiment of the application, the target splicing path is determined based on the intersection of the predicted driving path and the fusion region, including:

[0020] In a case where the predicted driving path and the fusion region intersect, the target splicing path is determined based on the position information of the obstacle projection and the position information of the predicted driving path.

[0021] According to an embodiment of the application, the target splicing path is determined based on the position information of the obstacle projection and the position information of the predicted driving path, including:

[0022] The intersection of the obstacle projection and the predicted driving path is determined based on the position information of the obstacle projection and the position information of the predicted driving path.

[0023] determine the target splicing path according to intersection of the obstacle projection and the predicted driving path.

[0024] According to one embodiment of the present application, the target splicing path is determined based on position information of the obstacle projection and position information of the predicted driving path, including:

[0025] In the case that at least one of the obstacle projections does not intersect with the predicted driving path, the obstacle projection that does not intersect with the predicted driving path is deleted;

[0026] The target splicing path is determined based on the deleted obstacle projection information.

[0027] According to one embodiment of the present application, the target splicing path is determined based on the deleted obstacle projection information, including:

[0028] In the case that no remaining obstacle projection exists based on the deleted obstacle projection information, the target splicing path is determined based on the fusion area.

[0029] According to one embodiment of the present application, the target splicing path is determined based on the deleted obstacle projection information, including:

[0030] In the case that a remaining obstacle projection exists based on the deleted obstacle projection information, the target splicing path is determined based on the remaining obstacle projection and the fusion area.

[0031] According to one embodiment of the present application, the target splicing path is determined based on the remaining obstacle projection and the fusion area, including:

[0032] The initial splicing path is adjusted based on the remaining obstacle projection and the fusion area.

[0033] According to one embodiment of the present application, the initial splicing path is adjusted based on the remaining obstacle projection and the fusion area, including:

[0034] In the case that at least one of the remaining obstacle projections intersects with the initial splicing path, the angle of the initial splicing path is adjusted;

[0035] In the case that the adjusted initial splicing path does not intersect with each of the remaining obstacle projections, the adjusted initial splicing path is determined as the target splicing path.

[0036] According to an embodiment of the present application, the adjusting the initial splicing path based on the remaining obstacle projections and the fusion region comprises:

[0037] adjusting an angle of the initial splicing path in a case where it is determined that at least one of the remaining obstacle projections intersects with the initial splicing path;

[0038] obtaining at least one candidate splicing path corresponding to the fusion region by using a path search algorithm in a case where each of the adjusted initial splicing paths intersects with at least one of the remaining obstacle projections;

[0039] determining a target candidate splicing path as the target splicing path in a case where the target candidate splicing path does not intersect with the remaining obstacle projections in the at least one candidate splicing path.

[0040] According to an embodiment of the present application, the determining the target splicing path based on the position information of the obstacle projections and the position information of the predicted driving path comprises:

[0041] determining the target splicing path based on at least one of a priority corresponding to each of the obstacle projections and a distance between each of the obstacle projections and the target vehicle in a case where all of the obstacle projections intersect with the predicted driving path.

[0042] According to an embodiment of the present application, the driving parameter of the target vehicle comprises at least one of a steering angle of the target vehicle and a driving direction of the vehicle.

[0043] According to an embodiment of the present application, the generating the initial splicing path based on the fusion region comprises:

[0044] determining a straight-line splicing path generated based on the fusion region as the initial splicing path, wherein the straight-line splicing path is located in the fusion region, and the straight-line splicing path has only one intersection point with any straight line parallel to two side boundary directions of the fusion region.

[0045] In a second aspect, the present application provides a panoramic surround view image generation device, which comprises:

[0046] a first processing module configured to determine obstacle projection information and a fusion region between target adjacent environment images in a plurality of environment images of a target vehicle under a plurality of viewing angles;

[0047] a second processing module configured to generate an initial splicing path based on the fusion region;

[0048] The third processing module is configured to determine a target splicing path based on the initial splicing path and the obstacle projection information.

[0049] The fourth processing module is configured to process the target adjacent environment image based on the target splicing path to obtain a panoramic surround view image.

[0050] According to the panoramic surround view image generation apparatus, the target splicing path corresponding to the target adjacent environment image is determined by combining the initial splicing path generated based on the fusion region and the obstacle projection information, the splicing path can be obtained by more flexibly avoiding multiple obstacle projections, the blind area elimination effect is better, the user experience is improved, and the driving safety is ensured.

[0051] In a third aspect, the present application provides a vehicle, comprising:

[0052] an image sensor;

[0053] The vehicle generates a panoramic surround view image based on the panoramic surround view image generation method according to the first aspect.

[0054] In a fourth aspect, the present application provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the panoramic surround view image generation method according to the first aspect.

[0055] In a fifth aspect, the present application provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements the panoramic surround view image generation method according to the first aspect.

[0056] The one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0057] According to the panoramic surround view image generation apparatus, the target splicing path corresponding to the target adjacent environment image is determined by combining the initial splicing path generated based on the fusion region and the obstacle projection information, the splicing path can be obtained by more flexibly avoiding multiple obstacle projections, the blind area elimination effect is better, the user experience is improved, and the driving safety is ensured.

[0058] Further, by combining the driving parameters of the target vehicle to search for the splicing path, when the obstacle projection cannot be avoided, the target splicing path that can ensure that the blind area of the obstacle in the driving track is smaller is obtained to perform splicing, the blind area of the obstacle in the driving track is ensured to be smaller, the blind area elimination effect is better, and the robustness and driving safety are further improved.

[0059] Further, in the scene where the obstacle boundary is not standard or there are multiple obstacles, etc., the irregular splicing path can be generated to flexibly avoid the obstacle projection, and better adapt to the scene where multiple obstacles exist.

[0060] Further, in the scene where the blind area cannot be eliminated, the splicing path search is performed in combination with the steering wheel angle of the target vehicle, the speed of the target vehicle, the gear of the target vehicle, the distance between the target vehicle and the obstacle projection, and the category of the obstacle, and the like, so as to preferentially ensure that the obstacle in the vehicle driving track or the obstacle closest to the vehicle body, or the obstacle with a higher priority, is fully displayed, and the robustness and driving safety are further improved.

[0061] Additional aspects and advantages of the application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0062] The above and / or additional aspects and advantages of the application will become apparent and be readily appreciated from the following description, including the appended drawings.

[0063] FIG. 1 is one of flow schematic diagrams of a panoramic surround view image generation method provided by an embodiment of the application;

[0064] FIG. 2 is another of flow schematic diagrams of the panoramic surround view image generation method provided by an embodiment of the application;

[0065] FIG. 3 is one of principle schematic diagrams of the panoramic surround view image generation method provided by an embodiment of the application;

[0066] FIG. 4 is another of principle schematic diagrams of the panoramic surround view image generation method provided by an embodiment of the application;

[0067] FIG. 5 is a third of principle schematic diagrams of the panoramic surround view image generation method provided by an embodiment of the application;

[0068] FIG. 6 is a fourth of principle schematic diagrams of the panoramic surround view image generation method provided by an embodiment of the application;

[0069] FIG. 7 is a fifth of principle schematic diagrams of the panoramic surround view image generation method provided by an embodiment of the application;

[0070] FIG. 8 is a sixth of principle schematic diagrams of the panoramic surround view image generation method provided by an embodiment of the application;

[0071] FIG. 9 is one of structure schematic diagrams of a panoramic surround view image generation device provided by an embodiment of the application;

[0072] FIG. 10 is another of structure schematic diagrams of the panoramic surround view image generation device provided by an embodiment of the application;

[0073] FIG. 11 is a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0074] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of them. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0075] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a category and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally represents a "or" relationship between the front and rear associated objects.

[0076] The panoramic surround view image generation method, panoramic surround view image generation device, electronic device and readable storage medium provided by the embodiments of the present application will be described in detail below in combination with the drawings and specific embodiments and their application scenarios.

[0077] The panoramic surround view image generation method can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.

[0078] The panoramic surround view image generation method provided by the embodiments of the present application can be executed by a vehicle or a functional module or functional entity in the vehicle capable of realizing the panoramic surround view image generation method, or can also be executed by an electronic device or a server in communication connection with the vehicle, etc. The electronic device mentioned in the embodiments of the present application includes but is not limited to a mobile phone, a tablet computer, a computer, a camera and a wearable device, etc. The panoramic surround view image generation method provided by the embodiments of the present application will be described below taking the vehicle as an example.

[0079] As shown in FIG. 1, the panoramic surround view image generation method includes steps 110, 120, 130 and 140.

[0080] In step 110, based on a plurality of environment images of a target vehicle under a plurality of viewing angles, obstacle projection information and a fusion region between target adjacent environment images in the plurality of environment images are determined.

[0081] In this step, the environment image can be environment information of the surroundings of the target vehicle collected by image sensors arranged at different positions of the target vehicle, for example, can be an environment image for sensing the environment in the parking space when the vehicle is parking.

[0082] The target adjacent environment image can be any two adjacent environment images in the plurality of environment images, for example, can be the front corresponding environment image and the left corresponding environment image, or can be the front corresponding environment image and the right corresponding environment image, etc.

[0083] The obstacle projection information, i.e., the obstacle information, is used to represent the projection of the obstacles around the target vehicle onto the ground plane.

[0084] The obstacle projection information includes the obstacle projection, the number of obstacle projections, and the intersection between the obstacle projections, etc., wherein the obstacle projection is the projection image of the obstacle on the ground plane.

[0085] The number of obstacle projections can include 0, 1 or more, etc.

[0086] It can be understood that in the case of the number of obstacle projections being 0, it can be approximately considered that there is no obstacle in the environment image.

[0087] FIGS. 4-6 illustrate various scenarios of obstacle projection, wherein FIG. 4 illustrates a scenario of a single obstacle projection, FIGS. 5 and 6 respectively illustrate scenarios of multiple obstacle projections, the obstacle projections in FIG. 5 do not intersect with each other, and the obstacle projections in FIG. 6 have intersection.

[0088] In some embodiments, the obstacle can include but is not limited to a person, an animal, another vehicle, a building, etc.

[0089] The fusion area can be the overlapping area between the projections of adjacent environment images after the environment image is projected onto the ground plane, as shown in FIG. 3.

[0090] For example, in actual execution, the front, rear, and left and right corresponding environment images of the target vehicle can be collected by a fisheye camera; the 4-way environment images are subjected to obstacle detection to obtain the category and position of the obstacle in each environment image.

[0091] The 4-way environment images and the position coordinates of the obstacles are subjected to inverse perspective transformation and projected onto the ground plane to obtain the environment image projection and the obstacle projection information.

[0092] Step 120, generating an initial splicing path based on the fusion area;

[0093] In this step, the initial stitching path can be obtained by an Around View Monitor (AVM) stitching algorithm.

[0094] The AVM is a kind of panoramic image system, which captures images through multiple super wide-angle fisheye lenses, and forms a panoramic image around the vehicle through data processing and image correction technology.

[0095] In some embodiments, step 120 can include:

[0096] The straight-line stitching path generated based on the fusion region is determined as the initial stitching path.

[0097] In this embodiment, the initial stitching path is a straight-line path.

[0098] The straight-line stitching path is located within the fusion region, and the straight-line stitching path has only one intersection point with any straight line parallel to the two side boundary directions of the fusion region.

[0099] Step 130, based on the initial stitching path and the obstacle projection information, determines a target stitching path.

[0100] In this step, the target stitching path is a stitching path determined within the fusion region, which is used to stitch and fuse the target adjacent environmental images.

[0101] It can be understood that for any two adjacent environmental images, a target stitching path can be determined.

[0102] The target stitching paths determined by different adjacent environmental images can be different, but the ways of determining the target stitching paths by different adjacent environmental images are similar, and the following will be described by taking the target adjacent environmental images including the environmental images corresponding to the front and left of the target vehicle as an example.

[0103] It should be noted that the target stitching path can be a straight-line path, or can also be an irregular curve path.

[0104] The target stitching path is a stitching path generated within the fusion region, which can cause less loss of obstacle information.

[0105] It can be understood that adjusting the initial obstacle stitching path according to the obstacle projection information can be manifested as making the stitching path bypass all obstacles, or bypass a large number of obstacles, or bypass obstacles with a higher priority, so that the image stitched based on the stitching path can retain more obstacles, so that the obstacle information loss is less to retain all obstacle information, or retain a large number of obstacle information, or retain obstacle information with a higher priority, etc.

[0106] After obtaining the initial stitching path, it is determined whether the number of obstacle projections and the intersection between the initial stitching path and the obstacle projections.

[0107] In some embodiments, in a case where it is determined based on the obstacle projection information that no obstacle projection intersects with the initial stitching path, the initial stitching path is determined as the target stitching path.

[0108] As shown in FIG. 2, for example, in a case where it is determined based on the obstacle projection information that the number of obstacle projections is 0, or the number of obstacle projections is not 0 and each obstacle projection does not intersect with the initial stitching path, the initial stitching path can be taken as the final target stitching path.

[0109] In a case where it is determined based on the obstacle projection information that at least one obstacle projection intersects with the initial stitching path, the initial stitching path or a new stitching path can be adjusted in combination with the obstacle projection information and the driving parameter of the target vehicle, so as to obtain the target stitching path.

[0110] The driving parameter is the driving parameter of the target vehicle at the collection time corresponding to the current environment image.

[0111] In actual execution, the target stitching path can be determined in combination with the number of obstacle projections and the coincidence between the obstacle projections and the fusion region.

[0112] Alternatively, the target stitching path can also be determined based on the prediction of the intersection between the target vehicle and the obstacle projection in the future in combination with the coincidence between the obstacle projection and the fusion region and the driving parameter of the target vehicle.

[0113] In this step, the number of obstacle projections and the intersection between the obstacle projections and the initial stitching path are used to select the corresponding stitching path optimization mode, so that the adaptive stitching path optimization mode can be selected flexibly based on different scenes, the multi-obstacle projection can be avoided as much as possible, the blind area elimination effect is improved, and the target stitching path can be determined quickly and accurately.

[0114] The implementation of step 130 will be described in detail below.

[0115] As shown in FIG. 2, in some embodiments, step 130 can include:

[0116] In a case where it is determined based on the obstacle projection information that at least one obstacle projection intersects with the initial stitching path, the target stitching path is determined based on the driving parameter of the target vehicle.

[0117] In this embodiment, the driving parameter of the target vehicle is the driving parameter of the target vehicle at the collection time corresponding to the current environment image.

[0118] In some embodiments, the driving parameter of the target vehicle can include at least one of a steering angle of the target vehicle and a driving direction of the vehicle.

[0119] In this embodiment, the driving direction of the vehicle is used to represent the driving direction of the target vehicle in the following period of time, such as keeping straight, turning left, turning right, U-turn, reversing, and stopping, etc.

[0120] The steering angle is used to represent the change angle of the driving direction of the target vehicle compared with the last time. In the case of 0 or fluctuation around 0, it can be approximately considered that the target vehicle keeps straight or is stationary, etc.

[0121] Of course, in some embodiments, the driving parameter can also include the steering wheel angle of the target vehicle, the wheel steering angle, the speed of the target vehicle, and the gear of the target vehicle, etc.

[0122] It can be understood that in actual execution, the search of the splicing path can be preferentially based on the projections of the obstacles, such as adjusting the angle of the initial splicing path or generating irregular candidate splicing paths through a path search algorithm, etc.

[0123] The path search algorithm can include a breadth-first search algorithm and a depth-first search algorithm, etc.

[0124] The irregular candidate splicing path includes a curved path. By setting the curved path, compared with the straight path, it can more flexibly avoid irregular obstacle boundaries and a large number of obstacles, etc., so as to avoid more obstacles as much as possible, so that the spliced image retains more obstacle information.

[0125] In the case that a splicing path that does not intersect with each projection of the obstacle can be searched in the fusion area, the splicing path can be directly taken as the target splicing path.

[0126] In the case that a splicing path that does not intersect with each projection of the obstacle cannot be searched, the splicing path search can be combined with the driving parameter of the target vehicle, so that the splicing path can avoid most of the obstacle projections or preferentially avoid the obstacle projections of the obstacles with higher priority.

[0127] The target splicing path determined based on each projection of the obstacle can be a straight path or a curved path.

[0128] In this application, by considering different scenarios such as the number of obstacles and whether the blind area can be eliminated, the target splicing path matched with the current scenario is obtained, which is suitable for various scenarios such as no obstacle, single obstacle, multiple obstacles, and obstacles that cannot be completely avoided, etc., and has a wide application scenario and high flexibility.

[0129] According to the panoramic surround view image generation method provided in the embodiments of the present application, the target splicing path that can ensure a smaller obstacle blind area in the driving track is obtained by searching the splicing path in combination with the driving parameter of the target vehicle when the obstacle projection cannot be avoided, so that the obstacle blind area in the driving track is ensured to be smaller, the effect of eliminating the blind area is better, and the robustness and driving safety are further improved.

[0130] With reference to FIG. 2, in some embodiments, determining the target splicing path based on the driving parameter of the target vehicle can include:

[0131] Determining a predicted driving path corresponding to the target vehicle based on the driving parameter.

[0132] Determining the target splicing path based on the intersection of the predicted driving path and the fusion region.

[0133] In this embodiment, the predicted driving path is a driving path of the target vehicle in a subsequent period of time.

[0134] As shown in FIG. 7, the predicted driving path can include a plurality of track lines of the target vehicle, and a driving region is formed between the plurality of track lines.

[0135] In some embodiments, the predicted driving path of the target vehicle can be calculated based on the steering wheel angle of the target vehicle, the speed of the target vehicle, and the gear of the target vehicle.

[0136] The intersection includes intersection or non-intersection.

[0137] Determining whether the predicted driving path passes through the fusion region, so as to select a corresponding generation mode of the target splicing path based on whether the predicted driving path and the fusion region intersect.

[0138] The predicted driving path and the fusion region do not intersect, that is, the predicted driving path does not overlap with any part of the entire fusion region.

[0139] In this embodiment, in the case where the splicing path that avoids all obstacle projections cannot be searched, the splicing path that can ensure a smaller obstacle blind area in the driving track is searched in combination with the driving parameter, and on this basis, panoramic splicing is performed based on the target splicing path searched, so that the panoramic surround view image obtained by the final splicing has a better effect of eliminating the blind area, and is suitable for a multi-obstacle scene and a scene where the blind area cannot be eliminated.

[0140] According to the panoramic surround view image generation method provided in the embodiments of the present application, the target splicing path that can ensure a smaller obstacle blind area in the driving track is obtained by searching the splicing path in combination with the driving parameters of the target vehicle, so that the obstacle blind area in the driving track is ensured to be smaller, the effect of eliminating the blind area is better, and the robustness and driving safety are further improved.

[0141] With reference back to FIG. 2, in some embodiments, determining the target splicing path based on the intersection of the predicted driving path and the fusion area can include:

[0142] In the case where the predicted driving path and the fusion area do not intersect, the target splicing path is determined based on at least one of the priority corresponding to each obstacle projection and the distance between each obstacle projection and the target vehicle.

[0143] In this embodiment, the priority corresponding to the obstacle projection is the priority level that needs to be reserved in the splicing process.

[0144] The priority corresponding to the obstacle projection can be customized by the user, or can be determined based on the category of the obstacle.

[0145] For example, in the case where the obstacle projection includes a projection corresponding to a pedestrian and a projection corresponding to a stone, the priority corresponding to the pedestrian can be higher than the priority corresponding to the stone, that is, the projection corresponding to the pedestrian should be given priority in the image fusion.

[0146] The distance between the obstacle projection and the target vehicle can be the pixel distance on the projection plane, or can also be the actual physical distance between the obstacle and the target vehicle.

[0147] In some embodiments, the smallest rectangular frame surrounding the projection of the target vehicle on the ground plane is taken as the reference, such as the rectangular frame (1) shown in FIG. 8, which is continuously expanded outward in a fixed aspect ratio until the first intersection point with the obstacle projection is generated, and the position of the intersection point is recorded as P1, and the rectangular frame in which the intersection point P1 is located is the rectangular frame (2) shown in FIG. 8.

[0148] The position of the projection of the center point of the rear axle of the target vehicle on the ground plane is recorded as P0, and the pixel distance between the point P0 and the point P1 is calculated, so as to obtain the distance between the obstacle projection and the target vehicle.

[0149] In some embodiments, the pixel distance can also be converted into the actual physical distance.

[0150] In some embodiments, the pixel distance between each pixel point in the projection of the target vehicle on the ground plane and each pixel point in the obstacle projection can also be calculated, and the pixel distance with the shortest pixel distance is taken as the distance between the obstacle projection and the target vehicle.

[0151] Of course, in other embodiments, other methods of calculating distance can also be used, which are not limited herein.

[0152] In some embodiments, after the distance between each obstacle projection and the target vehicle is calculated, the target splicing path can be generated by giving priority to avoiding one or more obstacle projections closest to the target vehicle, that is, when image fusion is performed, the projections corresponding to the first one or a certain number of obstacles closest to the target vehicle are preferentially retained.

[0153] In actual execution, when it is predicted that the driving path does not pass through the fusion area, it can be approximately considered that the possibility of subsequent collision between the target vehicle and the obstacles in the fusion area is low, and in this case, based on the priority of each obstacle projection in the fusion area and the distance between each obstacle projection and the target vehicle, a target splicing path can be generated so that the spliced image can effectively retain the obstacle projection with higher priority or the obstacle projection closer to the target vehicle.

[0154] For example, after obtaining the priority of each obstacle projection and the distance between each obstacle projection and the target vehicle, the obstacle projections can be arranged in ascending order according to the distance from far to near to obtain the obstacle projection closest to the target vehicle. If there are multiple shortest distances, the obstacle projections corresponding to the multiple shortest distances can be further sorted from important to less important according to the category of the obstacle corresponding to the obstacle projection, so as to filter the most important obstacle projection, that is, the obstacle projection with the highest priority.

[0155] In some embodiments, based on at least one of the priority of each obstacle projection and the distance between each obstacle projection and the target vehicle, the target splicing path can also include:

[0156] The priority of each obstacle projection and the distance between each obstacle projection and the target vehicle are weighted and calculated to obtain a score corresponding to each obstacle projection.

[0157] Based on the target number of obstacle projections with the highest score, the target splicing path is determined.

[0158] In this embodiment, the target number can be customized by the user, such as being set to 1 or 3, etc.

[0159] The higher the score is, the more the obstacle projection corresponding to the score should be preferentially retained in the fusion process, that is, in the process of generating the splicing path, the generated splicing path should preferentially avoid the obstacle projection corresponding to the score.

[0160] According to the panoramic surround view image generation method provided in the embodiments of the present application, in the scene where the blind area cannot be eliminated, the target vehicle's steering wheel angle, the target vehicle's speed, the target vehicle's gear position, the distance of the target vehicle and the obstacle projection, and the category of the obstacle are combined to search for a splicing path, so as to preferentially ensure that the obstacle in the vehicle driving track or the obstacle closest to the vehicle body or the obstacle with a higher priority is fully displayed, and the robustness and driving safety are further improved.

[0161] With reference back to FIG. 2, in some embodiments, determining the target splicing path based on the intersection of the predicted driving path and the fusion area can include:

[0162] In the case where the predicted driving path and the fusion area intersect, the target splicing path is determined based on the position information of the obstacle projection and the position information of the predicted driving path.

[0163] In this embodiment, the position information of the obstacle projection is used to represent the position of the obstacle projection on the projection plane where the obstacle projection is located.

[0164] The position information of the predicted driving path is used to represent the position of the predicted driving path on the projection plane where the obstacle projection is located.

[0165] In some embodiments, the position information can be determined based on the following steps:

[0166] Obtaining a pixel region set of each obstacle projection and a parametric equation of at least two trajectory lines corresponding to the predicted driving path;

[0167] Based on the parametric equation of the at least two trajectory lines, a driving region pixel set in a driving region formed between the at least two trajectory lines is obtained;

[0168] Based on the driving region pixel set, the position information of the predicted driving path is determined; and based on the pixel region set of the obstacle projection, the position information of the obstacle projection is determined.

[0169] In this embodiment, the predicted driving path can include a plurality of trajectory lines of the target vehicle, and the plurality of trajectory lines form a driving region.

[0170] The driving region pixel set is a pixel position set of the pixel points of the driving region on the projection plane where the obstacle projection is located.

[0171] The pixel region set of the obstacle projection is a pixel position set of the pixel points of the obstacle projection on the projection plane where the obstacle projection is located.

[0172] In some embodiments, determining the target splicing path based on the position information of the obstacle projection and the position information of the predicted driving path can include:

[0173] determine intersection of the obstacle projection and the predicted driving path based on the position information of the obstacle projection and the position information of the predicted driving path;

[0174] determine the target stitching path according to the intersection of the obstacle projection and the predicted driving path.

[0175] In this embodiment, based on the position information of the obstacle projection and the position information of the predicted driving path, it can be determined whether the predicted driving path intersects with each obstacle projection.

[0176] After obtaining the pixel region set of each obstacle projection and the pixel set of the driving region, the pixel region set of each obstacle projection is traversed to perform intersection operation with the pixel set of the driving region;

[0177] If the result is an empty set, it indicates that the obstacle projection does not intersect with the predicted driving path;

[0178] If the result is not empty, it indicates that the obstacle projection intersects with the predicted driving path.

[0179] For example, in the case that the position information of the predicted driving path and the position information of the obstacle projection partially coincide, it can be considered that the predicted driving path at least partially intersects with the obstacle projection; for another example, in the case that the position information of the predicted driving path and the position information of the obstacle projection are different, it can be approximately considered that the predicted driving path does not intersect with the obstacle projection.

[0180] With reference to FIG. 2, in some embodiments, based on the position information of the obstacle projection and the position information of the predicted driving path, the target stitching path can be determined, which can include: in the case that all obstacle projections intersect with the predicted driving path, the target stitching path is determined based on at least one of the priority corresponding to each obstacle projection and the distance between each obstacle projection and the target vehicle.

[0181] In this embodiment, in the case that the predicted driving path passes through the fusion region, as shown in FIG. 6, it can be approximately considered that the possibility of subsequent collision between the target vehicle and the obstacle in the fusion region is high, in which case, it is determined whether there is a stitching path that can avoid all obstacle projections, so that the stitched image can retain all obstacle projections; in the case that there is no stitching path that can avoid all obstacle projections, a stitching path is generated in combination with the priority corresponding to each obstacle projection and the distance between each obstacle projection and the target vehicle, so as to preferentially ensure that the obstacle blind area in the predicted driving path is small when the obstacle cannot be avoided.

[0182] The specific determination manner of the target splicing path is determined based on at least one of the priority corresponding to each obstacle projection and the distance between each obstacle projection and the target vehicle. The specific determination manner of the target splicing path has been described in the above embodiments, and thus will not be repeated here.

[0183] With reference back to FIG. 2, in some embodiments, determining the target splicing path based on the position information of the obstacle projections and the position information of the predicted driving path can include:

[0184] In a case where at least one of all the obstacle projections does not intersect with the predicted driving path, deleting the obstacle projection that does not intersect with the predicted driving path;

[0185] Determining the target splicing path based on the deleted obstacle projection information.

[0186] In this embodiment, the deleted obstacle projection information is used to represent the number of the remaining obstacle projections after deleting the obstacle projection that does not intersect with the predicted driving path, and the intersection between the remaining obstacle projections and the fusion region in a case where there are remaining obstacle projections.

[0187] It can be understood that at least one of all the obstacle projections does not intersect with the predicted driving path can include: all the obstacle projections do not intersect with the predicted driving path; and part of the obstacle projections do not intersect with the predicted driving path and the other part of the obstacle projections intersect with the predicted driving path.

[0188] Based on the above different cases, different target splicing path determination manners can be selected.

[0189] In some embodiments, determining the target splicing path based on the deleted obstacle projection information can include:

[0190] In a case where it is determined based on the deleted obstacle projection information that there is no remaining obstacle projection, determining the target splicing path based on the fusion region.

[0191] In this embodiment, for the case where all the obstacle projections do not intersect with the predicted driving path, after deleting the obstacle projection that does not intersect with the predicted driving path, there is no remaining obstacle projection.

[0192] In this case, the target splicing path can be generated based on the fusion region only, such as generating a straight splicing path by using a method similar to the initial splicing path, and taking the generated straight splicing path as the target splicing path.

[0193] With reference back to FIG. 2, in some embodiments, determining the target splicing path based on the deleted obstacle projection information can include:

[0194] In a case where it is determined that there is a remaining obstacle projection based on the deleted obstacle projection information, a target splicing path is determined based on the remaining obstacle projection and the fusion area.

[0195] In this embodiment, the remaining obstacle projection is an obstacle projection intersecting the predicted driving path.

[0196] The step 130 can be repeatedly performed based on the obstacle projection intersecting the predicted driving path and the fusion area to determine the target splicing path.

[0197] For example, the intersection of the remaining obstacle projection and the initial splicing path can be re-determined, and in a case where at least one of the remaining obstacle projections intersects the initial splicing path, the target splicing path can be re-determined by adjusting the angle of the initial splicing path, obtaining at least one candidate splicing path corresponding to the fusion area by using a path search algorithm, or in combination with the driving parameters of the target vehicle.

[0198] In some embodiments, determining the target splicing path based on the remaining obstacle projection and the fusion area can further include:

[0199] Adjusting the initial splicing path based on the remaining obstacle projection and the fusion area.

[0200] In this embodiment, adjusting the initial splicing path is dynamic splicing.

[0201] Adjusting the initial splicing path can include but is not limited to adjusting the position of the initial splicing path and adjusting the line shape of the initial splicing path.

[0202] The initial splicing path is adjusted to select a path that exists in the fusion area and does not intersect the remaining obstacle projection.

[0203] The manner of adjusting the initial splicing path is described below.

[0204] In some embodiments, adjusting the initial splicing path based on the remaining obstacle projection and the fusion area can include:

[0205] Adjusting the angle of the initial splicing path in a case where it is determined that at least one of the remaining obstacle projections intersects the initial splicing path.

[0206] In a case where the adjusted initial splicing path does not intersect each of the remaining obstacle projections, the adjusted initial splicing path is determined as the target splicing path.

[0207] In this embodiment, the obstacle projection intersects with the initial stitching path, which can be manifested as at least a partial region of the obstacle projection overlaps with at least a partial region of the initial stitching path, such as the initial stitching path passing through the obstacle projection, or the initial stitching path being located within the obstacle projection, and the like.

[0208] The obstacle projection does not intersect with the initial stitching path, that is, the initial stitching path does not overlap with any part of the entire obstacle projection.

[0209] In the case where the initial stitching path intersects with the obstacle projection, the angle of the initial stitching path can be adjusted based on the clockwise or counterclockwise direction to move the initial stitching path into the fusion region until a stitching path that does not intersect with each obstacle projection in the fusion region is obtained, and the stitching path is determined as the target stitching path, as shown in FIG. 4.

[0210] In the case where the initial stitching path cannot find a stitching path that does not intersect with each obstacle projection by adjusting the angle of the initial stitching path, for example, in the case where the number of obstacle projections is large, the obstacle projections are irregular, or there are intersecting obstacle projections, an irregular stitching path can be generated to avoid each obstacle projection as much as possible.

[0211] According to the panoramic surround view image generation method provided in the embodiments of the present application, the angle of the initial stitching path can be adjusted to quickly obtain a target stitching path that avoids the obstacle projection in a single obstacle scene.

[0212] In some embodiments, adjusting the initial stitching path based on the remaining obstacle projections and the fusion region can include:

[0213] adjusting the angle of the initial stitching path in the case where it is determined that at least one remaining obstacle projection intersects with the initial stitching path;

[0214] obtaining at least one candidate stitching path corresponding to the fusion region by using a path search algorithm in the case where each adjusted initial stitching path intersects with at least one remaining obstacle projection;

[0215] determining a target candidate stitching path as the target stitching path in the case where the target candidate stitching path does not intersect with the remaining obstacle projections in the at least one candidate stitching path.

[0216] In this embodiment, the angle of the initial stitching path can be adjusted based on the clockwise or counterclockwise direction to move the initial stitching path into the fusion region.

[0217] The path search algorithm can include a breadth-first search algorithm and a depth-first search algorithm, and the like.

[0218] The candidate splicing path searched based on the path searching algorithm can be an irregular splicing path.

[0219] With reference back to FIG. 2, in a case where a splicing path that does not intersect with each of the obstacle projections cannot be found in the fusion region by adjusting the angle of the initial splicing path, a path searching algorithm can be used to find an irregular candidate splicing path that meets the condition of not meeting each of the obstacle projections, thereby obtaining the target splicing path, as shown in FIG. 5.

[0220] In a case where a candidate splicing path that does not intersect with each of the obstacle projections still cannot be searched by the path searching algorithm, for example, in a case where the number of obstacle projections is large, the driving parameters of the target vehicle can be used to search for a candidate splicing path that gives priority to ensuring that the blind area of the obstacle in the driving track is small, so as to achieve the effect of ensuring the driving safety of the user.

[0221] According to the panoramic surround view image generation method provided in the embodiments of the present application, in a case where the boundary of the obstacle is irregular or multiple obstacles exist, an irregular splicing path can be generated to flexibly avoid the obstacle projections, better adapt to the case where multiple obstacles exist, and further improve the effect of eliminating the blind area.

[0222] With reference back to FIG. 2, in some embodiments, step 130 can further include:

[0223] adjusting the initial splicing path based on the obstacle projections and the fusion region;

[0224] in a case where a splicing path that does not intersect with each of the obstacle projections does not exist, determining the target splicing path based on the driving parameters of the target vehicle.

[0225] In this embodiment, the specific implementation manner of adjusting the initial splicing path based on the obstacle projections and the fusion region has been described in the above embodiments, and will not be repeated here.

[0226] For example, the target splicing path can be found by adjusting the angle or using a path searching algorithm, and in a case where a splicing path that does not intersect with each of the obstacle projections does not exist, the target splicing path can be determined based on the intersection between the predicted driving path and the fusion region and the driving parameters of the target vehicle.

[0227] The specific implementation manner of determining the target splicing path based on the intersection between the predicted driving path and the fusion region has been described in the above embodiments, and will not be repeated here.

[0228] According to the panoramic surround view image generation method provided in the embodiment of the present application, by providing a plurality of splicing path adjustment algorithms, different scenes such as the number of obstacles and whether the blind area can be eliminated can be flexibly selected, the target splicing path matched with the current scene is obtained, and the method is suitable for various scenes such as no obstacle, single obstacle, multiple obstacles, and obstacles that cannot be completely avoided, and has a wide application scenario and high flexibility.

[0229] In step 140, the target adjacent environment images are processed based on the target splicing path, and the panoramic surround view image is obtained.

[0230] In this step, the panoramic surround view image (AVM) is obtained by splicing the four fisheye images after distortion removal and inverse perspective transformation, and is a panoramic image for the user to observe the situation around the vehicle.

[0231] Through steps 110 to 130, the target splicing path corresponding to any two adjacent environment images can be obtained.

[0232] After obtaining the target splicing path, the projection images of the target adjacent environment images corresponding to the target splicing path on the horizontal plane are spliced and fused based on the target splicing path, and the panoramic surround view image is obtained.

[0233] In the present application, the initial splicing path is generated by fusing the regions, the target splicing path is further optimized by combining the initial splicing path and the obstacle projection information, the number of obstacles and other factors are considered, the target splicing path matched with the current scene is obtained, the method is suitable for various scenes such as no obstacle, single obstacle, and multiple obstacles, and has a wide application scenario and high flexibility.

[0234] In addition, by generating the target splicing path to splice the images, compared with the image splicing method in the related art, in which the segmented sub-images are screened to fuse the screened sub-images to realize image splicing, the present application can utilize the flexible characteristics of the splicing path, has higher flexibility and obstacle avoidance accuracy, and further improves the splicing effect.

[0235] According to the panoramic surround view image generation method provided in the embodiment of the present application, by combining the initial splicing path generated based on the fused regions and the obstacle projection information to determine the target splicing path corresponding to the target adjacent environment image, the splicing path can be more flexibly obtained by avoiding multiple obstacle projections, has better blind area elimination effect, thereby helping to improve the user experience and ensure the driving safety; and the method is suitable for a wide range of scenes.

[0236] In some embodiments, the AVM synthesis module as shown in FIG. 9 can be used to perform the steps of the method of the present application.

[0237] The input end of the AVM synthesis module is configured to input a plurality of environment images and driving parameters of a target vehicle, to determine a target splicing path based on the input plurality of environment images and the driving parameters of the target vehicle, and to process a target adjacent environment image based on the target splicing path, to generate the panoramic surround view image.

[0238] The panoramic surround view image generation method provided in the embodiments of the present application can be executed by a panoramic surround view image generation device. In the embodiments of the present application, the panoramic surround view image generation method is executed by a panoramic surround view image generation device as an example, to illustrate the panoramic surround view image generation device provided in the embodiments of the present application.

[0239] The embodiments of the present application further provide a panoramic surround view image generation device.

[0240] As shown in FIG. 10, the panoramic surround view image generation device includes a first processing module 1010, a second processing module 1020, a third processing module 1030, and a fourth processing module 1040.

[0241] The first processing module 1010 is configured to determine, based on a plurality of environment images of a target vehicle at a plurality of viewing angles, obstacle projection information and a fusion region between a target adjacent environment image in the plurality of environment images.

[0242] The second processing module 1020 is configured to generate an initial splicing path based on the fusion region.

[0243] The third processing module 1030 is configured to determine a target splicing path based on the initial splicing path and the obstacle projection information.

[0244] The fourth processing module 1040 is configured to process the target adjacent environment image based on the target splicing path, to obtain the panoramic surround view image.

[0245] According to the panoramic surround view image generation device provided in the embodiments of the present application, the target splicing path corresponding to the target adjacent environment image is determined by combining the initial splicing path generated based on the fusion region and the obstacle projection information, which can more flexibly avoid the splicing path projected by multiple obstacles, has a better blind area elimination effect, and thus helps to improve the user experience and ensure driving safety, and is suitable for a wide range of scenarios.

[0246] In some embodiments, the third processing module 1030 can be further configured to:

[0247] determine the target splicing path based on the initial splicing path and the obstacle projection information, including:

[0248] in a case where it is determined based on the obstacle projection information that at least one obstacle projection intersects with the initial splicing path, determining the target splicing path based on the driving parameters of the target vehicle.

[0249] In some embodiments, the third processing module 1030 can be further configured to:

[0250] determine a predicted driving path corresponding to the target vehicle based on the driving parameter;

[0251] determine the target splicing path based on an intersection of the predicted driving path and the fusion region.

[0252] In some embodiments, the third processing module 1030 can be further configured to:

[0253] determine the target splicing path based on at least one of a priority corresponding to each obstacle projection and a distance between each obstacle projection and the target vehicle, in a case that the predicted driving path and the fusion region do not intersect.

[0254] In some embodiments, the third processing module 1030 can be further configured to:

[0255] determine the target splicing path based on position information of the obstacle projection and position information of the predicted driving path, in a case that the predicted driving path and the fusion region intersect.

[0256] In some embodiments, the third processing module 1030 can be further configured to:

[0257] determine an intersection of the obstacle projection and the predicted driving path based on the position information of the obstacle projection and the position information of the predicted driving path;

[0258] determine the target splicing path according to the intersection of the obstacle projection and the predicted driving path.

[0259] In some embodiments, the third processing module 1030 can be further configured to:

[0260] delete the obstacle projection that does not intersect with the predicted driving path, in a case that at least one of the obstacle projections does not intersect with the predicted driving path among all the obstacle projections;

[0261] determine the target splicing path based on the deleted obstacle projection information.

[0262] In some embodiments, the third processing module 1030 can be further configured to:

[0263] determine the target splicing path based on the fusion region, in a case that it is determined that there is no remaining obstacle projection based on the deleted obstacle projection information.

[0264] In some embodiments, the third processing module 1030 can be further configured to:

[0265] In a case where it is determined that there is a remaining obstacle projection based on the deleted obstacle projection information, a target splicing path is determined based on the remaining obstacle projection and the fusion region.

[0266] In some embodiments, the third processing module 1030 can be further configured to:

[0267] adjust the initial splicing path based on the remaining obstacle projection and the fusion region.

[0268] In some embodiments, the third processing module 1030 can be further configured to:

[0269] adjust an angle of the initial splicing path in a case where it is determined that there is at least one remaining obstacle projection intersecting with the initial splicing path;

[0270] determine the adjusted initial splicing path as the target splicing path in a case where the adjusted initial splicing path does not intersect with each of the remaining obstacle projections.

[0271] In some embodiments, the third processing module 1030 can be further configured to:

[0272] adjust an angle of the initial splicing path in a case where it is determined that there is at least one remaining obstacle projection intersecting with the initial splicing path;

[0273] obtain at least one candidate splicing path corresponding to the fusion region by using a path search algorithm in a case where each of the adjusted initial splicing paths intersects with the at least one remaining obstacle projection;

[0274] determine a target candidate splicing path as the target splicing path in a case where the target candidate splicing path does not intersect with the remaining obstacle projection among the at least one candidate splicing path.

[0275] In some embodiments, the third processing module 1030 can be further configured to:

[0276] determine the target splicing path based on at least one of a priority corresponding to each obstacle projection and a distance between each obstacle projection and the target vehicle in a case where all the obstacle projections intersect with the predicted driving path.

[0277] In some embodiments, the second processing module 1020 can be further configured to:

[0278] determine a straight line splicing path generated based on the fusion region as the initial splicing path, wherein the straight line splicing path is located in the fusion region, and the straight line splicing path has only one intersection point with any straight line parallel to the two side boundary directions of the fusion region.

[0279] The panoramic surround view image generation apparatus in the embodiments of the present application can be an electronic device or a component in an electronic device, for example, an integrated circuit or a chip. The electronic device can be a terminal or other device than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and the like. The electronic device can also be a server, a Network Attached Storage (NAS), or a personal computer (PC), and the like. The embodiments of the present application are not limited in this regard.

[0280] The panoramic surround view image generation apparatus in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, an IOS operating system, or other possible operating system. The embodiments of the present application are not limited in this regard.

[0281] The panoramic surround view image generation apparatus provided in the embodiments of the present application can implement each process implemented by the method embodiments of FIGS. 1 to 8. To avoid repetition, the details are not described herein.

[0282] The embodiments of the present application also provide a vehicle.

[0283] The vehicle includes an image sensor.

[0284] In this embodiment, the number of image sensors can be multiple.

[0285] The multiple image sensors are configured to collect multiple environment images of the target vehicle from multiple perspectives.

[0286] The vehicle generates panoramic surround view images based on the panoramic surround view image generation method according to any of the above embodiments.

[0287] According to the vehicle provided in the embodiments of the present application, the target adjacent environment image corresponding target stitching path is determined by combining the initial stitching path generated based on the fusion region and the obstacle projection information, which can more flexibly avoid the multi-obstacle projection to obtain the stitching path, has a better blind area elimination effect, thereby helping to improve the user experience and ensuring driving safety. The vehicle is suitable for a wide range of scenarios.

[0288] In some embodiments, as shown in FIG. 11, the electronic device 1100 according to the embodiments of the present application includes a processor 1101, a memory 1102, and a computer program stored in the memory 1102 and executable on the processor 1101, which, when executed by the processor 1101, implements each process of the panoramic surround view image generation method embodiments described above and achieves the same technical effects. To avoid repetition, details are not described herein.

[0289] It should be noted that the electronic device in the embodiments of the present application includes the mobile electronic device and the non-mobile electronic device described above.

[0290] The embodiments of the present application also provide a non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements each process of the panoramic surround view image generation method described above and achieves the same technical effects. To avoid repetition, details are not described herein.

[0291] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0292] The embodiments of the present application also provide a computer program product including a computer program, which, when executed by a processor, implements the panoramic surround view image generation method described above.

[0293] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0294] The embodiments of the present application also provide a chip including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement each process of the panoramic surround view image generation method described above and achieve the same technical effects. To avoid repetition, details are not described herein.

[0295] It should be understood that the chip mentioned in the embodiments of the present application can also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip, etc.

[0296] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a", "comprising", or "comprises" does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the terms "one embodiment", "some embodiments", "certain embodiments", "certain examples", or "some examples" in the present document are not necessarily referring to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0297] From the above description of the embodiments, it is clear that the above-described method of the embodiments can be realized by means of software and the necessary universal hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) execute the method described in each embodiment of the present application.

[0298] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-described specific embodiments, which are only illustrative but not restrictive, and those of ordinary skill in the art can make many forms without departing from the purpose of the present application and the scope protected by the claims.

[0299] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "illustrative embodiment", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples.

[0300] While the embodiments of the application have been shown and described, it is to be understood that the embodiments can be varied, modified, substituted and changed by those skilled in the art without departing from the principles and spirit of the application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for generating a surround view image, the method comprising: The method comprises: determining obstacle projection information and a fusion area between target adjacent environment images based on a plurality of environment images of a target vehicle at a plurality of perspectives; generating an initial stitching path based on the fusion area; determining a target stitching path based on the initial stitching path and the obstacle projection information; processing the target adjacent environment images based on the target stitching path to obtain a panoramic surround view image. 2.The surround view image generation method of claim 1, wherein, The method further comprises: in a case where it is determined based on the obstacle projection information that at least one obstacle projection intersects with the initial stitching path, determining the target stitching path based on a driving parameter of the target vehicle.

3. The surround view image generation method of claim 2, wherein, The method further comprises: determining a predicted driving path corresponding to the target vehicle based on the driving parameter; determining the target stitching path based on an intersection between the predicted driving path and the fusion area.

4. The surround view image generation method of claim 3, wherein, The method further comprises: in a case where the predicted driving path and the fusion area do not intersect, determining the target stitching path based on at least one of a priority corresponding to each obstacle projection and a distance between each obstacle projection and the target vehicle. 5.The surround view image generation method of claim 3, wherein, The method further comprises: in a case where the predicted driving path and the fusion area intersect, determining the target stitching path based on position information of the obstacle projection and position information of the predicted driving path.

6. The surround view image generation method of claim 5, wherein, The method further comprises: determining an intersection between the obstacle projection and the predicted driving path based on the position information of the obstacle projection and the position information of the predicted driving path; determining the target stitching path according to the intersection between the obstacle projection and the predicted driving path.

7. The surround view image generation method of claim 5, wherein, The method further comprises: in a case where at least one of the obstacle projections does not intersect with the predicted driving path, deleting the obstacle projection that does not intersect with the predicted driving path; determining the target stitching path based on the obstacle projection information after deletion.

8. The surround view image generation method of claim 7, wherein, The method further comprises: in a case where it is determined based on the obstacle projection information after deletion that there is no remaining obstacle projection, determining the target stitching path based on the fusion area. 9.The surround view image generation method of claim 7, wherein, The method further comprises: in a case where it is determined based on the obstacle projection information after deletion that there is a remaining obstacle projection, determining the target stitching path based on the remaining obstacle projection and the fusion area.

10. The surround view image generation method of claim 9, wherein, The determining the target stitching path based on the remaining obstacle projections and the fusion region comprises: The adjusting the initial stitching path based on the remaining obstacle projections and the fusion region comprises:

11. The surround view image generation method of claim 10, wherein, The adjusting the initial stitching path based on the remaining obstacle projections and the fusion region comprises: adjusting an angle of the initial stitching path in a case where it is determined that at least one of the remaining obstacle projections intersects with the initial stitching path; determining the adjusted initial stitching path as the target stitching path in a case where the adjusted initial stitching path does not intersect with each of the remaining obstacle projections. 12.The surround view image generation method of claim 10, wherein, The adjusting the initial stitching path based on the remaining obstacle projections and the fusion region comprises: adjusting an angle of the initial stitching path in a case where it is determined that at least one of the remaining obstacle projections intersects with the initial stitching path; adopting a path search algorithm to obtain at least one candidate stitching path corresponding to the fusion region in a case where each of the adjusted initial stitching paths intersects with at least one of the remaining obstacle projections; determining a target candidate stitching path as the target stitching path in a case where the target candidate stitching path does not intersect with each of the remaining obstacle projections among the at least one candidate stitching path.

13. The surround view image generation method of claim 5, wherein, The determining the target stitching path based on the position information of the obstacle projections and the position information of the predicted driving path comprises: determining the target stitching path based on at least one of a priority corresponding to each of the obstacle projections and a distance between each of the obstacle projections and the target vehicle in a case where all of the obstacle projections intersect with the predicted driving path.

14. The surround view image generation method of any one of claims 2-13, wherein, The driving parameter of the target vehicle comprises at least one of a steering angle of the target vehicle and a driving direction of the vehicle.

15. The surround view image generation method of any one of claims 1-14, wherein, The generating the initial stitching path based on the fusion region comprises: determining a straight-line stitching path generated based on the fusion region as the initial stitching path, wherein the straight-line stitching path is located in the fusion region, and the straight-line stitching path has only one intersection point with any straight line parallel to two side boundary directions of the fusion region.

16. A vehicle characterized by comprising: comprises: an image sensor; The vehicle generates a surround view image based on the surround view image generation method according to any one of claims 1-15.

17. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the surround view image generation method according to any one of claims 1-15.

18. A computer program product comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the surround view image generation method according to any one of claims 1-15.

Images