Method, apparatus and system for generating surround view images, and computer program product and vehicle

Abstract

The present disclosure provides a method, apparatus, and system for generating a surround view image, as well as a computer program product and a vehicle. An image can be captured from an image capturing area where no shadows exist, and the captured image can be used to reconstruct an image covering a second filled area corresponding to a first filled area, i.e., an image covering an area corresponding to the vehicle's underside. Because the image captured from the image capturing area does not contain shadows, after performing an affine transformation on the image, the reconstructed image covering the second filled area and including an image of the vehicle's underside does not contain shadows. Therefore, it is possible to effectively prevent the reconstructed image of the vehicle's underside from including numerous shadows or distorted shadows, which can adversely affect the driver's ability to observe and interpret information about the vehicle's underside.

Description

[Technical Field]

[0001] The present invention relates to a method for generating a surround-view image, a panoramic surround-view device, a panoramic surround-view system, a computer program product, and a vehicle, which are capable of generating a panoramic surround-view image without a blind spot under a vehicle. [Background technology]

[0002] Currently, more and more panoramic imaging systems are being put into practical use in the automotive field, especially in the field of road vehicles. Panoramic imaging systems can provide a complete surround-view image that reflects the vehicle's surrounding environment. Therefore, especially in scenes with complex obstacles around the vehicle, such as parking lots and busy roads, panoramic imaging systems can intuitively provide the driver with information about the vehicle's surrounding environment and effectively avoid collisions between the vehicle and surrounding obstacles.

[0003] Currently, in addition to displaying the surrounding environment of a vehicle, panoramic imaging systems can also provide a view from the underside of the vehicle, allowing the driver of the vehicle to obtain information related to the underside of the vehicle. In this field, an affine transformation is typically performed on an image of the surrounding environment of the vehicle acquired according to the vehicle's driving parameters to reconstruct an underside image of the vehicle. However, if there are shadows around the vehicle, performing an affine transformation on the image containing the shadows will result in the reconstructed underside image containing multiple shadows and distorted shadows. This will affect the driver's observation and judgment of information related to the underside of the vehicle.

[0004] Therefore, there is a need for a panoramic imaging system that can provide an under-vehicle image that is not affected by shadows. Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, an object of the present invention is to provide a method, an apparatus and system for generating a surround view image, a computer program product, and a vehicle that can provide a shadow-free surround view image of the underside of a vehicle, thereby eliminating the impact of shadows under the vehicle on the driver of the vehicle. [Means for solving the problem]

[0006] In the sense of the present disclosure, a motor vehicle may be any vehicle. Preferably, the motor vehicle is, for example, a road vehicle or a train. Particularly preferably, the motor vehicle is a road vehicle such as a car or a truck.

[0007] A first aspect of the present disclosure relates to a method for generating a surround view image, comprising: determining a first environmental image around the vehicle at a first time point, the first environmental image being covered with a first filling area, the first filling area covering an area corresponding to an underside of the vehicle in the first environmental image and covering a shadow area around the vehicle in the first environmental image; Cropping an image from an image crop region in the first environment image, the image crop region being outside the first fill extent of the first environment image. obtaining operating parameters of the vehicle; determining a second environmental image of the surroundings of the vehicle at a second time point after the first time point, the second environmental image being covered by a second filling area corresponding to the first filling area; and mapping the image to the second filling range of the second environment image according to the operating parameters to generate a surround view image; Includes.

[0008] According to a more detailed embodiment of the method of the present disclosure, the method further includes cropping a third filling range image from the surround view image, the third filling range image corresponding to a third filling range, wherein the second environment image is covered with the third filling range, the third filling range corresponding to an area under the vehicle in the second environment image and not covering a shadow area of ​​the vehicle in the second environment image; and compositing the third filling range image with the second environment image to generate a surround view image including the shadow area.

[0009] According to a more detailed embodiment of the method of the present disclosure, in the direction of travel of the vehicle, the image cropping region is adjacent to the first filling range.

[0010] According to a more detailed embodiment of the method of the present disclosure, the method further includes rendering a transparent vehicle model corresponding to the vehicle in an area corresponding to an underside of the vehicle in the surround view image or in a surround view image including the shadow region.

[0011] According to a more detailed embodiment of the method of the present disclosure, the method further includes determining the first filling range and / or the second filling range according to the size of a shadow area around the vehicle and the size of the vehicle.

[0012] A second aspect of the present disclosure relates to a panoramic surround view device, one or more processors; and Contains one or more memories, The memory has stored therein computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the methods according to the present disclosure as described above.

[0013] A third aspect of the present disclosure relates to a computer program product including computer-readable instructions that, when executed by a processor, cause the processor to perform a method according to the present disclosure as described above.

[0014] A fourth aspect of the present disclosure relates to a panoramic surround view system, an image acquisition module configured to determine a first environmental image around a vehicle at a first time point and a second environmental image around the vehicle at a second time point after the first time point, the first environmental image being covered with a first filling range, the first filling range covering an area corresponding to an underside of the vehicle in the first environmental image and covering a shadow area around the vehicle in the first environmental image, and the second environmental image being covered with a second filling range corresponding to the first filling range; and an image reconstruction module configured to crop an image from an image crop region in the first environment image and map the image to the second fill region of the second environment image in accordance with the operational parameters to generate a surround view image, the image crop region being outside the first fill region of the first environment image; Includes:

[0015] According to a more detailed embodiment of the panoramic surround view system of the present disclosure, the image reconstruction module is further configured to: crop a third filling range image from the surround view image, the third filling range image corresponding to a third filling range, wherein the second environment image is covered with the third filling range, the third filling range corresponds to an area corresponding to the underside of the vehicle in the second environment image and does not cover a shadow area of ​​the vehicle in the second environment image; and composite the third filling range image with the second environment image to generate a surround view image including a shadow area.

[0016] A fifth aspect of the present disclosure relates to an automobile, which is equipped with the above-described panoramic surround view device or the above-described panoramic surround view system.

[0017] By using the method, apparatus, system, computer program product, and vehicle for generating a surround view image according to the present disclosure, an image can be cropped from a shadow-free image cropping region and used to reconstruct an image covering a second filling range corresponding to the first filling range, where the second filling range also covers an area below the vehicle. Because the image cropped from the image cropping region does not contain shadows, the image of the second filling range including the underside image reconstructed after affine transformation also does not contain shadows. Therefore, the negative impact on the driver of the vehicle's ability to observe and interpret information about the underside of the vehicle caused by the inclusion of numerous shadows and distorted shadows in the reconstructed image of the underside of the vehicle can be effectively prevented. [Brief explanation of the drawings]

[0018] The above and other objects, features, and advantages of the embodiments of the present disclosure will become more apparent from the detailed description of the embodiments of the present disclosure with reference to the drawings. The drawings are used to provide a further understanding of the embodiments of the present disclosure, constitute a part of the specification, and are used to explain the present disclosure together with the embodiments of the present disclosure, but are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0019] [Figure 1A] 10 exemplarily shows an automobile surround view image in the first frame. [Figure 1B] An exemplary automobile surround view image is shown in the second frame. [Figure 1C] 10 exemplarily shows an automobile surround view image in the third frame. [Figure 1D] 10 exemplarily shows an automobile surround view image in a subsequent frame. [Figure 2] 1 shows a schematic representation of an automobile surround view image with multiple shadows in the under-vehicle image. [Figure 3] 1 shows a flowchart of a method for generating a surround view image according to an embodiment of the present disclosure. [Figure 4A]1A and 1B illustrate schematic diagrams of an automobile surround view image generated corresponding to a first time point according to an embodiment of the present disclosure. [Figure 4B] 10A and 10B are schematic illustrations of automobile surround view images generated corresponding to a second time point according to an embodiment of the present disclosure. [Figure 4C] 10A and 10B are schematic illustrations of automobile surround view images generated corresponding to a third time point according to an embodiment of the present disclosure. [Figure 4D] 1 shows a schematic representation of a surround view image including a full fill range image and an under-vehicle area image. [Figure 5A] 1 shows a first environmental image of a vehicle surrounding determined by a method for generating a surround view image according to an embodiment of the present disclosure. [Figure 5B] 1 illustrates a surround-view image including an under-vehicle image generated by a method for generating a surround-view image according to an embodiment of the present disclosure. [Figure 6] 10 shows a flowchart of a method for generating a surround view image according to another embodiment of the present disclosure. [Figure 7] 10 shows an automobile surround view image generated corresponding to a second time point according to another embodiment of the present disclosure. [Figure 8] 10 shows a surround view image generated according to another embodiment of the present disclosure. [Figure 9] 1 illustrates a panoramic surround view device according to an embodiment of the present disclosure. [Figure 10] 1 illustrates a computer program product according to an embodiment of the present disclosure. [Figure 11] 1 illustrates a panoramic surround view system according to an embodiment of the present disclosure. [Figure 12] 1 illustrates a vehicle according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0020] The technical solutions in the embodiments of the present disclosure are clearly and completely described below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part, not all, of the embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present disclosure.

[0021] Furthermore, in this specification and drawings, the terms "first / second" are used only to distinguish between similar objects and do not represent a particular order of objects. Where permissible, "first / second" can be substituted for a particular order or sequence, and it is understood that the embodiments of the present disclosure described herein can be performed in an order other than that illustrated or described herein.

[0022] Furthermore, when flowcharts are used in the present specification and drawings to describe steps of a method according to an embodiment of the present disclosure, it should be understood that the preceding or following steps are not necessarily performed in the exact order. Conversely, unless explicitly defined in the embodiments of the present disclosure, various steps may be processed in the reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more steps may be deleted from these processes.

[0023] 1A-1D are schematic illustrations of automobile surround-view images including an image showing the underside of an automobile. When the solar elevation angle is low, such as in winter or in the evening, the automobile is typically surrounded by shadows. FIG. 1A exemplarily shows automobile surround-view image 101 in a first frame when the sun is shining on automobile 110 from directly behind it. Therefore, a first shadow 121 of the automobile is projected directly in front of automobile 110. First shadow 121 is shown here schematically as a triangle.

[0024] At the time of the first frame, a conventional panoramic imaging system with a vehicle underside image display function typically first acquires an image 141 within an image cropping region 130 in front of the automobile 110. Here, the image cropping region 130 is roughly outlined by a dashed line, and the image 141 is located within the dashed line frame. As shown in FIG. 1A , the image cropping region 130 includes the first shadow 121 of the automobile, and therefore the image 141 acquired at this time also includes the first shadow 121 of the automobile.

[0025] Then, according to vehicle driving parameters such as the current speed, current position, and current steering of vehicle 110, image 141 is mapped to a corresponding position 152 of the area below the vehicle by an affine transformation and displayed in vehicle surround-view image 102 in the second frame to be displayed. Figure 1B exemplarily shows vehicle surround-view image 102 in the second frame.

[0026] Because the mapped image 141 contains the first shadow 121 of the vehicle in the first frame, after mapping, the image displayed at the corresponding position 152 in the underside region of the vehicle 110 also contains the first shadow 121. This therefore causes the reconstructed underside image of the vehicle to contain the first shadow 121, which affects the use of the panoramic image system by the vehicle driver.

[0027] Similarly, at the time of the second frame, a conventional panoramic imaging system with an under-vehicle image display capability would typically continue to acquire an image 142 within the image crop region 130 in front of the automobile 110. Because the solar elevation angle is still small at the time of the second frame, the automobile's second shadow 122 continues to be cast in front of the automobile 110. Here, the second shadow 122 is also schematically represented by a triangle. The second shadow 122 is included in the image crop region 130. Therefore, the image 142 acquired at this time also includes the automobile's second shadow 122.

[0028] Then, similarly, image 142 is mapped to corresponding position 152 in the region below the vehicle by an affine transformation according to the vehicle driving parameters, and displayed in vehicle surround-view image 103 in the displayed third frame. Furthermore, similarly, image 141 is mapped to corresponding position 153 in the region below the vehicle by an affine transformation according to the vehicle driving parameters, and displayed in vehicle surround-view image 103 in the displayed third frame. Figure 1C exemplarily shows vehicle surround-view image 103 in the third frame.

[0029] Since the image 142 acquired at the time of the second frame to reconstruct the image of the underside of the vehicle includes the second shadow 122 and the image 141 acquired at the time of the first frame to reconstruct the image of the underside of the vehicle includes the first shadow 121, after mapping, the images displayed at the corresponding positions 152 and 153 of the underside area of ​​the automobile 110 also include the second shadow 122 and the first shadow 121, which affects the use of the panoramic image system by the automobile driver.

[0030] Similarly, in subsequent frames, the shadows around the vehicle 110 continue to be mapped to corresponding positions in the region below the vehicle and displayed in the vehicle surround-view image 10N in the subsequent frame. FIG. 1D exemplarily illustrates the vehicle surround-view image 10N in the subsequent frame. As shown in FIG. 1D, the image of the region below the vehicle contains numerous shadows 121, ..., and 12N-1. These shadows accumulate in the image of the region below the vehicle 110, affecting the driver's observation of the region below the vehicle and information about the region below the vehicle, such as whether there are obstacles or bumps underneath the vehicle, preventing the panoramic image system from providing useful reference information to the driver.

[0031] An automobile surround view image containing numerous shadows in the vehicle underside image is also shown in Figure 2. As shown in Figure 2, shadows 121, ..., and 12N-1 in the vehicle underside portion can seriously affect a motorist's observation of the bottom of the vehicle when using a panoramic imaging system.

[0032] Therefore, this disclosure proposes a method for generating a surround-view image, which effectively prevents the reconstructed under-vehicle image from containing many shadows by cropping an image from a shadow-free image crop region and using that image to reconstruct an image of a region corresponding to the underside of the vehicle at a subsequent time point.

[0033] 3 illustrates a flowchart of a method 300 for generating a surround-view image according to an embodiment of the present disclosure. The method 300 for generating a surround-view image according to an embodiment of the present disclosure can be used in vehicles equipped with a panoramic surround-view system, such as, for example, a car or a truck, but the embodiments of the present disclosure are not limited thereto. The method 300 for generating a surround-view image according to an embodiment of the present disclosure can also be used in any other automobile in which a surround-view image of the surroundings needs to be generated.

[0034] 4A and 4B further illustrate automobile surround view images 401 and 402 generated by performing method 300. For clarity, the method 300 for generating surround view images according to an embodiment of the present disclosure will be described below with combined reference to FIGS. 3, 4A, and 4B.

[0035] FIG. 4A shows an automobile surround-view image 401 generated corresponding to a first time point. As shown in FIGS. 3 and 4A, first, in step S310, a first environment image 421 around the automobile 410 is determined at a first time point, e.g., the time point of a first frame. Here, for example, images reflecting multiple different orientation environments around the automobile 410, such as four-directional video images, may be obtained using multiple camera devices installed on the automobile 410 to capture different orientation environments of the automobile 410. Next, image stitching is performed on the multiple different orientation environment images, and then perspective transformation is performed to obtain an automobile surround-view image 400 from a bird's-eye view as shown in FIG. 4A, and the first environment image 421 around the automobile 410 is determined. Here, as described above, because the environment image is formed by stitching together images captured by multiple camera devices installed on the automobile, the area below the automobile cannot be represented in the environment image. Then, an image of the area below the automobile needs to be reconstructed. It should be noted here that the present disclosure does not limit the method of generating the first environment image 421 or the particular viewpoint (such as a bird's-eye view, a side rear oblique view, etc.) from which the automobile surround view image 400 is displayed.

[0036] As shown in FIG. 4A , the first environment image 421 is covered by a first filling range 431. In the actual first environment image 421, the area of ​​the first environment image 421 covered by the first filling range 431 can no longer be displayed because it is covered by the first filling range 431. However, in order to clearly show the area covered by the first filling range 431, in FIG. 4A , the first filling range 431 is schematically represented by a transparent rectangular area with a triangular grid. As shown in FIG. 4A , in addition to completely covering the image of the area below the automobile 410, the first filling range 431 also covers a shadow area 441 around the automobile in the first environment image 421. Here, the shadow area 441 is similarly represented by a triangle. Therefore, because the shadow area 441 is covered, the first environment image 421 after the cover is no longer able to display the area of ​​the first environment image 421 covered by the first filling range 431, i.e., the shadow area 441.

[0037] Here, the first filling range (and the second filling range referred to in subsequent step S340) may be determined according to the size of the shadow area around the vehicle and the size of the vehicle, so that the filling range can completely cover the shadow area around the vehicle. For example, the size of the shadow area around the vehicle can be statistically determined in advance, and a filling range of this size is determined in advance and always used to cover the environment image in the subsequent method of generating a surround view image. The size of the filling range may be dynamically adjusted by determining the shadow area in the environment image in the method of generating a surround view image. Therefore, the method 300 for generating a surround view image according to an embodiment of the present disclosure may further include, in step S310′, determining the first filling range 431 and / or the second filling range 432 referred to below according to the size of the shadow area around the vehicle and the size of the vehicle.

[0038] Furthermore, in step S320, an image 461 is cropped from the image cropping region 450 in the first environment image 421. Here, as shown in FIG. 4A , in the first environment image 421, the image cropping region 450 is located outside the first filling range 431. Specifically, the position of the cropped image 461 in the first environment image 421, i.e., the position of the image cropping region 450, may be determined according to the driving parameters of the automobile 410, such as the driving direction, driving speed, steering angle, etc. Preferably, in the driving direction of the automobile, the image cropping region 450 is adjacent to the first filling range 431. That is, for example, when the automobile 410 is moving forward, the image cropping region 450 may be located directly in front of the first filling range 431, as shown in FIG. 4 . It is also conceivable that when the automobile 410 is moving backward, the image cropping region 450 should be located directly behind the first filling range 431. Here, the present disclosure does not limit the specific position of the image clipping region 450 in the first environment image 421. Considering the difficulty of performing an affine transformation, it is an optional embodiment that the image clipping region 450 is adjacent to the first filling range 431 in the driving direction, because this allows the affine transformation to be completed with a smaller distance / time difference and can avoid inaccurate mapping results caused by unpredictable vehicle movements within a longer time / distance difference.

[0039] 4A, the cropped image 461 is schematically represented by a rectangle with a diagonal line therein. As described above, the shadow region 441 is already covered by the first filling range 431, and the image cropping region 450 is located outside the first filling range 431, so the shadow region 441 is not included in the cropped image 461.

[0040] Furthermore, in step S330, motion parameters of the automobile 410, such as the driving direction, driving speed, steering angle, etc., are acquired. Here, by way of example, the motion parameters may be determined by corresponding sensor devices installed on the automobile 410, or may be provided to the automobile 410 by a navigation positioning system (GPS, Galileo, BDS, or other navigation system). Here, the present disclosure does not limit the specific method for acquiring the motion parameters. Then, an affine transformation is performed on the image 461 according to the acquired motion parameters to fill the area below the automobile 410.

[0041] Next, at a second time point after the first time point, i.e., a subsequent frame time point after the first frame, e.g., the second frame, step S340 determines a second environment image 422 around the automobile 410. Figure 4B shows an automobile surround-view image 402 generated corresponding to the second time point. Here, for example, the second environment image 422 may be determined in the same manner as the first environment image 421, and for the sake of brevity, this will not be described again here.

[0042] 4B , corresponding to the first environment image 421, the second environment image 422 is similarly covered by a second filling area 432 corresponding to the first filling area 431. That is, the filling area of ​​the first filling area 431 in the first environment image 421 is equivalent to the filling area of ​​the second filling area 432 in the second environment image 422, and the first filling area 431 and the second filling area 432 have the same filling area. The second filling area 432 not only completely covers the image of the area below the automobile 410, but also covers a shadow area 442 around the automobile 410 in the second environment image 422. Because the shadow area 442 is covered, the second environment image 422 cannot display the area covered by the second filling area 432 in the second environment image 422, i.e., the shadow area 442.

[0043] Furthermore, in step S350, the cropped image 461 at the first time point is mapped to a second filling range 432 of the second environment image 422 according to the motion parameters acquired in step S330 to generate a surround view image 402, particularly a surround view image 402 including an image of the lower region of the vehicle. Here, for example, predicted positions 472 within the second filling range 432 to which pixel points in the image 461 will move at the second time point may be estimated according to the motion parameters, and an affine transformation is performed accordingly for each pixel point in the image 461. The pixel points are displayed at the corresponding predicted positions 472, thereby reconstructing an image within the second filling range 432.

[0044] The reconstructed image of the second filled area 432 is then stitched (combined) with the second environment image 422 to generate a surround view image 402 corresponding to a second time point and including the reconstructed image of the second filled area 432, i.e., the image of the area below the vehicle.

[0045] As described above, image 461 cropped at the first time point does not include shadow region 441, and therefore after image 461 is mapped and transformed as shown in Figure 4B, the image in second filling range 432 reconstructed using that image does not include a shadow. Furthermore, because the area below the automobile is covered by second filling range 432, the image of the area below the automobile reconstructed as shown in Figure 4B also does not include a shadow.

[0046] Furthermore, in this embodiment, for simplicity, it should be noted that FIGS. 4A and 4B schematically illustrate only a portion of the processing performed at the first and second time points in the method 300 for generating a surround-view image according to an embodiment of the present disclosure. That is, as shown in FIG. 4B, at the second time point, only the image 461 cropped at the first time point is mapped to a subregion (a region corresponding to the predicted position 472) of the second filling range 432, and an image of the second filling range 432 in the subregion is reconstructed. However, since the method 300 for generating a surround-view image according to an embodiment of the present disclosure is continuously and periodically executed, it is conceivable that a surround-view image including a complete filling range image can be generated. That is, at a time point after the second time point, for example, at a third time point after the second time point as shown in FIG. 4C, an image 463 within the image crop region 450 is continuously acquired. Similarly, according to the driving parameters of the vehicle, the cropped image 462 at the second time point may be mapped to the predicted position 472 of the third filling area 433 by an affine transformation, and the image 461 captured at the first time point may be mapped to the predicted position 473 of the third filling area 433. Image stitching is then performed to generate the surround-view image 402 corresponding to the third time point. Then, similarly, by continuously and periodically performing the method 300 for generating a surround-view image according to an embodiment of the present disclosure, the filled area image is completely reconstructed, and a surround-view image including the complete filled area image and the vehicle underside area image, as shown in FIG. 4D , is generated.

[0047] For clarity, reference is made to FIG. 2 to further illustrate the actual surround-view image generated by the method 300 according to an embodiment of the present disclosure. FIG. 5A illustrates a first environment image 421 around a vehicle 410 determined in step S310 of the method 300 for generating a surround-view image according to an embodiment of the present disclosure. Here, because the environment image is formed by stitching together images captured by multiple camera devices disposed on the vehicle, the area below the vehicle cannot be represented in the actual environment image. A region 471 corresponding to the underside of the vehicle in the environment image needs to be reconstructed in a subsequent step and is currently shown in gray in FIG. 5A.

[0048] As shown in FIG. 5A , the first environment image 421 is covered by a first filling area 431. The first filling area 431 covers an area 471 in the first environment image 421 that corresponds to the underside of the vehicle. Furthermore, corresponding to what was described in FIG. 4A , the image crop area 450 is located directly in front of the first filling area 431. At subsequent time points, the image cropped by the image crop area 450 is periodically used to reconstruct images within the first filling area 431, thereby reconstructing a filling area image that includes the area that corresponds to the underside of the vehicle. The reconstructed filling area image is then combined with the environment image corresponding to that time point to generate a surround view image that includes the underside of the vehicle that corresponds to that time point. FIG. 5B illustrates a surround view image that includes the underside of the vehicle generated by the method 300 according to an embodiment of the present disclosure. As shown in FIG. 5B , the generated surround view image, particularly the area that corresponds to the underside of the vehicle, does not include any shadows. Compared with FIG. 2 , the method 300 for generating a surround view image according to an embodiment of the present disclosure effectively prevents the reconstructed vehicle underside image from containing multiple shadows and distorted shadows, thereby eliminating the adverse effects on the automobile driver's observation and judgment of information about the underside of the vehicle.

[0049] Because the reconstructed filled area covers the shadows around the vehicle, the generated surround-view image does not include the shadows after reconstruction, as shown in Figures 4A to 4C and 5B. However, in order to make the generated surround-view image match the actual image of the environment around the vehicle, it is necessary to display the shadows around the vehicle in the generated surround-view image.

[0050] A method 600 for generating a surround-view image according to another embodiment of the present disclosure aims to solve the above problem. FIG. 6 schematically illustrates a flowchart of the method 600 for generating a surround-view image according to another embodiment of the present disclosure. FIG. 7 schematically illustrates an automobile surround-view image 402 generated corresponding to a second time point according to another embodiment of the present disclosure. FIG. 8 illustrates a surround-view image generated according to another embodiment of the present disclosure. The method 600 according to another embodiment of the present disclosure will be described below with reference to FIGS. 6 to 8. It should be noted that the method 600 for generating a surround-view image according to another embodiment of the present disclosure may further process a surround-view image corresponding to any time point. Those skilled in the art should understand that the time point targeted by the method 600 according to another embodiment of the present disclosure is not limited to the second time point schematically illustrated in FIG. 7.

[0051] 6, steps S610-S640 of method 600 for generating a surround-view image according to another embodiment of the present disclosure correspond to steps S310-S350 of the method described in FIG. 3 and will not be described again here for brevity. According to another embodiment of the present disclosure, method 600 further includes, in step S660, cropping third filling area image 492 corresponding to third filling area 433 from the generated surround-view image, for example, surround-view image 402 generated corresponding to the second time point.

[0052] As shown on the left side of FIG. 7 , in the surround view image 402 generated in step S650, the second environment image 422 is covered by a third filling area 433. The third filling area 433 corresponds to the area below the automobile in the second environment image 422 and does not cover the shadow area of ​​the automobile in the second environment image, such as the shadow area 442 corresponding to the second time point. For clarity, the third filling area 433 is schematically indicated by a gray rectangle. The fact that the third filling area 433 corresponds to the area below the automobile in the second environment image 422 can be understood as the third filling area 433 covering the area below the automobile 410. Preferably, the third filling area 433 completely covers the area below the automobile 410. Compared to FIG. 4B , FIG. 7 illustrates an embodiment in which the third filling area 433 completely covers the area below the automobile 410. However, it should be noted that the present disclosure is not limited to selecting a particular size and location for third filling area 433. Those skilled in the art should understand that third filling area 433 may be located in an area corresponding to second filling area 432 in generated surround view image 702, may cover the area below automobile 410, and may not cover shadow areas.

[0053] Furthermore, according to another embodiment of the present disclosure, the method 600 further includes, in step S670, combining the third filled range image 492 with the second environment image 422 to generate a surround view image including the shadow region.

[0054] 7, a surround view image 402′ (see FIG. 8) including a vehicle under-area image can be generated by cropping third filled area image 492 within third filled area 433 and using it to fill second environment image 422, which corresponds to that time (second time point) and is not covered by any filled area, particularly vehicle under-area 471, which cannot be represented in second environment image 422 (see the right side of FIG. 7). Because second environment image 422, which is not covered by any filled area, includes shadow area 442, the filled image includes shadow area 442 in addition to vehicle under-area image 492, making the generated surround view image including the shadow area more realistic.

[0055] For clarity, referring to FIG. 8 , an actual surround-view image 402′ including a shadow region generated by the method 300 according to another embodiment of the present disclosure is further illustrated. Here, the surround-view image 402′ not only displays the vehicle underside image 492 but also displays a shadow region 442. Compared to the automobile surround-view image 10N shown in FIG. 2 , which includes numerous shadows, the surround-view image 402′ generated by the method 300 according to another embodiment of the present disclosure not only does not include numerous shadows in the vehicle underside region but also displays shadow regions that are consistent with the actual environment around the automobile, such as the shadow region in front of the automobile. This makes the generated surround-view image 402′ more realistic and further reduces the impact on the driver caused by the difference between the surround-view image generated by the panoramic imaging system and reality.

[0056] 8, in a method for generating a surround view image according to another embodiment of the present disclosure, a transparent vehicle model 411 corresponding to the vehicle is rendered in a region corresponding to the underside of the vehicle in the surround view image or the surround view image including a shadow region 402'. Therefore, when the underside region no longer contains many shadows, on the one hand, the transparent vehicle model 411 allows the driver to better observe the underside of the vehicle through the transparent vehicle model 411. On the other hand, the transparent vehicle model 411 can define a region corresponding to the underside of the vehicle in the surround view image, which makes it convenient for the driver to determine the underside information of the vehicle.

[0057] In summary, the method for generating a surround view image according to the present disclosure can effectively prevent the reconstructed vehicle underside image from containing a large number of shadows, provide a shadow-free perspective of the underside of the vehicle, and effectively prevent the shadows in the surround view image from affecting the vehicle driver.

[0058] The present disclosure further provides a panoramic surround view device 900. FIG. 9 schematically illustrates the panoramic surround view device 900 according to an embodiment of the present disclosure. Here, the panoramic surround view device 900 includes one or more processors 910 and one or more memories 920, which store computer-readable instructions that, when executed by the one or more processors 910, cause the one or more processors 910 to perform the method for generating a surround-view image described above with reference to FIGS. 3 to 8. For a specific description of various features and advantages of the panoramic surround view device 900, please refer to the description of the corresponding method above, and for the sake of brevity, they will not be described again here.

[0059] The present disclosure further provides a computer program product 1000. FIG. 10 schematically illustrates the computer program product 1000 according to an embodiment of the present disclosure. Here, the computer program product 1000 includes computer-readable instructions 1020 stored on a computer-readable storage medium 1010, which, when executed by a processor, causes the processor to perform the method for generating a surround-view image described above with reference to FIGS. 3 to 8. The computer-readable storage medium may include, but is not limited to, a volatile memory and / or a non-volatile memory. Examples of the volatile memory include, but are not limited to, a random access memory (RAM), a cache memory, etc. Examples of the non-volatile memory include, but are not limited to, a read-only memory (ROM), a hard disk, a flash memory, an optical disk, etc. For a specific description of various features and advantages of the computer program product 1000, please refer to the description of the corresponding method above, and for the sake of brevity, they will not be described again here.

[0060] The present disclosure further provides a panoramic surround view system 1100. Figure 11 illustrates a schematic diagram of a panoramic surround view system 1100 according to an embodiment of the present disclosure.

[0061] Here, the panoramic surround view system 1100 includes an image acquisition module 1110 configured to determine a first environmental image around the automobile at a first time point and to determine a second environmental image around the automobile at a second time point after the first time point, wherein the first environmental image is covered with a first filling range, which covers an area corresponding to the underside of the automobile in the first environmental image and covers a shadow area around the automobile in the first environmental image, and the second environmental image is covered with a second filling range corresponding to the first filling range.

[0062] Further, the panoramic sound view system 1100 includes an image reconstruction module 1120 configured to crop an image from an image crop region in the first environment image, perform an affine transformation on pixels of the image, and map the image to a second filling range of the second environment image according to the operating parameters to generate a surround view image, wherein the image crop region is outside the first filling range of the first environment image.

[0063] Here, for a specific description of the various features and advantages of the panoramic surround view system 1100, please refer to the description of the corresponding method above with reference to Figures 3 to 5B, and for the sake of brevity, they will not be described again here.

[0064] Furthermore, in another embodiment of the panoramic surround view system 1100 according to the present disclosure, the image reconstruction module 1120 is further configured to crop a third filling range image corresponding to a third filling range from the surround view image, wherein the second environment image is covered by the third filling range, the third filling range corresponds to an area corresponding to the underside of the vehicle in the second environment image and does not cover a shadow area of ​​the vehicle in the second environment image, and synthesize the third filling range image with the second environment image to generate a surround view image including the shadow area.

[0065] Here, for a specific description of the various features and advantages of the panoramic surround view system 1100 in the above embodiment, please refer to the description of the corresponding method above with reference to Figures 6 to 8, and for the sake of brevity, they will not be described again here.

[0066] The present disclosure further provides an automobile 1200. Fig. 12 schematically illustrates the automobile 1200 according to an embodiment of the present disclosure. As shown in Fig. 12, the automobile is equipped with the panoramic surround view device 900 described above with reference to Fig. 9 or the panoramic surround view system 1100 described above with reference to Fig. 11. For a specific description of the various features and advantages of the automobile 1200, please refer to the description of the panoramic surround view device 900 or the panoramic surround view system 1100 above, and for the sake of brevity, they will not be described again here.

[0067] Unless otherwise defined in this specification, all terms (including technical and scientific terms) used in this specification have the same meaning as commonly understood by those skilled in the art. Furthermore, terms generally defined in dictionaries, etc. should be interpreted as having the same meaning as in the context of the relevant technology, and should not be interpreted in an idealized or extremely formalized sense unless explicitly defined in this specification.

[0068] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented as modified and altered without departing from the spirit and scope of the present disclosure, which is determined by the claims. Therefore, the description in this specification is for illustrative purposes only and does not have any limiting meaning on the present disclosure.

Claims

1. 1. A method for generating a surround view image, comprising: determining a first environmental image around the vehicle at a first time point, the first environmental image being covered with a first filling range, the first filling range covering an area corresponding to an underside of the vehicle in the first environmental image and covering a shadow area around the vehicle in the first environmental image; cropping an image from an image crop region in the first environment image, the image crop region being outside the first fill extent of the first environment image; obtaining operational parameters of the vehicle; determining a second environmental image of the surroundings of the vehicle at a second time point after the first time point, the second environmental image being covered by a second fill area corresponding to the first fill area; mapping the image to the second filling area of ​​the second environment image according to the operating parameters to generate a surround view image; A method for providing the above.

2. cropping a third filled area image from the surround view image, the third filled area image corresponding to a third filled area, the second environment image being covered with the third filled area, the third filled area corresponding to an area under the vehicle in the second environment image and not covering the shadow area of ​​the vehicle in the second environment image; combining the third filled area image with the second environment image to generate a surround view image including the shadow region; The method of claim 1 further comprising:

3. In the direction of travel of the vehicle, the image cropping region is adjacent to the first filling range. The method of claim 1.

4. rendering a transparent vehicle model corresponding to the vehicle in an area corresponding to an underside of the vehicle in the surround-view image or the surround-view image including the shadow region; The method of any of claims 1 to 3, further comprising:

5. determining the first filling range and / or the second filling range according to a size of a shadow area around the vehicle and a size of the vehicle; The method of any of claims 1 to 3, further comprising:

6. A panoramic surround view device, comprising: one or more processors; one or more memories; The memory stores computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform a method according to any one of claims 1 to 5. Panoramic surround view device.

7. 1. A computer program product comprising: computer-readable instructions stored on a computer-readable storage medium, the computer-readable instructions, when executed by a processor, causing the processor to perform the method of any of claims 1 to 5; Computer program products.

8. A panoramic surround view system, comprising: an image acquisition module configured to determine a first image of an environment surrounding the vehicle at a first time point and a second image of an environment surrounding the vehicle at a second time point after the first time point; the first environmental image is covered with a first filling range, the first filling range covering an area corresponding to an underside of the vehicle in the first environmental image and covering a shadow area around the vehicle in the first environmental image, and the second environmental image is covered with a second filling range corresponding to the first filling range; an image reconstruction module configured to crop an image from an image crop region in the first environment image and map the image to the second fill region of the second environment image according to the operational parameters to generate a surround view image, wherein the image crop region is outside the first fill region of the first environment image; Panoramic surround view system.

9. The image reconstruction module further comprises: cropping a third filled area image from the surround view image, the third filled area image corresponding to a third filled area, the second environment image being covered with the third filled area, the third filled area corresponding to an area under the vehicle in the second environment image and not covering the shadow area of ​​the vehicle in the second environment image; combining the third filled area image with the second environment image to generate a surround view image including the shadow region; 9. The panoramic surround view system of claim 8, configured to:

10. 10. A vehicle equipped with the panoramic surround view device according to claim 6 or the panoramic surround view system according to claim 8 or 9.

Images