Method, device and system for generating surround view images, computer program product and automobile

Abstract

The present disclosure provides a method, apparatus, and system for generating a surround view image, as well as a computer program product and a vehicle. An image may be captured from a shadowless image capture area, and this image may be used to reconstruct an image that also covers a second fill area corresponding to a first fill area, i.e., an area corresponding to the vehicle floor of the vehicle. Since the image captured from the image capture area has no shadows, after an affine transformation is performed on the image, the reconstructed image covering the second fill area and including the vehicle underside image does not contain shadows. Accordingly, it is effectively prevented that the observation and judgment of information related to the vehicle underside by the driver of the vehicle is affected by the reconstructed vehicle underside image containing many shadows and distorted shadows.

Description

Technology Field

[0001] The present invention relates to a method for generating a surround view image, a panoramic surround view device, a panoramic surround view system, a computer program product, and an automobile, capable of generating a panoramic surround view image without blind spots under a vehicle. Background Technology

[0002] Currently, an increasing number of panoramic imaging systems are being used in the automotive sector, particularly in road vehicles. Panoramic imaging systems can provide complete surround-view images that reflect the vehicle's surrounding environment. Therefore, especially in scenes with complex obstacles around the vehicle, such as parking and congested road conditions, panoramic imaging systems intuitively provide the driver with information about the vehicle's surroundings, enabling effective avoidance of collisions between the vehicle and surrounding obstacles.

[0003] Currently, panoramic imaging systems can provide a perspective from the underside of the vehicle in addition to displaying the vehicle's surrounding environment. Consequently, drivers can obtain information regarding the vehicle's underside. In this field, affine transformations are typically performed on acquired images of the vehicle's surrounding environment based on driving parameters to reconstruct images of the vehicle's underside. However, if shadows are present around the vehicle, performing an affine transformation on images containing these shadows results in the reconstructed underside image containing multiple shadows and distorted shadows. Consequently, the driver's observation and judgment regarding information related to the vehicle's underside are affected.

[0004] Therefore, a panoramic imaging system is required that can provide images of the vehicle's undercarriage without being affected by shadows.

[0005] Accordingly, the object of the present invention is to provide a method, apparatus, and system for generating a surround view image, a computer program product, and a vehicle, which can eliminate the effect of shadows on the vehicle's undercarriage by providing a surround view image of a shadow-free vehicle undercarriage image.

[0006] In the sense of the present disclosure, a motor vehicle may be any vehicle. Preferably, the motor vehicle is, for example, a road vehicle or a train. Particularly preferably, the motor vehicle is a road vehicle such as a passenger car or a truck.

[0007] A first aspect of the present disclosure relates to a method for generating a surround view image, comprising: determining a first environment image around a vehicle at a first time point—wherein the first environment image is covered by a first fill range, wherein the first fill range covers an area corresponding to the lower part of the vehicle in the first environment image and covers a shadow area around the vehicle in the first environment image—; intercepting an image from an image intercept area of ​​the first environment image—wherein the image intercept area is outside the first fill range of the first environment image—; obtaining motion parameters of the vehicle; determining a second environment image around the vehicle at a second time point after the first time point—wherein the second environment image is covered by a second fill range corresponding to the first fill range—and mapping an image to a second fill range of the second environment image according to the motion parameters to generate a surround view image.

[0008] According to a more detailed embodiment of the method of the present disclosure, the method further comprises intercepting a third fill range image corresponding to a third fill range from a surround view image—wherein the second environment image is covered by the third fill range, and the third fill range corresponds to an area corresponding to the lower part of the vehicle in the second environment image and does not cover the shadow area of ​​the vehicle in the second environment image—and stitching the third fill range image to the second environment image to generate a surround view image including the shadow area.

[0009] According to a more detailed embodiment of the method of the present disclosure, in the direction of movement of the vehicle, the image intercept area is adjacent to the first fill area.

[0010] According to a more detailed embodiment of the method of the present disclosure, the method further comprises rendering a transparent vehicle model corresponding to a vehicle in an area corresponding to the lower part of a vehicle in a surround view image or a surround view image including a shadow area.

[0011] According to a more detailed embodiment of the method of the present disclosure, the method further comprises determining a first fill range and / or a second fill range based on the size of the shadow area around the vehicle and the size of the vehicle.

[0012] A second aspect of the present disclosure relates to a panoramic surround view device comprising one or more processors; and one or more memories, wherein computer-readable instructions are stored in the memories, and when the computer-readable instructions are executed by the one or more processors, the one or more processors perform the method according to the present disclosure described above.

[0013] A third aspect of the present disclosure relates to a computer program product comprising computer-readable instructions, wherein when the computer-readable instructions are executed by a processor, the processor causes the processor to perform the method according to the present disclosure described above.

[0014] A fourth aspect of the present disclosure relates to a panoramic surround view system:

[0015] An image acquisition module configured to determine a first environment image around a vehicle at a first time point and to determine a second environment image around a vehicle at a second time point after the first time point - where, the first environment image is covered by a first fill range, the first fill range covers the area corresponding to the lower part of the vehicle in the first environment image and covers the shadow area around the vehicle in the first environment image, and the second environment image is covered by a second fill range corresponding to the first fill range - and;

[0016] It includes an image reconstruction module configured to generate a surround view image by intercepting an image from an image intercept area of ​​a first environment image and mapping the image to a second fill range of a second environment image according to operation parameters—wherein the image intercept area is outside the first fill range of the first environment image.

[0017] According to a more detailed embodiment of the panoramic surround view system of the present disclosure, the image reconstruction module is further configured to: intercept a third fill range image corresponding to a third fill range from a surround view image, wherein the second environment image is covered by the third fill range, and the third fill range corresponds to an area corresponding to the lower part of the vehicle in the second environment image and does not cover the shadow area of ​​the vehicle in the second environment image; and to stitch the third fill range image to the second environment image to generate a surround view image including the shadow area.

[0018] A fifth aspect of the present disclosure relates to a vehicle. The vehicle is equipped with the panoramic surround view device or the panoramic surround view system described above.

[0019] By using the method, apparatus, and system for generating a surround view image according to the present disclosure, a computer program product, and a vehicle, an image can be intercepted from a shadowless image intercept area, and said image can be used to reconstruct an image covering a second fill area corresponding to a first fill area, said second fill area also covering an area corresponding to the lower part of the vehicle. Since the image intercepted from the image intercept area has no shadows, the image of the second fill area containing the lower image reconstructed after affine transformation also does not contain shadows. Accordingly, this effectively prevents the reconstructed lower image of the vehicle from containing a number of shadows and distorted shadows that affect the driver's observation and judgment regarding information related to the lower part of the vehicle. Brief explanation of the drawing

[0020] The above and other objects, features, and advantages of the embodiments of the present disclosure will become more apparent from the detailed description of the embodiments of the present disclosure with reference to the drawings. The drawings, which are used to provide a further understanding of the embodiments of the present disclosure and constitute part of the specification, are used to describe the present disclosure together with the embodiments of the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally indicate the same components or steps. In the drawings: FIG. 1a illustrates an exemplary car surround view image in the first frame; FIG. 1b illustrates an exemplary car surround view image in the second frame; FIG. 1c exemplarily illustrates a car surround view image in the third frame; FIG. 1d exemplarily illustrates a car surround view image in a subsequent frame; FIG. 2 schematically illustrates a car surround view image in which the vehicle underside image includes a plurality of shadows; FIG. 3 illustrates a flowchart of a method for generating a surround view image according to an embodiment of the present disclosure; FIG. 4a schematically illustrates an automobile surround view image generated corresponding to a first time point (moment) according to an embodiment of the present disclosure; FIG. 4b schematically illustrates an automobile surround view image generated corresponding to a second time point according to an embodiment of the present disclosure; FIG. 4c schematically illustrates an automobile surround view image generated corresponding to a third time point according to an embodiment of the present disclosure; FIG. 4d schematically illustrates a surround view image including a full fill area image and a vehicle underside area image; FIG. 5a illustrates a first environment image of a vehicle determined by a method for generating a surround view image according to an embodiment of the present disclosure; FIG. 5b illustrates a surround view image including a vehicle undercarriage image generated by a method for generating a surround view image according to an embodiment of the present disclosure; FIG. 6 illustrates a flowchart of a method for generating a surround view image according to another embodiment of the present disclosure; FIG. 7 illustrates an automobile surround view image generated corresponding to a second time point according to another embodiment of the present disclosure; FIG. 8 illustrates a surround view image generated according to another embodiment of the present disclosure; FIG. 9 illustrates a panoramic surround view device according to an embodiment of the present disclosure. FIG. 10 illustrates a computer program product according to an embodiment of the present disclosure; FIG. 11 illustrates a panoramic surround view system according to an embodiment of the present disclosure; FIG. 12 illustrates a vehicle according to an embodiment of the present disclosure. Specific details for implementing the invention

[0021] The technical solution in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. By way, the described embodiments are not all embodiments of the present disclosure, but only some embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative effort are included within the scope of protection of the present disclosure.

[0022] Additionally, in this specification and drawings, the terms “first / second” are used merely to distinguish similar objects and do not indicate a specific arrangement of objects. Where permitted, “first / second” may be interchanged with a specific order or sequence, which is understood to be so that embodiments of the disclosure described herein may be implemented in a different order than those shown or described herein.

[0023] Additionally, in this specification and drawings, where flowcharts are used to describe steps of a method according to an embodiment of the present disclosure, it should be understood that preceding or subsequent steps are not necessarily performed in an exact sequence. Conversely, unless explicitly defined in an embodiment of the present disclosure, various steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more steps may be removed from these processes.

[0024] FIGS. 1a through 1d schematically illustrate a car surround view image including an image showing the underside of the car. In the case of a low solar elevation angle, such as in winter and evening, the car is usually surrounded by shadow. FIG. 1a exemplarily illustrates a car surround view image (101) in the first frame when the sun is shining on the car (110) directly behind the car (110). Thus, the first shadow (121) of the car is projected directly in front of the car (110). The first shadow (121) is schematically illustrated here as a triangle.

[0025] At the point in time of the first frame, a conventional panoramic imaging system having a vehicle underbody image display function typically first acquires an image (141) in an image intercept area (130) located in front of the vehicle (110). Here, the image intercept area (130) is schematically framed by a dotted line, and the image (141) is located within the range of the dotted line frame. As illustrated in FIG. 1a, since the image intercept area (130) includes the first shadow (121) of the vehicle, the image (141) acquired at this point also includes the first shadow (121) of the vehicle.

[0026] After that, depending on the vehicle driving parameters such as the current speed, current position, and current steering of the vehicle (110), the image (141) is mapped to a corresponding position (152) in the lower region of the vehicle by affine transformation and is displayed in the vehicle surround view image (102) in the second frame to be displayed. FIG. 1b illustrates the vehicle surround view image (102) in the second frame as an example.

[0027] Since the mapped image (141) includes the first shadow (121) of the vehicle in the first frame, the image displayed at the corresponding location (152) of the lower area of ​​the vehicle (110) after mapping also includes the first shadow (121). Therefore, this causes the reconstructed vehicle lower image to include the first shadow (121), thereby hindering the use of the panoramic imaging system by the vehicle driver.

[0028] Likewise, at the point in time of the second frame, a conventional panoramic imaging system having a vehicle underbody image display function continues to acquire an image (142) in an image intercept area (130) located in front of the vehicle (110). Since the sun elevation angle is still small at the point in time of the second frame, the second shadow (122) of the vehicle continues to be projected in front of the vehicle (110). Here, the second shadow (122) is also schematically depicted as a triangle. The second shadow (122) is contained within the image intercept area (130). Accordingly, likewise, the second shadow (122) of the vehicle is also contained in the image (142) acquired at this point in time.

[0029] Then, similarly, the image (142) is mapped to a corresponding location (152) of the lower region of the vehicle by affine transformation according to the vehicle driving parameters and is displayed as a vehicle surround view image (103) in the third frame to be displayed. Also, similarly, the image (141) is mapped to a corresponding location (153) of the lower region of the vehicle by affine transformation according to the vehicle driving parameters and is displayed as a vehicle surround view image (103) in the third frame to be displayed. FIG. 1c illustrates an exemplary vehicle surround view image (103) in the third frame.

[0030] Since the image (142) acquired at the point of view of the second frame to reconstruct the vehicle underside image includes the second shadow (122) and the image (141) acquired at the point of view of the first frame to reconstruct the vehicle underside image includes the first shadow (121), the images displayed at the corresponding locations (152 and 153) of the lower area of ​​the vehicle (110) after mapping also include the second shadow (122) and the first shadow (121), thereby hindering the use of the panoramic imaging system by the vehicle driver.

[0031] Likewise, at the point in subsequent frames, shadows around the vehicle (110) continue to be mapped to corresponding locations in the vehicle's lower region and displayed in the vehicle surround view image (10N) in the subsequent frames. FIG. 1d illustrates the vehicle surround view image (10N) in subsequent frames as an example. As illustrated in FIG. 1d, the image of the vehicle's lower region includes a number of shadows (121, ..., and 12N-1). These shadows enrich the image of the vehicle's lower region (110), affecting the driver's observation of the vehicle's lower region and vehicle under information, such as whether there are obstacles, potholes, etc., under the vehicle, and consequently, the panoramic imaging system can no longer provide a valid reference to the driver.

[0032] A vehicle surround view image containing multiple shadows of the vehicle underside is also shown in FIG. 2. As shown in FIG. 2, shadows (121, ..., and 12N-1) of the vehicle underside can seriously affect the driver's observation of the vehicle's floor when using a panoramic imaging system.

[0033] To this end, the present disclosure proposes a method for generating a surround view image. In this method, the image is intercepted from a shadow-free image interception area, and the image is used to reconstruct an image of an area corresponding to the underside of a vehicle at a subsequent point in time, thereby effectively preventing the reconstructed image of the underside of the vehicle from containing multiple shadows.

[0034] FIG. 3 illustrates a flowchart of a method (300) for generating a surround view image according to an embodiment of the present disclosure. The method (300) for generating a surround view image according to an embodiment of the present disclosure may be used in a vehicle equipped with a panoramic surround view system, such as, for example, a passenger car or a truck, but the embodiment of the present disclosure is not limited thereto. The method (300) for generating a surround view image according to an embodiment of the present disclosure may also be used in any other vehicle that needs to generate a surround view image of its surroundings.

[0035] Additionally, FIGS. 4a and 4b further illustrate vehicle surround view images (401 and 402) generated by performing the method (300). Hereinafter, for clarity, a method (300) for generating a surround view image according to an embodiment of the present disclosure will be described with reference to FIGS. 3, 4a, and 4b in combination.

[0036] FIG. 4a illustrates a vehicle surround view image (401) generated corresponding to a first time point. As illustrated in FIG. 3 and FIG. 4a, first, in step S310, a first environment image (421) around the vehicle (410) is determined at a first time point, for example, a time point of the first frame. Here, for example, to acquire different orientation environments of the vehicle (410), a plurality of camera devices installed on the vehicle (410) can be used to acquire images reflecting multiple different orientation environments around the vehicle (410), such as four-directional video images. Then, by performing image stitching on images of multiple different orientation environments and additionally performing perspective transformation, a vehicle surround view image (400) at a top-view perspective as illustrated in FIG. 4a is obtained, and a first environment image (421) around the vehicle (410) is determined. Here, as previously described, since the environment image is formed by stitching images captured by a plurality of camera devices placed in the vehicle, the area below the vehicle cannot be represented in the environment image. Subsequently, it is necessary to reconstruct the image of the area below the vehicle. Here, it should be noted that the present disclosure is not intended to limit the method for generating the first environment image (421) and the specific viewpoint (top view, side-rear oblique view, etc.) for displaying the vehicle surround view image (400).

[0037] As illustrated in FIG. 4a, the first environment image (421) is covered by a first fill area (431). In the actual first environment image (421), since it is covered by the first fill area (431), the area of ​​the first environment image (421) covered by the first fill area (431) can no longer be displayed. However, here, to clearly illustrate the area covered by the first fill area (431), in FIG. 4a, the first fill area (431) is schematically illustrated as a transparent rectangular area having a triangular grid. As illustrated in FIG. 4, in addition to completely covering the image of the lower area of ​​the car (410), the first fill area (431) further covers the shadow area (441) around the car in the first environment image (421). Here, similarly, the shadow area (441) is schematically illustrated as a triangle. Therefore, since the shadow area (441) is covered, the first environment image (421) after covering cannot display the area of ​​the first environment image (421) covered by the first fill range (431), i.e., the shadow area (441).

[0038] Here, the first fill range (and the second fill range to be mentioned in the subsequent step S340) may be determined according to the size of the shadow area around the vehicle and the size of the vehicle, so that the fill range may completely cover the shadow area around the vehicle. For example, the size of the shadow area around the vehicle may be determined statistically in advance, and the fill range is determined in advance, and a fill range having this size is always used to cover the environment image in the subsequent method for generating a surround view image. The size of the fill range may also be determined and dynamically adjusted in the method for generating a surround view image by determining the shadow area in the environment image. Accordingly, the method (300) for generating a surround view image according to an embodiment of the present disclosure may further include, in step S310', determining the first fill range (431) and / or the second fill range (432) to be described later according to the size of the shadow area around the vehicle and the size of the vehicle.

[0039] Additionally, in step S320, an image (461) is cut out from an image intercept area (450) within the first environment image (421). Here, as illustrated in FIG. 4a, in the first environment image (421), the image intercept area (450) is located outside the first fill area (431). Specifically, the location of the cut image (461) in the first environment image (421), i.e., the location of the image intercept area (450), can be determined according to driving parameters of the vehicle (410), such as driving direction, driving speed, steering angle, etc. Preferably, in the driving direction of the vehicle, the image intercept area (450) is adjacent to the first fill area (431). That is, for example, when the vehicle (410) is moving forward, the image intercept area (450) may be located immediately in front of the first fill range (431), as illustrated in FIG. 4. When the vehicle (410) is moving backward, one might think that the image intercept area (450) should be located immediately behind the first fill range (431). Here, the present disclosure does not limit the specific location of the image intercept area (450) in the first environment image (421). Given the difficulty of performing an affine transformation, having the image intercept area (450) adjacent to the first fill range (431) in the driving direction is an optional implementation, as the affine transformation can be completed with a smaller distance / time difference, thereby avoiding inaccurate mapping results caused by the unpredictable movement of the vehicle within a longer time / distance difference.

[0040] As illustrated in FIG. 4a, the cropped image (461) is schematically depicted as a rectangle with a diagonal inside. As previously mentioned, since the shadow area (441) is covered by the first fill area (431) and the image intercept area (450) is located outside the first fill area (431), the cropped image (461) does not include the shadow area (441).

[0041] Additionally, at step S330, motion parameters of the vehicle (410), such as the driving direction, driving speed, and steering angle of the vehicle (410), are obtained. Here, for example, the motion parameters may be determined by a corresponding sensor device installed in the vehicle (410), or the motion parameters may be provided to the vehicle (410) by a navigation positioning system (GPS, Galileo, BDS, or other navigation system). Here, the present disclosure also does not limit the specific method for obtaining the motion parameters. Subsequently, an affine transformation is performed on the image (461) according to the obtained motion parameters to fill the lower region of the vehicle (410).

[0042] Then, at a second time point after the first time point, that is, at the time of a subsequent frame after the first frame, for example, at the time of the second frame, a second environment image (422) around the vehicle (410) is determined in step S340. FIG. 4b illustrates a vehicle surround view image (402) generated corresponding to the second time point. Here, for example, the second environment image (422) may be determined in the same way as the first environment image (421), which is not described again here for brevity.

[0043] As illustrated in FIG. 4b, corresponding to the first environment image (421), the second environment image (422) is similarly covered by a second fill range (432) corresponding to the first fill range (431). That is, the fill range of the first fill range (431) in the first environment image (421) is equivalent to the fill range of the second fill range (432) in the second environment image (422): the first fill range (431) and the second fill range (432) have the same fill range. The second fill range (432) not only completely covers the image of the lower area of ​​the car (410), but also covers the shadow area (442) around the car (410) in the second environment image (422). Because the shadow area (442) is covered, the second environment image (422) also cannot display the area covered by the second fill range (432) in the second environment image (422), i.e., the shadow area (442).

[0044] Additionally, at step S350, the image (461) cropped at the first time point is mapped to the second fill range (432) of the second environment image (422) according to the motion parameters acquired at step S330, and a surround view image (402) including an image of the lower area of ​​the vehicle is generated. Here, for example, the predicted position (472) within the second fill range (432) where the pixel points of the image (461) move to the second time point can be estimated according to the motion parameters, and accordingly, an affine transformation is performed for each pixel point of the image (461). The pixel points are displayed at the corresponding predicted position (472) to reconstruct the image within the second fill range (432).

[0045] Then, the reconstructed image of the second fill range (432) is stitched with the second environment image (422) to create a surround view image (402) that corresponds to the second time point and includes the reconstructed image of the second fill range (432), i.e., an image of the lower area of ​​the vehicle.

[0046] As described above, since the image (461) intercepted at the first point in time does not include a shadow area (441), the image within the second fill range (432) reconstructed using the image after the image (461) is mapped and transformed as shown in FIG. 4b does not include a shadow. Additionally, since the lower area of ​​the car is covered by the second fill range (432), the reconstructed image of the lower area of ​​the car as shown in FIG. 4b also does not include a shadow.

[0047] Additionally, in this embodiment, for the sake of brevity, FIGS. 4a and 4b schematically illustrate only a portion of the processing performed at the first and second time points in the method (300) for generating a surround view image according to an embodiment of the present disclosure. That is, as shown in FIG. 4b, at the second time point, only the image (461) intercepted at the first time point is mapped to a partial area (an area corresponding to the predicted location (472)) of the second fill range (432), and the image of the second fill range (432) of the partial area is reconstructed. However, it can be assumed that the method (300) for generating a surround view image according to an embodiment of the present disclosure is performed continuously and periodically so that a surround view image including the entire fill range image can be generated. That is, at a time point after the second time point, for example, at a third time point after the second time point, as shown in FIG. 4c, the image (463) within the image intercept area (450) is continuously acquired. Similarly, depending on the driving parameters of the vehicle, the image (462) intercepted at the second time point can be mapped to the predicted position (472) of the third fill range (433) by affine transformation, and the image (461) captured at the first time point can be mapped to the predicted position (473) of the third fill range (433). Here, image stitching is performed to generate a surround view image (402) corresponding to the third time point. Then, similarly, the method (300) for generating a surround view image according to an embodiment of the present disclosure is performed continuously and periodically so that the image of the fill range is completely reconstructed and a surround view image including the entire fill range image and the vehicle under-region image is generated as illustrated in FIG. 4d.

[0048] For clarity, with reference to FIG. 2, an actual surround view image generated by the method (300) according to an embodiment of the present disclosure is further illustrated. FIG. 5a illustrates a first environment image (421) around a vehicle (410) determined in step S310 of the method (300) for generating a surround view image according to an embodiment of the present disclosure. Here, since the environment image is formed by stitching images captured by a plurality of camera devices placed on the vehicle, the area below the vehicle cannot be represented in the actual environment image. The area (471) corresponding to the lower part of the vehicle in the environment image needs to be reconstructed in a subsequent step and is currently displayed in gray in FIG. 5a.

[0049] As illustrated in FIG. 5a, a first environment image (421) is covered by a first fill area (431). The first fill area (431) covers an area (471) corresponding to the underside of the vehicle in the first environment image (421). Additionally, corresponding to what is described in FIG. 4a, an image intercept area (450) is located immediately in front of the first fill area (431). At a subsequent point in time, the image intercepted in the image intercept area (450) is periodically used to reconstruct an image within the first fill area (431), so that a fill area image including an area corresponding to the underside of the vehicle can be reconstructed. Then, the reconstructed fill area image is stitched with an environment image corresponding to a point in time to generate a surround view image including an image of the underside of the vehicle corresponding to a point in time. FIG. 5b illustrates a surround view image including an image of the underside of the vehicle generated by a method (300) for generating a surround view image according to an embodiment of the present disclosure. As illustrated in FIG. 5b, the generated surround view image, particularly the area corresponding to the underside of the vehicle, does not contain shadows. Compared to FIG. 2, the method (300) for generating a surround view image according to an embodiment of the present disclosure effectively prevents a number of shadows and distorted shadows that affect the driver's observation and judgment of information related to the underside of the vehicle from being included in the reconstructed underside image of the vehicle.

[0050] Since the fill area to be reconstructed covers the shadows around the car, after reconstruction, the generated surround view image does not contain any shadow parts, as shown in FIGS. 4a to 4c and FIG. 5b. However, in order to match the generated surround view image with the actual image of the car's surrounding environment, it is necessary to display the shadows around the car in the generated surround view image.

[0051] A method (600) for generating a surround view image according to another embodiment of the present disclosure is intended to solve the above problem. FIG. 6 schematically illustrates a flowchart of a method (600) for generating a surround view image according to another embodiment of the present disclosure. FIG. 7 schematically illustrates a vehicle surround view image (402) generated corresponding to a second time point according to another embodiment of the present disclosure. FIG. 8 illustrates a surround view image generated according to another embodiment of the present disclosure. A method (600) according to another embodiment of the present disclosure will be described below with reference to FIG. 6 through FIG. 8. Herein, it should be noted that in the method (600) for generating a surround view image according to another embodiment of the present disclosure, a surround view image corresponding to any time point may be further processed. Those skilled in the art should understand that the time point targeted by the method (600) according to another embodiment of the present disclosure is not limited to the second time point schematically illustrated in FIG. 7.

[0052] As illustrated in FIG. 6, steps S610 to S640 of the method (600) for generating a surround view image according to another embodiment of the present disclosure correspond to steps S310 to S350 of the method described in FIG. 3, and are not described again here for brevity. According to another embodiment of the present disclosure, the method (600) further comprises, at step S660, intercepting a third fill range image (492) corresponding to a third fill range (433) from a generated surround view image, for example, a surround view image (402) generated corresponding to a second time point.

[0053] As illustrated on the left side of FIG. 7, in the surround view image (402) generated in step S650, the second environment image (422) is covered by a third fill area (433). The third fill area (433) corresponds to the area corresponding to the lower part of the vehicle in the second environment image (422) and does not cover the shadow area of ​​the vehicle in the second environment image, for example, the shadow area (442) corresponding to the second viewpoint. Here, for clarity, the third fill area (433) is schematically illustrated as a gray-filled rectangle. The third fill area (433) corresponds to the area corresponding to the lower part of the vehicle in the second environment image (422), which can be understood as the third fill area (433) covering the lower part of the vehicle (410). Preferably, the third fill area (433) covers the lower part of the vehicle (410) perfectly. Compared to FIG. 4b, FIG. 7 illustrates an embodiment in which the third fill area (433) perfectly covers the lower area of ​​the vehicle (410). However, it should be noted that the present disclosure does not limit the selection of a specific size and location of the third fill area (433). Those skilled in the art should understand that the third fill area (433) may be located in the area corresponding to the second fill area (432) in the generated surround view image (702) and only needs to be able to cover the lower area of ​​the vehicle (410) without covering the shadow area.

[0054] Additionally, according to another embodiment of the present disclosure, the method (600) further comprises, in step S670, stitching a third fill range image (492) to a second environment image (422) to generate a surround view image including a shadow area.

[0055] As illustrated in FIG. 7, the third fill range image (492) within the third fill range (433) is cropped and used to fill a second environment image (422) that does not cover any fill range, including a vehicle under area (471) that corresponds to a viewpoint (second viewpoint) and cannot be represented in the second environment image (422), as illustrated on the right side of FIG. 7, thereby generating a surround view image (402') (see FIG. 8) that includes a vehicle under area image. Since the second environment image (422) that is not covered by any fill range includes a shadow area (442), the filled image includes the shadow area (442) in addition to the vehicle under area image (492), so the generated surround view image including the shadow area is more realistic.

[0056] For clarity, referring to FIG. 8, a real surround view image (402') including a shadow area generated by the method (300) according to another embodiment of the present disclosure is further illustrated. Here, the surround view image (402') not only displays a vehicle under-region image (492), but also additionally displays a shadow area (442). Compared to the vehicle surround view image (10N) including a plurality of shadows shown in FIG. 2, the surround view image (402') generated by the method (300) according to another embodiment of the present disclosure not only does not include a plurality of shadows in the vehicle under-region, but also displays a shadow area that matches the actual environment around the vehicle, for example, in front of the vehicle, thereby making the generated surround view image (402') more realistic and further preventing the impact on the driver due to the difference between the surround view image generated by the panoramic imaging system and the reality.

[0057] Additionally, as illustrated in FIG. 8, in a method for generating a surround view image according to another embodiment of the present disclosure, a transparent vehicle model (411) corresponding to a vehicle is rendered in an area corresponding to the lower part of the vehicle in the surround view image or a surround view image (402') including a shadow area. Thus, when the lower area no longer contains a number of shadows, on the one hand, the driver can clearly observe the lower part of the vehicle through the transparent vehicle model (411); on the other hand, the area corresponding to the lower part of the vehicle can be defined in the surround view image by the transparent vehicle model (411), making it convenient for the driver to determine information about the lower part of the vehicle.

[0058] In summary, the method for generating a surround view image according to the present disclosure can effectively prevent the reconstructed vehicle underbody image from containing a number of shadows, provides a shadow-free vehicle underbody view, and effectively prevents shadows of the surround view image from affecting the vehicle driver.

[0059] The present disclosure further provides a panoramic surround view device (900). FIG. 9 schematically illustrates a panoramic surround view device (900) according to an embodiment of the present disclosure. Herein, the panoramic surround view device (900) comprises one or more processors (910); and one or more memories (920), wherein the memory has computer-readable instructions stored therein, and when the computer-readable instructions are executed by one or more processors (910), the one or more processors (910) perform the method of generating a surround view image as described above in relation to FIGS. 3 through 8. For a specific description of various features of the panoramic surround view device (900) and their advantages, refer to the description of the corresponding method described above, and for brevity, will not be described again here.

[0060] The present disclosure further provides a computer program product (1000). FIG. 10 schematically illustrates a computer program product (1000) according to an embodiment of the present disclosure. Herein, the computer program product (1000) comprises a computer-readable instruction (1020) stored in a computer-readable storage medium (1010), and when the computer-readable instruction (1020) is executed by a processor, the processor performs the method of generating a surround view image as described above with respect to FIGS. 3 through 8. Estimation method. The computer-readable storage medium includes, for example, volatile memory and / or non-volatile memory, but is not limited thereto. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, optical disk, etc. For a detailed description of the various features and advantages of the computer program product (1000), refer to the description of the corresponding method described above, and for brevity, it will not be described again here.

[0061] The present disclosure further provides a panoramic surround view system (1100). FIG. 11 schematically illustrates a panoramic surround view system (1100) according to an embodiment of the present disclosure.

[0062] Here, the panoramic surround view system (1100) includes an image acquisition module (1110) configured to determine a first environment image around a vehicle at a first time point and to determine a second environment image around a vehicle at a second time point after the first time point, wherein the first environment image is covered by a first fill range, wherein the first fill range covers an area corresponding to the lower part of the vehicle in the first environment image and covers a shadow area around the vehicle in the first environment image, and wherein the second environment image is covered by a second fill range corresponding to the first fill range.

[0063] Additionally, the panoramic surround view system (1100) further includes an image reconstruction module (1120) configured to generate a surround view image by cutting an image from an image intercept area of ​​a first environment image, performing an affine transformation on pixel points of the image according to operation parameters, and mapping the image to a second fill area of ​​a second environment image, wherein the image intercept area is outside the first fill area of ​​the first environment image.

[0064] Here, regarding the specific description of the various features and advantages of the panoramic surround view system (1100), refer to the description of the corresponding method with reference to FIGS. 3 to 5b described above, and for brevity, it will not be described again here.

[0065] Additionally, in another embodiment of the panoramic surround view system (1100) according to the present disclosure, the image reconstruction module (1120) is further configured to intercept a third fill range image corresponding to a third fill range from a surround view image—wherein the second environment image is covered by the third fill range, and the third fill range corresponds to an area corresponding to the lower part of the vehicle in the second environment image and does not cover the shadow area of ​​the vehicle in the second environment image—and to stitch the third fill range image to the second environment image to generate a surround view image including the shadow area.

[0066] Here, regarding the specific description of the various features and advantages of the panoramic surround view system (1100) in the above embodiment, refer to the description of the corresponding method with reference to FIGS. 6 to 8 described above, and for brevity, it will not be described again here.

[0067] Additionally, the present disclosure further provides a vehicle (1200). FIG. 12 schematically illustrates a vehicle (1200) according to an embodiment of the present disclosure. As illustrated in FIG. 12, the vehicle is equipped with a panoramic surround view device (900) described above with respect to FIG. 9 or a panoramic surround view system (1100) described above with respect to FIG. 11. For a specific description of the various features of the vehicle (1200) and their advantages, refer to the description of the panoramic surround view device (900) or the panoramic surround view system (1100) above, which is not described again here for brevity.

[0068] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as generally understood by a person skilled in the art. Furthermore, it should be understood that terms such as those generally defined in advance should be interpreted as having the same meaning in the context of the relevant technology, rather than being interpreted in an idealized or highly formalized sense, unless explicitly defined so in this specification.

[0069] Although the present disclosure has been described in detail above, it will be apparent to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure may be implemented by modification and change without departing from the essence and scope of the present disclosure as determined by the claims. Accordingly, the description in this specification is for illustrative purposes only and is not meant to be restrictive to the present disclosure.

Claims

Claim 1 A method for generating a surround view image comprises: determining a first environment image around a vehicle at a first time point — said first environment image is covered by a first fill range, said first fill range covers an area corresponding to the lower part of the vehicle in said first environment image and covers a shadow area around the vehicle in said first environment image —; intercepting an image from an image interception region of said first environment image — said image interception region is outside the first fill range of said first environment image and, in the direction of movement of said vehicle, said image interception region is adjacent to said first fill range —; acquiring motion parameters of said vehicle; and determining a second environment image around the vehicle at a second time point after said first time point — said second environment image is covered by a second fill range corresponding to the first fill range, said second fill range corresponds to the lower part of said vehicle in said second environment image A method for generating a surround view image, comprising covering an area and covering a shadow area around the vehicle in the second environment image; and mapping the image intercepted from the image intercept area of ​​the first environment image to the second fill range of the second environment image according to the operation parameter to generate a surround view image. Claim 2 A method for generating a surround view image according to claim 1, further comprising: intercepting a third fill range image corresponding to a third fill range from the surround view image — wherein the second environment image is covered by the third fill range, and the third fill range corresponds to an area corresponding to the lower part of the vehicle in the second environment image and does not cover the shadow area of ​​the vehicle in the second environment image — and stitching the third fill range image to the second environment image to generate a surround view image including the shadow area. Claim 3 delete Claim 4 A method for generating a surround view image according to claim 1, further comprising rendering a transparent vehicle model corresponding to the vehicle in an area corresponding to the lower part of the vehicle in the surround view image or the surround view image including the shadow area. Claim 5 A method for generating a surround view image according to claim 1, further comprising determining at least one of the first fill range and the second fill range according to the size of the shadow area around the vehicle and the size of the vehicle. Claim 6 A panoramic surround view device comprising: one or more processors; and one or more memories, wherein the memory has computer-readable instructions stored therein, and when the computer-readable instructions are executed by the one or more processors, the one or more processors perform the method described in claim 1. Claim 7 A computer program comprising computer-readable instructions stored in a computer-readable storage medium, wherein, when executed by a processor, the computer-readable instructions cause the processor to perform the method described in claim 1. Claim 8 In a panoramic surround view system, an image acquisition module configured to determine a first environmental image around a vehicle at a first time point and to determine a second environmental image around the vehicle at a second time point after the first time point — wherein the first environmental image is covered by a first fill range, the first fill range covers an area corresponding to the lower part of the vehicle in the first environmental image and covers a shadow area around the vehicle in the first environmental image, and the second environmental image is covered by a second fill range corresponding to the first fill range, the second fill range covers an area corresponding to the lower part of the vehicle in the second environmental image and covers a shadow area around the vehicle in the second environmental image — and an image reconstruction module configured to intercept an image from an image intercept area of ​​the first environmental image and to generate a surround view image by mapping the image intercepted from the image intercept area of ​​the first environmental image to a second fill range of the second environmental image according to operation parameters. A panoramic surround view system comprising: ― The image intercept area is outside the first fill range of the first environment image, and in the direction of movement of the vehicle, the image intercept area is adjacent to the first fill range. Claim 9 A panoramic surround view system according to claim 8, wherein the image reconstruction module is further configured to intercept a third fill range image corresponding to a third fill range from a surround view image ― the second environment image is covered by the third fill range, and the third fill range corresponds to an area corresponding to the lower part of the vehicle in the second environment image and does not cover a shadow area of ​​the vehicle in the second environment image ― and to stitch the third fill range image to the second environment image in order to generate a surround view image including a shadow area. Claim 10 An automobile having a panoramic surround view device as described in claim 6 or a panoramic surround view system as described in claim 8 or 9.

Images