Method, device and system for generating surround-view image, and computer program product and motor vehicle

Abstract

Provided in the present disclosure are a method, device and system for generating a surround-view image, and a computer program product and a motor vehicle. An image can be captured from an image capture region where there is no shadow, and the image can be used to reconstruct an image which covers a second filling range corresponding to a first filling range, that is, also covering a region corresponding to the vehicle bottom of a motor vehicle. Since there is no shadow in an image which is captured from an image capture region, after affine transformation is performed on the image, a reconstructed image, which covers the second filling region and comprises an under-vehicle image, does not comprise a shadow. Therefore, the observation and determination of under-vehicle-related information by a driver of a motor vehicle being affected due to a reconstructed under-vehicle image of the vehicle containing a lot of shadows and distorted shadows is effectively prevented.

Description

Method, device, system, computer program product, and motor vehicle for generating surround view images Technical Field

[0001] The present invention relates to a method for generating a surround view image, a panoramic surround view device, a panoramic surround view system, a computer program product and a motor vehicle, which can generate a panoramic surround view image without a blind spot under the vehicle. Background Art

[0002] Currently, panoramic imaging systems are increasingly being used in the automotive sector, particularly in automobiles. These systems provide a comprehensive surround view of the vehicle's surroundings. This intuitively provides drivers with information about the vehicle's surroundings, particularly in scenarios with complex obstacles, such as parking and congested roads, effectively preventing collisions between the vehicle and surrounding obstacles.

[0003] Currently, panoramic imaging systems not only display the vehicle's surroundings but also provide a view from underneath the vehicle. This allows the driver to obtain information about the vehicle's underbody. In the field, reconstructing an image of the vehicle's underbody is typically done by performing an affine transformation on the captured image of the vehicle's surroundings based on the vehicle's driving parameters. However, if shadows are present around the vehicle, performing an affine transformation on the image containing these shadows will result in the reconstructed image of the vehicle's underbody containing a large number of shadows and distorted shadows. This can affect the driver's ability to observe and interpret information about the vehicle's underbody.

[0004] Therefore, there is a need for a panoramic imaging system that can provide an image of the vehicle bottom without being affected by shadows.

[0005] Summary of the Invention

[0006] Therefore, the object of the present invention is to provide a method, device, system, computer program product and motor vehicle for generating a surround view image, which can provide a surround view image without a shadow under the vehicle, thereby eliminating the influence of the shadow under the vehicle on the motor vehicle driver.

[0007] In the sense of the present disclosure, a motor vehicle can be any vehicle. Preferred motor vehicles are, for example, cars, trains. Particularly preferred are motor vehicles, such as passenger cars or trucks.

[0008] A first aspect of the present disclosure relates to a method for generating a surround view image, comprising: determining, at a first moment, a first environment image around a motor vehicle, wherein the first environment image is covered by a first filling range, wherein the first filling range covers an area corresponding to the bottom of the motor vehicle in the first environment image and covers a shadow area around the motor vehicle in the first environment image; capturing an image from an image capture area in the first environment image, wherein the image capture area is outside the first filling range of the first environment image; collecting operating parameters of the motor vehicle; determining, at a second moment after the first moment, a second environment image around the motor vehicle, wherein the second environment image is covered by a second filling range corresponding to the first filling range; and mapping the image to the second filling range of the second environment image according to the operating parameters to generate a surround view image.

[0009] According to a more detailed embodiment of the method disclosed herein, the method further includes: cutting out a third filling range image corresponding to a third filling range from the surround view image, wherein the second environmental image is covered by the third filling range, wherein the third filling range corresponds to an area corresponding to the bottom of the motor vehicle in the second environmental image and does not cover a shadow area of ​​the motor vehicle in the second environmental image; and splicing the third filling range image into the second environmental image to generate a surround view image including a shadow area.

[0010] According to a more detailed embodiment of the method of the present disclosure, in the driving direction of the motor vehicle, the image capture area is adjacent to the first filling range.

[0011] According to a more detailed embodiment of the method disclosed herein, the method further includes: rendering a transparent vehicle model corresponding to the motor vehicle in the area corresponding to the bottom of the motor vehicle in the surround view image or the surround view image including the shadow area.

[0012] According to a more detailed embodiment of the method of the present disclosure, the method further includes: determining the first filling range and / or the second filling range according to the size of the shadow area around the motor vehicle and the size of the motor vehicle.

[0013] A second aspect of the present disclosure relates to a panoramic surround view device, comprising: one or more processors; and one or more memories, wherein the memories store computer-readable instructions, which, when executed by the one or more processors, enable the one or more processors to execute the method according to the present disclosure as described above.

[0014] A third aspect of the present disclosure relates to a computer program product, comprising computer-readable instructions, which, when executed by a processor, cause the processor to perform the method according to the present disclosure as described above.

[0015] A fourth aspect of the present disclosure relates to a panoramic surround view system, comprising:

[0016] an image acquisition module configured to determine a first environmental image around a motor vehicle at a first moment, and to determine a second environmental image around the motor vehicle at a second moment after the first moment, wherein the first environmental image is covered by a first filling range, wherein the first filling range covers an area corresponding to the bottom of the motor vehicle in the first environmental image and covers a shadow area around the motor vehicle in the first environmental image, and wherein the second environmental image is covered by a second filling range corresponding to the first filling range;

[0017] An image reconstruction module is configured to capture an image from an image capture area in the first environmental image and map the image to a second filling range of the second environmental image according to the operating parameters to generate a surround view image, wherein the image capture area is located outside the first filling range of the first environmental image.

[0018] According to a more detailed embodiment of the panoramic surround view system disclosed herein, the image reconstruction module is further configured to: cut out a third filling range image corresponding to a third filling range from the surround view image, wherein the second environment image is covered by the third filling range, wherein the third filling range corresponds to an area corresponding to the bottom of the motor vehicle in the second environment image and does not cover a shadow area of ​​the motor vehicle in the second environment image; and splice the third filling range image into the second environment image to generate a surround view image including a shadow area.

[0019] A fifth aspect of the present disclosure relates to a motor vehicle equipped with the panoramic view device or the panoramic view system as described above.

[0020] Using the method, apparatus, system, computer program product, and motor vehicle for generating surround view images disclosed herein, it is possible to capture an image from a shadow-free image capture area and use this image to reconstruct an image covering a second fill range corresponding to the first fill range, i.e., the area under the motor vehicle. Because the image captured from the image capture area is shadow-free, the image of the second fill range reconstructed after affine transformation, including the vehicle underbody image, also contains no shadows. This effectively prevents the reconstructed vehicle underbody image from containing a large number of shadows and distorted shadows, which could affect the driver's observation and judgment of information related to the vehicle underbody. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other purposes, features and advantages of the embodiments of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps. In the drawings:

[0022] FIG1A exemplarily shows a first frame of a motor vehicle surround view image;

[0023] FIG1B exemplarily shows a second frame of a motor vehicle surround view image;

[0024] FIG1C exemplarily shows a third frame of a motor vehicle surround view image;

[0025] FIG1D exemplarily shows a subsequent frame of a motor vehicle surround view image;

[0026] FIG2 schematically shows a surround view image of a motor vehicle in which the bottom image contains a large amount of shadows;

[0027] FIG3 shows a flow chart of a method for generating a surround view image according to an embodiment of the present disclosure;

[0028] FIG4A schematically shows a generated motor vehicle surround view image corresponding to a first moment according to an embodiment of the present disclosure;

[0029] FIG4B schematically shows a generated motor vehicle surround view image corresponding to a second moment according to an embodiment of the present disclosure;

[0030] FIG4C schematically shows a generated motor vehicle surround view image corresponding to a third moment according to an embodiment of the present disclosure;

[0031] FIG4D schematically shows a surround view image including a complete fill range image and an image of the vehicle bottom area;

[0032] FIG5A shows a first environment image around a motor vehicle determined by a method for generating a surround view image according to an embodiment of the present disclosure;

[0033] FIG5B shows a surround view image including a vehicle bottom image generated by the method for generating a surround view image according to an embodiment of the present disclosure;

[0034] FIG6 shows a flowchart of a method for generating a surround view image according to another embodiment of the present disclosure;

[0035] FIG7 shows a generated motor vehicle surround view image corresponding to a second moment according to another embodiment of the present disclosure;

[0036] FIG8 shows a surround view image generated according to another embodiment of the present disclosure;

[0037] FIG9 shows a panoramic surround view device according to an embodiment of the present disclosure;

[0038] FIG10 shows a computer program product according to an embodiment of the present disclosure;

[0039] FIG11 shows a panoramic surround view system according to an embodiment of the present disclosure;

[0040] FIG. 12 illustrates a motor vehicle according to an embodiment of the disclosure. DETAILED DESCRIPTION

[0041] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0042] In this specification and the drawings, the terms "first\second" are only used to distinguish similar objects and do not represent a specific order for the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0043] In addition, in this specification and the accompanying drawings, if flowcharts are used to illustrate the steps of the methods according to the embodiments of the present disclosure, it should be understood that the preceding or following steps are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, unless otherwise specified in the embodiments of the present disclosure. Furthermore, other operations may be added to these processes, or one or more steps may be removed from these processes.

[0044] Figures 1A to 1D schematically illustrate a motor vehicle surround view image including an image of the underside of the motor vehicle. In conditions such as winter and in the evening when the sun's altitude is low, a motor vehicle is often surrounded by shadows. Figure 1A exemplarily illustrates a first frame of a motor vehicle surround view image 101, where the sun shines directly behind motor vehicle 110. This creates a first shadow 121 of the motor vehicle directly in front of motor vehicle 110. This first shadow 121 is schematically illustrated as a triangle.

[0045] At the first frame, a conventional panoramic imaging system with underbody image display typically captures image 141 within image capture area 130 in front of motor vehicle 110. Image capture area 130 is schematically outlined with dashed lines, and image 141 is located within this dashed frame. As shown in FIG1A , since image capture area 130 includes the first shadow 121 of the motor vehicle, image 141 captured at this moment also includes the first shadow 121 of the motor vehicle.

[0046] Thereafter, based on motor vehicle driving parameters, such as the current speed, current position, and current direction of the motor vehicle 110, the image 141 is mapped to a corresponding position 152 of the underbody region of the motor vehicle via affine transformation and displayed in the second frame of the motor vehicle surround view image 102 to be displayed. FIG1B exemplarily illustrates the second frame of the motor vehicle surround view image 102.

[0047] Because the mapped image 141 includes the first shadow 121 of the motor vehicle in the first frame, the image displayed at the corresponding position 152 of the bottom area of ​​the motor vehicle 110 after mapping also includes the first shadow 121. As a result, the reconstructed bottom area image includes the first shadow 121, reflecting the motor vehicle driver's use of the panoramic imaging system.

[0048] Similarly, at the second frame, a conventional panoramic imaging system with underbody image display capabilities typically continues to capture image 142 in image capture area 130 in front of motor vehicle 110. Because the sun's altitude is still relatively low at the second frame, a second shadow 122 of the motor vehicle continues to be cast in front of motor vehicle 110. This second shadow 122 is also schematically illustrated as a triangle. This second shadow 122 is included in image capture area 130, and therefore, similarly, image 142 captured at this time also includes the second shadow 122 of the motor vehicle.

[0049] Thereafter, similarly, image 142 is mapped to a corresponding position 152 of the underbody region of the motor vehicle using an affine transformation based on the motor vehicle's driving parameters and displayed in the third frame of the motor vehicle surround view image 103 to be displayed. Furthermore, similarly, image 141 is mapped to a corresponding position 153 of the underbody region of the motor vehicle using an affine transformation based on the motor vehicle's driving parameters and displayed in the third frame of the motor vehicle surround view image 103 to be displayed. FIG1C exemplarily illustrates the third frame of the motor vehicle surround view image 103.

[0050] Since the image 142 captured at the second frame moment for reconstructing the underbody image includes the second shadow 122, and the image 141 captured at the first frame moment for reconstructing the underbody image includes the first shadow 121, after mapping, the images displayed at the corresponding positions 152 and 153 of the underbody area of ​​the motor vehicle 110 also include the second shadow 122 and the first shadow 121, reflecting the use of the panoramic imaging system by the driver of the motor vehicle.

[0051] Similarly, at the time of the subsequent frame, the shadows around the motor vehicle 110 will continue to be mapped to the corresponding positions of the motor vehicle's underbody area and displayed in the motor vehicle surround view image 10N of the subsequent frame. FIG1D exemplarily illustrates the motor vehicle surround view image 10N of the subsequent frame. As shown in FIG1D , the image of the motor vehicle's underbody area includes a large number of shadows 121, ..., 12N-1. These shadows are enriched in the image of the motor vehicle's underbody area 110, affecting the motor vehicle driver's observation of the underbody area and the underbody information, such as whether there are obstacles or potholes under the vehicle, resulting in the panoramic imaging system no longer being able to provide effective reference for the motor vehicle driver.

[0052] Figure 2 also shows a vehicle surround view image with a large number of shadows on the vehicle bottom. As shown in Figure 2, the shadows 121, ..., 12N-1 on the vehicle bottom can seriously affect the driver's observation of the vehicle bottom when using the panoramic imaging system.

[0053] To this end, the present disclosure proposes a method for generating a surround view image. This method captures an image in a shadow-free image capture area and subsequently uses this image to reconstruct an image of the area corresponding to the underside of the motor vehicle, effectively avoiding the inclusion of a large number of shadows in the reconstructed underside image of the vehicle.

[0054] Figure 3 shows a flowchart of a method 300 for generating a surround view image according to an embodiment of the present disclosure. The method 300 can be used, for example, in vehicles equipped with a surround view system, such as passenger cars or trucks, but the present disclosure is not limited thereto. The method 300 can also be used for any other motor vehicle requiring surround view images of its surroundings.

[0055] 4A and 4B also show motor vehicle surround view images 401 and 402 generated by executing the method 300. For clarity, the method 300 for generating surround view images according to an embodiment of the present disclosure will be described below in conjunction with FIG3 , FIG4A and FIG4B .

[0056] FIG4A shows a generated motor vehicle surround view image 401 corresponding to a first moment. As shown in FIG3 and FIG4A , first, in step S310, at a first moment, i.e., the first frame, a first environmental image 421 surrounding motor vehicle 410 is determined. For example, multiple cameras mounted on motor vehicle 410 for capturing the environment at different angles of the motor vehicle 410 can be used to capture images reflecting the environment at multiple angles of the motor vehicle 410, such as four video streams. Then, by stitching these multiple images at different angles and further performing perspective transformation, a top-down motor vehicle surround view image 400 is generated as shown in FIG4A , and a first environmental image 421 surrounding motor vehicle 410 is determined. As described above, since the environmental image is stitched together from multiple images captured by the cameras mounted on the motor vehicle, the area beneath the motor vehicle is not represented in the environmental image. Subsequently, an image of the area beneath the motor vehicle needs to be reconstructed. It should be noted that the present disclosure does not intend to limit the method for generating the first environment image 421 and the specific viewing angle (top view, side and rear oblique view, etc.) for displaying the motor vehicle surround view image 400 .

[0057] As shown in FIG4A , the first environment image 421 is overlaid with a first fill area 431. In the actual first environment image 421, the area covered by the first fill area 431 is no longer visible due to the coverage of the first fill area 431. However, to clearly illustrate the area covered by the first fill area 431, FIG4A schematically illustrates the first fill area 431 as a transparent rectangular area with a triangular mesh. As shown in FIG4 , in addition to completely covering the image of the underbody area of ​​the motor vehicle 410, the first fill area 431 also covers a shadow area 441 surrounding the motor vehicle in the first environment image 421. Here, the shadow area 441 is also schematically illustrated as triangles. Therefore, due to the coverage of the shadow area 441, the first environment image 421 cannot display the area covered by the first fill area 431, i.e., the shadow area 441.

[0058] Here, the first filling range (and the second filling range mentioned in the subsequent step S340) can be determined based on the size of the shadow area around the motor vehicle and the size of the motor vehicle, so that the filling range can completely cover the shadow area around the motor vehicle. For example, the size of the shadow area around the motor vehicle can be counted in advance, the filling range can be predetermined, and the filling range of this size can always be used to cover the environmental image in the subsequent method of generating a surround view image. The size of the filling range can also be determined and dynamically adjusted in the method of generating a surround view image by determining the shadow area in the environmental image. Therefore, the method 300 for generating a surround view image according to an embodiment of the present disclosure can also include, in step S310', determining the first filling range 431 and / or the second filling range 432 to be mentioned below based on the size of the shadow area around the motor vehicle and the size of the motor vehicle.

[0059] Furthermore, in step S320, an image 461 is captured from the image capture area 450 in the first environmental image 421. As shown in FIG4A , in the first environmental image 421, the image capture area 450 is located outside the first filling range 431. Specifically, the location of the captured image 461 in the first environmental image 421, i.e., the location of the image capture area 450, can be determined based on driving parameters of the motor vehicle 410, such as driving direction, driving speed, steering angle, etc. Preferably, the image capture area 450 is adjacent to the first filling range 431 in the direction of the motor vehicle's travel. For example, when the motor vehicle 410 is traveling forward, as shown in FIG4 , the image capture area 450 may be located directly in front of the first filling range 431. However, when the motor vehicle 410 is reversing, it is also conceivable that the image capture area 450 should be located directly behind the first filling range 431. The present disclosure does not limit the specific location of the image capture area 450 in the first environment image 421. Considering the difficulty of performing an affine transformation, placing the image capture area 450 adjacent to the first filling range 431 in the direction of travel is an optional implementation. This allows the affine transformation to be completed with a smaller distance / time difference, thus avoiding inaccurate mapping results caused by unpredictable vehicle movement within a longer time / distance difference.

[0060] As shown in FIG4A , a rectangle with diagonal lines therein schematically illustrates the cutout image 461. As described above, since the shadow area 441 is covered by the first filling range 431 and the image cutout area 450 is outside the first filling range 431, the cutout image 461 does not include the shadow area 441.

[0061] Furthermore, in step S330, operating parameters of the motor vehicle 410 are collected, such as the driving direction, driving speed, and steering angle of the motor vehicle 410. For example, the operating parameters can be determined using corresponding sensor equipment installed on the motor vehicle 410, or provided to the motor vehicle 410 using a navigation and positioning system (GPS, Galileo, BDS, or other navigation systems). The present disclosure does not limit the specific method for collecting the operating parameters. Subsequently, an affine transformation is performed on the image 461 based on the collected operating parameters to fill the bottom area of ​​the motor vehicle 410.

[0062] Then, at a second moment after the first moment, i.e., a subsequent frame after the first frame, for example, the second frame, in step S340, a second environment image 422 surrounding the motor vehicle 410 is determined. FIG4B shows the generated motor vehicle surround image 402 corresponding to the second moment. Here, for example, the second environment image 422 can be determined using the same method as the first environment image 421, and for the sake of brevity, further description is omitted here.

[0063] As shown in FIG4B , corresponding to the first environment image 421, the second environment image 422 is also covered with a second fill range 432 corresponding to the first fill range 431. Specifically, the fill range of the first fill range 431 in the first environment image 421 is equivalent to the fill range of the second fill range 432 in the second environment image 422: the first fill range 431 and the second fill range 432 have the same fill range. The second fill range 432 not only completely covers the image of the underbody area of ​​the motor vehicle 410, but also covers the shadow area 442 surrounding the motor vehicle 410 in the second environment image 422. Because the shadow area 442 is covered, the second environment image 422 is also unable to display the area covered by the second fill range 432, i.e., the shadow area 442.

[0064] Furthermore, in step S350, based on the operating parameters collected in step S330, image 461 captured at the first moment is mapped to second filling range 432 of second environment image 422, generating surround view image 402, particularly encompassing the area under the vehicle. For example, based on the operating parameters, predicted positions 472 within second filling range 432 to which pixels in image 461 will move at the second moment can be estimated. Based on this, an affine transformation is then performed on each pixel in image 461, causing it to be displayed at the corresponding predicted position 472, thereby reconstructing the image within second filling range 432.

[0065] Then, the reconstructed image of the second filling range 432 is spliced ​​with the second environment image 422 to generate a surround view image 402 corresponding to the second moment, including the reconstructed image of the second filling range 432 , that is, including the image of the bottom area of ​​the motor vehicle.

[0066] As described above, since image 461 captured at the first moment does not include shadow region 441, the image reconstructed within second filling range 432 using this image 461 after the mapping transformation, as shown in FIG4B , does not include shadows. Furthermore, since the underbody region of the vehicle is covered by second filling range 432, the reconstructed image of the underbody region of the vehicle, as shown in FIG4B , also does not include shadows.

[0067] In addition, it should be noted that, for the sake of simplicity, FIG4A and FIG4B only schematically illustrate the partial processing performed by the method 300 for generating a surround view image according to an embodiment of the present disclosure at the first and second moments. That is, as shown in FIG4B , at the second moment, only the image 461 intercepted at the first moment is mapped to a partial area of ​​the second filling range 432 (the area corresponding to the predicted position 472) and the image of the second filling range 432 of the partial area is reconstructed. However, it is conceivable that the method 300 for generating a surround view image according to an embodiment of the present disclosure can be cyclically executed to generate a surround view image including a complete filling range image. That is, at a moment after the second moment, for example, as shown in FIG4C , at a third moment after the second moment, the image 463 in the image interception area 450 is continued to be collected, and similarly based on the driving parameters of the motor vehicle, the image 462 intercepted at the second moment can be mapped to the predicted position 472 of the third filling range 433 by means of an affine transformation, and the image 461 intercepted at the first moment can be mapped to the predicted position 473 of the third filling range 433. Image stitching is performed here to generate a surround view image 402 corresponding to the third moment. Similarly, the surround view image generation method 300 according to an embodiment of the present disclosure is then looped and executed, whereby the image of the filled area is completely reconstructed, generating a surround view image as shown in FIG4D , including the complete filled area image and the vehicle bottom area image.

[0068] For clarity, FIG2 further illustrates a real surround view image generated by method 300 according to an embodiment of the present disclosure. FIG5A illustrates a first environment image 421 of the surroundings of a motor vehicle 410, as determined in step S310 of method 300 for generating a surround view image according to an embodiment of the present disclosure. Because the environment image is composed of images captured by multiple cameras mounted on the motor vehicle, the area under the motor vehicle cannot be represented in the real environment image. The area 471 corresponding to the underside of the motor vehicle in the environment image needs to be reconstructed in a subsequent step and is currently displayed in gray in FIG5A.

[0069] As shown in Figure 5A, the first environmental image 421 is overlaid with a first filling range 431. This first filling range 431 covers the area 471 in the first environmental image 421 corresponding to the underside of the motor vehicle. Furthermore, corresponding to the description of Figure 4A, the image capture area 450 is located directly in front of the first filling range 431. At subsequent moments, the images captured in the image capture area 450 are cyclically used to reconstruct the images within the first filling range 431, thereby reconstructing a filling range image that includes the area corresponding to the underside of the motor vehicle. The reconstructed filling range image is then spliced ​​with the environmental image corresponding to that moment to generate a surround view image that includes the underside image corresponding to that moment. Figure 5B shows a surround view image that includes the underside image generated by the method 300 for generating a surround view image according to an embodiment of the present disclosure. As shown in Figure 5B, the generated surround view image, particularly the area corresponding to the underside of the motor vehicle, does not contain shadows. Compared with FIG. 2 , the method 300 for generating a surround view image according to an embodiment of the present disclosure effectively avoids the reconstructed vehicle bottom image from containing a large number of shadows and distorted shadows that may affect the motor vehicle driver's observation and judgment of vehicle bottom related information.

[0070] Because the infill area to be reconstructed covers the shadows around the motor vehicle, the resulting surround view image, as shown in Figures 4A to 4C and 5B, does not contain any shadows. However, to ensure that the generated surround view image is consistent with the actual image of the motor vehicle's surroundings, the shadows around the motor vehicle must be displayed in the generated surround view image.

[0071] The method 600 for generating a surround view image according to another embodiment of the present disclosure is intended to solve the above-mentioned problems. FIG6 schematically shows a flow chart of the method 600 for generating a surround view image according to another embodiment of the present disclosure. FIG7 schematically shows a surround view image 402 of a motor vehicle generated corresponding to a second moment according to another embodiment of the present disclosure. FIG8 shows a surround view image generated according to another embodiment of the present disclosure. The method 600 according to another embodiment of the present disclosure is described below in conjunction with FIG6 to FIG8. It should be pointed out here that the method 600 for generating a surround view image according to another embodiment of the present disclosure can be further processed for the surround view image corresponding to any moment, and those skilled in the art should understand that the moment targeted by the method 600 according to another embodiment of the present disclosure is not limited to the second moment schematically shown in FIG7.

[0072] As shown in FIG6 , steps S610 to S640 of a method 600 for generating a surround view image according to another embodiment of the present disclosure correspond to steps S310 to S350 of the method described in FIG3 , and are not further described here for the sake of brevity. According to another embodiment of the present disclosure, method 600 further includes, at step S660, extracting a third fill range image 492 corresponding to the third fill range 433 from the generated surround view image, such as surround view image 402 generated at the second moment.

[0073] As shown on the left side of FIG. 7 , in the surround view image 402 generated in step S650, the second environment image 422 is covered by a third fill area 433. This third fill area 433 corresponds to the area under the motor vehicle in the second environment image 422 and does not cover the shadow area of ​​the motor vehicle in the second environment image, such as the shadow area 442 corresponding to the second moment. For clarity, the third fill area 433 is schematically illustrated as a grayscale filled rectangle. The fact that the third fill area 433 corresponds to the area under the motor vehicle in the second environment image 422 can be understood as covering the area under the motor vehicle 410. Preferably, the third fill area 433 exactly covers the area under the motor vehicle 410. Compared to FIG. 4B , FIG. 7 illustrates an embodiment in which the third fill area 433 exactly covers the area under the motor vehicle 410. However, it should be noted that this disclosure does not limit the specific size or location of the third fill area 433. Those skilled in the art will appreciate that the third filling range 433 only needs to be located within the area corresponding to the second filling range 432 in the generated surround view image 702 and cover the bottom area of ​​the motor vehicle 410 without covering the shadow area.

[0074] In addition, according to another embodiment of the present disclosure, the method 600 further includes: in step S670, stitching the third filling range image 492 into the second environment image 422 to generate a surround view image including a shadow area.

[0075] As shown in Figure 7 , third fill range image 492 within third fill range 433 is extracted and used to fill in second environment image 422 (shown on the right side of Figure 7 ), which also corresponds to the same moment (second moment) and is not covered by any fill range. Specifically, this fills in under-vehicle region 471, which is not represented in second environment image 422. This generates surround view image 402′ (see Figure 8 ) that includes an under-vehicle region image. Because second environment image 422, which is not covered by any fill range, includes shadow region 442, the filled image includes shadow region 442 in addition to under-vehicle region image 492, making the generated surround view image that includes the shadow region more realistic.

[0076] For clarity, FIG8 further illustrates a realistic surround view image 402' including shadow areas, generated by method 300 according to another embodiment of the present disclosure. In addition to displaying the underbody image 492, surround view image 402' also displays shadow areas 442. Compared to the motor vehicle surround view image 10N shown in FIG2 , which contains numerous shadows, surround view image 402' generated by method 300 according to another embodiment of the present disclosure not only eliminates numerous shadows in the underbody area but also displays shadow areas consistent with the real environment around the motor vehicle, such as directly in front of it. This makes generated surround view image 402' more realistic and further mitigates the impact on the driver caused by differences between the surround view image generated by the panoramic imaging system and reality.

[0077] Furthermore, as shown in FIG8 , in a method for generating a surround-view image according to another embodiment of the present disclosure, a transparent vehicle model 411 corresponding to the motor vehicle is rendered in the surround-view image or the surround-view image 402′ including a shadowed area, in the area corresponding to the underside of the motor vehicle. Thus, with the underside of the vehicle no longer heavily shadowed, transparent vehicle model 411 allows the driver to clearly see through it and also defines the area corresponding to the underside of the vehicle in the surround-view image, facilitating the driver's assessment of the vehicle's underside information.

[0078] In summary, the method for generating surround view images according to the present disclosure can effectively avoid the reconstructed vehicle bottom image from containing a large number of shadows, provide a shadow-free bottom view of the vehicle, and effectively avoid the shadows in the surround view image from affecting the motor vehicle driver.

[0079] The present disclosure also provides a panoramic surround view device 900. FIG9 schematically illustrates a panoramic surround view device 900 according to an embodiment of the present disclosure. The panoramic surround view device 900 includes one or more processors 910 and one or more memories 920, wherein the memories store computer-readable instructions that, when executed by the one or more processors 910, cause the one or more processors 910 to execute the method for generating surround view images as described above with respect to FIG3 through FIG8. For a detailed description of the various features and advantages of the panoramic surround view device 900, please refer to the description of the corresponding method above, which will not be repeated here for the sake of brevity.

[0080] The present disclosure also provides a computer program product 1000. Figure 10 schematically shows a computer program product 1000 according to an embodiment of the present disclosure. Here, the computer program product 1000 includes computer-readable instructions 1020 stored in a computer-readable storage medium 1010. When the computer-readable instructions 1020 are executed by a processor, the processor executes the method for generating surround view images as described above with respect to Figures 3 to 8. Estimation method. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, optical disk, etc. For a detailed description of the various features and advantages of the computer program product 1000, please refer to the description of the corresponding method above, which will not be repeated here for the sake of brevity.

[0081] The present disclosure further provides a panoramic view system 1100. Fig. 11 schematically shows a panoramic view system 1100 according to an embodiment of the present disclosure.

[0082] Here, the panoramic surround view system 1100 includes: an image acquisition module 1110, which is configured to determine a first environmental image around a motor vehicle at a first moment, and determine a second environmental image around the motor vehicle at a second moment after the first moment, wherein the first environmental image is covered by a first filling range, wherein the first filling range covers an area corresponding to the bottom of the motor vehicle in the first environmental image, and covers a shadow area around the motor vehicle in the first environmental image, and wherein the second environmental image is covered by a second filling range corresponding to the first filling range.

[0083] In addition, the panoramic surround view system 1100 also includes: an image reconstruction module 1120, which is configured to capture an image from an image capture area in the first environmental image, and perform an affine transformation on the pixel points of the image according to the operating parameters, map the image to a second filling range of the second environmental image, and generate a surround view image, wherein the image capture area is located outside the first filling range of the first environmental image.

[0084] Here, for a detailed description of the various features and advantages of the panoramic surround view system 1100, please refer to the description of the corresponding methods in conjunction with Figures 3 to 5B above, which will not be repeated here for the sake of brevity.

[0085] In addition, according to another embodiment of the panoramic surround view system 1100 disclosed herein, the image reconstruction module 1120 is further configured to: cut out a third filling range image corresponding to a third filling range from the surround view image, wherein the second environment image is covered by the third filling range, wherein the third filling range corresponds to an area corresponding to the bottom of the motor vehicle in the second environment image and does not cover the shadow area of ​​the motor vehicle in the second environment image; and splice the third filling range image into the second environment image to generate a surround view image including a shadow area.

[0086] Here, for a detailed description of the various features and advantages of the panoramic surround view system 1100 according to the above embodiment, please refer to the description of the corresponding method in conjunction with Figures 6 to 8 above, which will not be repeated here for the sake of brevity.

[0087] The present disclosure also provides a motor vehicle 1200. FIG12 schematically illustrates a motor vehicle 1200 according to an embodiment of the present disclosure. As shown in FIG12 , the motor vehicle is equipped with the panoramic view device 900 described above with respect to FIG9 or the panoramic view system 1100 described above with respect to FIG11 . A detailed description of the features and advantages of the motor vehicle 1200 is provided above with respect to the panoramic view device 900 or the panoramic view system 1100, which will not be repeated here for the sake of brevity.

[0088] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and should not be interpreted in an idealized or highly formal sense, unless explicitly defined as such herein.

[0089] While the present disclosure has been described in detail above, it will be apparent to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure may be implemented in various modifications and variations without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the descriptions in this specification are for illustrative purposes only and are not intended to be limiting of the present disclosure.

Claims

1. A method for generating a surround view image, comprising: At a first moment, determining a first environment image around a motor vehicle, wherein the first environment image is covered by a first filling range, wherein the first filling range covers an area corresponding to the bottom of the motor vehicle in the first environment image and covers a shadow area around the motor vehicle in the first environment image; intercepting an image from an image interception area in the first environment image, wherein the image interception area is outside a first filling range of the first environment image; collecting operating parameters of the motor vehicle; determining, at a second time instant after the first time instant, a second image of the surroundings of the motor vehicle, wherein the second image of the surroundings is covered with a second filling range corresponding to the first filling range; According to the operating parameters, the image is mapped to a second filling range of the second environment image to generate a surround view image.

2. The method according to claim 1, further comprising: Extracting a third filling range image corresponding to a third filling range from the surround view image, wherein the second environment image is covered by the third filling range, wherein the third filling range corresponds to an area corresponding to the bottom of the motor vehicle in the second environment image and does not cover a shadow area of the motor vehicle in the second environment image; and The third filling range image is stitched into the second environment image to generate a surround view image including a shadow area.

3. The method according to claim 1, wherein In the traveling direction of the motor vehicle, the image cutout area is adjacent to the first filling range.

4. The method according to any one of claims 1 to 3, further comprising: In the surround view image or the surround view image including the shadow area, a transparent vehicle model corresponding to the motor vehicle is rendered in an area corresponding to the bottom of the motor vehicle.

5. The method according to any one of claims 1 to 3, further comprising: The first filling range and / or the second filling range is determined according to the size of the shadow area around the motor vehicle and the size of the motor vehicle.

6. A panoramic view device, comprising: one or more processors; as well as One or more memories, wherein computer-readable instructions are stored in the memories, and when the computer-readable instructions are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 5.

7. A computer program product comprising computer-readable instructions stored in a computer-readable storage medium, wherein when the computer-readable instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 5.

8. A panoramic view system comprising: an image acquisition module configured to determine a first image of the surroundings of the motor vehicle at a first moment in time, and to determine a second image of the surroundings of the motor vehicle at a second moment in time after the first moment in time, wherein the first environment image is covered by a first filling range, wherein the first filling range covers an area corresponding to the bottom of the motor vehicle in the first environment image and covers a shadow area around the motor vehicle in the first environment image, and wherein the second environment image is covered by a second filling range corresponding to the first filling range; An image reconstruction module is configured to capture an image from an image capture area in the first environmental image and map the image to a second filling range of the second environmental image according to the operating parameters to generate a surround view image, wherein the image capture area is located outside the first filling range of the first environmental image.

9. The panoramic view system according to claim 8, wherein: The image reconstruction module is further configured to: A third filling range image corresponding to a third filling range is cut out from the surround image, wherein the second environment image is covered by the third filling range, wherein the third filling range The range corresponds to an area corresponding to the bottom of the motor vehicle in the second environment image and does not cover a shadow area of the motor vehicle in the second environment image; and The third filling range image is stitched into the second environment image to generate a surround view image including a shadow area. 10 . A motor vehicle equipped with the panoramic view device according to claim 6 or the panoramic view system according to claim 8 or 9.

Images