IMAGE PROCESSING METHOD, IMAGE DISPLAY METHOD, IMAGE PROCESSING APPARATUS, AND IMAGE DISPLAY APPARATUS

Abstract

According to this image processing method, first imaging devices (2L, 2R) installed on the sides of a vehicle outside the cabin and forwardly of the rear end of the vehicle are used to capture first images of areas including laterally rearward areas of the vehicle (S1); a second imaging device (3) installed on the rear end of the vehicle is used to capture a second image of an area behind the vehicle (S1); the first images and the second image are combined to generate a combined image as a rearward image of the vehicle (S7); and contour lines that are line segments along the contours of the sides of the vehicle are superimposed in the combined image (S8).

Description

[Technical field]

[0001] The present invention relates to an image processing method, an image display method, an image processing device, and an image display device. [Background technology]

[0002] The following Patent Document 1 describes an image display device that generates a composite image by combining an image of a rear area of ​​a vehicle with an image of an area on one side of the rear of the vehicle, and displays the generated composite image on a display device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2019-101879 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the synthetic image generated in Patent Document 1 above has a problem in that it is difficult for a driver to grasp the relative positional relationship between the vehicle and surrounding objects when driving and parking the vehicle, making it difficult to park the vehicle while looking at the synthetic image. The present invention has an object to make it easier to grasp the relative positional relationship between a vehicle and surrounding objects from an image captured behind the vehicle. [Means for solving the problem]

[0005] In one embodiment of the image processing method of the present invention, a first image is captured using a first imaging device provided on the side outside the vehicle cabin and forward of the rear end of the vehicle, and a second image is captured using a second imaging device provided at the rear end of the vehicle, and the first and second images are combined to generate a composite image of the area behind the vehicle, and a contour line, which is a line segment that follows the contour of the side of the vehicle, is superimposed on the composite image. In an image display method according to another aspect of the present invention, a composite image generated by the above image processing method and having a contour line superimposed thereon is displayed on a display device within a vehicle cabin. Effect of the Invention

[0006] According to the present invention, it becomes easier to grasp the relative positional relationship between a vehicle and surrounding objects from a composite image. The objects and advantages of the invention will be realized and attained by means of the elements and combinations recited in the appended claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed. [Brief description of the drawings]

[0007] [Figure 1] 1 is a schematic configuration diagram of an example of an image display device according to an embodiment; [Figure 2A] 3 is a schematic diagram showing the mounting positions of a left side camera, a right side camera, and a rear camera, the range of the angle of view, and the range of the angle of view of an image after viewpoint conversion. [Figure 2B] 3 is a schematic diagram showing the mounting positions of a left side camera, a right side camera, and a rear camera, the range of the angle of view, and the range of the angle of view of an image after viewpoint conversion. [Figure 3A] FIG. 13 is a schematic diagram of a left side camera image. [Figure 3B] 13 is a schematic diagram of a left side camera image after distortion correction, viewpoint conversion processing, and left-right inversion processing. FIG. [Figure 3C] 3C is a schematic diagram of a partial left side camera image cut out from the left side camera image of FIG. 3B. [Figure 4A] 13 is a schematic diagram of a rear camera image after distortion correction, viewpoint conversion processing, and left-right inversion processing. FIG. [Figure 4B] 10 is a schematic diagram of a left rear camera image obtained by moving a virtual viewpoint of the rear camera image to a lateral position equivalent to the position of the left side camera in the vehicle width direction. FIG. [Figure 4C]10 is a schematic diagram of a right rear camera image obtained by moving a virtual viewpoint of the rear camera image to a lateral position equivalent to the position of the right side camera in the vehicle width direction. FIG. [Figure 5A] 11 is a schematic diagram of a left rear side composite image obtained by combining a left rear camera image with a partial left side camera image. FIG. [Figure 5B] 11 is a schematic diagram of a right rear side composite image obtained by combining a right rear camera image with a partial right side camera image. FIG. [Figure 5C] 1 is a schematic diagram of a composite image in which the outline of a vehicle and a transparent image are superimposed. [Figure 6] 1 is a flowchart of an example of an image display method according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] (composition) 1 is a schematic configuration diagram of an example of an image display device according to an embodiment. The image display device 1 is mounted on a vehicle 10 together with a left side camera 2L, a right side camera 2R, and a rear camera 3. For example, the image display device 1 may be an electronic circuit including a processor (e.g., a CPU or an MPU) and a storage device (e.g., a semiconductor storage device, a magnetic storage device, an optical storage device, etc.). The image display device 1 may realize the information processing described below by executing a computer program stored in the storage device by the processor. 2A and 2B, the left side camera 2L is provided on the left side outside the vehicle cabin and in front of the rear end of the vehicle 10, and generates a "left side camera image" that is an image of an area including the left rear side of the vehicle. The right side camera 2R is provided on the right side outside the vehicle cabin and in front of the rear end of the vehicle 10, and generates a "right side camera image" that is an image of an area including the right rear side of the vehicle. For example, the left side camera 2L and the right side camera 2R are attached below the left and right side mirrors, respectively. In addition, the rear camera 3 is provided at the rear end of the vehicle 10 to generate a "rear camera image" which is an image capturing the area behind the vehicle. For example, the rear camera 3 is attached above the rear license plate.

[0009] For example, dashed lines 6Lr, 6Rr and 7r in FIGS. 2A and 2B typically indicate the ranges of the angles of view of the left side camera 2L, the right side camera 2R and the rear camera 3, respectively. The optical axes of the left side camera 2L and the right side camera 2R are perpendicular to the front-rear direction of the vehicle 10 and directed diagonally downward to the left and right, respectively. The angle between these optical axes and the horizontal direction may be, for example, 45 degrees. The optical axis of the rear camera 3 is directed obliquely downward toward the rear of the vehicle 10, and the angle between the optical axis and the horizontal direction may be, for example, 45 degrees. The angle of view of the left side camera 2L, the right side camera 2R, and the rear camera 3 may be, for example, about 180 degrees (e.g., about 185 degrees). That is, the left side camera 2L, the right side camera 2R, and the rear camera 3 may be wide-angle cameras equipped with fisheye lenses with wide angles of view. For example, the left side camera 2L, the right side camera 2R, and the rear camera 3 may be around-view monitor cameras that generate images that are converted into a bird's-eye view (around-view monitor image).

[0010] 1, the image display device 1 includes an image processing device 4 and a display device 5. The image processing device 4 performs image processing on the left side camera image, right side camera image, and rear camera image generated by the left side camera 2L, right side camera 2R, and rear camera 3, to generate a "left rear side composite image," which is an image of the rear side area on the left side of the vehicle 10, and a "right rear side composite image," which is an image of the rear side area on the right side of the vehicle 10. The display device 5 is provided in the passenger compartment of the vehicle 10, and displays the left rear side composite image and the right rear side composite image generated by the image processing device 4 on a display screen.

[0011] The following is a detailed description of the image processing device 4. The image processing device 4 includes a distortion correction unit 4a, a viewpoint conversion unit 4b, an image inversion unit 4c, a clipping unit 4d, a synthesis unit 4e, and a superimposition unit 4f. The distortion correction unit 4a corrects distortions contained in the left side camera image, the right side camera image and the rear camera image generated by the left side camera 2L, the right side camera 2R and the rear camera 3, which are wide-angle cameras.

[0012] 3A is a schematic diagram of a left side camera image before distortion correction by the distortion correction unit 4a. The parts indicated by reference numerals 20, 21, 22, and 23 show the side of the left front end, the side of the left rear end, the left front wheel, and the left rear wheel of the vehicle 10, respectively. The part indicated by reference numeral 24 shows the parking space behind the vehicle 10, and the part indicated by reference numeral 25 shows the plants around the parking space 24. The part indicated by reference numeral 26 shows the road around the vehicle 10, and the part indicated by reference numeral 27 shows the plants growing along the road.

[0013] Moreover, an area 28 indicated by relatively thick dot hatching is an area in which parts around the installation location of the left side camera 2L are captured, or an area outside the image circle of the optical system of the left side camera 2L. As shown in FIG. 3A, images captured by the wide-angle cameras, the left side camera 2L, the right side camera 2R, and the rear camera 3, are distorted, so the distortion correction unit 4a corrects the distortion caused by the wide-angle lenses. The viewpoint conversion unit 4b performs viewpoint conversion processing on the left side camera image to generate a virtual viewpoint image in which the viewing direction is directed horizontally and rearward from the vehicle 10 with the position of the left side camera 2L as the viewpoint. Also, the viewpoint conversion unit 4b performs viewpoint conversion processing on the right side camera image to generate a virtual viewpoint image in which the viewing direction is directed horizontally and rearward from the vehicle 10 with the position of the right side camera 2R as the viewpoint. In FIG. 2A and FIG. 2B dash-dotted line 6Lv and 6Rv respectively show the ranges of the angles of view of the left side camera image and the right side camera image after the viewpoint conversion.

[0014] The image inverter 4c inverts the left side camera image after the viewpoint conversion. Fig. 3B is a schematic diagram of the left side camera image after the distortion correction, viewpoint conversion processing, and left-right inversion processing are performed. The area 28 shown with relatively thick dot hatching is an area showing parts around the installation location of the left side camera 2L and an area outside the image circle of the optical system of the left side camera 2L. The area 29 shown with diagonal hatching is an area consisting of pixels not included in the left side camera image before viewpoint conversion. Moreover, the area 30 shown with relatively thin dot hatching corresponds to a part of the left side camera image before viewpoint conversion in FIG. 3A other than the area on the left rear side of the vehicle 10, but is not shown here for ease of explanation.

[0015] An image showing the left rear side area of ​​the vehicle 10 is generated in the image area 31 of Fig. 3B. In the example of the schematic diagram of Fig. 3B, the left rear side surface 21 of the vehicle 10, the parking space 24, and the plants 25 and 27 are shown in the image area 31. 3B indicates the position of the outline (outer line) of the vehicle 10 actually captured in the left side camera image. Also, dashed line 33 indicates the position where the outline (outer line) of the vehicle 10 should be captured in an image that is not actually captured in the left side camera image but has the same line of sight and angle of view as the left side camera image after viewpoint conversion.

[0016] The cropping unit 4d crops out a part 34 of the image area 31 of the left side camera image after the viewpoint conversion process and the left-right inversion process as a "partial left side camera image". The partial left side camera image is synthesized with the rear camera image as described later. Figure 3C is a schematic diagram of a partial left side camera image 40L cropped out from the left side camera image of Figure 3B. For example, the cropping unit 4d may crop, as the partial left side camera image, pixels in a range outside in the horizontal direction of the image from positions (positions of dashed line 32 and dashed dotted line 33 in the example of FIG. 3B) where pixels representing the contour of the vehicle 10 should be displayed in an image having the same line of sight direction and angle of view as the left side camera image after viewpoint conversion, among the pixels included in the image region 31. A part 35 of the edges (borders, boundaries) of the partial left side camera image may coincide with the contour (dashed line 32) actually shown in the left side camera image after viewpoint conversion. The viewpoint conversion unit 4b and the image inversion unit 4c perform the viewpoint conversion process and the left-right inversion process on the right side camera image in the same manner. The cutout unit 4d cuts out a "partial right side camera image" from the right side camera image after the viewpoint conversion process and the left-right inversion process have been performed.

[0017] On the other hand, the viewpoint conversion unit 4b performs viewpoint conversion processing on the rear camera image to generate a virtual viewpoint image in which the position of the rear camera 3 is used as the viewpoint and the line of sight is directed horizontally backward of the vehicle 10. The image inversion unit 4c inverts the rear camera image after viewpoint conversion. FIG. 4A is a schematic diagram of the rear camera image after distortion correction, viewpoint conversion processing, and left-right inversion processing. The rear camera image shows a parking space 24, plants 25 and 27, and a road 26. The rear bumper of the vehicle 10 is shown in the portion indicated by reference numeral 36. Furthermore, the viewpoint conversion unit 4b performs viewpoint conversion processing on the rear camera image in Figure 4A to generate a virtual viewpoint image (hereinafter referred to as the ``left side rear camera image'') in which the virtual viewpoint of the rear camera image is moved to a lateral position equivalent to the vehicle width position of the left side camera 2L.

[0018] Similarly, the viewpoint conversion unit 4b performs viewpoint conversion processing on the rear camera image of Figure 4A to generate a virtual viewpoint image (hereinafter referred to as the ``right side rear camera image'') in which the virtual viewpoint of the rear camera image is moved to a lateral position equivalent to the vehicle width position of the right side camera 2R. 2A and 2B are schematic diagrams showing the positions of the virtual viewpoints PvL and PvR of the left rear camera image and the right rear camera image, and the ranges of the viewing angles 7Lv and 7Rv. FIG. 4B and FIG. 4C are schematic diagrams of the left rear camera image 41L and the right rear camera image 41R, respectively. In this way, by moving the position of the virtual viewpoint of the rear camera image, it becomes easier to synthesize the partial left side camera image cut out from the left side camera image and the partial right side camera image cut out from the right side camera image into the rear camera image.

[0019] Please refer to Fig. 1. The synthesis unit 4e generates a synthetic image by synthesizing the left rear camera image 41L with the partial left side camera image 40L. Hereinafter, the left rear camera image 41L synthesized with the partial left side camera image 40L may be referred to as the "left rear lateral synthetic image 41L". 5A is a schematic diagram of the left rear side composite image 41L. For example, the composite unit 4e composites the left rear camera image 41L and the partial left side camera image 40L by replacing pixels in the range of the left rear camera image 41L that is occupied by the angle of view of the partial left side camera image 40L with the partial left side camera image 40L. See Fig. 5B. Similarly, the synthesis unit 4e generates a synthetic image by synthesizing the partial right side camera image 40R with the right rear camera image 41R. Hereinafter, the right rear camera image 41R synthesized with the partial right side camera image 40R may be referred to as a "right rear lateral synthetic image 41R."

[0020] Please refer to Fig. 1. The superimposing unit 4f superimposes a contour line, which is a line segment along the contour of the vehicle 10, on the left rear side composite image 41L and the right rear side composite image 41R. 5C is a schematic diagram of the left rear side composite image 41L and the right rear side composite image 41R on which the contour line of the vehicle 10 is superimposed. The horizontal and vertical directions of the left rear side composite image 41L and the right rear side composite image 41R are defined as the "X direction" and the "Y direction", respectively. The superimposing unit 4f superimposes a contour line 50L, which is a line segment along the contour of the left side of the vehicle 10, onto the left rear side composite image 41L. The position at which the contour line 50L is superimposed onto the left rear side composite image 41L may be determined by, for example, calculating in advance the position at which the contour of the vehicle 10 is captured when an image of the range captured in the left rear side composite image 41L is generated with the position of the left side camera 2L as the viewpoint.

[0021] 3B, the superimposing unit 4f may detect the contour (dashed line 32) of the vehicle 10 actually captured in the left side camera image. The superimposing unit 4f may determine the position to superimpose the contour line 50L based on the position to composite the partial left side camera image 40L into the left rear lateral composite image 41L and the position of the contour detected from the left side camera image, so that the contour line 50L is a line segment along the contour of the vehicle 10. The superimposing unit 4f further superimposes a transparent image 51L of the shape of the vehicle 10 on an area of ​​the left rear side composite image 41L that is inside the contour line 50L in the lateral direction (X direction) of the image. Similarly, the superimposing unit 4f superimposes a contour line 50R, which is a line segment along the contour of the right side of the vehicle 10, and a transparent image 51R of the shape of the vehicle 10 on the right rear side composite image 41R.

[0022] In addition, the area surrounded by dashed line 52 in the left rear lateral composite image 41L is an image area including pixels extracted from the partial left side camera image 40L, and the area surrounded by dotted line 53 is an image area including pixels extracted from the left rear camera image 41L in Figure 4B. In addition, the area surrounded by dashed line 54 in the right rear lateral composite image 41R is an image area including pixels extracted from the partial right side camera image 40R, and the area surrounded by dotted line 55 is an image area including pixels extracted from the right rear camera image 41R shown in Figure 4C.

[0023] Consider the coordinates P1, P2, and P3 on the boundary between image area 52 and image area 53. The Y direction positions (i.e., vertical positions) of coordinates P1, P2, and P3 are y1, y2, and y3, respectively, and the X direction positions (i.e., horizontal positions) are x1, x2, and x3, respectively. In the left rear lateral composite image 41L, among the pixels at the Y direction position y1, pixels in the area outside in the X direction from the X direction position x1 are extracted from the partial left side camera image 40L, and pixels in the area inside are extracted from the left rear camera image 41L. Similarly, among the pixels at the Y-direction position y2, pixels in an area outside the X-direction position x2 are extracted from the partial left side camera image 40L, and pixels in the inner area are extracted from the left rear camera image 41L. Among the pixels at the Y-direction position y3, pixels in an area outside the X-direction position x3 are extracted from the partial left side camera image 40L, and pixels in the inner area are extracted from the left rear camera image 41L. The X-direction positions x1, x2, and x3 of the coordinates P1, P2, and P3 are each an example of a "predetermined lateral position" described in the claims.

[0024] As described above, the partial left side camera image 40L is a partial image extracted by cutting out pixels in the range outside the contour of the vehicle 10 from the left side camera image after viewpoint conversion. Therefore, at Y-direction positions y1 and y2, X-direction positions x1 and x2 are positions outside a contour line 50L that follows the contour of the vehicle 10 in the vehicle width direction (horizontal direction of the image). 3C, a part 35 of the edge of the partial left side camera image 40L may coincide with the contour (dashed line 32) that is actually captured in the left side camera image after the viewpoint conversion. Therefore, the X-direction position x3 of the boundary line between the image area 52 and the image area 53 at the Y-direction position y3 may be the same as the position of the contour line 50L.

[0025] 1, the superimposing unit 4f outputs to the display device 5 a left rear side composite image 41L and a right rear side composite image 41R in which the contour lines 50L and 50R of the vehicle 10 and the transparent images 51L and 51R are superimposed, respectively. The display device 5 displays the left rear side composite image 41L and the right rear side composite image 41R on a display screen in the cabin of the vehicle 10. For example, the display device 5 may display an image of the rear area of ​​the vehicle 10, such as the rear camera image in Fig. 4A, between the left rear side composite image 41L and the right rear side composite image 41R. Alternatively, the left rear side composite image 41L and the right rear side composite image 41R may be displayed side by side as shown in Fig. 5C.

[0026] (operation) FIG. 6 is a flowchart of an example of an image display method according to the embodiment. In step S1, the left side camera 2L, the right side camera 2R, and the rear camera 3 generate a left side camera image, a right side camera image, and a rear camera image, respectively. In step S2, the distortion correction unit 4a corrects distortions contained in the left side camera image, the right side camera image, and the rear camera image.

[0027] In step S3, the viewpoint conversion unit 4b performs viewpoint conversion processing on the left side camera image and the right side camera image to generate a virtual viewpoint image in which the viewpoint is the position of the left side camera 2L and the right side camera 2R, respectively, and the line of sight is directed horizontally backward of the vehicle 10. The viewpoint conversion unit 4b also performs viewpoint conversion processing on the rear camera image to generate a virtual viewpoint image in which the viewpoint is the position of the rear camera 3, and the line of sight is directed horizontally backward of the vehicle 10.

[0028] In step S4, the image inverting unit 4c inverts the left side camera image, the right side camera image and the rear camera image after the viewpoint conversion. In step S5, the viewpoint conversion unit 4b performs viewpoint conversion processing on the rear camera image after the left-right inversion processing of step S4, thereby generating a left rear camera image and a right rear camera image in which the virtual viewpoints PvL and PvR are moved to lateral positions corresponding to the vehicle width positions of the left side camera 2L and the right side camera 2R, respectively. In step S6, the cutout unit 4d cuts out a partial left side camera image 40L showing the left rear lateral area from the left side camera image after the left-right reversal processing in step S4. Similarly, the cutout unit 4d cuts out a partial right side camera image 40R showing the right rear lateral area from the right side camera image after the left-right reversal processing in step S4.

[0029] In step S7, the synthesis unit 4e generates a left rear lateral synthesized image 41L in which a partial left side camera image 40L is synthesized with the left rear camera image, and a right rear lateral synthesized image 41R in which a partial right side camera image 40R is synthesized with the right rear camera image. In step S8, the superimposing unit 4f superimposes the contour lines 50L and 50R of the vehicle 10 onto the left rear side composite image 41L and the right rear side composite image 41R. In step S9, the superimposing unit 4f superimposes the transparent images 51L and 51R of the shape of the vehicle 10 on the left rear side composite image 41L and the right rear side composite image 41R. In step S10, the display device 5 displays the left rear side composite image 41L and the right rear side composite image 41R, in which the contour lines 50L and 50R and the transparent images 51L and 51R of the shape of the vehicle 10 are superimposed, on a display screen in the cabin of the vehicle 10. Then, the process ends.

[0030] (Effects of the embodiment) (1) The left side camera 2L and the right side camera 2R, which are provided on the sides outside the passenger compartment of the vehicle 10 and in front of the rear end of the vehicle 10, capture a first image, which is an image of the area including the rear side of the vehicle 10. The rear camera 3, which is provided at the rear end of the vehicle 10, captures a second image, which is an image of the area behind the vehicle 10. The image processing device 4 combines the first image and the second image to generate a composite image as an image of the rear of the vehicle, and superimposes a contour line, which is a line segment that follows the contour of the side of the vehicle 10, on the composite image. This makes it easier to grasp the relative positional relationship between the vehicle 10 and surrounding objects from a composite image captured of the rear side of the vehicle 10. Also, by combining the first images captured by the left side camera 2L and the right side camera 2R, which are provided in a position forward of the rear end of the vehicle 10, a composite image can be generated that captures objects on the sides of the vehicle 10 that are not captured in the second image captured by the rear camera 3.

[0031] (2) The image processing device 4 may extract from the first image an image of an area outside a predetermined lateral position in the lateral direction of the composite image, and may extract from the second image an image of an area inside the predetermined lateral position. The predetermined lateral position may include a position outside a line segment along the contour of the vehicle 10 in the lateral direction of the image. This makes it possible to generate a composite image that not only captures the rear lateral area captured in the first images from the left side camera 2L and the right side camera 2R, but also captures the area hidden by the vehicle body from the left side camera 2L and the right side camera 2R.

[0032] (3) The predetermined horizontal positions may be set for each vertical position in the composite image, and at least a portion of the predetermined horizontal positions may be equal to the position of the contour line. This makes it possible to generate a composite image that captures areas close to the sides of the vehicle 10. (4) The image processing device 4 may detect the contour of the vehicle 10 from the first image. The contour line superimposed in the composite image may be a line segment that follows the contour of the vehicle 10 detected from the first image. This makes it possible to determine the position at which to superimpose the contour line on the composite image according to the position of the contour of the vehicle 10 actually captured in the first image.

[0033] (5) The image processing device 4 may superimpose a see-through image of the shape of the vehicle 10 onto an area of ​​the composite image that is laterally inward of the contour line of the image. This makes it easier to grasp the contour lines in the composite image. (6) The display device 5 may display the composite image, which is generated by the image processing device 4 and on which the contour line is superimposed, on a display device in the cabin of the vehicle 10. This makes it easier for the occupants of the vehicle 10 to grasp the relative positional relationship between the vehicle 10 and surrounding objects on the rear sides of the vehicle 10.

[0034] All examples and conditional terms described herein are intended for educational purposes to help the reader understand the present invention and the concepts provided by the inventor for the advancement of technology, and should be interpreted without being limited to the above specifically described examples and conditions, and the configuration of the examples in this specification with respect to showing the advantages and disadvantages of the present invention. Although the embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0035] 1...image display device, 2L...left side camera, 2R...right side camera, 4...image processing device, 4a...distortion correction unit, 4b...viewpoint conversion unit, 4c...image inversion unit, 4d...cut-out unit, 4e...synthesizing unit, 4f...superimposing unit, 5...display device

Claims

1. capturing a first image, which is an image of an area including a rear side of the vehicle, by a first imaging device provided at a position on a side surface outside a cabin of the vehicle and forward of a rear end of the vehicle; capturing a second image, which is an image of an area behind the vehicle, by a second imaging device provided at a rear end of the vehicle; synthesizing the first image and the second image to generate a synthetic image as an image behind the vehicle; and superimposing a contour line, which is a line segment along a contour of the vehicle side surface, on the composite image; an image of an area of ​​the composite image that is outside a predetermined lateral position in a lateral direction of the composite image is extracted from the first image, and an image of an area of ​​the composite image that is inside the predetermined lateral position in the lateral direction is extracted from the second image; An image processing method, comprising the steps of: setting the predetermined horizontal position for each vertical position in the composite image; and wherein at least a portion of the predetermined horizontal position is equal to a position of the contour line.

2. (delete)

3. 2. The image processing method according to claim 1, wherein the predetermined lateral position includes a position outside a line segment along a contour of the vehicle in the lateral direction.

4. Detecting an outline of the vehicle from the first image; the contour line superimposed in the composite image is a line segment along the contour of the vehicle detected from the first image; 4. The image processing method according to claim 1, wherein the first and second pixels are arranged in a first direction.

5. 5. The image processing method according to claim 1, wherein a transparent image of the shape of the vehicle is superimposed on an area of ​​the composite image that is laterally inside the contour line.

6. 6. An image display method, comprising: displaying the composite image, generated by the image processing method according to claim 1, 3, 4 or 5 and having the contour line superimposed thereon, on a display device in a cabin of the vehicle.

7. a process of generating a composite image as an image behind the vehicle by combining a first image, which is an image of an area including the rear side of the vehicle, captured by a first imaging device provided at a position on the side outside the passenger compartment of the vehicle and forward of the rear end of the vehicle, and a second image, which is an image of the area behind the vehicle, captured by a second imaging device provided at the rear end of the vehicle; a process of superimposing a contour line, which is a line segment that follows a contour of the vehicle side surface, on the composite image; an image of an area of ​​the composite image that is outside a predetermined lateral position in a lateral direction of the composite image is extracted from the first image, and an image of an area of ​​the composite image that is inside the predetermined lateral position in the lateral direction is extracted from the second image; The image processing device according to claim 1, wherein the predetermined horizontal positions are set for each vertical position in the composite image, and at least a portion of the predetermined horizontal positions is equal to a position of the contour line.

8. The image processing device according to claim 7 ; a display screen for displaying the composite image, generated by the image processing device and having the contour line superimposed thereon, in a passenger compartment of the vehicle; An image display device comprising:

Images