Image processing device

Abstract

To provide an image processing device which facilitates comparison of an own vehicle and another vehicle, and is capable of generating a comparison image while considering privacy.SOLUTION: An image processing device includes a processor. The processor extracts a profile of another vehicle from an image of the other vehicle, estimates a position and a posture of the other vehicle from the image of the other vehicle, generates a wireframe model of the other vehicle on the basis of the profile, the position and the posture of the other vehicle, and composes the wireframe model of the other vehicle to an image of the own vehicle.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an image processing apparatus.

Background Art

[0002] Patent Document 1 discloses a technique for creating a skeleton model using the indirect position of a human body from an image in which a person is photographed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, in practices such as rally and race, or driving lessons, the driving of one's own vehicle is compared with that of other vehicles. As one of the comparison methods at that time, for example, one's own vehicle and other vehicles are photographed with a fixed-point camera, and two videos are synthesized (overlaid). However, there is a problem that the area where the two vehicles overlap becomes difficult to see. In addition, since the other vehicle to be compared appears in the video, it is difficult to widely obtain comparison targets from the viewpoint of privacy.

[0005] The present disclosure has been made in view of the above, and an object thereof is to provide an image processing apparatus that makes it easier to compare one's own vehicle with other vehicles and can generate a comparison image considering privacy.

Means for Solving the Problems

[0006] The image processing apparatus according to the present disclosure includes a processor, and the processor extracts the contour of another vehicle from an image of the other vehicle, estimates the position and orientation of the other vehicle from the image of the other vehicle, generates a wireframe model of the other vehicle based on the contour, position, and orientation of the other vehicle, and synthesizes the wireframe model of the other vehicle with an image of the host vehicle.

Advantages of the Invention

[0007] According to the present disclosure, by converting an image of another vehicle into a wireframe model and synthesizing it with an image of the host vehicle, the image of the host vehicle is no longer blocked by the image of the other vehicle, making it easier to compare the host vehicle and the other vehicle. Further, by converting an image of another vehicle into a wireframe model, a comparison image considering privacy can be generated.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Embodiments for Carrying Out the Invention

[0009] The image processing apparatus according to an embodiment of the present disclosure will be described with reference to the drawings. Note that the components in the following embodiments include those that can be easily replaced by those skilled in the art or those that are substantially the same.

[0010] (Image Processing Apparatus) The configuration of the image processing apparatus according to the embodiment will be described with reference to FIG. 1. The image processing apparatus according to the embodiment is for synthesizing a plurality of images of a vehicle in motion. These images include still images and moving images, but hereinafter, the description will be made on the assumption of moving images. Further, hereinafter, an example in the case of synthesizing a moving image of the host vehicle and a moving image of another vehicle will be described.

[0011] The image processing apparatus according to the embodiment is realized by, for example, a general-purpose computer such as a workstation or a personal computer. Further, as shown in FIG. 1, the image processing apparatus 1 includes at least an arithmetic unit 11 and a storage unit 12.

[0012] Specifically, the arithmetic unit 11 includes a processor composed of a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), etc., and a memory (main storage unit) composed of a RAM (Random Access Memory), a ROM (Read Only Memory), etc.

[0013] For example, the arithmetic unit 11 loads a program stored in the storage unit 12 into the working area of the main storage unit and executes it, and controls each component through the execution of the program, thereby realizing a function that meets a predetermined purpose. The arithmetic unit 11 functions as a wireframe processing unit 111 and a video synthesis unit 112 through the execution of the above-described program.

[0014] The wireframe processing unit 111 converts another vehicle included in the video captured by the imaging device 2 into a wireframe model. This wireframe model is a representation of the outline (outer shape) of an object by lines.

[0015] Specifically, the wireframe processing unit 111 extracts the outline of another vehicle from the video of the other vehicle and estimates the position and orientation of the other vehicle from the video of the other vehicle. Then, based on the extracted outline of the other vehicle and the estimated position and orientation of the other vehicle, the wireframe processing unit 111 generates a wireframe model of the other vehicle. Note that the above processing performed by the wireframe processing unit 111 can be performed using known image processing techniques.

[0016] The video synthesis unit 112 synthesizes the wireframe model of the other vehicle generated by the wireframe processing unit 111 with the video of the host vehicle captured by the imaging device 2.

[0017] The storage unit 12 is composed of recording media such as, for example, an EPROM (Erasable Programmable ROM), a hard disk drive (HDD), and a removable medium. Examples of the removable medium include disk recording media such as a USB (Universal Serial Bus) memory, a CD (Compact Disc), a DVD (Digital Versatile Disc), and a BD (Blu-ray (registered trademark) Disc). In addition, an operating system (OS), various programs, various tables, various databases, etc. can be stored in the storage unit 12.

[0018] The storage unit 12 stores, as necessary, for example, images (still images, videos) captured by the imaging device 2, images of the wireframe model generated by the wireframe processing unit 111, images synthesized by the video synthesis unit 112, and the like.

[0019] The imaging device 2 is a camera incorporating an imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The imaging device 2 independently captures images of the host vehicle and other vehicles passing through a specific point and outputs the captured images to the image processing device 1.

[0020] The display device 3 is a display composed of, for example, an LCD (liquid crystal display), an OLED (organic EL display), or the like. This display device 3 displays, as necessary, images (still images, moving images) captured by the imaging device 2, images of wireframe models generated by the wireframe processing unit 111, images synthesized by the video synthesis unit 112, and the like.

[0021] (Image processing method) The flow of the image processing method executed by the image processing apparatus according to the embodiment will be described with reference to FIG. 2.

[0022] First, the imaging device 2 captures a moving image of the host vehicle (step S1). Subsequently, the imaging device 2 captures a moving image of another vehicle (step S2). Note that the order of the processes in steps S1 and S2 may be reversed or they may be performed simultaneously.

[0023] Subsequently, the wireframe processing unit 111 performs wireframe modeling on the moving image of the other vehicle (step S3). Details of the wireframe modeling will be described later (see FIGS. 3 and 4). Subsequently, the video synthesis unit 112 synthesizes the moving image of the other vehicle with the moving image of the host vehicle (step S4). In step S4, specifically, the video synthesis unit 112 synthesizes the wireframe model of the other vehicle with the moving image of the host vehicle. Subsequently, the video synthesis unit 112 outputs the synthesized (superimposed) moving image by the display device 3 (step S5), and this process is completed.

[0024] (Method for generating wireframe model 1) The flow of the method for generating the wireframe model 1 in step S3 of FIG. 2 will be described with reference to FIG. 3.

[0025] First, the wireframe processing unit 111 performs preprocessing on the video of the other vehicle (step S11). Subsequently, the wireframe processing unit 111 extracts the contour of the other vehicle from the video of the other vehicle (step S12). Subsequently, the wireframe processing unit 111 performs postprocessing on the video of the other vehicle (step S13).

[0026] Subsequently, the wireframe processing unit 111 determines whether or not the above steps S11 to S13 have been performed for all frames of the video of the other vehicle (step S14). In step S14, if it is determined that the above steps S11 to S13 have been performed for all frames (Yes in step S14), the wireframe processing unit 111 outputs a wireframe model (step S15) and completes this process. On the other hand, in step S14, if it is determined that the above steps S11 to S13 have not been performed for all frames (No in step S14), the wireframe processing unit 111 returns to the process of step S11.

[0027] As described above, in the wireframe model generation method 1, the wireframe model is directly generated from the video by image processing.

[0028] (Wireframe model generation method 2) The flow of the wireframe model generation method 2 in step S3 of FIG. 2 will be described with reference to FIG. 4.

[0029] First, the wireframe processing unit 111 performs preprocessing on the video of the other vehicle (step S21). Subsequently, the wireframe processing unit 111 extracts the contour of the other vehicle from the video of the other vehicle (step S22). Subsequently, the wireframe processing unit 111 estimates the position and orientation of the other vehicle from the video of the other vehicle (step S23). In the process of step S23, for example, a Kalman filter or the like may be used based on the orientation estimated in the past frames.

[0030] Subsequently, the wireframe processing unit 111 places a wireframe model that matches the vehicle position and orientation estimated in step S23 onto the video of the other vehicle (step S24). In the process of step S24, for example, a wireframe model is separately generated from CAD data of a vehicle whose vehicle type or shape is similar to that of the other vehicle. Then, a model obtained by changing the orientation, rotating, and deforming the wireframe model so as to match the position and orientation of the other vehicle in the video is used as the wireframe model of the other vehicle.

[0031] Subsequently, the wireframe processing unit 111 determines whether or not the above steps S21 to S24 have been performed for all frames of the video of the other vehicle (step S25). In step S25, if it is determined that the above steps S21 to S24 have been performed for all frames (Yes in step S25), the wireframe processing unit 111 outputs the wireframe model (step S26) and completes this process. On the other hand, in step S25, if it is determined that the above steps S21 to S24 have not been performed for all frames (No in step S25), the wireframe processing unit 111 returns to the process of step S21.

[0032] As described above, in the wireframe model generation method 2, a wireframe model generated from CAD data of a vehicle similar to the other vehicle is used as the wireframe model of the other vehicle. In this generation method, unlike the generation method 1, it is necessary to estimate the position and orientation of the other vehicle. However, for example, it is possible to suppress the instability of the wireframe model or the blurring of the lines due to the paint or stickers on the vehicle body, the reflection of the background, etc. Further, in this generation method, since the position and orientation of the other vehicle are estimated at all times, for example, blurring between frames can be improved by means such as a Kalman filter.

[0033] According to the image processing apparatus according to the embodiment described above, by converting the image of another vehicle into a wireframe model and synthesizing it with the image of the host vehicle, the image of the host vehicle is not blocked by the image of the other vehicle, so that it becomes easier to compare the host vehicle with the other vehicle. Further, by converting the image of the other vehicle into a wireframe model, a comparison image considering privacy can be generated.

[0034] Further effects and modifications can be easily derived by those skilled in the art. Therefore, a broader aspect of the present invention is not limited to the specific details and representative embodiments shown and described as above. Accordingly, various changes can be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and their equivalents.

Explanation of Reference Numerals

[0035] 1 Image processing apparatus 11 Arithmetic unit 111 Wireframe processing unit 112 Video synthesis unit 12 Storage unit 2 Photographing device 3 Display device

Claims

Claim 1 A processor is provided, wherein the processor extracts the contour of the other vehicle from an image of the other vehicle, estimates the position and orientation of the other vehicle from an image of the other vehicle, generates a wireframe model of the other vehicle based on the contour, position, and orientation of the other vehicle, and composes the wireframe model of the other vehicle with an image of the host vehicle. An image processing apparatus.

Images