IMAGE SYNTHESIS METHOD, COMPUTER PROGRAM, AND IMAGE SYNTHESIS DEVICE

Abstract

To provide an image composition method allowing a user to change a composition position and a depth distance on a plane with intuitive operations.SOLUTION: An image composition method for composing a second image on a first image has: a position detection step of detecting a specified position on a first image to compose the second image; a composition and display step of, based on information on the subject distance of the first image at the specified position and information on the size of a subject in the second image, changing the size of the subject in the second image to be a size corresponding to the subject distance of the first image at the specified position, and composing and displaying the images; a depth distance change detection step of detecting, on the specified position, depth distance change operations for arranging the second image at a depth distance different from the subject distance at the specified position; and a size change step of changing the size of the second image in response to the depth distance change operations.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an image synthesis method and the like. [Background technology]

[0002] Conventionally, there is a technique for compositing and arranging an object in a captured image. Patent Document 1 proposes a method of generating real coordinate space of the shooting space by acquiring the distance to the subject when shooting an image, and processing and displaying an object according to the specified real space coordinates. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6685814 Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, an object image is processed and synthesized according to the coordinates of a specified synthesis position in a real coordinate space generated from a photographed image and distance information. The synthesis position is specified on a two-dimensional image in which three-dimensional coordinates are projected onto a two-dimensional photographed image, and the Z coordinate (depth position) is determined at the time when the XY coordinates (planar position) are specified. Therefore, it is not possible to synthesize an object image by arbitrarily changing the depth position (Z coordinate) while keeping the planar position (XY coordinates) on the photographed image fixed.

[0005] In view of the above problems, the present invention aims to provide an image synthesis method that solves the above problems and allows the synthesis position and depth distance on a plane to be changed by an intuitive operation. [Means for solving the problem]

[0006] In order to achieve the above object, the image synthesis method of the present invention comprises the steps of: 1. An image synthesis method for synthesizing a second image on a first image, comprising the steps of: a position detection step of detecting a designated position on the first image for synthesizing the second image; a composite display process for changing a size of the subject of the second image based on subject distance information of the first image at the specified position and subject size information of the second image so that the size corresponds to the subject distance of the first image at the specified position, and displaying the composite image; a depth distance change detection step of detecting a depth distance change operation for disposing the second image at a depth distance different from the subject distance at the specified position on the specified position; a size changing step of changing a size of the second image in response to the depth distance changing operation; a depth distance information display step of displaying changed depth distance information in response to the depth distance change operation; Have death, In the depth distance information display step, a region of the first image corresponding to the depth distance is highlighted. The present invention is characterized by the above. Effect of the Invention

[0007] According to the present invention, it is possible to realize an image synthesis method that allows the synthesis position and depth distance on a plane to be changed by an intuitive operation. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a main functional configuration for executing an image synthesis method according to an embodiment of the present invention; [Diagram 2] 4 is a flowchart illustrating a process executed in the embodiment. [Diagram 3] FIG. 11 is a diagram showing a display method relating to a first image selection executed in the embodiment. [Figure 4] FIG. 11 is a diagram showing a display method relating to a second image selection executed in the embodiment. [Diagram 5] 13A to 13C are diagrams illustrating subject extraction related to the second image selection executed in the embodiment. [Figure 6] FIG. 13 is a diagram for explaining the position of a reference point on a second image in the embodiment. [Figure 7] 11A to 11C are diagrams for explaining a method of synthesizing a first image by changing the size of the second image according to the subject distance of the first image in an embodiment. [Figure 8] 13A to 13C are diagrams illustrating an example of a change in size of a second image to be synthesized when the depth distance is changed, and a method of displaying the depth distance to be synthesized. [Figure 9] 13A and 13B are diagrams illustrating another method of displaying the depth distance synthesized in the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In each drawing, the same members or elements are designated by the same reference numerals, and duplicated descriptions will be omitted or simplified.

[0010] In the embodiment described below, an example will be described in which the image synthesis method of the present embodiment is executed using a computer as an image processing device. That is, the present embodiment can be executed in a desktop computer, a laptop computer, a tablet computer, a smartphone, or the like.

[0011] The image synthesis method of this embodiment acquires a photographed image and distance information (distance map information) corresponding to the photographed image from an imaging device such as a digital camera, and enables intuitive synthesis of another image at a desired planar position and depth position (depth distance) of the photographed image. In addition, at that time, an image of an appropriate size according to the depth position (depth distance) is synthesized.

[0012] The configuration of a computer as an image processing device that executes the image compositing method of this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the main functional configuration for executing the image compositing method of this embodiment, and does not show peripheral components. 1 has a user interface for acquiring operation input by a user, such as a mouse, a keyboard, a joystick, a touch panel display, etc. The input unit 10 also has an interface (I / F) for acquiring images and distance information (distance map information) from a digital camera or other external device.

[0013] The control unit 11 controls data in the input unit 10, the calculation unit 12, the storage unit 13, and the output unit 14, and controls each unit of the entire image processing device 100. That is, the control unit 11 has a built-in CPU as a computer, and controls each unit of the entire image processing device 100 via a bus based on a computer program stored in a non-volatile memory. The calculation unit 12 has a calculation processing device and executes a calculation processing program stored in the storage unit 13, for example.

[0014] The storage unit 13 is composed of a primary storage device such as a semiconductor memory and a secondary storage device such as a hard disk. The storage unit 13 stores an arithmetic processing program as a computer program, and also stores various information such as image data, distance information, subject size information, parameters, etc. input via the input unit 10. The storage unit 13 also stores the image synthesized by the arithmetic unit 12.

[0015] The output unit 14 includes a display as a display unit, and an I / F for outputting images and the like to the outside. The display of the output unit 14 in this embodiment is, for example, a touch panel display that also has a user interface function.

[0016] Next, FIG. 2 is a flow chart for explaining the processing executed in this embodiment, and the image synthesis processing of this embodiment will be explained using the flow chart of FIG. The processing shown in the flowchart of FIG. 2 is realized by the control unit 11 reading out a computer program stored in, for example, a non-volatile memory in the storage unit 13, expanding it into a volatile memory in the control unit 11, and executing it, thereby causing each unit to operate.

[0017] In step S201, the control unit 11 displays a plurality of images stored in the storage unit 13 as candidates for a first image (background image) on the touch panel display of the output unit 14. Note that Fig. 3 is a diagram showing a display method related to the first image selection executed in this embodiment, and the display is a thumbnail display as shown in Fig. 3, or a single image is displayed so that the displayed image can be changed by a user operation. When the user performs a tap operation on the display area of ​​a desired image on the displayed images via the input unit 10, a first image (background image) is selected.

[0018] In this embodiment, the images stored in memory unit 13 are images to which subject distance information (subject distance map information) has been added, and are images input to image processing device 100 from an imaging device (camera) and distance measuring unit (not shown) via the interface of input unit 10. Alternatively, the images are images captured by an external imaging device and to which subject distance information (subject distance map information) has been added, which are temporarily stored in an external device, and then acquired via the interface of input unit 10 and stored in memory unit 13. Here, subject distance information (subject distance map information) is, for example, information on subject distance recorded for each predetermined pixel.

[0019] In step S202, control unit 11 displays the first image (background image) selected in step S201 in a part (area 401) of the touch panel display of output unit 14, as shown in FIG. FIG. 4 is a diagram showing a display method relating to the second image selection executed in the embodiment. In an area 402 of the touch panel display different from an area 401 displaying the first image, a plurality of candidate images to be combined with the first image are displayed so that the user can select them. Note that although thumbnail display is shown as an example in FIG. 4, other display methods may be used.

[0020] Here, the first image is selected in step S201, and the first image is displayed in step S202 along with multiple images to be combined. However, the order is not necessarily limited to this. Multiple images to be combined can also be displayed in area 402 in advance.

[0021] It is also possible to store a plurality of first image candidates in a specific storage area, display one of the images in area 401, and change to another first image by swiping area 401. Alternatively, it is also possible to display, for example, a left-pointing arrow and a right-pointing arrow on the left and right periphery of area 401, and to sequentially display other images in the storage area by tapping on either arrow.

[0022] With this configuration, it is possible to easily change the first image used as the background image without using complex menu selection or the like. In step S203, the user operates the touch panel display of input unit 10 and selects an image to be combined with the first image (background image) from area 402 by tapping or touching. The selected image to be combined is set as the second image. That is, the image displayed at the coordinates on the touch panel display touched by the user's finger becomes the selected second image.

[0023] The control unit 11 acquires the selected second image and meta information associated with the second image. The meta information is saved as tag data saved in Exif format or maker note information. The meta information acquired in step S203 includes information related to the size of the subject present in the second image (size information), etc.

[0024] The subject size information includes area information (coordinate information) of the subject recognized in the second image and information on its size. The subject size information may include information on how many mm one pixel of the image corresponds to in the object space, distance information to the object (subject) recognized by face recognition or object recognition, focal length information when the subject is photographed, or other information that allows the size of the subject to be calculated. Alternatively, the subject size information may include information on a map of layers when the image is divided into layers according to the distance to the subject.

[0025] In the case of an image to which object recognition information or distance layer information is attached as meta information, it is also possible to extract a specific subject or a specific layer from the image displayed at the coordinates on the touch panel display where the user's finger touches and use it as a second image. 5A and 5B are diagrams for explaining subject extraction related to the second image selection executed in the embodiment. For example, as shown in FIG. 5A, two people 502 and 503 are shown in an image 501, and when the area of ​​the person 502 is touched and selected, only the subject area is cut out as shown in FIG. 5B as 502' and used as the second image. In this way, the second image may be a predetermined subject image extracted from a predetermined image.

[0026] When the second image is selected, the control unit 11 sets a point in the selected second image that will be used as a reference for image synthesis. Fig. 6 is a diagram for explaining the position of a reference point on a second image according to an embodiment. For example, when the second image is rectangular, the reference point is set to any one of the vertices of the upper right 601, lower right 602, upper left 603, and lower left 604, any one of the midpoints of each side (605 to 608), or the center point 609 of the second image, as shown in Fig. 6(A). Alternatively, the reference point may be set to the coordinates 610 of the contact position on the second image when the user selects the second image by a touch operation.

[0027] In particular, since the subject is often in contact with the ground or the like, it is preferable to set the reference point at the midpoint 607 of the lower side. If the second image has the shape of a specific object in the selected image, one of the topmost end 611, bottommost end 612, rightmost end 613, or leftmost end 614 of the specific object is selected, as shown in FIG. 6(B).

[0028] Alternatively, the reference point is set to any one of the top end 615 of the midpoint of the shape width of the specific subject, the bottom end 616 of the midpoint of the shape width, the right end 617 of the midpoint of the shape height, the left end 618 of the midpoint of the shape height, and the midpoints 619 of both the shape height and the shape width. It is particularly preferable to set the reference point to the bottom end 616 of the midpoint of the shape width. Thus, step S203 includes a reference selection step of selecting a reference position in the second image while displaying the reference position of the second image.

[0029] In step S204, the user drags the second image selected by touch operation in step S203 on the touch panel display, for example, and drops it by releasing the finger on the first image displayed on the touch panel display. In this case, the position of the finger on the dropped first image (background image), i.e., the position where the finger is released, is set as the synthesis position (designated position) on the plane where the second image is synthesized.

[0030] Here, step S204 functions as a position detection process (position detection means) that detects a designated position on the first image for synthesizing the second image, and detects the position where the user drags and then drops the second image on the touch panel display. The control unit 11 acquires coordinates (composite position coordinates) on the first image (background image) corresponding to the coordinates of the composite position on the touch panel display. That is, the control unit 11 acquires distance information of the subject in the first image corresponding to the composite position coordinates from the meta information of the first image.

[0031] Note that instead of drag-and-drop, in step S203, the second image may be selected, for example, by a tap operation, and then the finger may be released, and the combination position may be specified by touching (tapping) the touch panel display on which the first image is displayed. In addition, when specifying the synthesis position, if the position touched (tapped) on the touch panel display is set as the synthesis position, it may be difficult to finely adjust the synthesis position. Therefore, in order to make the synthesis position clearly identifiable, a position that is shifted from the contact position and does not overlap with the user's finger may be highlighted as the synthesis position.

[0032] In step S205, control unit 11 transmits information on the subject distance of the first image at the synthesis position coordinates on the acquired first image (background image) and information on the size of the second image to calculation unit 12. Based on the input information, calculation unit 12 calculates size information when the second image is placed at the subject distance of the synthesis position coordinates on the first image.

[0033] That is, based on the subject distance D1 at the synthesis position coordinates on the first image, the focal length f1 when the first image was captured, and the size of one pixel in the first image (image side pixel size) s1, S1 is calculated using the following equation 1. Here, S1 is the size of one pixel in the first image at the distance of the subject in the synthesis position coordinates (object-side pixel size). S1 = D1s1 / f1 (1)

[0034] In addition, based on the size information of the subject in the second image, a converted pixel number PN1 is calculated by converting the size Os of the subject in the second image (image to be synthesized) into how many pixels it corresponds to using the object-side pixel size S1. Next, in step S206, calculation unit 12 calculates a scaling ratio R of the second image using the ratio between the number of pixels PN2 of the subject in the second image and the converted number of pixels PN1 calculated in step S205, and enlarges / reduces the second image based on the calculated scaling ratio R. Here, the reference position for image synthesis in the second image set in step S203 is reset to match the enlarged / reduced second image.

[0035] In step S207, the control unit 11 performs synthesis so that the reference point of the second image after enlargement / reduction calculated in step S206 coincides with the synthesis position coordinates on the first image specified in step S205. That is, synthesis is performed by matching the reference position selected in step S203 with the specified position on the first image. The compositing is performed by replacing the first image in the area where the scaled second image is to be composited with the scaled second image.

[0036] However, in the area of ​​the first image into which the second image is to be composited, the area that is at the subject distance in front of the composite position of the second image is not replaced with the second image, but the first image is displayed as is. That is, in the area of ​​the first image into which the second image is composited, the second image is displayed in the area at the depth distance of the specified position, and the first image is displayed in the area in front of the depth distance of the specified position. This enables composition that takes into account the depth relationship in front of and behind the subject.

[0037] By such processing, when the subject of the second image is placed at a position on the first image at a depth distance from the imaging device (camera) to the subject in the synthesis position coordinates, the size of the subject of the second image is changed to the size observed from the imaging device and synthesized. That is, when synthesizing the subject of the second image, if the depth distance (subject distance) of the first image at the synthesis position is close, the subject of the second image is synthesized in a large size on the first image. On the other hand, if the depth distance (subject distance) of the first image at the synthesis position is far, the subject of the second image is synthesized in a small size.

[0038] In this way, step S207 functions as a composite display step (composite display means). Also, based on the subject distance information of the first image at the specified position and the subject size information of the second image, the size of the subject of the second image is changed so as to correspond to the subject distance of the first image at the specified position, and the composite display is performed.

[0039] FIG. 7 is a diagram for explaining a method for changing the size of a second image in accordance with the subject distance of a first image and synthesizing the first image in the embodiment. As shown in Fig. 7, the selected second image 701 is resized and synthesized according to the depth distance (subject distance) of a synthesis position 702 on the first image. 703 in Fig. 7 is the second image 701 that has been resized and synthesized.

[0040] The processes from step S204 to step S207 may be executed after the user completes the drag-and-drop operation, or the processes from step S204 to step S207 may be executed periodically while the user is performing the drag operation. In other words, the position detection step and the composition display step may be executed periodically while the user is moving the designated position.

[0041] By executing periodically, the size of the second image can be changed according to the distance on the first image, and the user can change the drop position while checking the size of the second image. Also, since the composite position on the first image and the reference point of the second image are always displayed to match, it is possible to determine the placement (composite position) while recognizing these positions.

[0042] Next, in step S208, the control unit 11 determines whether or not to change the depth distance (depth position, subject distance) at which the second image arranged in step S207 is synthesized in the first image. When determining whether to change the depth distance, the synthesis position is determined, for example, when the user drags the second image onto the first image and releases his / her finger at the desired synthesis position coordinates.

[0043] At that time, if it is detected that the finger is released after touching the synthesis position coordinates for a period longer than the predetermined time, or that the finger is released after pressing the synthesis position coordinates with a pressure of a predetermined value or more, it is determined that the depth distance (subject distance) is changed. Alternatively, if the second image is dragged to the desired synthesis position coordinates and the display is touched with another finger while still being touched, that is, if two or more touches are detected, it may be determined that the depth distance is changed.

[0044] The method for determining whether the depth distance is changed is not limited to the above method, and other methods may be used. Here, step S208 functions as a depth distance change detection process (depth distance change detection means) that detects a depth distance change operation for placing the second image at a depth distance different from the subject distance at the specified position on the specified position. As described above, in this embodiment, the depth distance change detection process detects that a depth distance change operation has been performed by the time or pressure that the user touches the touch panel display or by touching multiple points.

[0045] If it is determined in step S208 that the depth distance for placement is to be changed, the process proceeds to step S209, and if it is determined that the depth distance is not to be changed, the process proceeds to step S210. In step S209, the control unit 11 changes the depth distance to be placed at the synthesis position coordinates, for example, depending on the touch time or touch pressure, and changes the size of the second image depending on the changed depth distance, and displays it synthesized on the first synthesis position coordinates.

[0046] That is, step S209 functions as a size changing step (size changing means) for changing the size of the second image in response to the depth distance changing operation. Also, it functions as a depth distance information display step (depth distance information display means) for displaying changed depth distance information in response to the depth distance changing operation. In this embodiment, in the depth distance change detection step, the depth distance is changed in response to the time or pressure with which the user touches the touch panel display or a pinch operation.

[0047] The change in the depth distance may be set so as to gradually change the distance away from the object, or so as to gradually change the distance toward the object. Note that the direction of change may be selectively set in advance. Furthermore, the size of the second image to be combined with the depth distance may be changed at a predetermined touch time interval and displayed, and when the set maximum or minimum distance is reached, the depth distance may be gradually changed toward the minimum or maximum distance.

[0048] The change in depth distance may be made according to the touch pressure other than the touch time. The change interval of the depth distance may be equal or may be changed according to the distance. In this case, it is preferable to make the change interval of the depth distance wider as the distance becomes longer.

[0049] In addition, if the distance is changed when two or more touch points are detected in step S208, a pinch operation (operation to change the distance between two touch points) may be detected and the depth distance may be changed in accordance with the pinch operation. In a pinch operation, for example, when the distance between two touch points is increased, the depth distance is made closer, and when the distance between two touch points is decreased, the depth distance is made farther. In this case, the size of only the second image, which is a specific subject, is changed, and the corresponding depth distance is displayed on the screen, for example, as a number, which makes it easier to synthesize the three-dimensional position, unlike conventional zooming in and out using a pinch operation.

[0050] In order to allow the user to visually recognize the changed depth distance, it is preferable to highlight the subject in the first image that is at the same depth distance as the changed depth distance. In this case, it is preferable to highlight the image area at the corresponding depth distance in a specific color or with a different brightness.

[0051] FIG. 8 is a diagram for explaining a change in size of the second image to be synthesized when the depth distance is changed, and an example of a method of displaying the depth distance to be synthesized. For example, as shown in FIG. 8(A), when the size of the synthesized second image 801a is changed to be larger than the state in FIG. 7, the area indicated by the dotted line 802a in the first image is highlighted as the area of ​​the corresponding depth distance.

[0052] In this way, by highlighting the area of ​​the first image corresponding to the depth distance, it is easy to visually recognize that the size of the changed second image is the size when it is placed at the same depth distance as the highlighted area of ​​the first image. Also, as shown in Fig. 8(B), when the depth distance at which the second image is synthesized is brought closer, the size of the synthesized second image 801b changes, and the dotted line 802b indicating the subject position at the same distance as the depth distance in the first image is changed accordingly. Note that in Fig. 8(B), the depth distance at which the second image is synthesized is brought closer to the imaging device (camera) compared to Fig. 8(A), so it is displayed larger and synthesized.

[0053] As a method of displaying the depth distance to be synthesized, a numerical value of the depth distance to be synthesized can be displayed in any area on the touch panel display, as in 803a or 803b in FIG.

[0054] FIG. 9 is a diagram for explaining another method of displaying the depth distance synthesized in the embodiment. 9, distance bars 901a and 901b are displayed, indicating the maximum and minimum depth distances that can be arranged. By displaying the depth distance as a bar in this way, it is possible to show the user the currently set depth distance for composition by displaying the position of the depth distance for composition. FIG. 9(A) shows an example in which the second image is placed at the same distance as in FIG. 8(A), and FIG. 9(B) shows an example in which the second image is placed at the same distance as in FIG. 8(B).

[0055] The first image may be converted into a luminance signal and a pseudo color corresponding to the depth distance may be superimposed on the image, and the distance bars 901a and 901b may be displayed as similar pseudo color bars to clearly show the distance. In addition, after changing the depth distance for synthesis, it is necessary to provide an operation for returning the size of the synthesized second image to the size at the distance on the synthesis position coordinates on the first image.

[0056] For example, after changing the depth distance, by detecting a double tap operation in the synthesized second image area, the depth distance to be synthesized may be reset to the depth distance of the first image on the synthesis coordinates, and the size of the second image may be changed to match the depth distance of the first image on the synthesis coordinates.

[0057] In step S210, the control unit 11 judges whether to move the coordinate positions in the XY directions on the first image with respect to the second image. If it is judged that the movement is to be executed, the movement is executed without changing the depth distance (depth position) to be synthesized, that is, without changing the size of the second image after the depth distance to be synthesized is changed. For example, if two points in the synthesized second image area are touched simultaneously, it is judged that this is an operation for moving the second image, and the second image is moved by moving the two points simultaneously. At that time, the movement is performed while maintaining the relationship between the reference point and the two selected points. After the movement, the process returns to step S204, and the processes from step S204 onwards are repeated.

[0058] In step S211, the control unit 11 determines whether the user has touched the area 402 of the image displayed on the touch panel display to select a composite image, and determines whether or not to add another image to the first image. If it is determined that additional composition is to be performed, the process returns to step S203 and executes the processes from step S203 onward. If additional composition is not to be performed, the process waits for an input from the user until the generation of the composite image is completed.

[0059] When deleting the composite second image from the first image, two points on the second image composited with the first image are touched and dragged out of the first image area. Alternatively, when the second image composited with the first image is touched and then moved at a high speed of a predetermined speed or faster, it may be determined as a delete operation and deleted. As described above, according to this embodiment, by using the shooting information and the like attached to the captured image, it becomes possible to more freely synthesize the size of a subject existing at a desired position in the image.

[0060] The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the gist of the present invention, and these modifications are not excluded from the scope of the present invention. A computer program for implementing all or part of the control in this embodiment may be provided to the image processing device via a network or various storage media. The computer (or CPU, MPU, etc.) in the image processing device may then read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0061] 100 Computers 10 Input section 11 Control section 12 Arithmetic section 13 Storage section 14 Output section

Claims

1. 1. An image synthesis method for synthesizing a second image on a first image, comprising the steps of: a position detection step of detecting a designated position on the first image for synthesizing the second image; a composite display step of changing a size of the subject of the second image based on subject distance information of the first image at the specified position and subject size information of the second image so that the size corresponds to the subject distance of the first image at the specified position, and displaying the composite image; a depth distance change detection step of detecting a depth distance change operation for disposing the second image at a depth distance different from the subject distance at the specified position on the specified position; a size changing step of changing a size of the second image in response to the depth distance change operation; a depth distance information display step of displaying changed depth distance information in response to the depth distance change operation; having The image synthesis method according to claim 1, wherein in said depth distance information display step, a region of said first image corresponding to said depth distance is highlighted.

2. 2. The image synthesis method according to claim 1, wherein said depth distance information display step displays a numerical value of said depth distance.

3. 2. The image synthesis method according to claim 1, wherein the depth distance is displayed as a bar in the depth distance information display step.

4. 2. The image synthesis method according to claim 1, wherein, in the synthesis display step, the second image is displayed in an area of ​​the first image into which the second image is synthesized, the area being at the depth distance from the specified position, and the first image is displayed in an area in front of the depth distance from the specified position.

5. 2. The image synthesis method according to claim 1, wherein the depth distance change detection step detects that the depth distance change operation has been performed based on the duration or pressure of the user's touch on the touch panel display, or the user's touch on multiple points.

6. 2. The image synthesis method according to claim 1, wherein in the depth distance change detection step, the depth distance is changed in accordance with a time or pressure with which the user touches the touch panel display, or a pinch operation.

7. 2. The image synthesis method according to claim 1, further comprising a reference selection step of selecting a reference position in the second image, and displaying the synthesized image by aligning the selected reference position with the specified position on the first image.

8. 8. The image synthesis method according to claim 7, wherein in the reference selection step, the reference position of the second image is displayed.

9. 2. The image synthesis method according to claim 1, wherein the second image is a predetermined subject image extracted from a predetermined image.

10. 2. The image synthesis method according to claim 1, wherein the position detection step and the synthesis display step are performed periodically while the user is moving the designated position.

11. 2. The image synthesis method according to claim 1, wherein the position detection step detects a position where the user drags and drops the second image on a touch panel display.

12. A computer program for controlling each step of the image synthesis method according to any one of claims 1 to 11 by a computer.

13. An image synthesis device for synthesizing a second image on a first image, comprising: a position detection means for detecting a designated position on the first image for combining the second image; a composite display means for changing a size of the subject of the second image based on subject distance information of the first image at the specified position and subject size information of the second image so that the size corresponds to the subject distance of the first image at the specified position, and displaying the composite image; a depth distance change detection means for detecting a depth distance change operation for disposing the second image at a depth distance different from the subject distance at the specified position on the specified position; a size changing means for changing a size of the second image in response to the depth distance changing operation; a depth distance information display means for displaying changed depth distance information in response to the depth distance change operation, an area of ​​the first image corresponding to the depth distance is highlighted by the depth distance information display means;

Images