Image processing program, image processing apparatus, and image processing method

Abstract

To perform image processing to replace a part of a captured image with an object.SOLUTION: An image processing program acquires a captured image, identifies a subject in the captured image, determines one of a plurality of unit areas obtained by dividing the captured ranged, in which the identified subject is located, sets at least one of the unit areas in which the subject is determined to be positioned as a target region, determine a region to be replaced from among the regions other than the target region, acquires a replacement image including an object extracted from an object arrangement image obtained by arranging an object in advance in a replacement area and photographing the object, and pixels surrounding the object, generates a clipped image from which pixel data of a portion placed with the replacement image is deleted in each of the replacement areas, obtains a background image, and superimposes and synthesizes the background image, the clipped image, and the replacement image. Instead of a process of replacing with the replacement image, the subject in the replacement area may be erased.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for replacing a part of a photographed image with an object.

Background Art

[0002] Conventionally, there has been a business model in which a photographed picture taken at a preset fixed shooting location is presented to a customer who is the subject, and if the customer likes it, they are asked to purchase it. As an example, a photographed picture automatically taken inside an attraction of an amusement facility is presented to the customers who were on board after the attraction ends, and the photographed picture is sold to those who wish. It is also conceivable to perform some kind of image processing on such a photographed picture before selling it.

[0003] For example, Patent Document 1 discloses a technique for detecting a plurality of types of objects from image data and switching image processing according to the type of the detected object and the detected region.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] By the way, in the above-described business model of photo sales, it is conceivable to photograph a plurality of groups simultaneously in one shooting. For example, in a situation where photo shooting is performed so that a plurality of riding locations are included in the shooting range at the same time in an attraction having a plurality of riding locations, photo shooting may be performed with different groups boarding at the plurality of riding locations. Whether or not to purchase the photographed photo is up to each customer, but there is a problem that the purchase rate of the photo decreases because there are customers who do not want to purchase a photo in which others are reflected.

[0006] The present invention has been made in view of the above problems, and an object thereof is to provide an image processing program, an image processing apparatus, and an image processing method capable of replacing a part of a photographed image with an object.

Means for Solving the Problems

[0007] The image processing program according to the present invention is an image processing program for causing a computer to realize a process of erasing a part of a photographed image. The computer is caused to: acquire a photographed image generated by performing photographing at a predetermined timing for a photographing range determined by a predetermined photographing position and a photographing angle, which is a photographing image acquisition function; specify a subject shown in the photographed image, which is a subject specification function; divide the photographing range into a plurality of preset unit regions, and determine in which location of the plurality of unit regions the specified subject is located, which is a subject position determination function; focus on at least one of the unit regions (hereinafter referred to as a subject region) determined to have the subject located therein (hereinafter referred to as a focused region), and determine a subject region (hereinafter referred to as a subject erasure region) to be a target for erasing the subject based on a predetermined rule from among the subject regions other than the focused region, which is a subject erasure region determination function; generate a clipped image in which pixel data of the subject is deleted in each of the subject erasure regions, which is a clipped image generation function; acquire a background image that is photographed and stored in a storage means with only a background where no subject appears at the same photographing position and photographing angle as the photographed image, which is a background image acquisition function; and generate an erased photographed image in which the subject in the subject erasure region is erased by overlapping and synthesizing the background image and the clipped image, which is an erased photographed image generation function.

[0008] Further, when a moving object on which a subject can board is a shooting target, the image processing program according to the present invention further causes the computer to have a moving object position specifying function for specifying the position of the moving object shown in the captured image, and a moving object image acquisition function for acquiring a moving object image obtained by shooting a moving object without a subject on board at the same shooting position and shooting angle as the captured image, extracting only the moving object, and storing it in a storage means. In the subject position determination function, the boarding possible locations of the subject on the moving object are divided into a plurality of unit areas. In the background image acquisition function, a background image in a state where the moving object does not exist is acquired. In the generated captured image generation function, the erased captured image is generated by overlapping and synthesizing the background image, the moving object image, and the cut-out image using the position information of the specified moving object.

[0009] The image processing program according to the present invention is an image processing program for causing a computer to realize a process of replacing a part of a photographed image with an object. The computer is caused to: acquire a photographed image generated by performing photographing at a predetermined timing for a photographing range determined by a predetermined photographing position and a photographing angle, which is a photographing image acquisition function; specify a subject shown in the photographed image, which is a subject specification function; divide the photographing range into a plurality of preset unit areas, and determine in which location of the plurality of unit areas the specified subject is located, which is a subject position determination function; focus on at least one of the unit areas (hereinafter referred to as a subject area) determined to have the subject located therein (hereinafter referred to as a focused area), and determine a subject area and / or a unit area to be a replacement target (hereinafter referred to as a replacement area) based on a predetermined rule from among the subject areas other than the focused area and / or the unit areas, which is a replacement area determination function; acquire a replacement image for storage by extracting an image of a predetermined range including the object and its peripheral pixels from an object arrangement image obtained by arranging an object in advance at the location of the replacement area and performing photographing at the same photographing position and photographing angle as the photographed image, which is a replacement image acquisition function; generate a cut-out image by deleting pixel data of the location to be replaced with the replacement image in each of the replacement areas, which is a cut-out image generation function; acquire a background image by photographing only the background where no subject appears at the same photographing position and photographing angle as the photographed image and storing it in a storage means, which is a background image acquisition function; and generate a photographed image after replacement in which the object is replaced at the position of the replacement target by overlapping and synthesizing the background image, the cut-out image, and the replacement image, which is a photographed image after replacement generation function.

[0010] Further, when the moving object on which the subject can board is the imaging target, the image processing program according to the present invention further causes the computer to have a moving object position specifying function for specifying the position of the moving object shown in the captured image, and a moving object image acquisition function for acquiring a moving object image in which only the moving object is extracted by capturing a moving object without a subject on board at the same imaging position and imaging angle as the captured image and storing it in the storage means. In the subject position determination function, the mountable locations of the subject on the moving object are divided into a plurality of unit areas. In the background image acquisition function, a background image in a state where the moving object does not exist is acquired. In the generated captured image generation function, the replaced captured image is generated by overlapping and synthesizing the background image, the moving object image, the cut-out image, and the replacement image using the position information of the specified moving object.

[0011] Further, in the replacement image acquisition function of the image processing program according to the present invention, an image in a predetermined range including the object and the characteristic constituent parts for alignment existing around the object is extracted from the object arrangement image as the replacement image. In the generated captured image generation function, alignment is performed using the characteristic constituent parts for alignment and they are overlapped and synthesized.

[0012] Further, in the moving object position specifying function of the image processing program according to the present invention, a plurality of characteristic configurations in the appearance of the moving object are selected in advance, and the position of the moving object is specified by specifying the positions at which the plurality of characteristic configurations exist in the captured image.

[0013] In addition, in the image processing program according to the present invention, in the subject identification function, pixels corresponding to a subject are identified in a distinguishable manner for each individual based on a learned model that has been pre-trained for subject identification. In the subject position determination function, for each subject identified in the subject identification function, a minimum rectangular region that encompasses all the pixels identified as the same individual is set, and based on which of the unit regions the center point of the minimum rectangular region belongs to, it is determined at which location in any of the plurality of unit regions the identified subject is located.

[0014] The image processing apparatus according to the present invention is an image processing apparatus for performing a process of erasing a part of a captured image. The image processing apparatus includes: a captured image acquisition unit that acquires a captured image generated by performing a capture at a predetermined timing for a capture range determined by a predetermined capture position and a capture angle; a subject identification unit that identifies a subject appearing in the captured image; a subject position determination unit that divides the capture range into a plurality of preset unit regions and determines at which location in any of the plurality of unit regions the identified subject is located; a subject erasure region determination unit that focuses on at least one of the unit regions (hereinafter referred to as the focused region) determined to have a subject located therein (hereinafter referred to as the subject region), and determines a subject erasure region, which is a subject region to be erased based on a predetermined rule, from among the subject regions other than the focused region; a clipped image generation unit that generates a clipped image by deleting the pixel data of the subject in each of the subject erasure regions; a background image acquisition unit that acquires a background image that has been captured and stored in a storage means by capturing only the background where no subject appears at the same capture position and capture angle as the captured image; and a captured image with erasure generation unit that generates a captured image with erasure in which the subject in the subject erasure region is erased by overlapping and synthesizing the background image and the clipped image.

[0015] The image processing apparatus according to the present invention is an image processing apparatus for performing a process of replacing a part of a photographed image with an object. The image processing apparatus includes: a photographed image acquisition unit that acquires a photographed image generated by performing photographing at a predetermined timing with respect to a photographing range determined by a predetermined photographing position and a photographing angle; a subject identification unit that identifies a subject appearing in the photographed image; a subject position determination unit that divides the photographing range into a plurality of preset unit regions and determines in which location of the plurality of unit regions the identified subject is located; a replacement region determination unit that focuses on at least one of the unit regions (hereinafter referred to as a subject region) determined to have the subject located therein (hereinafter referred to as a focused region), and determines a subject region and / or a unit region (hereinafter referred to as a replacement region) to be a replacement target based on a predetermined rule from among the subject regions other than the focused region and / or the unit regions; a replacement image acquisition unit that acquires a replacement image obtained by extracting an image of a predetermined range including the object and its peripheral pixels from an object arrangement image obtained by arranging an object in advance at the location of the replacement region and performing photographing at the same photographing position and photographing angle as the photographed image and storing it in a storage means; a clipped image generation unit that generates a clipped image obtained by deleting pixel data at the location to be replaced with the replacement image in each of the replacement regions; a background image acquisition unit that acquires a background image obtained by photographing only a background in which no subject appears at the same photographing position and photographing angle as the photographed image and storing it in a storage means; and a replaced photographed image generation unit that generates a replaced photographed image in which the object is replaced at the position of the replacement target by overlapping and synthesizing the background image, the clipped image, and the replacement image.

[0016] The image processing method according to the present invention is an image processing method for realizing, by a computer, a process of erasing a part of a captured image. The method includes: a captured image acquisition procedure for acquiring a captured image generated by performing shooting at a predetermined timing with respect to a shooting range determined by a predetermined shooting position and a shooting angle; a subject identification procedure for identifying a subject shown in the captured image; a subject position determination procedure for dividing the shooting range into a plurality of preset unit areas and determining in which location of the plurality of unit areas the identified subject is located; a subject erasing area determination procedure for focusing on at least one of the unit areas (hereinafter referred to as the subject area) determined to have the subject located therein (hereinafter referred to as the focused area), and determining, based on a predetermined rule, a subject area (hereinafter referred to as the subject erasing area) to be the target of erasing the subject from among the subject areas other than the focused area; a clipped image generation procedure for generating a clipped image in which pixel data of the subject is deleted in each of the subject erasing areas; a background image acquisition procedure for acquiring a background image that is captured and stored in a storage means with only the background where the subject does not appear at the same shooting position and shooting angle as the captured image; and a generated captured image generation procedure for generating an erased captured image in which the subject in the subject erasing area is erased by overlapping and synthesizing the background image and the clipped image.

[0017] The image processing method according to the present invention is an image processing method for realizing, by a computer, a process of replacing a part of a photographed image with an object. The method includes: a photographed image acquisition procedure for acquiring a photographed image generated by performing photographing at a predetermined timing with respect to a photographing range determined by a predetermined photographing position and a photographing angle; a subject identification procedure for identifying a subject appearing in the photographed image; a subject position determination procedure for dividing the photographing range into a plurality of preset unit areas and determining in which location of the plurality of unit areas the identified subject is located; a replacement area determination procedure for focusing on at least one of the unit areas (hereinafter referred to as the subject area) determined to have a subject located therein (hereinafter referred to as the focused area), and determining, based on a predetermined rule, the subject area and / or the unit area to be a replacement target (hereinafter referred to as the replacement area) from among the subject areas other than the focused area and / or the unit areas; a replacement image acquisition procedure for acquiring, from an object arrangement image obtained by arranging an object in advance at the location of the replacement area and performing photographing at the same photographing position and photographing angle as the photographed image, an image of a predetermined range including the object and its peripheral pixels, and storing the image in a storage means; a cutout image generation procedure for generating a cutout image in which pixel data at the location to be replaced with the replacement image is deleted in each of the replacement areas; a background image acquisition procedure for acquiring a background image obtained by photographing only a background where no subject appears at the same photographing position and photographing angle as the photographed image and storing the background image in a storage means; and a replaced photographed image generation procedure for generating a replaced photographed image in which the object is replaced at the position of the replacement target by overlapping and synthesizing the background image, the cutout image, and the replacement image. [Effects of the Invention]

[0018] According to the present invention, since a replaced captured image is generated by replacing an object with respect to the position of the object to be replaced, it is possible to obtain a replaced captured image in which a part of the captured image is replaced with an object. That is, when a plurality of groups are simultaneously captured in the captured image, a replaced captured image can be generated in which one group is left in the captured image and the other groups are replaced with replacement images. Therefore, it is possible for the customer to obtain a captured image in which only they are captured. As a result, for example, in a business model in which a captured photo taken at a preset fixed shooting location is presented to a customer who is the subject, and if the customer likes it, they are asked to purchase it. For a customer who does not want to purchase a photo in which someone else is reflected, it is possible to provide a photo in which only they are captured. Furthermore, if the replacement image is created based on content that is attractive to the customer, the replaced captured image becomes a more attractive image, so it is possible to stimulate the customer's willingness to purchase the photo. Furthermore, in a facility that operates an attraction, there is a possibility that the captured image will be dull during the slack season because the empty seats in the moving body are conspicuous. In such a case, by replacing the empty seat area with a replacement image, it is possible to obtain a lively captured image.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0020] [First Embodiment] Hereinafter, an example of an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of the image processing apparatus 100 according to the present invention. As shown in FIG. 1, the image processing apparatus 100 includes a photographed image acquisition unit 11, a subject identification unit 12, a moving body position identification unit 13, a subject position determination unit 14, a replacement area determination unit 15, a replacement image acquisition unit 16, a cut-out image generation unit 17, a background image acquisition unit 18, a moving body image acquisition unit 19, a photographed image generation unit 20 after replacement, and a storage unit 21. In this example, the description will be made on the premise of a configuration in which a moving body carrying a person automatically takes a photo at the timing when the moving body reaches a predetermined position, but this is merely an example, and the present invention can also be applied to photography without the appearance of a moving body.

[0021] Note that the image processing apparatus 100 may be a device designed as a dedicated machine, but it is assumed that it can be realized by a general computer. In this case, the image processing apparatus 100 is assumed to include a CPU (Central Processing Unit), a memory, and a storage such as a hard disk drive, which are typically provided in a general computer (not shown). Also, a GPU (Graphics Processing Unit) may be provided as needed. Needless to say, various processes are executed by a program to cause these general computers to function as the image processing apparatus 100 of this example.

[0022] Further, the image processing apparatus 100 may function as a stand-alone independent of a communication network, but a part or all of the functions of the image processing apparatus 100 may be provided in a server apparatus, and the image processing apparatus 100 may be used by connecting from a terminal apparatus via a communication network.

[0023] The captured image acquisition unit 11 has a function of acquiring a captured image generated by performing imaging at a predetermined timing for an imaging range determined by a predetermined imaging position and imaging angle. Since the place where imaging is executed is various, the timing of imaging is also appropriately set. However, when imaging a person appearing in a moving body as in this example, adjustment is made so that imaging is performed at the timing when the moving body enters the imaging range.

[0024] The subject identification unit 12 has a function of identifying a subject shown in the captured image. Any means may be used as long as the subject to be imaged in the captured image can be identified. For example, it may be configured to identify the subject based on a learned model that has been pre-trained to identify the target subject from the captured image. When a plurality of subjects are shown, it is preferably possible to identify them separately one by one. It is still more preferable if it is possible to determine whether each pixel of the captured image data is a subject or not. When using a learned model, the learned model may be configured to be stored in the storage unit 21 described later, or may be configured to be stored in another storage means such as a server device connectable via a communication network.

[0025] The mobile body position identification unit 13 has a function of identifying the position of the mobile body shown in the captured image. Any method may be used as long as the exact position of the mobile body can be identified. For example, a plurality of characteristic configurations in the appearance of the mobile body may be selected in advance, and the position of the mobile body may be identified by a so-called template matching method that identifies where the plurality of characteristic configurations exist in the captured image. In this case, the identified position of the mobile body is stored as coordinates in the captured image. At this time, it may be stored according to the coordinate positions of each characteristic configuration.

[0026] The subject position determination unit 14 has a function of dividing the shooting range into a plurality of preset unit regions and determining in which location of the plurality of unit regions the specified subject is located. Here, the unit region refers to a region that is a unit for replacement of an object. The unit region may be set in any manner, but it is preferable to use a unit that can easily distinguish the regions where the subject is located from each other. For example, in the case of a moving body having a plurality of boarding locations as in this example, it is conceivable to set the first row, the second row, and the third row as unit regions, respectively. Then, it is determined in which unit region each subject specified by the subject specifying unit 12 is located. Any method may be used as long as it can specify to which unit region the subject belongs. For example, when setting the smallest rectangle that can entirely enclose the range determined as the subject, a method of determining that the region where the center of the rectangle is located is the unit region where the subject is located is conceivable.

[0027] The replacement area determination unit 15 focuses on at least one of the unit areas (hereinafter referred to as the subject areas) in which the subject is determined to be located (hereinafter referred to as the focused area), and has a function of determining a subject area and / or a unit area to be replaced (hereinafter referred to as the replacement area) based on a predetermined rule from the subject areas other than the focused area and / or the unit areas. Here, the focused area refers to the subject area that remains as it is in the captured image without replacement among the subject areas. In the image processing of this example, since a focused area is always set to one, an image in which at least one subject is captured is finally generated. However, it is not necessarily required that the focused area is one, so it is also possible to select a plurality of focused areas. In this replacement area determination unit 15, at least one replacement area to be replaced with an object is determined from the subject areas other than the focused area and / or the unit areas that do not include the subject. The method for determining the replacement area may be any method. The determination of the replacement area may be, for example, a method of automatically determining based on a preset predetermined rule, or a method selected by the user. Note that, finally, one pattern of determination of the replacement area is performed for one output of the captured image after replacement, and of course, an operation of determining a plurality of patterns and outputting a plurality of patterns of captured images after replacement is also conceivable. Regarding the determination of the replacement area, when the number of subject areas other than the focused area and the unit areas is small, a configuration may be adopted in which captured images after replacement are output for all combinations regarding whether or not to determine the replacement area.

[0028] The replacement image acquisition unit 16 has a function of acquiring a replacement image by extracting an image of a predetermined range including an object and its peripheral pixels from an object arrangement image obtained by photographing with the same photographing position and photographing angle as the photographed image by previously arranging an object at the position of the replacement area and storing it in the storage means. Here, the object refers to any object to be replaced with the replacement area. The object may be anything, for example, a person wearing a character costume may be used as the object. In this case, a person wearing a character costume is actually made to board a moving body to take the object arrangement image. Then, a replacement image including the object and its peripheral pixels is extracted from the object arrangement image and stored in advance. The storage target may be the storage unit 21 described later or other storage means. The replacement image does not necessarily have to be one, and a plurality of replacement images may be stored in advance and appropriately selected and used. Also, in the case of an example of a moving body as in this example, the replacement images to be replaced may be set to be different for each of the first column, the second column, and the third column. For example, it is conceivable to set the characters to be replaced to be different in the first column, the second column, and the third column. Note that, regarding the range for extracting the replacement image from the object arrangement image, only the portion corresponding to the object may be extracted.

[0029] Here, in a situation where attractions are operated by a plurality of moving objects with different shapes, colors, etc., it is necessary to determine which of the plurality of moving objects appears in the captured image. Therefore, based on the unique shape and unique color for each moving object, the moving object appearing in the captured image is identified. Also, since the way the moving object appears in the captured photo changes depending on where the moving object is located within the shooting range, the way the object mounted on the moving object appears also changes. Therefore, in order to further reduce the sense of incongruity in the composite image, a plurality of object arrangement images for the same moving object with the same object mounted at the same mounting position are prepared so that the positions of the moving objects in the shooting range are different from each other, and a plurality of replacement images are extracted from these plurality of object arrangement images and stored. By doing so, it becomes possible to select and use the replacement image closest to the position of the moving object in the captured image.

[0030] The cutout image generation unit 17 has a function of generating a cutout image in which the pixel data at the location to be replaced with the replacement image is deleted in each of the replacement regions. The cutout image is an image in which pixel data that becomes unnecessary when synthesizing images is deleted in advance. For example, since the subject appearing in the subject region set in the replacement region is replaced with an object during image synthesis, the pixel data corresponding to the subject is deleted, and also, if necessary, the surrounding pixel data is deleted. Regarding which range in the replacement region to delete the pixel data, it can be appropriately set by determining a predetermined rule in advance. Note that if the pixel region corresponding to the subject is completely included in the region of the replacement image, there will be no influence even if the cutout process (deletion process of pixel data) is not performed. Therefore, the cutout process in the cutout image generation unit 17 is not necessarily an essential configuration. In some cases, after determining whether or not the pixel region corresponding to the subject is completely included in the region of the replacement image, it may be configured to perform the cutout process only on the portion where the pixel region corresponding to the subject protrudes from the region of the replacement image.

[0031] In addition, when the subject extraction image generation unit 17 performs the process of extracting the subject, there are cases where the subject appearing in the replacement area straddles the adjacent target area. For example, when the person 1 to be extracted in the replacement area stretches out their hand and the hand extends behind the head of the person 2 to be left in the captured image appearing in the target area, if the areas of person 1 and person 2 cannot be distinguished, there is a risk of extracting both persons. Therefore, it is preferable that the person areas can be specified by distinguishing each person. Then, by extracting only the area of person 1 and supplementing it with the background image or the moving object image, it becomes possible to appropriately leave the person 2 who is desired to be left in the captured image without being extracted.

[0032] The background image acquisition unit 18 has a function of acquiring a background image that is captured and stored in the storage means by capturing only the background where the subject does not appear at the same shooting position and shooting angle as the shooting in the shooting function. Although the background also appears in the captured image, if there is background data that is lost when the cut-out image and the replacement image are superimposed, a sense of incongruity will occur. Therefore, by preparing a background image, it is possible to prevent a sense of incongruity from occurring in the final output, which is the replaced captured image.

[0033] The moving object image acquisition unit 19 has a function of acquiring a moving object image by photographing a moving object without a subject on board at the same photographing position and photographing angle as the photographed image, extracting only the moving object, and storing it. Similar to the background image, if there is moving object image data that will be lost when the cutout image and the replacement image are superimposed in the moving object image, a sense of incongruity will occur. Therefore, by preparing the moving object image, it is possible to prevent a sense of incongruity from occurring in the finally output replaced photographed image. The object for storing the moving object image may be the storage unit 21 described later or other storage means. Note that since it is difficult to apply a single moving object image when the position of the moving object is greatly different, a plurality of moving object images with slightly different positions and angles of the moving object are extracted and stored in advance, so that a more natural replaced photographed image can be obtained. Also, in the case of a situation including a plurality of moving objects with different shapes and colors, it is necessary to uniquely identify the moving objects shown in the photographed image based on the shape and color. Specifically, the moving objects shown in the photographed image are identified by comparing the shapes and colors of the preset determination locations. Then, the moving object images stored in association with the identified moving objects are extracted. That is, when there are a plurality of moving objects with different shapes and colors, a plurality of moving object images with slightly different positions and angles of the moving object are extracted and stored in advance for each moving object.

[0034] The replaced captured image generation unit 20 has a function of generating a replaced captured image in which an object is replaced at the position of the replacement target by superimposing and synthesizing a background image, a moving object image, a cut-out image, and a replacement image. When synthesizing images, it is necessary to perform accurate alignment. For example, an image of a predetermined range including the object and characteristic constituent parts for alignment existing around the object is extracted from the object arrangement image as a replacement image, and in this replaced captured image generation unit 20, it is conceivable to perform alignment using the characteristic constituent parts for alignment. As a specific example, in the case of the moving object in this example, it is conceivable to set the end portions of the handrails provided in front of each row of the moving object as the characteristic constituent parts for alignment. Also, for the alignment of the moving object image, alignment may be performed using the characteristic constituent parts used in template matching. By accurately performing alignment and synthesizing the background image, the moving object image, the cut-out image, and the replacement image, a replaced captured image in which the replacement area is replaced with an object can be obtained.

[0035] Alignment will be described in more detail. First, regarding the alignment of the cut-out image and the moving object image, in the moving object position specifying unit 13, a plurality of characteristic configurations in the appearance of the moving object are selected in advance, and the position of the moving object is specified and aligned by the method of template matching for specifying in which position in the captured image the plurality of characteristic configurations exist. Next, regarding the alignment of the replacement image with respect to the moving object image and the cut-out image, using the coordinates obtained by template matching in the moving object position specifying unit 13 as input, the SVM estimates the approximate positions of the characteristic constituent parts for alignment, for example, the end portions of the handrails installed in front of each sheet. Then, by executing template matching considering the alpha value representing the opacity of the pixels within a predetermined range near the estimated position of the estimated position, it is conceivable to execute superposition after specifying the exact matching positions of the characteristic constituent parts for alignment.

[0036] Note that the replacement image does not necessarily have to be an image including the characteristic constituent parts for alignment. For example, the replacement image may extract only the object parts, accurately memorize the positional relationship between the characteristic constituent parts for alignment and the replacement image, perform alignment accurately based on the characteristic constituent parts for alignment, and then synthesize the replacement image from which only the object parts are extracted.

[0037] The storage unit 21 has a function of storing data necessary for various processes performed in the image processing apparatus 100 and data obtained as a result of the processes.

[0038] Next, the flow of image processing in the image processing apparatus 100 will be described. FIG. 2 is a flowchart showing the flow of image processing in the image processing apparatus 100 according to the present invention. Note that the processing order of the flowchart shown in FIG. 2 is merely an example, and it goes without saying that the order can be appropriately changed within the range where the same result can be obtained.

[0039] As shown in FIG. 2, the image processing is started by performing shooting in the image processing apparatus 100 to obtain a captured image (step S101). Next, the image processing apparatus 100 identifies a subject from the obtained captured image (step S102). Next, the image processing apparatus 100 identifies the position of the moving object (step S103). Next, when the image processing apparatus 100 sets a unit area based on a predetermined rule set in advance for the captured image, it determines in which unit area the identified subject is located (step S104). Next, the image processing apparatus 100 determines a replacement area from the subject area and / or unit area outside the area of interest (step S105). Next, the image processing apparatus 100 obtains a replacement image corresponding to the position of the replacement area (step S106). Next, the image processing apparatus 100 deletes pixel data in a predetermined range in the replacement area to generate a cut-out image (step S107). Next, the image processing apparatus 100 obtains a background image (step S108). Next, the image processing apparatus 100 obtains a moving object image (step S109). Then, the image processing apparatus 100 overlays and synthesizes the background image, the moving object image, the cut-out image, and the replacement image to output a replaced captured image (step S110), and ends the image processing.

[0040] Next, the image processing in the image processing apparatus 100 will be described based on a specific example. FIG. 3 is an explanatory diagram showing an example of a captured image in the image processing apparatus 100 according to the present invention. The captured image shown in FIG. 3 represents, as an example, a captured image taken during an attraction in which people are enjoying themselves while riding on a moving object having three rows of seats.

[0041] FIG. 4 is an explanatory diagram showing an example of an image after performing subject identification processing in the image processing apparatus 100 according to the present invention. When subject identification processing is performed on the captured image shown in FIG. 3 to identify the subject, for example, as shown in FIG. 4, the pixels corresponding to each subject are identified in a form distinguishable from each other. In FIG. 4, four people are identified as subjects.

[0042] FIG. 5 is an explanatory diagram showing an example of subject position determination processing in the image processing apparatus 100 according to the present invention. As a premise, in a situation where three rows of sheets of a moving body are divided and each is set as a unit area, it is determined in which unit area each subject is located. In this case, as shown in FIG. 5, when the smallest rectangle that includes all the pixels corresponding to the subject is set, the unit area in which the center point of the rectangle is located is determined to be the unit area in which the subject is located.

[0043] FIG. 6 is an explanatory diagram showing an example of an object arrangement image in the image processing apparatus 100 according to the present invention. The example shown in FIG. 6 represents an object arrangement image obtained by actually photographing a person wearing a character costume on a moving body.

[0044] FIG. 7 is an explanatory diagram showing an example of a replacement image in the image processing apparatus 100 according to the present invention. FIG. 7(a) represents a replacement image obtained by extracting an object and its surrounding pixels from the object arrangement image shown in FIG. 6. FIG. 7(b) represents a replacement image prepared for replacement of the second row of the moving body by the same method.

[0045] FIG. 8 is an explanatory diagram showing an example of image synthesis in the image processing apparatus 100 according to the present invention. This FIG. 8 shows a state in which a background image, a moving body image, a cut-out image, and a replacement image are superimposed and synthesized in the replaced photographed image generation unit 20.

[0046] FIG. 9 is an explanatory diagram showing an example of a replaced photographed image in the image processing apparatus 100 according to the present invention. In the replaced photographed image generation unit 20, by accurately aligning and synthesizing each image shown in FIG. 8, the replaced photographed image shown in FIG. 9 is obtained. This replaced photographed image is an image in which the characters of dogs are replaced for two persons shown in the first row of the sheet. When the first row and the second row are different groups, the replaced photographed image shown in FIG. 9 becomes a commemorative photograph in which only the customers of the second row group are shown.

[0047] As described above, according to the image processing apparatus 100 of the present invention, a captured image is acquired, a subject shown in the captured image is specified, it is determined in which location of a plurality of unit regions obtained by dividing the shooting range the specified subject is located, at least one of the unit regions determined to have the subject located therein is set as a region of interest, a region to be replaced is determined from among those other than the region of interest, a replacement image including an object and its peripheral pixels extracted from an object placement image obtained by placing an object in advance at the location of the replacement region and then shooting is acquired, a cutout image is generated by deleting the pixel data of the portion to be replaced with the replacement image in each of the replacement regions, a background image is acquired, and a replaced captured image in which the object is replaced at the position of the replacement target is generated by overlapping and synthesizing the background image, the cutout image, and the replacement image. Thus, it is possible to obtain a replaced captured image in which a part of the captured image is replaced with an object.

[0048] That is, when a plurality of groups are simultaneously shown in a captured image, a replaced captured image can be generated in which one group is left in the captured image and the other groups are replaced with a replacement image. Therefore, it is possible for a customer to obtain a captured image in which only the customer himself / herself is shown.

[0049] Thereby, for example, in a business model in which a captured photo taken at a preset fixed shooting location is presented to a customer who is the subject, and if the customer likes it, the customer is asked to purchase it, it is possible to provide a photo in which only the customer himself / herself is shown to a customer who does not want to purchase a photo in which someone else is reflected. Furthermore, if the replacement image is created based on content that is attractive to the customer, the replaced captured image becomes a more attractive image, so it is possible to stimulate the customer's willingness to purchase the photo.

[0050] Furthermore, in a facility that operates an attraction, there is a possibility that the captured image may become dull during the slack period because the empty seats in the moving body are conspicuous. In such a case, it is possible to obtain a lively captured image by replacing the empty seat locations with a replacement image.

[0051] [Second Embodiment] In the first embodiment, the case where a person is used as a subject has been described as an example. However, the present invention is not limited to this, and the image processing of the present invention can be applied to various subjects other than persons.

[0052] Also, in the first embodiment, the image processing of replacing the replacement area with another object or the like has been described. However, the present invention is not limited to this, and image processing for erasing the subject shown in the replacement area may be used. Regarding which range in the replacement area to execute the pixel data erasing process, it can be appropriately set by previously determining a predetermined rule. In this case, the replacement area determination unit 15 in the first embodiment functions as a subject erasing area determination unit that determines the area for erasing the subject instead of determining the area to be replaced. Also, since no replacement is performed, the replacement image acquisition unit 16 is unnecessary. Note that in the subject erasing area determination unit, it is also possible to select one target area where the subject remains and erase all the subjects shown in other unit areas. In this case, it can be said that in the subject erasing area determination unit, all unit areas other than the target area where the subject remains are selected as the subject erasing area.

[0053] That is, for a shooting range determined by a predetermined shooting position and shooting angle, a shooting image acquisition unit that acquires a shooting image generated by performing shooting at a predetermined timing, a subject identification unit that identifies a subject shown in the shooting image, a shooting range is divided into a plurality of preset unit regions, and a subject position determination unit that determines in which location of the plurality of unit regions the identified subject is located, at least one of the unit regions (hereinafter referred to as a subject region) in which the subject is determined to be located is focused on (hereinafter referred to as a focused region), and a subject elimination region determination unit that determines a subject region (hereinafter referred to as a subject elimination region) that is a target for eliminating a subject from among the subject regions other than the focused region based on a predetermined rule, a cutout image generation unit that generates a cutout image in which the pixel data of the subject is deleted in each of the subject elimination regions, a background image acquisition unit that acquires a background image that is shot and stored in a storage means only for the background where no subject appears at the same shooting position and shooting angle as the shooting image, and an erased shooting image generation unit that generates an erased shooting image in which the subject in the subject elimination region is eliminated by overlapping and synthesizing the background image and the cutout image. An image processing apparatus may be provided.

[0054] In the first embodiment, the replacement process of replacing the subject part with an object is described, and in the second embodiment, the elimination process of eliminating the subject part is described, but a process combining these may also be used. That is, in the replacement region determination unit in the first embodiment, if it is configured to be able to determine both the replacement region and the subject elimination region at the same time, it is possible to generate a replaced shooting image including both the part to be replaced and the part to be eliminated.

Explanation of Signs

[0055] 10 Image processing apparatus 11 Shooting image acquisition unit 12 Subject identification unit 13 Moving body position identification unit 14 Subject position determination unit 15 Replacement region determination unit 16 Replacement image acquisition unit 17 Cropped Image Generation Unit 18 Background Image Acquisition Unit 19 Moving Object Image Acquisition Unit 20 Generated Captured Image after Replacement Unit 21 Memory Unit

Claims

1. An image processing program for causing a computer to implement a process of erasing a part of a captured image, wherein the computer has a captured image acquisition function of acquiring a captured image generated by performing imaging on a imaging range determined by a predetermined imaging position and imaging angle, a subject identification function of identifying a subject shown in the captured image, a subject position determination function of dividing the imaging range into a plurality of preset unit regions and determining in which location of the plurality of unit regions the identified subject is located, a subject erasure region determination function of focusing on at least one of the unit regions (hereinafter referred to as the subject region) in which the subject is determined to be located (hereinafter referred to as the focused region), and determining all the other subject regions except the focused region as the subject regions (hereinafter referred to as the subject erasure regions) to be the target of erasing the subject, a clipped image generation function of generating a clipped image in which the pixel data of the subject is deleted in each of the subject erasure regions, a background image acquisition function of acquiring a background image that is obtained by imaging only the background where no subject appears at the same imaging position and imaging angle as the captured image and storing it in a storage means, and a generated image generation function of generating a generated captured image with the subject in the subject erasure region erased by overlapping and synthesizing the background image and the clipped image An image processing program for realizing the above.

2. When a moving body on which a subject can board is the imaging target, the computer has a moving body position identification function of identifying the position of the moving body shown in the captured image, and a moving body image acquisition function of acquiring a moving body image that is obtained by imaging a moving body on which no subject is boarding at the same imaging position and imaging angle as the captured image, extracting only the moving body, and storing it in a storage means, and further realizes, in the subject position determination function, the boarding possible locations of the subject on the moving body are preset as the plurality of unit regions, in the background image acquisition function, a background image in a state where the moving body does not exist is acquired, and in the generated captured image generation function, the generated captured image is generated by overlapping and synthesizing the background image, the moving body image, and the clipped image using the position information of the identified moving body. The image processing program according to Claim 1.

3. An image processing program for causing a computer to realize a process of replacing a part of a photographed image with an object, wherein the computer is caused to: have a photographed image acquisition function of acquiring a photographed image generated by performing photographing on a photographing range determined by a predetermined photographing position and a photographing angle; have a subject identification function of identifying a subject shown in the photographed image; have a subject position determination function of dividing the photographing range into a plurality of preset unit areas and determining in which location of the plurality of unit areas the identified subject is located; focus on at least one of the unit areas (hereinafter referred to as the subject area) in which the subject is determined to be located (hereinafter referred to as the focused area), and (1) determine the subject area and / or the unit area (hereinafter referred to as the replacement area) to be the replacement target from among the subject areas and the unit areas other than the focused area by user selection, (2) automatically select one pattern from all the combination patterns of which of the subject areas and the unit areas other than the focused area are to be determined as the replacement area, or (3) determine all the subject areas and the unit areas other than the focused area as the replacement area, which is a replacement area determination function; have a replacement image acquisition function of extracting, from an object placement image obtained by placing an object in advance at the location of the replacement area and performing photographing at the same photographing position and photographing angle as the photographed image, an image of a preset predetermined range including the object and its peripheral pixels, and storing the image in a storage means; have a clipped image generation function of generating a clipped image in which the pixel data of the location to be replaced with the replacement image is deleted in each of the replacement areas; have a background image acquisition function of acquiring a background image obtained by photographing only the background where no subject appears at the same photographing position and photographing angle as the photographed image and storing the image in a storage means; and have a replaced photographed image generation function of generating a replaced photographed image in which the object is replaced at the position of the replacement target by overlapping and synthesizing the background image, the clipped image, and the replacement image. An image processing program for realizing the above.

4. An image processing program for causing a computer to realize a process of replacing a part of a photographed image with an object, wherein the computer is caused to: An imaging image acquisition function for acquiring an imaging image generated by performing imaging on an imaging range determined by a predetermined imaging position and imaging angle, A subject identification function for identifying a subject shown in the imaging image, A subject position determination function for dividing the imaging range into a plurality of preset unit areas and determining in which location of the plurality of unit areas the identified subject is located, Focus on at least one of the unit areas (hereinafter referred to as the subject area) determined to have a subject located therein (hereinafter referred to as the focused area), and (1) determine the subject area to be replaced (hereinafter referred to as the replacement area) from among the subject areas other than the focused area by user selection, (2) automatically select and determine one pattern from all the combination patterns regarding which one to determine as the replacement area from among the subject areas other than the focused area, or (3) determine all the subject areas other than the focused area as the replacement area; a replacement area determination function, A replacement image acquisition function for acquiring a replacement image by extracting, from an object placement image obtained by placing an object in advance at the location of the replacement area and performing imaging at the same imaging position and imaging angle as the imaging image, an image of a preset predetermined range including the object and its surrounding pixels and storing it in a storage means, A cutout image generation function for generating a cutout image in which the pixel data of the location to be replaced with the replacement image is deleted in each of the replacement areas, A background image acquisition function for acquiring a background image obtained by imaging only the background where no subject appears at the same imaging position and imaging angle as the imaging image and storing it in a storage means, A replaced imaging image generation function for generating a replaced imaging image in which the object is replaced at the position of the replacement target by superimposing and synthesizing the background image, the cutout image, and the replacement image, An image processing program for realizing the above.

5. An image processing program for causing a computer to realize a process of replacing a part of an imaging image with an object, In the computer, An imaging image acquisition function for acquiring an imaging image generated by performing imaging on an imaging range determined by a predetermined imaging position and imaging angle, A subject identification function for identifying a subject shown in the imaging image, Divide the shooting range into a plurality of preset unit areas, and determine in which location of the plurality of unit areas the specified subject is located, and a subject position determination function; Focus on at least one of the unit areas (hereinafter referred to as the subject area) determined to have a subject located therein (hereinafter referred to as the focused area), and (1) select the unit area to be replaced (hereinafter referred to as the replacement area) from among the unit areas other than the focused area by the user's selection, (2) automatically select one pattern from all the combination patterns regarding which one to determine as the replacement area from among the unit areas other than the focused area, or (3) determine all the unit areas other than the focused area as the replacement area, and a replacement area determination function; From the object placement image obtained by placing an object in advance at the location of the replacement area and performing shooting at the same shooting position and shooting angle as the shooting image, extract an image of a preset predetermined range including the object and its surrounding pixels, and obtain a replacement image stored in the storage means, and a replacement image acquisition function; Generate a clipped image by deleting the pixel data of the location to be replaced with the replacement image in each of the replacement areas, and a clipped image generation function; Obtain a background image that is stored in the storage means by shooting only the background where the subject does not appear at the same shooting position and shooting angle as the shooting image, and a background image acquisition function; Generate a photographed image after replacement in which the object is replaced at the position of the replacement target by overlapping and synthesizing the background image, the clipped image, and the replacement image, and a photographed image generation function after replacement An image processing program for realizing the above.

6. When a moving object on which a subject can board is the shooting target, In the computer, A moving object position specifying function for specifying the position of the moving object shown in the shooting image; A moving object image acquisition function for obtaining a moving object image that is stored in the storage means by shooting a moving object without a subject on board at the same shooting position and shooting angle as the shooting image and extracting only the moving object; Further realize the above, In the subject position determination function, divide the mountable locations of the subject on the moving object into a plurality of unit areas; In the background image acquisition function, obtain a background image in a state where the moving object does not exist. In the replacement image acquisition function, an object is arranged in advance at a location on the moving body where a subject can board, and from an object arrangement image obtained by taking a picture at the same shooting position and shooting angle as the captured image, an image of a predetermined range including the object and pixels of the moving body around it is extracted and stored in a storage means as a replacement image to be acquired. In the replaced captured image generation function, the replaced captured image is generated by overlapping and synthesizing the background image, the moving body image, the cut-out image, and the replacement image using the position information of the identified moving body. The image processing program according to any one of claims 3 to 5.

7. In the replacement image acquisition function, an image of a predetermined range including the object and a feature configuration part preset for alignment existing around the object is extracted from the object arrangement image as a replacement image. In the replaced captured image generation function, alignment is performed using the feature configuration part for alignment and overlapping synthesis is performed. The image processing program according to any one of claims 3 to 6.

8. In the moving body position identification function, a plurality of characteristic configurations in the appearance of the moving body are selected in advance, and the position of the moving body is identified by identifying the positions of the plurality of characteristic configurations in the captured image. The image processing program according to claim 2 or claim 6.

9. In the subject identification function, based on a learned model that has been pre-trained to distinguish and identify target subjects one by one from the input captured image, pixels corresponding to the subject are identified in a distinguishable form for each individual. In the subject position determination function, for each subject identified in the subject identification function, a minimum rectangular region that includes all the pixels identified as the same individual is set, and based on which unit region the center point of the minimum rectangular region belongs to, it is determined at which location in which of the plurality of unit regions the identified subject is located. The image processing program according to any one of claims 1 to 6.

10. An image processing apparatus for performing a process of erasing a part of a captured image, a captured image acquisition unit that acquires a captured image generated by taking a picture of a shooting range determined by a predetermined shooting position and shooting angle, A subject identification unit that identifies a subject shown in the captured image; A subject position determination unit that divides the imaging range into a plurality of preset unit regions and determines in which location of the plurality of unit regions the identified subject is located; Focus on at least one of the unit regions (hereinafter referred to as the subject region) in which the subject is determined to be located (hereinafter referred to as the focused region), and determine all the subject regions other than the focused region as the subject regions to be the target of subject elimination (hereinafter referred to as the subject elimination regions) by a subject elimination region determination unit; A clipped image generation unit that generates a clipped image by deleting the pixel data of the subject in each of the subject elimination regions; A background image acquisition unit that acquires a background image obtained by photographing only the background where no subject appears at the same photographing position and photographing angle as the captured image and storing it in a storage means; An erased captured image generation unit that generates an erased captured image in which the subject in the subject elimination region is erased by overlapping and synthesizing the background image and the clipped image An image processing apparatus comprising.

11. An image processing apparatus for performing a process of replacing a part of a captured image with an object, A captured image acquisition unit that acquires a captured image generated by performing imaging on an imaging range determined by a predetermined imaging position and imaging angle; A subject identification unit that identifies a subject shown in the captured image; A subject position determination unit that divides the imaging range into a plurality of preset unit regions and determines in which location of the plurality of unit regions the identified subject is located; Focus on at least one of the unit regions (hereinafter referred to as the subject region) in which the subject is determined to be located (hereinafter referred to as the focused region), and (1) determine the subject regions and / or the unit regions to be the replacement regions (hereinafter referred to as the replacement regions) from among the subject regions other than the focused region and the unit regions by user selection, (2) automatically select and determine one pattern from all the combination patterns of which of the subject regions other than the focused region and the unit regions are to be determined as the replacement regions, or (3) determine all the subject regions and the unit regions other than the focused region as the replacement regions by a replacement region determination unit; A replacement image acquisition unit that acquires a replacement image by arranging an object in advance at the location of the replacement area, performing shooting at the same shooting position and shooting angle as the captured image, extracting an image of a predetermined range including the object and its surrounding pixels from the object arrangement image obtained, and storing it in a storage means; A clipped image generation unit that generates a clipped image by deleting pixel data at the location to be replaced with the replacement image in each of the replacement areas; A background image acquisition unit that acquires a background image by shooting only the background where the subject does not appear at the same shooting position and shooting angle as the captured image and storing it in a storage means; A replaced captured image generation unit that generates a replaced captured image in which the object is replaced at the position of the replacement target by overlapping and synthesizing the background image, the clipped image, and the replacement image; An image processing apparatus comprising the above.

12. An image processing apparatus for performing a process of replacing a part of a captured image with an object, A captured image acquisition unit that acquires a captured image generated by performing shooting on a shooting range determined by a predetermined shooting position and shooting angle; A subject identification unit that identifies a subject appearing in the captured image; A subject position determination unit that divides the shooting range into a plurality of preset unit areas and determines in which location of the plurality of unit areas the identified subject is located; Focus on at least one of the unit areas (hereinafter referred to as the subject area) in which the subject is determined to be located (hereinafter referred to as the focused area), and (1) determine the subject area to be replaced (hereinafter referred to as the replacement area) from among the subject areas other than the focused area by user selection, (2) automatically select one pattern from all the combination patterns regarding which of the subject areas other than the focused area is determined as the replacement area, or (3) determine all the subject areas other than the focused area as the replacement area. A replacement area determination unit; A replacement image acquisition unit that acquires a replacement image by arranging an object in advance at the location of the replacement area, performing shooting at the same shooting position and shooting angle as the captured image, extracting an image of a predetermined range including the object and its surrounding pixels from the object arrangement image obtained, and storing it in a storage means; A clipped image generation unit that generates a clipped image by deleting pixel data at locations to be replaced with the replacement image in each of the replacement areas; A background image acquisition unit that acquires a background image obtained by photographing only the background where the subject does not appear at the same photographing position and photographing angle as the photographed image and storing it in a storage means; A photographed image generation unit that generates a photographed image with the object replaced at the position of the replacement target by superimposing and synthesizing the background image, the clipped image, and the replacement image; An image processing apparatus comprising:

13. An image processing apparatus for performing a process of replacing a part of a photographed image with an object, A photographed image acquisition unit that acquires a photographed image generated by photographing within a photographing range determined by a predetermined photographing position and photographing angle; A subject identification unit that identifies a subject appearing in the photographed image; A subject position determination unit that divides the photographing range into a plurality of preset unit areas and determines in which location of the plurality of unit areas the identified subject is located; Focus on at least one of the unit areas (hereinafter referred to as the subject area) in which the subject is determined to be located (hereinafter referred to as the focused area), and (1) determine the unit area to be replaced (hereinafter referred to as the replacement area) from among the unit areas other than the focused area by user selection, (2) automatically select and determine one pattern from all the combination patterns of which of the unit areas other than the focused area are to be determined as the replacement area, or (3) determine all the unit areas other than the focused area as the replacement area; a replacement area determination unit; A replacement image acquisition unit that extracts an image of a predetermined range including the object and its surrounding pixels from an object arrangement image obtained by arranging an object in advance at the location of the replacement area and photographing at the same photographing position and photographing angle as the photographed image, and stores it in a storage means; A clipped image generation unit that generates a clipped image by deleting pixel data at locations to be replaced with the replacement image in each of the replacement areas; A background image acquisition unit that acquires a background image obtained by photographing only the background where the subject does not appear at the same photographing position and photographing angle as the photographed image and storing it in a storage means; A photographed image generation unit that generates a photographed image with the object replaced at the position to be replaced by superimposing and synthesizing the background image, the cut-out image, and the replacement image An image processing apparatus comprising the same.

14. An image processing method for realizing, by a computer, a process of erasing a part of a photographed image, comprising: A photographed image acquisition procedure for acquiring a photographed image generated by performing photographing on a photographing range determined by a predetermined photographing position and a photographing angle; A subject identification procedure for identifying a subject appearing in the photographed image; A subject position determination procedure for dividing the photographing range into a plurality of preset unit areas and determining in which of the plurality of unit areas the identified subject is located; Focusing on at least one of the unit areas (hereinafter referred to as the subject area) in which the subject is determined to be located (hereinafter referred to as the focused area), and determining all the subject areas other than the focused area as the subject areas (hereinafter referred to as the subject erasure areas) to be the target for erasing the subject; A cut-out image generation procedure for generating a cut-out image in which the pixel data of the subject is deleted in each of the subject erasure areas; A background image acquisition procedure for acquiring a background image that is photographed and stored in a storage means with only the background where the subject does not appear at the same photographing position and photographing angle as the photographed image; An erased photographed image generation procedure for generating an erased photographed image with the subject in the subject erasure area erased by superimposing and synthesizing the background image and the cut-out image An image processing method including the same.

15. An image processing method for realizing, by a computer, a process of replacing a part of a photographed image with an object, comprising: A photographed image acquisition procedure for acquiring a photographed image generated by performing photographing on a photographing range determined by a predetermined photographing position and a photographing angle; A subject identification procedure for identifying a subject appearing in the photographed image; A subject position determination procedure for dividing the photographing range into a plurality of preset unit areas and determining in which of the plurality of unit areas the identified subject is located; Focus on at least one of the unit regions (hereinafter referred to as the subject region) in which the subject is determined to be located (hereinafter referred to as the focused region), and (1) determine the subject region and / or the unit region to be replaced (hereinafter referred to as the replacement region) to be replaced from among the subject regions and the unit regions other than the focused region by the user's selection, (2) determine one pattern automatically selected from all the combination patterns of which to determine as the replacement region from among the subject regions and the unit regions other than the focused region, or (3) a replacement region determination procedure for determining all the subject regions and the unit regions other than the focused region as the replacement region, A replacement image acquisition procedure for acquiring a replacement image in which an image of a predetermined range including the object and its peripheral pixels is extracted from an object arrangement image obtained by photographing with an object arranged in advance at the location of the replacement region at the same photographing position and photographing angle as the photographed image and stored in a storage means, A clipped image generation procedure for generating a clipped image in which the pixel data of the portion to be replaced with the replacement image is deleted in each of the replacement regions, A background image acquisition procedure for acquiring a background image obtained by photographing only the background where the subject does not appear at the same photographing position and photographing angle as the photographed image and stored in a storage means, A replaced photographed image generation procedure for generating a replaced photographed image in which the object is replaced at the position of the replacement target by overlapping and synthesizing the background image, the clipped image, and the replacement image, An image processing method including the above.

16. An image processing method for realizing, by a computer, a process of replacing a part of a photographed image with an object, A photographed image acquisition procedure for acquiring a photographed image generated by photographing within a photographing range determined by a predetermined photographing position and photographing angle, A subject identification procedure for identifying a subject appearing in the photographed image, A subject position determination procedure for dividing the photographing range into a plurality of preset unit regions and determining in which location of the plurality of unit regions the identified subject is located, Focus on at least one of the unit regions (hereinafter referred to as the subject regions) in which the subject is determined to be located (hereinafter referred to as the focused region), and (1) determine the subject region to be replaced (hereinafter referred to as the replacement region) from among the subject regions other than the focused region by the user's selection, (2) determine one pattern automatically selected from all the combination patterns regarding which of the subject regions other than the focused region is to be determined as the replacement region, or (3) a replacement region determination procedure for determining all the subject regions other than the focused region as the replacement region, A replacement image acquisition procedure for acquiring a replacement image in which an image of a predetermined range including the object and its peripheral pixels is extracted from an object arrangement image obtained by photographing with the same photographing position and photographing angle as the photographed image after arranging an object in advance at the position of the replacement region and storing it in a storage means, A cutout image generation procedure for generating a cutout image in which the pixel data of the portion to be replaced with the replacement image is deleted in each of the replacement regions, A background image acquisition procedure for acquiring a background image obtained by photographing only the background where the subject does not appear at the same photographing position and photographing angle as the photographed image and storing it in a storage means, A replaced photographed image generation procedure for generating a replaced photographed image in which the object is replaced at the position of the replacement target by overlapping and synthesizing the background image, the cutout image, and the replacement image, An image processing method including the above.

17. An image processing method for realizing, by a computer, a process of replacing a part of a photographed image with an object, A photographed image acquisition procedure for acquiring a photographed image generated by photographing within a photographing range determined by a predetermined photographing position and photographing angle, A subject identification procedure for identifying the subject appearing in the photographed image, A subject position determination procedure for dividing the photographing range into a plurality of preset unit regions and determining in which location of the plurality of unit regions the identified subject is located, Focus on at least one of the unit areas (hereinafter referred to as the subject area) in which the subject is determined to be located (hereinafter referred to as the focused area), and (1) determine the unit area to be replaced (hereinafter referred to as the replacement area) from among the unit areas other than the focused area by the user's selection, (2) automatically select and determine one pattern from all the combination patterns regarding which of the unit areas other than the focused area is to be determined as the replacement area, or (3) determine all the unit areas other than the focused area as the replacement area, which is a replacement area determination procedure, From an object arrangement image obtained by arranging an object in advance at the location of the replacement area and performing shooting at the same shooting position and shooting angle as the captured image, extract an image of a preset predetermined range including the object and its surrounding pixels, and obtain a replacement image stored in the storage means, which is a replacement image acquisition procedure, A cutout image generation procedure for generating a cutout image in which the pixel data at the location to be replaced with the replacement image is deleted in each of the replacement areas, A background image acquisition procedure for acquiring a background image obtained by shooting only the background where the subject does not appear at the same shooting position and shooting angle as the captured image and storing it in the storage means, A replaced captured image generation procedure for generating a replaced captured image in which the object is replaced at the position of the replacement target by overlapping and synthesizing the background image, the cutout image, and the replacement image, An image processing method including the above.

18. An image processing program for causing a computer to realize a process of replacing a part of a captured image with an object, The computer is caused to Have a captured image acquisition function for acquiring a captured image generated by performing shooting on a shooting range determined by a predetermined shooting position and shooting angle, Have a subject identification function for identifying the subject shown in the captured image, Divide the shooting range into a plurality of preset unit areas, and have a subject position determination function for determining in which location of the plurality of unit areas the identified subject is located, Focus on at least one of the unit areas (hereinafter referred to as subject areas) in which the subject is determined to be located (hereinafter referred to as the focused area), and (1) determine the subject areas and / or unit areas to be replaced (hereinafter referred to as replacement areas) from among the subject areas and unit areas other than the focused area by user selection, (2) determine one pattern automatically selected from all the combination patterns regarding which of the subject areas and unit areas other than the focused area are to be determined as replacement areas, or (3) a replacement area determination function for determining all the subject areas and unit areas other than the focused area as replacement areas, From an object arrangement image obtained by arranging an object in advance at the location of the replacement area and performing imaging at the same imaging position and imaging angle as the captured image, accurately store in advance the positional relationship between the object and the characteristic constituent parts preset for alignment existing around it, store in advance the images of the characteristic constituent parts, and obtain a replacement image from a storage means that has extracted the location of the object and stored it in advance as a replacement image. A replacement image acquisition function for acquiring a replacement image, A cutout image generation function for generating a cutout image in which the pixel data of the location to be replaced with the replacement image is deleted in each of the replacement areas, A background image acquisition function for acquiring a background image obtained by imaging only the background where the subject does not appear at the same imaging position and imaging angle as the captured image and storing it in a storage means, In the superposition synthesis of the background image, the cutout image, and the replacement image, perform position identification using the image of the characteristic constituent parts stored in the storage means, and based on the information on the positional relationship between the object stored in the storage means and the characteristic constituent parts, identify the superposition position of the replacement image and perform synthesis to generate a captured image after replacement in which the object is replaced at the position to be replaced. A captured image generation function for generating a captured image after replacement, An image processing program for realizing the above.

19. An image processing program for causing a computer to realize a process of replacing a part of a captured image with an object, The computer is provided with A captured image acquisition function for acquiring a captured image generated by performing imaging on an imaging range determined by a predetermined imaging position and imaging angle, A subject identification function for identifying a subject shown in the captured image, A subject position determination function that divides the shooting range into a plurality of preset unit areas and determines in which location of the plurality of unit areas the identified subject is located, Focus on at least one of the unit areas (hereinafter referred to as the subject area) in which the subject is determined to be located (hereinafter referred to as the focused area), and (1) determine the subject area to be replaced (hereinafter referred to as the replacement area) from among the subject areas other than the focused area by the user's selection, (2) automatically select one pattern from all the combination patterns of which one to determine as the replacement area from among the subject areas other than the focused area, or (3) determine all the subject areas other than the focused area as the replacement area. A replacement area determination function, From an object placement image obtained by placing an object in advance at the location of the replacement area and performing shooting at the same shooting position and shooting angle as the captured image, accurately store in advance the positional relationship between the object and the characteristic constituent parts preset for alignment existing around it, store in advance the images of the characteristic constituent parts, and extract the location of the object and store it in advance as a replacement image. A replacement image acquisition function for acquiring a replacement image from a storage means, A cutout image generation function for generating a cutout image in which the pixel data of the location to be replaced with the replacement image is deleted in each of the replacement areas, A background image acquisition function for acquiring a background image obtained by shooting only the background where no subject appears at the same shooting position and shooting angle as the captured image and storing it in a storage means, In the superposition synthesis of the background image, the cutout image, and the replacement image, perform position identification using the image of the characteristic constituent parts stored in the storage means, and based on the information on the positional relationship between the object stored in the storage means and the characteristic constituent parts, identify the superposition position of the replacement image and perform synthesis to generate a replaced captured image in which the object is replaced at the position to be replaced. A replaced captured image generation function, An image processing program for realizing the above.

20. An image processing program for causing a computer to realize a process of replacing a part of a captured image with an object, In the computer, An imaging image acquisition function for acquiring an imaging image generated by performing imaging on an imaging range determined by a predetermined imaging position and imaging angle, A subject identification function for identifying a subject shown in the imaging image, A subject position determination function for dividing the imaging range into a plurality of preset unit areas and determining in which location of the plurality of unit areas the identified subject is located, Focus on at least one of the unit areas (hereinafter referred to as the subject area) determined to have a subject located therein (hereinafter referred to as the focused area), and (1) determine the unit area to be replaced (hereinafter referred to as the replacement area) from among the unit areas other than the focused area by the user's selection, (2) automatically select and determine one pattern from all the combination patterns of which one of the unit areas other than the focused area is to be determined as the replacement area, or (3) determine all the unit areas other than the focused area as the replacement area. A replacement area determination function, From an object placement image obtained by placing an object in advance at the location of the replacement area and performing imaging at the same imaging position and imaging angle as the imaging image, accurately store in advance the positional relationship between the object and the characteristic constituent parts preset for alignment existing around it, store in advance the images of the characteristic constituent parts, and extract the location of the object and store it in advance as a replacement image. A replacement image acquisition function for acquiring a replacement image from the storage means, A cutout image generation function for generating a cutout image in which the pixel data of the location to be replaced with the replacement image is deleted in each of the replacement areas, A background image acquisition function for acquiring a background image that is obtained by imaging only the background where no subject appears at the same imaging position and imaging angle as the imaging image and storing it in the storage means, In the superposition synthesis of the background image, the cutout image, and the replacement image, perform position identification using the image of the characteristic constituent parts stored in the storage means, and based on the information on the positional relationship between the object stored in the storage means and the characteristic constituent parts, identify the superposition position of the replacement image and perform synthesis to generate a replaced imaging image in which the object is replaced at the position to be replaced. A replaced imaging image generation function An image processing program for realizing the above.

21. When a moving object on which a subject can board is the object to be photographed, the computer is caused to have a moving object position specifying function for specifying the position of the moving object shown in the photographed image, and a moving object image acquisition function for acquiring a moving object image that is obtained by photographing a moving object without a subject boarding at the same photographing position and photographing angle as the photographed image, extracting only the moving object, and storing it in a storage means. is further realized, in the subject position determination function, the boarding possible locations of the subject on the moving object are divided into a plurality of unit regions, in the background image acquisition function, a background image in a state where the moving object does not exist is acquired, in the replacement image acquisition function, an object is arranged in advance at a boarding possible location of the subject on the moving object, and from an object arrangement image obtained by photographing at the same photographing position and photographing angle as the photographed image, the position relationship between the object and a characteristic constituent part preset as an alignment target existing on the moving object around it is accurately stored in advance, the image of the characteristic constituent part on the moving object is stored in advance, and the replacement image is acquired from a storage means that extracts the location of the object and stores it in advance as a replacement image. In the generated photographed image generation function, position specification is performed using the image of the characteristic constituent part stored in the storage means, the overlapping position of the replacement image is specified based on the information on the position relationship between the object stored in the storage means and the characteristic constituent part, and then the background image, the moving object image, the cut-out image, and the replacement image are overlapped and synthesized using the specified position information to generate the generated photographed image. The image processing program according to any one of claims 18 to 20.

Images