Program, image processing apparatus, and image processing method

Abstract

To provide a program, an image processing apparatus, and an image processing method which can generate a motion picture which looks like one obtained by capturing a target from a fixed view point, from a motion picture obtained by capturing the target with tracking it.SOLUTION: A removal image generating unit 101 of an image processing apparatus 100 generates a plurality of removal images in which targets were removed from a plurality of frame images included in movies obtained by tracking and capturing the target. A panoramic image generating unit 102 executes collation of feature points included I the plurality of removal images to generate a panoramic image indicating a background of the target by overlapping the removal images of the plurality of respective removal images to which the matched feature points were converted by image conversion. A synthesis unit 103 subjects images of the target removed from the frame images of the plurality of respective frame images to the image conversion subjected to the removal images generated from the respective frame images to overlap them with the panoramic image, thereby creating the respective frame images of the plurality of frame images included in a fixed-viewpoint movie.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a program, an image processing apparatus, and an image processing method.

Background Art

[0002] Conventionally, various techniques for analyzing a target using a video obtained by tracking and photographing the target have been proposed. For example, Patent Document 1 discloses a technique for trimming a racehorse from a video obtained by tracking and photographing a racehorse walking in a paddock to create a trimmed video showing how the racehorse moves.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In general, a video obtained by tracking and photographing a racehorse in a paddock as described above is photographed so that the position of the racehorse is fixed within the photographing field of view. On the other hand, stride and walking rhythm are used to judge the state of the racehorse, but it is easier to observe them when shown in a fixed background, that is, when photographed from a fixed viewpoint. Therefore, depending on the analysis items of the target, a video photographed from a fixed viewpoint may be more desirable than a video obtained by tracking and photographing, but there are cases where a video photographed from a fixed viewpoint cannot be obtained even if a video obtained by tracking and photographing is obtained.

[0005] The present invention solves the above problems, and an object thereof is to provide a program, an image processing apparatus, and an image processing method capable of generating a video in which the target is photographed from a fixed viewpoint from a video obtained by tracking and photographing the target.

Means for Solving the Problems

[0006] The program according to the first aspect of the present invention is for a computer to generate a plurality of removed images obtained by removing the target from a plurality of frame images included in a tracking video in which the target is tracked and photographed, a removed image generation unit to execute collation between feature points included in the plurality of removed images, and to generate a panorama image representing the background of the target by overlapping, after performing image conversion on each of the plurality of removed images, the feature points that match each other a synthesis unit that synthesizes a fixed viewpoint video from the tracking video and the panorama image, for each of the frame images of the plurality of frame images, to perform the image conversion applied to the removed image generated from each of the frame images on the image of the target removed from each of the frame images, and to overlap the result on the panorama image, thereby obtaining each of the frame images included in the fixed viewpoint video the synthesis unit, characterized by causing the computer to function as such.

[0007] Further, in the program according to the above aspect, the removed image generation unit generates the removed image by making the area where the target is drawn transparent in each of the frame images of the plurality of frame images, the panorama image generation unit executes the collation while ignoring the feature points detected in the vicinity of the boundary of the transparent area characterized by this.

[0008] Further, in the program according to the above aspect, the target is a racehorse in a horse race, the tracking video is a video in which the racehorse is tracked and photographed in the paddock characterized by this.

[0009] Further, in the program according to the above aspect, The synthesizing unit includes an index used for determining the state of the runaway horse in the fixed-viewpoint video characterized in that

[0010] Also, in the program according to the above aspect the synthesizing unit includes the footprint of the runaway horse as the index in the fixed-viewpoint video characterized in that

[0011] Also, in the program according to the above aspect the synthesizing unit includes the footprint of the front leg of the runaway horse as the index in the fixed-viewpoint video characterized in that

[0012] Also, in the program according to the above aspect the synthesizing unit highlights the part of the runaway horse related to the evaluation based on the evaluation of the state of the runaway horse in the fixed-viewpoint video characterized in that

[0013] The image processing apparatus according to the second aspect of the present invention a removal image generation unit that generates a plurality of removed images from which the target has been removed from a plurality of frame images included in a tracking video in which the target is tracked; a panorama image generation unit that executes collation of feature points included in the plurality of removed images, and overlaps the feature points that match each other after performing image conversion on each of the plurality of removed images, thereby generating a panorama image representing the background of the target; a synthesizing unit that synthesizes a fixed-viewpoint video from the tracking video and the panorama image, for each of the plurality of frame images Applying the image conversion performed on the removed image of the object from each of the frame images to the removed image generated from each of the frame images, and superimposing it on the panoramic image to obtain each frame image of the plurality of frame images included in the fixed viewpoint video A composition unit, characterized by comprising.

[0014] An image processing method according to a third aspect of the present invention is an image processing method executed by an image processing apparatus, a removed image generation step of generating a plurality of removed images obtained by removing the object from a plurality of frame images included in a tracking video in which the object is tracked; a panoramic image generation step of performing collation of feature points included in the plurality of removed images, and generating a panoramic image representing the background of the object by superimposing the feature points that match each other after performing image conversion on each of the plurality of removed images; a composition step of composing a fixed viewpoint video from the tracking video and the panoramic image, for each of the plurality of frame images, Applying the image conversion performed on the removed image of the object from each of the frame images to the removed image generated from each of the frame images, and superimposing it on the panoramic image to obtain each frame image of the plurality of frame images included in the fixed viewpoint video A composition step, characterized by comprising.

[0015] The above program may be recorded on a non-transitory recording medium. The non-transitory recording medium can be distributed and sold independently of the computer. Here, the non-transitory recording medium refers to a tangible recording medium. Examples of the non-transitory recording medium include a compact disk, a flexible disk, a hard disk, a magneto-optical disk, a digital video disk, a magnetic tape, a semiconductor memory, and the like. Also, the transitory recording medium refers to the transmission medium (propagating signal) itself. Examples of the transitory recording medium include an electrical signal, an optical signal, an electromagnetic wave, and the like. Note that the temporary storage area is an area for temporarily storing data and programs, and is, for example, a volatile memory such as a RAM (Random Access Memory).

Effect of the Invention

[0016] According to the present invention, it is possible to provide a program, an image processing apparatus, and an image processing method capable of generating a video that seems to be taken of an object from a fixed viewpoint from a video obtained by tracking and photographing the object.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0018] (1. Overall Configuration) As shown in FIG. 1, the image processing apparatus 100 according to an embodiment of the present invention is communicably connected via a database 200, a terminal device 300, and a computer communication network 400 such as the Internet.

[0019] The image processing apparatus 100 is a device that generates a fixed viewpoint video as if the target is photographed from a fixed viewpoint from a tracking video in which the target is tracked and photographed. The image processing apparatus 100 provides the generated fixed viewpoint video in response to a request from a user of the terminal device 300.

[0020] The database 200 is a device that records the tracking video.

[0021] The terminal device 300 is a device used by a user who views the fixed viewpoint video. The terminal device 300 is, for example, a smartphone, a tablet computer, or the like.

[0022] (2. Hardware Configuration of Image Processing Apparatus) FIG. 2 is a block diagram showing the hardware configuration of the image processing apparatus 100.

[0023] As shown in FIG. 2, the image processing apparatus 100 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM 13, a recording medium 14, an output device 15, a communication device 16, and an input device 17. Each component is connected by a bus 18.

[0024] The CPU 11 controls the overall operation of the image processing apparatus 100, is connected to each component, and exchanges control signals and data.

[0025] The ROM 12 stores an operating program and various data necessary for the overall operation control of the image processing apparatus 100.

[0026] The RAM 13 is for temporarily recording data and programs, and holds programs and data read from the recording medium 14, as well as other data necessary for communication, etc.

[0027] The recording medium 14 is composed of a hard disk, a flash memory, etc., and records data to be processed by the image processing apparatus 100.

[0028] The output device 15 includes a display device such as an LCD (Liquid Crystal Display), and an audio output device such as a speaker. Under the control of the CPU 11, the output device 15 outputs, for example, data output from the CPU 11.

[0029] The communication device 16 includes a communication interface for connecting the image processing apparatus 100 to a computer communication network such as the Internet, and communicates with other information processing apparatuses, etc. via the communication device 16.

[0030] The input device 17 includes input devices such as buttons, keyboards, touch panels, and microphones. The input device 17 receives operation inputs from the user of the image processing apparatus 100 and outputs a signal corresponding to the received operation input to the CPU 11.

[0031] (3. Functional Configuration of the Image Processing Apparatus of the Embodiment) The functional configuration of the image processing apparatus 100 will be described with reference to FIG. 3.

[0032] Functionally, the image processing apparatus 100 includes a removal image generation unit 101, a panorama image generation unit 102, and a synthesis unit 103. In the present embodiment, the CPU 11 functions as the removal image generation unit 101, the panorama image generation unit 102, and the synthesis unit 103.

[0033] Hereinafter, the object to be tracked is the horse participating in a horse race, and the tracking video handled by the image processing apparatus 100 is a video in which the participating horse is tracked in the paddock (hereinafter referred to as the "paddock video"). The paddock video is stored in the database 200, and the image processing apparatus 100 acquires the tracking video from the database 200.

[0034] The removed image generation unit 101 generates a plurality of removed images obtained by removing the object from a plurality of frame images included in the tracking video in which the object is tracked.

[0035] For example, the tracking video X is composed of a plurality of frame images 510-1 to 510-N (N is a natural number of 1 or more). The removed image generation unit 101 generates removed images 520-1 to 520-N obtained by removing the area where the participating horse is shown from the frame images 510-1 to 510-N.

[0036] FIG. 4 shows an example of a frame image of the tracking video. The frame image 510-i in FIG. 4 includes, as a video, the participating horse 601 that is the object walking in the paddock.

[0037] Here, the removed image generation unit 101 generates a removed image by making the area where the object is drawn transparent in each of the plurality of frame images.

[0038] For example, the removed image generation unit 101 generates a removed image 520-i by making the area where the participating horse 601 in the frame image 510-i in FIG. 4 is drawn transparent. The removed image 520-i includes, as a video, the ground 602 and the fence 603 other than the participating horse 601 in the frame image 510-i.

[0039] The panorama image generation unit 102 performs collation between feature points included in the plurality of removed images, and generates a panorama image representing the background of the object by subjecting the feature points that match each other to image conversion in each of the plurality of removed images and then superimposing them.

[0040] A feature point is a point that has features distinguishable from other points in the image after removal, for example, a corner part of the subject. For the extraction of feature points in the image after removal, known methods are adopted.

[0041] Image conversion adopts known conversion methods used for generating panoramic images, such as perspective conversion, homography conversion, spherical projection conversion, and affine conversion.

[0042] Here, the panoramic image generation unit 102 executes collation while ignoring the feature points detected in the vicinity of the boundary of the transparent region.

[0043] For example, the panoramic image generation unit 102 performs collation without including the feature points detected in the vicinity of the boundary of the region where the running horse 601 is drawn in the collation of feature points with each other. Since feature points are points that can be distinguished from other points like the corners of the subject, it is considered that points in the vicinity of the region where the running horse 601 is drawn are likely to be extracted. However, the feature points in the vicinity of the region where the running horse 601 is drawn are not necessarily appropriate as background feature points, so they are not included in the collation. Note that the distance considered to be in the vicinity of the boundary of the transparent region is arbitrarily set by the user of the image processing apparatus 100 or the user who views the fixed viewpoint video.

[0044] The panoramic image generation unit 102 collates the feature points included in the removed images 520-1 to 520-N with each other, and overlaps the feature points that match each other after performing image conversion on the removed images 520-1 to 520-N, thereby generating a panoramic image showing the background of the running horse 601.

[0045] Fig. 5 shows an example of the panoramic image 700. The panoramic image 700 in Fig. 5 includes the ground 604 and the fence 605 where the ground 602 and the fence 603 of the removed images 520-1 to 520-N are joined together.

[0046] The composition unit 103 composes a fixed viewpoint video from the tracking video and the panoramic image.

[0047] For example, the synthesizing unit 103 synthesizes a fixed viewpoint video Y including frame images 530-1 to 530-N from a tracking video X including frame images 510-1 to 510-N and a panorama image 700.

[0048] Specifically, for each of the plurality of frame images, the synthesizing unit 103 applies an image transformation applied to the removed image from the respective frame image to the removed image generated from the respective frame image, and superimposes it on the panorama image to obtain each of the plurality of frame images included in the fixed viewpoint video.

[0049] For example, for the frame image 510-i, the synthesizing unit 103 applies the image transformation applied to the removed image 520-i to the image of the runaway horse 601 removed from the frame image 510-i, and superimposes the image of the runaway horse 601 removed from the frame image 510-i on the panorama image 700 to generate the frame image 530-i of the fixed viewpoint video.

[0050] Fig. 6 shows an example of a frame image of the fixed viewpoint video. The frame image 530-i in Fig. 6 includes an image of the runaway horse 601 removed from the frame image 510-i of the tracking video with the panorama image 700 as the background.

[0051] In this way, the synthesizing unit 103 superimposes the images of the runaway horses 601 removed from the frame images 510-1 to 510-N included in the tracking video X on the panorama image 700 to generate the frame images 530-1 to 530-N included in the fixed viewpoint video Y.

[0052] FIG. 7 shows a state where a fixed-viewpoint video synthesized from a tracking video and a panoramic image is being displayed. Page 800 of FIG. 7 is a page for providing information on racehorses participating in a horse race and is displayed on the screen of the terminal device 300. Page 800 of FIG. 7 includes a region 801 for displaying the fixed-viewpoint video, a region 802 for displaying information regarding the racehorse 601 (horse name “X1”) included in the fixed-viewpoint video displayed in region 801, a region 803 including options for displaying information on other racehorses participating in the same “Race A” as the racehorse 601, a button 804 for displaying a race card, a button 805 for changing the direction in which the racehorse 601 moves in the fixed-viewpoint video, a play / stop button 806, and a playback bar 807.

[0053] When button 805 is selected, a fixed-viewpoint video in which the racehorse 601 moves in the direction opposite to the direction in which the racehorse 601 moves in the fixed-viewpoint video displayed in region 801 is displayed in region 801. In the example of page 800 of FIG. 7, when button 805 is selected, a fixed-viewpoint video in which the racehorse 601 moves from left to right is displayed in region 801. The paddock video, which is a tracking video, shows the racehorse walking along an elliptical circuit, and is generally taken from a shooting location provided outside the circuit. Therefore, depending on the shooting timing, the paddock video may show the racehorse moving from right to left or from left to right. The image processing device 100 generates a fixed-viewpoint video in which the racehorse moves from right to left and a fixed-viewpoint video in which the racehorse moves from left to right for each racehorse, and switches between the two fixed-viewpoint videos in response to the selection of button 805. When loop playback is performed, the fixed-viewpoint video in which the racehorse moves from right to left and the fixed-viewpoint video in which the racehorse moves from left to right may be automatically switched and played back.

[0054] Also, the fixed-viewpoint video displayed in region 801 can be started and stopped and the playback time can be specified by operating the knobs of the play / stop button 806 and the playback bar 807.

[0055] Here, the synthesizing unit 103 includes an index used for determining the state of the racehorse in the fixed viewpoint video.

[0056] The index used for determining the state of the racehorse is an element serving as a criterion for determining whether the condition of the racehorse is good. For example, the state of the racehorse is determined by the stride of the racehorse, whether it has walked such that the footprint of the front leg is exceeded by the footprint of the hind leg, the neck stretch indicating the way the racehorse holds its neck, whether the racehorse is running around the outer periphery of the paddock, whether the walking of the racehorse has a certain rhythm, and the like.

[0057] Therefore, the synthesizing unit 103 includes the footprint of the racehorse in the fixed viewpoint video as an index. Thereby, the stride of the racehorse 601 can be easily confirmed.

[0058] For example, as shown in FIG. 7, the synthesizing unit 103 includes the footprint 808 of the front leg and the footprint 809 of the hind leg of the racehorse 601 in the fixed viewpoint video displayed in the region 801.

[0059] Alternatively, the synthesizing unit 103 includes the footprint of the front leg of the racehorse in the fixed viewpoint video as an index. Thereby, it can be easily confirmed whether the hind leg has exceeded the footprint of the front leg.

[0060] For example, the synthesizing unit 103 includes only the footprint 808 of the front leg of the racehorse 601 in the fixed viewpoint video displayed in the region 801. The footprint 808 of the front leg may disappear when the hind leg touches the ground, or may remain even after the hind leg touches the ground. For example, when the right hind leg touches the ground beyond the footprint 808 of the right front leg, the footprint 808 of the right front leg is made to emit light and then disappears, and when the right hind leg touches the ground without exceeding the footprint 808 of the right front leg, the footprint 808 of the right front leg does not emit light and disappears after the right hind leg touches the ground.

[0061] In addition, the synthesizing unit 103 includes a horizontal line close to the racehorse in the fixed viewpoint video as an index. Thereby, since it becomes easier to discriminate the angle of the neck indicated by the line from the head to the back of the racehorse, the neck stretch can be easily confirmed.

[0062] For example, the synthesizing unit 103 includes a horizontal line 810 in front of the starting horse 601 in the fixed-view video displayed in the area 801.

[0063] In addition, the synthesizing unit 103 includes an index indicating whether the starting horse is running around the outer periphery of the paddock in the fixed-view video.

[0064] For example, when the starting horse 601 has run around the outer periphery of the paddock for more than half a circle, the synthesizing unit 103 includes an index 811 indicating that it is running around the outer periphery in the fixed-view video displayed in the area 801.

[0065] In addition, the synthesizing unit 103 includes an index indicating whether the gait of the starting horse is at a constant rhythm in the fixed-view video.

[0066] For example, when the timing of the four legs of the starting horse touching the ground is set as one set and the walking rhythm is constant for 5 sets or more, the synthesizing unit 103 includes an index 812 indicating that the walking is at a constant rhythm in the fixed-view video displayed in the area 801.

[0067] In addition, in the fixed-view video, the synthesizing unit 103 highlights the part of the starting horse related to the evaluation based on the evaluation of the state of the starting horse.

[0068] For example, when the right hind leg crosses the footprint 808 of the right front leg, the synthesizing unit 103 determines that the state of the starting horse is good, and in the fixed-view video displayed in the area 801, the right hind leg that has crossed the footprint 808 of the right front leg is displayed so as to emit light.

[0069] In addition, the degree of light emission may be changed according to the front-back relationship between the footprint 808 of the front leg and the hind leg when the hind leg touches the ground. For example, the longer the distance that the right hind leg has crossed the footprint 808 of the right front leg, the stronger the right hind leg is displayed to emit light. Alternatively, the longer the distance that the right hind leg has crossed the footprint 808 of the right front leg, the stronger the footprint 808 of the right front leg may be displayed to emit light.

[0070] Also, for example, when the angle of the neck of the racehorse 601 is kept near horizontal for 5 seconds or more, the composition unit 103 determines that the state of the racehorse is good, and in the fixed-viewpoint video displayed in the area 801, the neck of the racehorse 601 is displayed so as to emit light.

[0071] (4. Operation of the Image Processing Apparatus According to the Embodiment) The operation of the image processing apparatus 100 according to the present embodiment will be described with reference to FIG. 8. For example, when the image processing apparatus 100 receives an instruction to create a fixed-viewpoint video from the tracking video, the image processing shown in FIG. 8 is started.

[0072] The removed image generation unit 101 generates a plurality of removed images from which the target has been removed from a plurality of frame images included in the tracking video in which the target is tracked (step S101).

[0073] For example, the removed image generation unit 101 generates removed images 520-1 to 520-N from which the area where the racehorse 601 is shown has been removed from the frame images 510-1 to 510-N included in the tracking video X.

[0074] The panorama image generation unit 102 performs collation of feature points included in the plurality of removed images, and overlaps the feature points that match each other after performing image conversion on each of the plurality of removed images, thereby generating a panorama image representing the background of the target (step S102).

[0075] For example, the panorama image generation unit 102 collates the feature points included in the removed images 520-1 to 520-N, and overlaps the feature points that match each other after performing image conversion on the removed images 520-1 to 520-N, thereby generating a panorama image 700 (FIG. 5) showing the background of the racehorse 601.

[0076] For each frame image of a plurality of frame images, the compositing unit 103 performs image conversion on the target image removed from each frame image and applied to the processed image generated from each frame image, and overlays it on the panoramic image, thereby generating each frame image of the plurality of frame images included in the fixed-viewpoint video (step S103).

[0077] For example, for the frame images 510-1 to 510-N included in the tracking video X, the compositing unit 103 overlays the image of the running horse 601 removed from the frame images 510-1 to 510-N on the panoramic image 700 to generate the frame images 530-1 to 530-N included in the fixed-viewpoint video Y.

[0078] According to this embodiment, it is possible to generate a video that seems to be taken of the target from a fixed viewpoint from a video that tracks the target. For example, from a video taken such that the position of the running horse is fixed within the shooting field like a paddock video, it is possible to generate a video showing the running horse moving the paddock as if taken from a fixed position.

[0079] Also, according to this embodiment, since an index used for determining the state of the running horse is included in the fixed-viewpoint video, the state of the running horse can be easily determined.

[0080] Also, according to this embodiment, in the fixed-viewpoint video, based on the evaluation of the state of the running horse, the part of the running horse related to the evaluation is highlighted, so that the state of the running horse can be presented in an easy-to-understand manner even to beginners who are not used to judging the state.

[0081] (5. Modification example) The embodiments of the present invention have been described above, but when implementing the present invention, various forms of modification and application are possible.

[0082] In the above embodiment, although one fixed-viewpoint video is displayed on the page 800 shown on the screen of the terminal device 300, it may include a plurality of fixed-viewpoint videos. For example, the page 800 in FIG. 7 may further include a fixed-viewpoint video synthesized from a past paddock video of the starting horse 601 (horse name "X1") and a panoramic image. Alternatively, the page 800 in FIG. 7 may further include a fixed-viewpoint video synthesized from a paddock video and a panoramic image of another starting horse that starts in the same "Race A" as the starting horse 601. When a plurality of fixed-viewpoint videos are displayed in parallel, the synthesizing unit 103 estimates the zoom level of the paddock video based on the size of an object such as a fence or a planter in the background, and corrects it so that the sizes of the starting horses included in the fixed-viewpoint videos are the same. Further, when a plurality of fixed-viewpoint videos are displayed in parallel, the synthesizing unit 103 may select a fixed-viewpoint video in which the starting horses included in the fixed-viewpoint videos move in the same direction and the walking start points are close.

[0083] Also, by applying the operation program that defines the operation of the image processing apparatus 100 according to the above embodiment to an existing personal computer or information terminal device, it is possible to make the personal computer or information terminal device function as the image processing apparatus 100 according to the embodiment.

[0084] The present invention can be implemented in various embodiments and modifications without departing from the broad spirit and scope of the present invention. Also, the above-described embodiments are for explaining the present invention and do not limit the scope of the present invention. That is, the scope of the present invention is indicated by the claims rather than the embodiments. And various modifications made within the scope of the claims and within the scope of the meaning of the invention equivalent thereto are considered to be within the scope of the present invention.

Industrial Applicability

[0085] According to the present invention, it is possible to provide a program, an image processing apparatus, and an image processing method capable of generating a video that captures an object from a fixed viewpoint from a video that captures the object while tracking it.

Explanation of Symbols

[0086] 11 CPU 12 ROM 13 RAM 14 Recording Medium 15 Output Device 16 Communication Device 17 Input Device 18 Bus 100 Image Processing Device 101 Removed Image Generation Unit 102 Panorama Image Generation Unit 103 Synthesis Unit 200 Database 300 Terminal Device 400 Computer Communication Network 510-1 to 510-N, 510-i Frame Images of Tracking Shooting Video 520-1 to 520-N, 520-i Images after Removal 530-1 to 530-N, 530-i Frame Images of Fixed Viewpoint Video 601 Runaway Horse 602, 604 Ground 603, 605 Fence 700 Panorama Image 800 Page 801, 802, 803 Area 804, 805 Button 806 Play / Pause Button 807 Play Bar 808, 809 Footprint 810 Horizontal Line 811, 812 Indicator

Claims

1. A computer A removed image generation unit that generates a plurality of removed images from which the target has been removed from a plurality of frame images included in a tracking video in which the target has been tracked and photographed; A panorama image generation unit that performs collation between feature points included in the plurality of removed images, and overlaps the feature points that match each other after performing image conversion on each of the plurality of removed images, thereby generating a panorama image representing the background of the target; A synthesis unit that synthesizes a fixed viewpoint video from the tracking video and the panorama image, For each of the plurality of frame images, By applying the image conversion applied to the removed image generated from each of the frame images to the image of the target removed from each of the frame images and overlapping it with the panorama image, each of the plurality of frame images included in the fixed viewpoint video is obtained. Synthesis unit, A program characterized by causing it to function as such.

2. The removed image generation unit generates the removed image by making the area where the target is drawn transparent in each of the plurality of frame images, The panorama image generation unit executes the collation while ignoring the feature points detected in the vicinity of the boundary of the transparent area. The program according to claim 1, characterized in that.

3. The target is a racehorse in a horse race, The tracking video is a video in which the racehorse is tracked and photographed in the paddock. The program according to claim 1 or 2, characterized in that.

4. The synthesis unit, Includes an index used for judging the state of the racehorse in the fixed viewpoint video. The program according to claim 3, characterized in that.

5. The synthesis unit, Includes the footprint of the racehorse in the fixed viewpoint video as the index. The program according to claim 4, characterized in that.

6. The synthesis unit, Includes the footprint of the front hoof of the racehorse in the fixed viewpoint video as the index. The program according to claim 4, characterized in that.

7. The synthesis unit, In the fixed viewpoint video, based on the evaluation of the state of the racehorse, the part of the racehorse related to the evaluation is highlighted. The program according to claim 3, characterized in that.

8. An object removal image generation unit that generates a plurality of removed images from which the object has been removed from a plurality of frame images included in a tracking video in which the object has been tracked. A panorama image generation unit that performs collation of feature points included in the plurality of removed images, and overlaps feature points that match each other after performing image conversion on each of the plurality of removed images to generate a panorama image representing the background of the object. A composition unit that composes a fixed viewpoint video from the tracking video and the panorama image, For each of the plurality of frame images, Apply the image conversion applied to the removed image generated from each of the frame images to the image of the object removed from each of the frame images, and overlap it with the panorama image to obtain each of the plurality of frame images included in the fixed viewpoint video. Composition unit, An image processing apparatus characterized by comprising.

9. An image processing method executed by an image processing apparatus, A removed image generation step of generating a plurality of removed images from which the object has been removed from a plurality of frame images included in a tracking video in which the object has been tracked, A panorama image generation step of performing collation of feature points included in the plurality of removed images, and overlapping feature points that match each other after performing image conversion on each of the plurality of removed images to generate a panorama image representing the background of the object, A composition step of composing a fixed viewpoint video from the tracking video and the panorama image, For each of the plurality of frame images, Apply the image conversion applied to the removed image generated from each of the frame images to the image of the object removed from each of the frame images, and overlap it with the panorama image to obtain each of the plurality of frame images included in the fixed viewpoint video. Composition step, An image processing method characterized by comprising.

Images