Image synthesis device and program

Abstract

To provide an image synthesis device and program for generating a more natural composite image of a 3D model and a 2D image.SOLUTION: An image synthesis device 1 includes a 3D model attribute acquisition unit 11 for acquiring a 3D model of an object, and a 3D model attribute including the type of object, the size of the object, a generation method of the 3D model, the number of points, and the type of shot, a 2D image attribute acquisition unit 12 for acquiring a 2D image and a 2D image resolution which is the resolution of the 2D image, an image quality determination unit 14 for determining, as an image quality, a resolution-lower one of an estimated resolution when a 3D model of the acquired 3D model attribute is converted into a 2D image and the 2D image resolution, and a video composite unit 15 in which when the image quality is the estimated resolution, the 2D image whose resolution has been converted is to that image quality is combined with the 3D model which has been converted into the 2D image, and, when the image quality is the 2D image resolution, the 2D image acquired by the 2D image attribute acquisition unit 12 is combined with the 3D model which has been converted into the 2D image of that image quality.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a video synthesizing device and a program. [Background technology]

[0002] As a method for synthesizing multiple images of different formats, there is a method of synthesizing after converting the format of one image to the format of the other image. For example, when synthesizing a 3D (three-dimensional) model and a 2D (two-dimensional) image, the 3D model is converted to a 2D image and then synthesized. Here, there is a technology for preventing the subjective image quality of the 3D model from decreasing before and after conversion when converting from a 3D model to a 2D image (for example, see Patent Document 1). In addition, as other image synthesis methods, there is a technology for generating a synthetic image by mapping a 2D image onto a VR image (for example, see Patent Document 2) and a technology for superimposing multiple high-quality images (for example, see Patent Document 3). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-063882 [Patent Document 2] Patent Publication No. 2022-21886 [Patent Document 3] JP 2018-109946 A Summary of the Invention [Problem to be solved by the invention]

[0004] When combining images of different formats, such as combining a 3D model with a 2D image, the combined image may look unnatural. This can be due to a variety of factors, including the position where the images are combined, the size of the combined image, and light reflections in the combined image. Until now, unnaturalness was eliminated by manually checking the combined image with the human eye and adjusting various parameters, which required a great deal of effort.

[0005] The technology of Patent Document 1 described above is a method of converting a point cloud into voxels, and since it takes into consideration image quality degradation in the point cloud alone, it does not mention synthesis with 2D images. In addition, the technology of Patent Document 2 partially increases the resolution of noteworthy parts of a VR image, and the technology of Patent Document 3 superimposes a flat image of high image quality on a spherical image of low image quality when integrating a still image of a spherical image and a flat image. However, these do not make the synthesized image look natural.

[0006] The present invention has been made in consideration of the above circumstances, and provides an image synthesis device and program capable of generating a more natural synthesized image of a 3D model and a 2D image. [Means for solving the problem]

[0007] [1] One aspect of the present invention is an image synthesis device including: a three-dimensional model, which is data in which an object is rendered by a point cloud; a three-dimensional model attribute acquisition unit that acquires three-dimensional model attributes including a type of the object, a size of the object, a generation method of the three-dimensional model, a number of points of the three-dimensional model, and a type of shot; a two-dimensional video attribute acquisition unit that acquires a two-dimensional video and a two-dimensional video resolution that is a resolution of the two-dimensional video; an image quality determination unit that determines an image quality to be the lower of an estimated resolution when the three-dimensional model of the three-dimensional model attributes is converted into a two-dimensional video and the two-dimensional video resolution; and, when the estimated resolution is determined as the image quality, a video synthesis unit that generates a synthetic image by synthesizing the two-dimensional video whose resolution has been converted to the image quality and the three-dimensional model converted into the two-dimensional video, and, when the two-dimensional video resolution is determined as the image quality, a video synthesis unit that generates a synthetic image by synthesizing the two-dimensional video acquired by the two-dimensional video attribute acquisition unit and the three-dimensional model converted into the two-dimensional video of the image quality.

[0008] [2] One aspect of the present invention is the above-mentioned image synthesis device, which includes an image quality memory unit that stores an estimated resolution in association with an object type, an object size, a three-dimensional model generation method, a number of points, and a shot type, and the image quality determination unit reads out from the image quality memory unit the estimated resolution corresponding to a word representing the type of object indicated by the three-dimensional model attributes, a synonym of the word representing the type of object, or a superordinate word of the word representing the type of object, and the object size, the generation method, the number of points, and the shot type indicated by the three-dimensional model attributes.

[0009] [3] One aspect of the present invention is the above-mentioned image synthesis device, wherein the image synthesis unit reduces the number of points of the three-dimensional model and converts the three-dimensional model with the reduced number of points into a two-dimensional image, thereby obtaining the three-dimensional model converted into a two-dimensional image of the above image quality.

[0010] [4] One aspect of the present invention is the above-mentioned image synthesis device, wherein the two-dimensional image attribute acquisition unit acquires a screen size of the acquired two-dimensional image, and if the aspect ratio of the acquired screen size is a predetermined ratio, the number of horizontal pixels of the screen size is set to the two-dimensional image resolution, and if the aspect ratio of the acquired screen size is not the predetermined ratio, the number of horizontal pixels in the screen size is not changed and the aspect ratio is set to the predetermined ratio, and the two-dimensional image resolution is set to the number of horizontal pixels in the screen size.

[0011] [5] One aspect of the present invention is a program for causing a computer to function as: a three-dimensional model attribute acquisition means for acquiring a three-dimensional model, which is data in which an object is rendered by a point cloud, and three-dimensional model attributes including the type of the object, the size of the object, a generation method of the three-dimensional model, the number of points in the three-dimensional model, and a type of shot; a two-dimensional video attribute acquisition means for acquiring a two-dimensional video and a two-dimensional video resolution that is the resolution of the two-dimensional video; an image quality determination means for determining, as an image quality, the lower of an estimated resolution when the three-dimensional model of the three-dimensional model attributes is converted into a two-dimensional video and the two-dimensional video resolution; and, when the estimated resolution is determined as the image quality, image synthesis means for generating a synthetic image by synthesizing the two-dimensional video whose resolution has been converted into the image quality and the three-dimensional model converted into the two-dimensional video, and, when the two-dimensional video resolution is determined as the image quality, image synthesis means for generating a synthetic image by synthesizing the two-dimensional video acquired by the two-dimensional video attribute acquisition means and the three-dimensional model converted into the two-dimensional video of the image quality. Effect of the Invention

[0012] According to the present invention, it is possible to generate a more natural composite image of a 3D model and a 2D image. [Brief description of the drawings]

[0013] [Figure 1] FIG. 2 is a diagram showing an overview of the processing of an image synthesizing apparatus according to an embodiment of the present invention. [Diagram 2] 1 is a functional block diagram of an image synthesizing device according to a first embodiment. [Diagram 3] FIG. 4 is a diagram showing an example of an image quality DB according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a voxel size DB according to the first embodiment. [Diagram 5] FIG. 2 is a flow diagram showing an image synthesis process of the image synthesis device according to the first embodiment. [Figure 6] FIG. 4 is a flow chart showing search processing of an image quality determination unit according to the first embodiment. [Figure 7]FIG. 2 is a diagram illustrating an example of processing performed by the image synthesizing device according to the first embodiment. [Figure 8] FIG. 2 is a diagram illustrating an example of processing performed by the image synthesizing device according to the first embodiment. [Figure 9] FIG. 11 is a functional block diagram of an image synthesizing device according to a second embodiment. [Figure 10] FIG. 11 is a flowchart showing an image synthesis process of the image synthesis device according to the second embodiment. [Figure 11] FIG. 1 is a diagram illustrating an example of a hardware configuration of an image synthesizing device according to first and second embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. As described above, there are various factors that can make a 3D model (also called a 3D object) look unnatural when compositing a 2D image, such as position, size, and light reflection. In this embodiment, the focus is on the resolution, among other factors that can make a 3D model look unnatural. For example, when a 3D model of a notable person is composited on a 2D live-action image of a park landscape, it is unnatural if the resolution of the person is lower than that of the background and appears blurry. However, only the number of points is currently treated as a resolution index of a point cloud. Generally, the higher the density of points, the higher the resolution is, but there is no clear indication of how high the resolution will be when composited with a different format. In addition, the amount of data in a point cloud is large, and it is also necessary to effectively omit points in order to perform efficient processing.

[0015] FIG. 1 is a diagram showing an overview of the processing of an image synthesizing device according to an embodiment of the present invention. When synthesizing a 3D model A in a point cloud format such as ply and a 2D image B in a tiff or PNG format, the image synthesizing device of this embodiment adjusts the 3D model A or the 2D image B so that the resolution of both images is balanced. That is, the image synthesizing device of this embodiment refers to a DB (database) that lists in advance what the resolution of the 3D model will look like when 2D rendering is performed, and obtains an estimated resolution of a rendered 2D image A' obtained by rendering the 3D model A into a 2D image. The image synthesizing device determines the resolution at the time of synthesis so as to balance the resolution between the estimated resolution of the rendered 2D image A' and the resolution of the 2D image B. The image synthesizing device downsamples the 3D model A to the determined resolution, or converts the resolution of the 2D image B, to obtain a synthetic image C that combines the resolution between the images of these different formats. Detailed embodiments of the present invention will be described below.

[0016] [First embodiment] Fig. 2 is a functional block diagram showing the configuration of the image synthesizer 1 according to the first embodiment. Fig. 2 shows only functional blocks related to this embodiment. The image synthesizer 1 can be realized, for example, by one or more computer devices. The image synthesizer 1 is configured to include a 3D model attribute acquisition unit 11, a 2D image attribute acquisition unit 12, an image quality information storage unit 13, an image quality determination unit 14, an image synthesis unit 15, a terminal-specific image quality conversion unit 16, and a terminal resolution acquisition unit 17.

[0017] The 3D model attribute acquisition unit 11 acquires a 3D model and 3D model attribute information. The 3D model attribute information includes information on the type of object represented by the 3D model, the size of the original model, the 3D model generation method, the number of points of the 3D model, and the shot type of the 3D model. The size of the original model represents the actual size of the object represented by the 3D model. The types of shots include, for example, a shot FS (full shot) that is a shot of the whole of a person or object, a shot WS (waist shot) that is a shot ranging from the waist to the head of a person, etc., a shot BS (bust shot) that is a shot ranging from the chest to the head of a person, etc., and a shot US (up shot) that is a shot ranging from the shoulder to the head of a person, etc. The 3D model attribute acquisition unit 11 may acquire each piece of information included in these 3D model attribute information from, for example, the header of a ply file in which the 3D model is described, may read it from a recording medium, or may receive it from another device connected via a network. Each piece of information about the 3D model attribute information may be input by an input unit (not shown) included in the image synthesis device 1, and the 3D model attribute acquisition unit 11 may analyze the 3D model to acquire some or all of the information about the 3D model attribute information. The 3D model attribute acquisition unit 11 may also acquire the size of the original model depending on the type of object.

[0018] The 2D video attribute acquisition unit 12 acquires 2D video and 2D video attribute information. The 2D video attribute information includes information on the resolution of the acquired 2D video. The resolution is represented by an image size, such as 7680×4320 (pixels). The 2D video attribute acquisition unit 12 acquires the image size from the 2D video by any existing method. As an example, .shape of OpenCV (Open Source Computer Vision Library), which is an open source library, can be used. Hereinafter, the resolution of the acquired 2D video is referred to as 2D video resolution. Note that the 2D video attribute acquisition unit 12 may read the 2D video attribute information from a recording medium, or may receive it from another device connected via a network. The 2D video attribute information may be input by an input unit (not shown) included in the video synthesis device 1.

[0019] The image quality information storage unit 13 stores an image quality DB. The image quality DB is information that associates attribute information of a 3D model with an estimated resolution of an image obtained by 2D rendering the 3D model of the attribute information. The estimated resolution is expressed as the number of horizontal pixels on a 16:9 screen. For example, an estimated resolution of "1920" means an image quality equivalent to 2K. The image quality DB is created in advance and stored in the image quality information storage unit 13.

[0020] The image quality determination unit 14 reads out the estimated resolution corresponding to the 3D model attribute information acquired by the 3D model attribute acquisition unit 11 from the image quality DB stored in the image quality information storage unit 13, and compares the read out estimated resolution with the 2D video resolution. The image quality determination unit 14 determines the lower of the estimated resolution and the 2D video resolution as the composite video resolution.

[0021] The image synthesis unit 15 renders the 3D model into a 2D image with a synthetic image resolution to generate a rendered 2D image. The image synthesis unit 15 synthesizes the rendered 2D image with the synthetic image resolution and the 2D image with the synthetic image resolution to generate a synthetic image. The image synthesis unit 15 has an image quality conversion unit 151, a voxel size information storage unit 152, a 2D rendering unit 153, and a 2D image synthesis unit 154.

[0022] When the composite video resolution is the estimated resolution of the 3D model, the image quality conversion unit 151 outputs the 3D model as is to the 2D rendering unit 153, and outputs the 2D video converted to the composite video resolution to the 2D video synthesis unit 154. When the composite video resolution is the 2D video resolution, the image quality conversion unit 151 outputs a 3D model downsampled to the composite video resolution after 2D rendering to the 2D rendering unit 153, and outputs the 2D video as is without changing the resolution to the 2D video synthesis unit 154. The image quality conversion unit 151 can use existing technology such as Voxel Grid Filter as a downsampling method for reducing the number of points.

[0023] The voxel size information storage unit 152 stores a voxel size DB. The voxel size DB is information that associates attribute information of a 3D model, an estimated resolution of an image obtained by 2D rendering the 3D model, and the voxel size of the 3D model. When downsampling a 3D model, the image quality conversion unit 151 reads out the 3D model attribute information acquired by the 3D model attribute acquisition unit 11 and a voxel size corresponding to the composite image resolution from the voxel size DB, and downsamples the 3D model using the read voxel size as a parameter of the Voxel Grid Filter.

[0024] The 2D rendering unit 153 renders the 3D model input from the image quality conversion unit 151 into a 2D image, and generates a rendered 2D image. The 2D rendering unit 153 outputs the generated rendered 2D image to the 2D image synthesis unit 154. The 2D image synthesis unit 154 synthesizes the rendered 2D image input from the 2D rendering unit 153 and the 2D image input from the image quality conversion unit 151, and generates a synthesized image. The 2D image synthesis unit 154 outputs the generated synthesized image to the terminal-specific image quality conversion unit 16.

[0025] The terminal-specific image quality conversion unit 16 converts the composite image input from the image composition unit 15 into an image that matches the terminal resolution acquired by the terminal-specific image quality conversion unit 16. The terminal resolution is the resolution of the viewing terminal 2. The terminal-specific image quality conversion unit 16 uses an existing method such as bicubic for this conversion. The terminal-specific image quality conversion unit 16 delivers the composite image converted to match the terminal resolution to the viewing terminal 2 by broadcast or via a communication network. The terminal resolution acquisition unit 17 acquires the terminal resolution from the viewing terminal 2, for example, via a communication network.

[0026] Fig. 3 is a diagram showing an example of an image quality DB. The image quality DB is information that associates attribute information, including the type of object, the size of the original model, the generation method of the 3D model, the number of points, and the type of shot, with an estimated resolution. The estimated resolution is expressed as the number of horizontal pixels in a 16:9 image. The technology in Reference 1 below is used to determine the estimated resolution to be set in the image quality DB.

[0027] (Reference 1) Kikuyoshi Miura et al., “Evaluation of Resolution of 3D Objects Displayed in 2D,” Institute of Electronics, Information and Communication Engineers General Conference D-11-15, March 2023

[0028] Specifically, the image obtained by 2D rendering the 3D model is compared with 2D images with gradually reduced resolution, and the person selects the 2D image with the same or closest resolution. In the image quality DB, the resolution of the 2D image selected for the 3D model is set as the estimated resolution, linked to the attribute information of the 3D model. In addition, the type of shot is related to the size of the object in the image, and is therefore a factor in determining the estimated resolution.

[0029] FIG. 4 is a diagram showing an example of a voxel size DB. The voxel size DB is information that associates the type of object, the size of the original model, the 3D model generation method, the type of shot, and the estimated resolution with the voxel size. The estimated resolution is expressed as the number of horizontal pixels in a 16:9 image. The technology in Reference 1 above is used to determine the voxel size to be set in the voxel size DB.

[0030] Specifically, a 2D image with a predetermined resolution is compared with each rendered 2D image obtained by stepwise downsampling a 3D model while changing the voxel size, followed by 2D rendering, and a person selects the rendered 2D image whose resolution matches or is closest to the 2D image. The voxel size DB is set by linking the object type of the 3D model from which the selected rendered 2D image was obtained, the size of the original model, the 3D model generation method, and the type of shot, the estimated resolution indicating the resolution of the 2D image, and the voxel size used to generate the selected rendered 2D image.

[0031] 5 is a flow diagram showing the processing of the image synthesizer 1. First, the 3D model attribute acquisition unit 11 of the image synthesizer 1 acquires a 3D model and 3D model attribute information of the 3D model (step S105). The acquired 3D model attribute information is described as acquired 3D model attribute information. In addition, the 2D video attribute acquisition unit 12 acquires a 2D video and 2D video attribute information of the 2D video (step S110).

[0032] The image quality determination unit 14 sets the object type, the size of the original model, the 3D model generation method, the number of points, and the type of shot indicated by the acquired 3D model attribute information as the estimated resolution search conditions. The image quality determination unit 14 reads out the estimated resolution associated with the attribute information that matches the estimated resolution search conditions from the image quality DB stored in the image quality information storage unit 13 (step S115). The image quality determination unit 14 compares the 2D image resolution indicated by the 2D image attribute information acquired in step S110 with the estimated resolution read out in step S115 (step S120).

[0033] When the image quality determination unit 14 determines that the 2D video resolution is higher than the estimated resolution (step S125: YES), it determines the estimated resolution as the composite video resolution and outputs the determined composite video resolution to the image synthesis unit 15 (step S130). The image quality conversion unit 151 of the image synthesis unit 15 converts the 2D video acquired in step S110 into a 2D video at the composite video resolution and outputs it to the 2D video synthesis unit 154 (step S135). Furthermore, the image quality conversion unit 151 outputs the 3D model as it is to the 2D rendering unit 153.

[0034] The 2D rendering unit 153 renders the 3D model input from the image quality conversion unit 151 into a 2D image, and generates a rendered 2D image (step S140). The 2D rendering unit 153 outputs the generated rendered 2D image to the 2D image synthesis unit 154. The 2D image synthesis unit 154 synthesizes the rendered 2D image input from the 2D rendering unit 153 and the 2D image input from the image quality conversion unit 151, and generates a synthesized image (step S145). The 2D image synthesis unit 154 outputs the generated synthesized image to the terminal-specific image quality conversion unit 16.

[0035] The terminal resolution acquisition unit 17 acquires the terminal resolution of the viewing terminal 2 and outputs it to the terminal-specific image quality conversion unit 16 (step S150). The terminal-specific image quality conversion unit 16 converts the composite image input from the image synthesis unit 15 into a distribution image of the terminal resolution acquired by the terminal resolution acquisition unit 17 (step S155). The terminal-specific image quality conversion unit 16 distributes the distribution image converted to the terminal resolution to the viewing terminal 2 (step S160).

[0036] On the other hand, when the image quality determination unit 14 determines that the estimated resolution is higher than the 2D image resolution (step S125: NO), it determines the 2D image resolution as the composite image resolution and outputs the determined composite image resolution to the image synthesis unit 15 (step S165). The image quality conversion unit 151 of the image synthesis unit 15 sets the type of object, the size of the original model, the generation method and shot type of the 3D model, and the composite image resolution, which are indicated by the acquired 3D model attribute information, as voxel size search conditions. The image quality conversion unit 151 of the image synthesis unit 15 reads out a voxel size corresponding to the voxel size search conditions from the voxel size DB stored in the voxel size information storage unit 152. The image quality conversion unit 151 downsamples the 3D model using the read out voxel size (step S170). The image quality conversion unit 151 outputs the downsampled 3D model to the 2D rendering unit 153, and outputs the 2D image as it is to the 2D image synthesis unit 154. The image synthesizer 1 performs the processes from step S140 onwards.

[0037] Note that the image synthesizer 1 may perform the process of step S110 at any timing before step S120. Also, the image synthesizer 1 may perform the process of step S150 at any timing before step S155.

[0038] In the process of step S115, the image quality determination unit 14 searches the image quality DB using the estimated resolution search conditions obtained from the acquired 3D model attribute information to obtain the estimated resolution, and determines the composite image resolution, which is the image quality of the composite image. However, there are cases where the image quality DB does not contain attribute information that completely matches the information set in the acquired 3D model attribute information. In this case, the image quality determination unit 14 reads out the estimated resolution corresponding to the attribute information that matches the acquired 3D model attribute information in terms of priority of (1) 3D model generation method, (2) shot type, (3) object type, and (4) number of points.

[0039] Specifically, for example, the image quality determination unit 14 reads out the estimated resolution as follows. First, the image quality determination unit 14 extracts 3D model attribute information in which the 3D model generation method included in the acquired 3D model attribute information is set from among the attribute information set in the image quality DB, and sets it as first extracted attribute information. The image quality determination unit 14 extracts the first extracted attribute information in which the shot type included in the acquired 3D model attribute information is set, and sets it as second extracted attribute information. If the shot type included in the acquired 3D model attribute information is not set in any of the first extracted attribute information, the image quality determination unit 14 sets all of the first extracted attribute information as second extracted attribute information.

[0040] Next, the image quality determination unit 14 extracts, from the second extracted attribute information, information in which the object type included in the acquired 3D model attribute information is set as the third extracted attribute information. If the object type included in the acquired 3D model attribute information is not set in any of the second extracted attribute information, the image quality determination unit 14 sets all of the second extracted attribute information as the third extracted attribute information.

[0041] Next, the image quality determination unit 14 extracts, from among the third extracted attribute information, information to which the number of points included in the acquired 3D model attribute information is set as the fourth extracted attribute information. If the object type included in the acquired 3D model attribute information is not set in any of the third extracted attribute information, the image quality determination unit 14 sets all of the third extracted attribute information as the fourth extracted attribute information. The image quality determination unit 14 sets the most frequently occurring estimated resolution among the estimated resolutions linked to the fourth extracted attribute information as the estimated resolution corresponding to the acquired 3D model attribute information. Alternatively, the image quality determination unit 14 may set the average of the estimated resolutions linked to the fourth extracted attribute information as the estimated resolution corresponding to the acquired 3D model attribute information.

[0042] Furthermore, when determining whether the object type in the acquired 3D model attribute information matches the object type set in the image quality DB, the image quality determination unit 14 may determine whether the object type matches based on similar concepts, not limited to character matching. For example, the image quality determination unit 14 may treat dog and canine as equivalent object type names using a thesaurus such as a classification vocabulary table or WordNet. Furthermore, when the object type name "wolf" set in the acquired 3D model attribute information is not used in the image quality DB, the image quality determination unit 14 may refer to the image quality DB using "dog" acquired from the thesaurus as a synonym or superordinate concept instead of "wolf". When the image quality determination unit 14 cannot acquire either a synonym or superordinate concept of the object type name set in the acquired 3D model attribute information, it acquires from the image quality DB an estimated resolution of an object with a similar size, or an object with a similar number of points if there is no size information. Synonym processing is described in Reference 2 below.

[0043] (Reference 2) "[Natural Language] Extracting Synonyms with Wordnet and Python," [online], [Retrieved June 21, 2023], Internet<https: / / www.yoheim.net / blog.php?q=20160201> , 2016

[0044] The resolution of a rendered 2D image generated from a 3D model varies depending on various factors, such as animal fur, shiny objects, objects with a lot of reflections, difficulty in reproducing color gradations, etc. Since images of similar types of objects are likely to have similar characteristics, in this embodiment, the image quality DB is searched using synonyms and superordinate concepts.

[0045] 6 is a flow chart showing an example of processing in which the image quality determination unit 14 searches the image quality DB using a similar concept. In step S115, the image quality determination unit 14 performs the processing shown in FIG.

[0046] If the image quality determination unit 14 determines that the object type name (hereinafter referred to as the acquired object type name) set in the acquired 3D model attribute information is included in the image quality DB (step S205: YES), it acquires an estimated resolution from the image quality DB using the object type, size of the original model, 3D model generation method, number of points, and type of shot indicated by the acquired 3D model attribute information as estimated resolution search conditions (step S210).

[0047] When the image quality determination unit 14 determines that the acquired object type name is not included in the image quality DB (step S205: NO), it reads out a synonym for the acquired object type name from the thesaurus. The thesaurus may be stored in a computer server or the like connected to the image synthesizer 1, or may be stored in a storage device of the image synthesizer 1. When the image quality determination unit 14 determines that the read out synonym is included in the image quality DB (step S215: YES), it obtains an estimated resolution from the image quality DB using the read out synonym and the size of the original model, the 3D model generation method, the number of points, and the type of shot indicated by the acquired 3D model attribute information as estimated resolution search conditions (step S220).

[0048] When the image quality determination unit 14 determines that the read synonym is not included in the image quality DB (step S215: NO), it reads out the superordinate concept of the acquired object type name from the thesaurus. When the image quality determination unit 14 determines that the read superordinate concept is included in the image quality DB (step S225: YES), it acquires the estimated resolution from the image quality DB using the read superordinate concept and the size of the original model, the 3D model generation method, the number of points, and the type of shot indicated by the acquired 3D model attribute information as the estimated resolution search conditions (step S230).

[0049] When the image quality determination unit 14 determines that the read superordinate concept is not included in the image quality DB (step S225: NO), it determines whether or not the size of the original model can be acquired from the acquired 3D model attribute information (step S235).When the image quality determination unit 14 determines that the size of the original model can be acquired (step S235; YES), it reads out from the image quality DB an estimated resolution that is associated with attribute information that matches the size of the original model, the 3D model generation method, the number of points, and the type of shot indicated by the acquired 3D model attribute information, and is closest to the size of the original model indicated by the acquired 3D model attribute information (step S240).

[0050] If the image quality determination unit 14 determines that it cannot obtain the size of the original model (step S235: NO), it reads out from the image quality DB an estimated resolution that matches the size of the original model, the 3D model generation method, and the shot type indicated by the obtained 3D model attribute information, and that is associated with attribute information that is closest to the number of points indicated by the obtained 3D model attribute information (step S245).

[0051] Even if the acquired object type name is not included in the voxel size DB, the image quality determination unit 14 performs the same process as in Fig. 6. That is, the image quality determination unit 14 searches the voxel size DB using synonyms, superordinate concepts, etc. instead of the acquired object type name, and if synonyms and superordinate concepts cannot be used, acquires a voxel size of a model similar in size to the original model.

[0052] FIG. 7 is a diagram showing a specific example of processing by the image synthesizer 1. FIG. 7 shows an example of a case where the resolution of the 3D model is lower than that of the 2D image. A specific example of processing by the image synthesizer 1 will be described with reference to FIG. 7 and FIG. 5. Here, as shown in FIG. 1, a case is considered in which a 3D model A (point cloud) of a person is synthesized with a 2D image B of an ocean background to obtain a synthetic image C. The image synthesizer 1 first acquires attribute information of both the 3D model A and the 2D image B. Next, the image synthesizer 1 acquires an estimated resolution that matches the attribute information of the 3D model A from the image quality DB, and determines the image quality by comparing it with the resolution of the 2D image B. The image synthesizer 1 performs size conversion of the 2D image B according to the determined image quality. The image synthesizer 1 converts the resolution of the synthetic image C, which is obtained by synthesizing the rendered 2D image A' obtained by rendering the 3D model A into a 2D image and the 2D image B, based on the terminal resolution of the viewing terminal 2, and distributes the result.

[0053] More specifically, the 3D model attribute acquisition unit 11 receives an input of the 3D model A and the shot type "FS". The 3D model attribute acquisition unit 11 acquires the object type "person", the size of the original model "170 cm", the object generation method "method A", and the number of points "2,000,000 (points)" from the file of the 3D model A (step S105). Meanwhile, the 2D video attribute acquisition unit 12 receives an input of the 2D video B. The 2D video attribute acquisition unit 12 analyzes the input 2D video B and acquires the 2D video resolution "7680×4320 (pixels)" (step S110).

[0054] The image quality determination unit 14 uses the object type "person", the size of the original model "170 cm", the object generation method "Method A", the number of points "2,000,000 (points)", and the shot type "FS" as the estimated resolution search conditions, and reads out the estimated image quality "960" from the image quality DB shown in Fig. 3 (step S115). Since the estimated resolution "960" is smaller than the 2D image resolution (number of horizontal pixels) "7680", the image quality determination unit 14 determines the composite image resolution to be "960" (steps S120, S125: YES, step S130).

[0055] The image quality conversion unit 151 converts the 2D image B into a 2D image B' with a resolution of "960 x 540" (step S135). The 2D rendering unit 153 renders the 3D model A into a 2D image to generate a rendered 2D image A' (step S140). The 2D image synthesis unit 154 synthesizes the rendered 2D image A' with the 2D image B' with a resolution of "960 x 540" to generate a synthesized image C with a resolution of "960 x 540" (step S145). The terminal-specific image quality conversion unit 16 converts the synthesized image C with a resolution of "960 x 540" into a distribution image C' with a terminal resolution of "1920 x 1080" and distributes it (steps S150, S155).

[0056] Fig. 8 is a diagram showing an example of specific processing by the image synthesizing device 1 when the perceived resolution of the 3D model is higher than that of the 2D video. A case where the perceived resolution of the 3D model is higher than that of the 2D video will be described with reference to Fig. 8 and Fig. 5. When the perceived resolution of the 3D model is higher than that of the 2D video, the image quality conversion unit 151 of the video synthesizing unit 15 adjusts the image quality by reducing the number of points of the 3D model.

[0057] Specifically, the 3D model attribute acquisition unit 11 receives an input of the 3D model A and the shot type "FS". The 3D model attribute acquisition unit 11 acquires the object type "raccoon dog", the size of the original model "80 cm", the object generation method "method B", and the number of points "30,000,000 (points)" from the file of the 3D model A (step S105). Meanwhile, the 2D video attribute acquisition unit 12 receives an input of the 2D video B. The 2D video attribute acquisition unit 12 analyzes the input 2D video B and acquires the 2D video resolution "1920 x 1080 (pixels)" (step S110).

[0058] The image quality determination unit 14 uses the object type "raccoon dog", the size of the original model "80 cm", the object generation method "Method B", the number of points "30,000,000 (points)", and the shot type "FS" as the estimated resolution search conditions, and reads out the estimated image quality "5760" from the image quality DB shown in Fig. 3 (step S115). Since the estimated resolution "5760">2D image resolution "1920", the image quality determination unit 14 determines the composite image resolution to be "1920" (steps S120, S125: NO, S165).

[0059] The image quality conversion unit 151 reads out the voxel size "0.08" corresponding to the object type "raccoon dog", the size of the original model "80 cm", the object generation method "Method B", the shot type "FS", and the composite image resolution "1920" from the voxel size DB stored in the voxel size information storage unit 152. The image quality conversion unit 151 downsamples the 3D model with a Voxel Grid Filter using the voxel size "0.08" as the parameter value, and obtains a 3D model A" with 2,000,000 points (step S170).

[0060] The image quality conversion unit 151 may read out from the image quality DB the number of points "2,000,000" corresponding to the object type "raccoon dog", the object generation method "method B", the shot type "FS", and the estimated resolution "1920". The image quality conversion unit 151 downsamples the 3D model A to generate a 3D model A" so that the number of points becomes "2,000,000". In this case, a technique is required to convert the 3D model to the target number of points.

[0061] The 2D rendering unit 153 renders the downsampled 3D model A" into a 2D image to generate a rendered 2D image A' (step S140). The 2D image synthesis unit 154 synthesizes the rendered 2D image A' with the 2D image B with a resolution of "1920 x 1080" to generate a synthesized image C with a resolution of "1920 x 1080" (step S145). Because the terminal resolution is "1920 x 1080", the terminal-specific image quality conversion unit 16 distributes the synthesized image C with a resolution of "1920 x 1080" as is as the distribution image C' (steps S150, S155).

[0062] The image synthesizer 1 may receive an input of an arbitrary composite video resolution. In this case, the image synthesizer 1 does not perform the processes of steps S115 to S130 and S165 in FIG. 5. The image synthesizer 1 performs the processes of steps S135 and S170 so as to obtain the input composite video resolution, and then performs the processes of steps S140 and after. The image synthesizer 1 may also receive input of different composite video resolutions for the 3G model and the 2D video. In this case, the image synthesizer 1 performs the process of step S135 so as to obtain the input first composite video resolution, performs the process of step S170 so as to obtain the input second composite video resolution, and then performs the processes of step S140 and after.

[0063] [Second embodiment] In the second embodiment, a case will be described in which the aspect ratio of a 2D image input to an image synthesizer differs from the aspect ratio assumed in the image quality DB. The second embodiment will be described focusing on the differences from the first embodiment.

[0064] Fig. 9 is a functional block diagram showing the configuration of an image synthesizer 1a according to the second embodiment. Fig. 9 shows only functional blocks related to this embodiment. The image synthesizer 1a shown in Fig. 9 differs from the image synthesizer 1 of the first embodiment shown in Fig. 2 in that it further includes a 2D correction calculation unit 18 and includes an image synthesizer 15a instead of the image synthesizer 15. The image synthesizer 15a differs from the image synthesizer 15 of the first embodiment shown in Fig. 2 in that it includes an image quality conversion unit 151a instead of the image quality conversion unit 151.

[0065] The estimated resolution set in the image quality DB of the image quality information storage unit 13 is expressed as the value of the number of horizontal pixels on a screen with a predetermined aspect ratio, such as 16:9. This ratio is referred to as the reference ratio. When the image size of the 2D video acquired by the 2D video attribute acquisition unit 12 is not the reference ratio, the 2D correction calculation unit 18 calculates a conversion pixel number, which is the number of horizontal pixels when the number of vertical pixels of the 2D video is fixed to the reference ratio. The 2D correction calculation unit 18 outputs the calculated conversion pixel number of the horizontal pixels and the number of vertical pixels of the 2D video to the image quality determination unit 14 as 2D video attribute information. Furthermore, the 2D correction calculation unit 18 calculates a correction value for converting the image size indicated by the 2D video attribute information notified to the image quality determination unit 14 into the image size of the 2D video, and notifies the image quality conversion unit 151a of the video synthesis unit 15a. This correction value is obtained by dividing the number of horizontal pixels of the 2D video by the conversion pixel number.

[0066] The image quality determination unit 14 uses the horizontal converted pixel count indicated by the 2D image attribute information input from the 2D correction calculation unit 18 as the 2D image resolution, and performs the same processing as in the first embodiment. When the composite image resolution determined by the image quality determination unit 14 is the same as the horizontal converted pixel count calculated by the 2D correction calculation unit 18, the image quality conversion unit 151a operates in the same manner as the image quality conversion unit 151 in the first embodiment. That is, the image quality conversion unit 151a outputs a 3D model that has been downsampled to have the composite image resolution after 2D rendering to the 2D rendering unit 153, and outputs the 2D image as it is to the 2D image composition unit 154 without changing the resolution of the 2D image.

[0067] On the other hand, when the composite video resolution is the estimated resolution of the 3D model, the image quality conversion unit 151a outputs the 3D model as it is to the 2D rendering unit 153. Furthermore, the image quality conversion unit 151a corrects the composite video resolution notified from the image quality determination unit 14 with the correction value notified from the 2D correction calculation unit 18 to calculate the number of horizontal pixels after the image quality conversion of the 2D video. The image quality conversion unit 151a calculates the number of vertical pixels after the image quality conversion of the 2D video when the number of horizontal pixels is the calculated number of horizontal pixels after the image quality conversion and the aspect ratio is the original aspect ratio of the 2D video. The image quality conversion unit 151a converts the 2D video into a 2D video with the number of horizontal pixels and the number of vertical pixels after the image quality conversion, and then outputs the 2D video to the 2D video synthesis unit 154.

[0068] FIG. 10 is a flow diagram showing the process of the image synthesizer 1a according to the second embodiment. In FIG. 10, the same processes as those in the first embodiment shown in FIG. 5 are denoted by the same reference numerals, and the description thereof will be omitted. The image synthesizer 1a performs the same processes as steps S105 to S110 in FIG. 5. When the image size set in the 2D image attribute information input from the 2D image attribute acquisition unit 12 is not the standard ratio, the 2D correction calculation unit 18 fixes the number of vertical pixels of the image size and calculates a conversion pixel number that is the number of horizontal pixels when the aspect ratio is the standard ratio. The 2D correction calculation unit 18 outputs the 2D image attribute information in which the calculated conversion pixel number of the horizontal pixel number and the number of vertical pixels of the 2D image are set to the image quality determination unit 14 (step S305). Furthermore, the 2D correction calculation unit 18 calculates a correction value for returning the image size indicated by the 2D image attribute information notified to the image quality determination unit 14 to the image size of the 2D image acquired in step S110 (step S310). The 2D correction calculation unit 18 notifies the calculated correction value to the image quality conversion unit 151a of the video synthesis unit 15a.

[0069] The image quality determination unit 14 reads out the estimated resolution from the image quality DB based on the acquired 3D model attribute information (step S115). The image quality determination unit 14 compares the 2D image resolution indicated as the horizontal converted pixel count of the 2D image attribute information notified from the 2D correction calculation unit 18 in step S305 with the estimated resolution read out in step S115 (step S120). When the image quality determination unit 14 determines that the 2D image resolution is higher than the estimated resolution (step S125: YES), it determines the estimated resolution as the composite image resolution and outputs the determined composite image resolution to the image composition unit 15a (step S130).

[0070] When the composite image resolution is the estimated resolution of the 3D model, the image quality conversion unit 151a outputs the 3D model as it is to the 2D rendering unit 153. Furthermore, the image quality conversion unit 151a corrects the composite image resolution with the correction value notified from the 2D correction calculation unit 18 to calculate the number of horizontal pixels of the 2D image after image quality conversion. The image quality conversion unit 151a calculates the number of vertical pixels when the number of horizontal pixels is the calculated number of horizontal pixels after image quality conversion and the aspect ratio is the aspect ratio of the 2D image acquired in step S110 as the number of vertical pixels after image quality conversion of the 2D image. The image quality conversion unit 151a converts the 2D image acquired in step S110 into a 2D image with the number of horizontal pixels and the number of vertical pixels after image quality conversion, and then outputs the 2D image composition unit 154 (step S315). Furthermore, the image quality conversion unit 151 outputs the 3D model as it is to the 2D rendering unit 153.

[0071] The process of the image synthesizer 1a from step S140 onwards is the same as that of the first embodiment. Moreover, in step S125, if the image quality determination unit 14 determines that the estimated resolution is higher than the 2D image resolution (step S125: NO), the process of the image synthesizer 1a from step S165 onwards is the same as that of the first embodiment.

[0072] A specific example of processing by the image synthesizing device 1a will be described with reference to Figures 9 and 10. Here, an example is shown in which the aspect ratio of the 2D video is 8:9 and the reference ratio is 16:9.

[0073] The 3D model attribute acquisition unit 11 receives an input of the 3D model A and the shot type "FS". The 3D model attribute acquisition unit 11 acquires the object type "person", the size of the original model "170 cm", the object generation method "method A", and the number of points "2,000,000 (points)" from the file of the 3D model A (step S105). Meanwhile, the 2D video attribute acquisition unit 12 receives an input of the 2D video B. The 2D video attribute acquisition unit 12 analyzes the input 2D video B and acquires the 2D video resolution "3849 x 4320 (pixels)" (step S110). The 2D correction calculation unit 18 calculates the image size "7660 x 4320" with a reference ratio of 16:9 when the height is 4320 pixels because the aspect ratio of the image size of the 2D video is 8:9. 2D correction calculation unit 18 outputs the 2D video attribute information in which the calculated image size is set to image quality determination unit 14 (step S305). 2D correction calculation unit 18 calculates information such as "multiply the width by 1 / 2" as a correction value for returning the image size "7660 x 4320" notified to image quality determination unit 14 to the original 2D video screen size "3849 x 4320 (pixels)", and notifies image quality conversion unit 151a.

[0074] The image quality determination unit 14 uses the object type "person", the size of the original model "170 cm", the object generation method "method A", and the shot type "FS" as the estimated resolution search conditions, and reads out the estimated image quality "960" from the image quality DB shown in Fig. 3 (step S115). Since the estimated resolution "960" is smaller than the 2D image resolution "7680", the image quality determination unit 14 determines the composite image resolution to be "960" (steps S120, S125: YES, step S130).

[0075] The image quality conversion unit 151a outputs the 3D model as is to the 2D rendering unit 153. Furthermore, the image quality conversion unit 151a corrects the composite image resolution "960" by the correction value "multiply the width by 1 / 2" notified by the 2D correction calculation unit 18, and calculates the number of horizontal pixels of the 2D image after image quality conversion to be "480". The image quality conversion unit 151a calculates the number of vertical pixels to be "540" when the aspect ratio is 8:9, which is the number of horizontal pixels of "480". The image quality conversion unit 151a converts the input 2D image B into a 2D image B' having a size of "480 x 540" (step S315).

[0076] The 2D rendering unit 153 renders the 3D model A into a 2D image to generate a rendered 2D image A' (step S140). The 2D image synthesis unit 154 synthesizes the rendered 2D image A' with a 2D image B' with a resolution of "480 x 540" to generate a synthesized image C with a resolution of "480 x 540" pixels (step S145). The terminal-specific image quality conversion unit 16 converts the synthesized image C into a distribution image C' with a resolution of "960 x 1080" to match the vertical size of the terminal resolution of "1920 x 1080", and distributes it (steps S150, S155).

[0077] According to the above-described embodiment, the image synthesizer 1, 1a can synthesize 3D models and 2D images of different formats into images with a more natural-looking resolution. In addition, the image synthesizer 1, 1a can not only align the resolution, but also arbitrarily specify the degree of blur of each of the 3D model and the 2D image according to the intention of the performance by referring to the image quality DB. In addition, the image synthesizer 1, 1a can reduce the load by reducing the 3D model, which is point cloud data, to the number of points required according to the purpose and then rendering it into a 2D image.

[0078] The image synthesizer 1, 1a may be realized by a plurality of computer devices connected to a network. In this case, it is possible to arbitrarily select which of the plurality of computer devices is used to realize each functional unit of the image synthesizer 1, 1a. For example, the computer device that realizes the 3D model attribute acquisition unit 11, the 2D image attribute acquisition unit 12, the image quality information storage unit 13, the image quality determination unit 14, and the image synthesis unit 15 of the image synthesizer 1 may be different from the computer device that realizes the terminal-specific image quality conversion unit 16 and the terminal resolution acquisition unit 17 of the image synthesizer 1. Also, the computer device that realizes the 3D model attribute acquisition unit 11, the 2D image attribute acquisition unit 12, the image quality information storage unit 13, and the image quality determination unit 14 of the image synthesizer 1 may be different from the computer device that realizes the image synthesis unit 15 of the image synthesizer 1. Also, the same functional unit of the image synthesizer 1, 1a may be realized by a plurality of computer devices. For example, the image quality information storage unit 13 may be realized by a plurality of computer devices.

[0079] FIG. 11 is a diagram showing an example of a hardware configuration of the image synthesizer 1, 1a. The image synthesizer 1, 1a includes a processor 71, a storage unit 72, a communication interface 73, and a user interface 74. The processor 71 is a central processing unit that performs calculations and control. The processor 71 is, for example, a central processing unit (CPU) or a graphics processing unit (GPU). The processor 71 reads out and executes a program from the storage unit 72. The storage unit 72 further has a work area and the like when the processor 71 executes various programs. The communication interface 73 is connected to other devices so as to be able to communicate with them. The user interface 74 is an input device such as a button, a keyboard, or a pointing device, a display device such as a display, or an audio output device such as a speaker.

[0080] All or part of the functions of the image synthesizers 1 and 1a are realized by the processor 71 reading and executing a program from the storage unit 72. Note that all or part of these functions may be realized using hardware such as an ASIC or a PLD.

[0081] According to the embodiment described above, the image synthesis device includes a three-dimensional model attribute acquisition unit, a two-dimensional image attribute acquisition unit, an image quality determination unit, and an image synthesis unit. For example, the three-dimensional model attribute acquisition unit corresponds to the 3D model attribute acquisition unit 11 of the embodiment, and the two-dimensional image attribute acquisition unit corresponds to the 2D image attribute acquisition unit 12 of the embodiment. The three-dimensional model attribute acquisition unit acquires a three-dimensional model, which is data in which an object is drawn by a point cloud, and three-dimensional model attributes including the type of object, the size of the object, the generation method of the three-dimensional model, the number of points of the three-dimensional model, and the type of shot. The two-dimensional image attribute acquisition unit acquires a two-dimensional image and a two-dimensional image resolution, which is the resolution of the two-dimensional image. The image quality determination unit determines the lower of the estimated resolution when the three-dimensional model of the three-dimensional model attribute is converted into a two-dimensional image and the two-dimensional image resolution as the image quality. When the image synthesis unit determines the estimated resolution as the image quality, it synthesizes a two-dimensional image whose resolution has been converted to that image quality with a three-dimensional model converted into the two-dimensional image to generate a synthetic image, and when it determines the two-dimensional image resolution as the image quality, it synthesizes the two-dimensional image acquired by the two-dimensional image attribute acquisition unit with a three-dimensional model converted into a two-dimensional image of that image quality to generate a synthetic image.

[0082] The image synthesis device may include an image quality storage unit that stores an estimated resolution in association with a type of object, a size of object, a generation method of a three-dimensional model, a number of points, and a type of shot. The image quality determination unit reads out from the image quality storage unit an estimated resolution corresponding to a word representing a type of object indicated by a three-dimensional model attribute, a synonym of the word representing the type of object, or a word of a higher concept of the word representing the type of object, and the size of object indicated by the three-dimensional model attribute, a generation method, a number of points, and a type of shot.

[0083] The image synthesis unit reduces the number of points of the three-dimensional model and converts the three-dimensional model with the reduced number of points into a two-dimensional image, thereby obtaining a three-dimensional model converted into a two-dimensional image of the determined image quality.

[0084] The two-dimensional video attribute acquisition unit acquires the screen size of the acquired two-dimensional video, and if the aspect ratio of the acquired screen size is a predetermined ratio, the number of horizontal pixels of the screen size is set as the two-dimensional video resolution, and if the aspect ratio of the acquired screen size is not the predetermined ratio, the number of horizontal pixels when the number of vertical pixels in the screen size is not changed is set as the two-dimensional video resolution.

[0085] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0086] 1, 1a Image synthesizer 11 3D model attribute acquisition section 12 2D video attribute acquisition section 13 Image quality information storage unit (image quality DB) 14 Image quality determination section 15, 15a Video synthesis section 16 Device-specific image quality conversion section 17 Terminal resolution acquisition unit 18 2D correction calculation unit 71 Processor 72 Memory section 73 Communication Interface 74 User Interface 151, 151a Image quality conversion unit 152 Voxel size information storage unit (voxel size DB) 153 2D Rendering Section 154 2D image synthesis section

Claims

1. a three-dimensional model attribute acquisition unit that acquires a three-dimensional model, which is data in which an object is rendered by a point cloud, and three-dimensional model attributes including the type of the object, the size of the object, a generation method of the three-dimensional model, the number of points in the three-dimensional model, and a type of shot; A two-dimensional image attribute acquisition unit that acquires a two-dimensional image and a two-dimensional image resolution that is a resolution of the two-dimensional image; an image quality determination unit that determines, as an image quality, the lower of an estimated resolution when the three-dimensional model having the three-dimensional model attribute is converted into a two-dimensional image and a resolution of the two-dimensional image; an image synthesis unit that, when the estimated resolution is determined as the image quality, synthesizes the two-dimensional image whose resolution has been converted to the image quality with the three-dimensional model converted into the two-dimensional image to generate a synthetic image, and, when the two-dimensional image resolution is determined as the image quality, synthesizes the two-dimensional image acquired by the two-dimensional image attribute acquisition unit with the three-dimensional model converted into the two-dimensional image of the image quality to generate a synthetic image; An image synthesis device comprising:

2. an image quality storage unit that stores an estimated resolution in association with a type of object, a size of the object, a method of generating a three-dimensional model, a number of points, and a type of shot; the image quality determination unit reads out from the image quality storage unit the estimated resolution corresponding to a word representing the type of the object indicated by the three-dimensional model attribute, a synonym of the word representing the type of the object, or a word of a superordinate concept of the word representing the type of the object, and the size of the object indicated by the three-dimensional model attribute, the generation method, the number of points, and the type of the shot; 2. The image synthesizing apparatus according to claim 1.

3. The image synthesis unit reduces the number of points of the three-dimensional model, and converts the three-dimensional model with the reduced number of points into a two-dimensional image, thereby obtaining the three-dimensional model converted into a two-dimensional image of the image quality.

2. The image synthesizing apparatus according to claim 1.

4. the two-dimensional video attribute acquisition unit acquires a screen size of the acquired two-dimensional video, and when an aspect ratio of the acquired screen size is a predetermined ratio, the number of horizontal pixels of the screen size is set as the two-dimensional video resolution, and when an aspect ratio of the acquired screen size is not the predetermined ratio, the number of horizontal pixels in the case where the aspect ratio is set to the predetermined ratio without changing the number of vertical pixels in the screen size is set as the two-dimensional video resolution.

2. The image synthesizing apparatus according to claim 1.

5. Computer, a three-dimensional model attribute acquisition means for acquiring a three-dimensional model, which is data in which an object is rendered by a point cloud, and three-dimensional model attributes including the type of the object, the size of the object, a generation method of the three-dimensional model, the number of points of the three-dimensional model, and a type of shot; A two-dimensional image attribute acquisition means for acquiring a two-dimensional image and a two-dimensional image resolution that is a resolution of the two-dimensional image; an image quality determination means for determining, as an image quality, the lower of an estimated resolution when the three-dimensional model having the three-dimensional model attribute is converted into a two-dimensional image and a resolution of the two-dimensional image; an image synthesis means for generating a synthetic image by synthesizing the two-dimensional image whose resolution has been converted to the image quality and the three-dimensional model converted to the two-dimensional image when the estimated resolution is determined as the image quality, and for generating a synthetic image by synthesizing the two-dimensional image acquired by the two-dimensional image attribute acquisition means and the three-dimensional model converted to the two-dimensional image of the image quality when the two-dimensional image resolution is determined as the image quality; A program to function as a

Images