Simulation device, computer program, and storage medium

Abstract

To provide a simulation device capable of smoothly acquiring an appropriate photographing condition or the like by using a virtual space.SOLUTION: A simulation device comprises: information input means of inputting information on an event as a photographing target, photographing information, and a desired photographing image; virtual space construction means of constructing a virtual space that an event situation is assumed by using an object model on the basis of the information input by the information input means; simulation image generation means of generating an image obtained in the case of photographing it at an event site as a simulation image by using the virtual space; evaluation means of evaluating at least one similarity of brightness and a blur amount in the simulation image and the desired photographing image; and notification means of notifying a user of the photographing condition in accordance with the similarity evaluated by the evaluation means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a simulation device using a virtual space. [Background technology]

[0002] To efficiently shoot events such as athletic meets and weddings, it is effective to do location hunting (hereinafter referred to as location scouting), where you visit the venue in advance and consider shooting conditions such as shooting position and angle of view to obtain the desired shots. Location scouting is the process of bringing multiple shooting equipment to the event venue in advance and taking repeated test shots to ensure smooth shooting on the day of the event.

[0003] On the other hand, simulation technology that reproduces spaces such as event venues using three-dimensional measurement data is becoming widespread. For example, Patent Document 1 discloses a simulation technology that uses images of the event venue taken from multiple viewpoints to generate image data of the event venue viewed from any position and any direction. Simulation services that allow you to check the event venue from any position and any direction have also appeared. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2012-113281 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in a simulation using the above-mentioned arbitrary viewpoint image synthesis technology, although it was possible to check the state of the venue, it was difficult to consider the shooting image taking into account the position and size of the person who would actually be the subject. In addition, in order to find the shooting conditions such as the shooting position, direction, and angle of view that would produce the desired shot image, it was necessary to search around the simulation screen using input operations such as a mouse and keyboard, which was time-consuming.

[0006] Furthermore, it was not possible to confirm the detailed shooting conditions, such as ISO sensitivity, aperture value, and shutter speed, and the combination of shooting equipment (camera, lens, etc.) required to obtain brightness and background blur close to the desired captured image. An object of the present invention is to provide a simulation device that can quickly obtain appropriate shooting conditions, etc. by using a virtual space. [Means for solving the problem]

[0007] The simulation device according to the present invention comprises: an information input means for inputting information on the event to be photographed, photographing information, and a desired photographed image; a virtual space constructing means for constructing a virtual space that embodies the event using a subject model based on the information input by the information inputting means; a simulation image generating means for generating a simulation image using the virtual space from an image obtained by photographing the event; an evaluation means for evaluating a similarity between the simulation image and a desired captured image in terms of at least one of brightness and blur amount; The similarity evaluated by the evaluation means A combination of photographing equipment necessary for photographing the simulation image having a value of at least a predetermined value and photographing conditions corresponding to the combination Notification method to inform the user and, The present invention is characterized by having the following. Effect of the Invention

[0008] According to the present invention, it is possible to provide a simulation device that can quickly obtain appropriate shooting conditions and the like by using a virtual space. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of the configuration of a simulation device according to a first embodiment; [Diagram 2] FIG. 11 is a diagram showing an example of a venue layout for an athletic meet in the first embodiment. [Diagram 3] FIG. 11 is a diagram showing an example of background information of a venue. [Figure 4] FIG. 4 is a diagram illustrating an example of light source information. [Diagram 5] 11 is a diagram for explaining an example of subject position information in a footrace; FIG. [Figure 6] FIG. 11 is a diagram showing an example of a method for specifying the subject direction. [Figure 7] FIG. 11 is a diagram illustrating an example of a format of subject position information. [Figure 8] FIG. 11 is a diagram illustrating an example of shooting position information. [Figure 9] FIG. 11 is a diagram showing an example of shooting information. [Figure 10] FIG. 2 is a diagram showing an example of a desired captured image. [Figure 11] 13 is a flowchart for constructing a virtual space. [Figure 12] FIG. 13 is a diagram showing an example of coordinate transformation of a venue layout. [Figure 13] FIG. 1 is a diagram showing an image of a subject model. [Figure 14] FIG. 2 is a diagram showing an example of subject model placement in a virtual space. [Figure 15] 13 is a flowchart of a simulation image generation process. [Figure 16] FIG. 13 is a diagram illustrating an example of a projected image. [Figure 17] FIG. 13 is a diagram showing an example of a simulation image in which a background is added to a projected image. [Figure 18] FIG. 13 is a diagram showing an example of a simulation image whose brightness has been adjusted. [Figure 19] FIG. 13 is a diagram showing an example of a simulation image to which background blur has been added. [Figure 20] 13 is a flowchart of a similarity calculation. [Figure 21] FIG. 13 is a diagram showing an example of a similarity calculation graph. [Figure 22] 11A and 11B are diagrams illustrating an example of a method for calculating a similarity regarding a face direction. [Diagram 23] FIG. 11 is a diagram showing an example of a two-dimensional filter characteristic for detecting the amount of blur. [Figure 24] FIG. 13 is a diagram illustrating an example of a weighting coefficient for similarity calculation. [Diagram 25] 10 is a flowchart showing an example of a flow for a shooting condition evaluation process of the simulation device. [Figure 26] FIG. 13 is a diagram showing an example of shooting information (combination 1). [Figure 27] FIG. 13 is a diagram showing an example of shooting information (combination 2). [Figure 28] FIG. 13 is a diagram showing an example of shooting information (combination 3). [Figure 29] FIG. 13 is a diagram showing an example of shooting information (combination 4). [Diagram 30] FIG. 13 is a diagram showing an example (full size) of a simulation image to which a background has been added. [Diagram 31] FIG. 13 is a diagram showing an example of a simulation image (APS-C) to which a background has been added. [Diagram 32] 13 is a flowchart showing an example of a flow for calculating an optimal equipment combination. [Diagram 33] FIG. 13 is a diagram showing an example of score calculation for each equipment combination. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In each drawing, the same members or elements are designated by the same reference numerals, and duplicated descriptions will be omitted or simplified. EXAMPLES

[0011] 1 is a block diagram showing an example of the configuration of a simulation device according to embodiment 1. In this embodiment, an explanation will be given assuming an athletic meet at an elementary school or the like as an event, but the present invention can be similarly applied to other events. In FIG. 1, the simulation device has information input means 100 for inputting information on an event to be photographed, photographing information, a desired photographed image, and the like. The system also includes a virtual space construction means 101 for constructing a virtual space that reproduces (assumes) an event situation using a subject model based on information inputted by the information input means 100. The system also includes a simulation image generation means 102 for generating a simulation image using the virtual space from an image obtained by photographing the event venue.

[0012] Furthermore, the apparatus includes an evaluation means 103 for evaluating the similarity between the simulation image and a desired captured image, and a notification means 104 for collecting shooting conditions according to the similarity so that the evaluation result by the evaluation means 103 is high, and notifying the user of the shooting conditions according to the similarity. In this embodiment, the evaluation means 103 is configured to evaluate the similarity of at least one of brightness and the amount of blur (of the background) between the simulation image and the desired captured image.

[0013] The control means 105 controls the virtual space construction means 101, the simulation image generation means 102, the evaluation means 103, and the notification means 104 based on the information inputted by the information input means 100. The control means 105 has a built-in CPU as a computer, and executes the operations of the entire device, for example, the flowcharts in Figures 11, 15, 20, 25, and 32, based on a computer program stored in a memory.

[0014] Moreover, the virtual space construction means 101, the simulation image generation means 102, the evaluation means 103, the notification means 104, etc. execute their respective functions by the computer of the control means 105 executing a computer program. Note that the virtual space construction means 101, the simulation image generation means 102, the evaluation means 103, the notification means 104, etc. may be partly or entirely included in the control means 105, or each function may be configured by a discrete circuit. Each of the components in FIG. 1 will be described in detail below.

[0015] The information input means 100 is an input means for inputting event information, photography information, desired photographed images, etc. The event information includes at least one of the layout of the venue where the event to be photographed is held, background information of the venue, light source information of the venue, event program information related to the event contents, event sequence, etc.

[0016] The information input means is capable of inputting at least one of information relating to the brightness of the event venue and a background image of the event venue. An event refers to an event that brings together a large number of people, such as athletic meets, races, plays, school performances, festivals, live music events, weddings, etc.

[0017] The venue layout includes an overhead view of the venue where the event will take place. Fig. 2 is a diagram showing an example of a venue layout for an athletic meet in the first embodiment. Reference numeral 200 in Fig. 2 denotes a playground (schoolyard) that serves as the event venue, and is represented in a two-dimensional (X, Y) coordinate space with the upper left corner as the origin (0,0). Reference numeral 201 in Fig. 2 denotes a track that is used as a running course or boundary line for each event in the athletic meet, 202 denotes a school building, and 203 to 205 denote objects that serve as the background during photography, such as trees planted in the playground (hereinafter referred to as background objects).

[0018] Fig. 3 is a diagram showing an example of background information of a venue. Background information of a venue is a group of images taken of the venue where the event is held and background objects placed around it. In the case of the venue layout of the athletic meet shown in Fig. 2, background information is, for example, images shown in Fig. 3(A) and (B) of a school building 202 placed in the schoolyard, and images shown in Fig. 3(C) and (D) of trees 203, 204, and 205 planted in the vicinity.

[0019] As shown in Figure 3, by capturing background information (background images) from multiple different viewpoints and inputting them, a background image from any viewpoint can be generated using synthesis processing. The images can be captured in advance at the event venue using a highly portable camera such as a smartphone, or, if images related to the event venue are already available on the cloud, they can be downloaded.

[0020] It is assumed that the input image has the appropriate brightness and is in focus. There are various known techniques for synthesizing an image from an arbitrary viewpoint based on images from multiple viewpoints, and for capturing multiple images suitable for synthesis without excess or deficiency, so detailed explanations will be omitted here. The light source information is information about the lighting conditions (for example, brightness and color temperature) of the venue where the event is held.

[0021] Fig. 4 shows an example of light source information, and a list of EV (Exposure Value) values ​​corresponding to the lighting conditions of an event venue as shown in Fig. 4 is created in advance and selected from that list. In cases such as outdoor events where the lighting conditions on the day may be affected by the weather, multiple lighting conditions may be selected and a simulation may be performed under each of the conditions. The brightness information as the light source information may be automatically set by downloading data such as a weather forecast for the date and time of the event. At that time, color temperature information may also be input as light source information according to the weather forecast.

[0022] The event program information includes the contents of the event, the order of the event, subject position information indicating the movement information of subjects participating in the event, and the like. The subject position information includes information regarding at least one of the position of the subject and the orientation of the subject at each time that the event is taking place. FIG. 5 is a diagram for explaining an example of subject position information in a footrace, and an overview of subject position information will be described with reference to FIG. 5 using a footrace at an athletic meet as an example.

[0023] In the figure, 501, 502, 503, and 504 represent four students (subjects) participating in the race. The figure also shows how each student moves from the start of a track 505 drawn in a playground 500 toward the finish line as the time progresses from T0, T1, T2, to TN. Here, the position of each student is expressed by XY coordinates, similar to the venue layout in Figure 2.

[0024] FIG. 6 is a diagram showing an example of a method for specifying the subject direction, and the direction of the student is expressed by the directions of arrows (D0 to D15) that equally divide 360 ​​degrees as shown in FIG. FIG. 7 is a diagram showing an example of the format of subject position information. As shown in FIG. 7, the subject position information input to the information input means 100 is composed of a set of information on the positions and orientations of a plurality of subjects at each time.

[0025] The subject position information can be stored as multiple preset data (registered data) in the information input means 100 for each event program (for each sport, such as a foot race or tug-of-war, in the case of an athletic meet). This allows the user to select the subject position information stored in advance from multiple preset data according to the event program.

[0026] The content and sequence of events will be entered as event program information, but the User Interface (hereafter abbreviated as UI) used for this will not be mentioned here. You can input pre-quantified data, or you can have the user input the information via a Graphical User Interface (hereafter abbreviated as GUI). Or you can download it via the Internet, etc.

[0027] Next, the shooting information will be described. The shooting information includes the shooting position, shooting direction, shooting angle of view, and also includes information on at least one of the aperture value, shutter speed, ISO sensitivity, and white balance data (color temperature data). The shooting positions include information about areas at the event venue where photographers or shooting equipment such as drones (hereinafter referred to as cameras) can enter (enter).

[0028] FIG. 8 is a diagram showing an example of shooting position information. FIG. 8(A) shows an overview of shooting positions, with P0 to P15 indicated by black circles around a playground 800 indicating positions where shooting is possible. Each shooting position is expressed using a list of XY coordinates as shown in Fig. 8(B). The shooting positions may be treated as area information rather than point information within the event venue. The shooting direction is the direction in which the camera shoots, and in this embodiment, is expressed by the directions of arrows (D0 to D15) that equally divide 360 ​​degrees, as in FIG. The angle of view is an index that indicates the shooting range of the lens attached to the camera. For example, it is expressed as the focal length (mm) when converted to a sensor size of 35 mm.

[0029] FIG. 9 is a diagram showing an example of shooting information. In this embodiment, as shown in FIG. 9A, discrete angles of view from wide angle (16 mm) to super telephoto (800 mm) are expressed in correspondence with F0 to F12. Aperture value is an index for adjusting the amount of exposure when a camera takes a photograph, and is expressed using the F-number. The smaller the F-number, the brighter the lens (the more exposure), but the smaller the area in front of and behind the subject that is in focus (depth of field) becomes.

[0030] The range of the minimum and maximum F-numbers varies depending on the lens attached to the camera. In this embodiment, the F-number N is expressed in correspondence with the AV (Aperture Value) value by the following formula 1. AV = 2 log2N (Equation 1) By using the AV value, it is possible to express discrete aperture values ​​from F1.0 to F32 in correspondence with continuous values ​​from AV0 to AV10, as shown in FIG. 9B.

[0031] Shutter speed is an index that indicates the exposure time of a camera that takes a picture. In a general camera, the setting range is from a few seconds to a few thousandths of a second. The slower the shutter speed, the brighter the image can be taken even in a dark place, but the effect of camera shake or subject motion during shooting becomes greater, so the possibility of failing to take a picture increases. In this embodiment, the shutter speed t is expressed in correspondence with a TV (Time Value) value using the following formula 2. TV=log2(1 / t)...(Equation 2) By using the TV value, it is possible to express discrete shutter speeds from 1 second to 1 / 8000 seconds in correspondence with continuous values ​​from Tv0 to Tv13, as shown in FIG. 9(C).

[0032] ISO sensitivity is the sensitivity setting of the sensor implemented in the camera used to take the photo. It is expressed with the number ISO at the beginning, such as ISO100 or ISO20. In general cameras, the ISO sensitivity setting ranges from several hundred to several tens of thousands. The higher the value, the brighter the image you can take even in dark places, but the greater the effect of noise, which degrades the image quality.

[0033] In this embodiment, the ISO sensitivity S is expressed in correspondence with the SV (Sensivity Value) value by the following Equation 3. SV=log2(S / 100)...(Formula 3) By using the SV value, it is possible to express the discrete ISO sensitivities from ISO100 to ISO51200 in correspondence with the continuous values ​​from SV0 to SV9, as shown in FIG. 9(D).

[0034] Next, the desired photographed image will be described. The desired photographed image is information about a sample image of an image that the user wants to photograph, and includes image information as a sample, and is stored in association with at least one of the number of subjects, face positions, face sizes, and face orientations. FIG. 10 is a diagram showing an example of a desired photographed image.

[0035] In this embodiment, a large number of sample images such as those shown in Fig. 10(A) and (B) are prepared as preset data. Then, the number of subjects, face positions, face sizes, face orientations, and background object positions, sizes, and orientations are linked (associated) with each sample image and saved. The saved data may be stored in a cloud on the Internet, etc. There are various methods for inputting the desired photographed image, but photographed images or illustrations of the same event held in the past are used as input. The user can download a plurality of sample images saved as preset data, for example via the Internet, and select a desired captured image from among them.

[0036] Information associated with the selected sample image is stored as input information. A plurality of desired captured images may be selected. It should be noted that the brightness information and the blur information of the face are also linked to each sample image and stored.Furthermore, the color and color temperature of the subject may also be linked to each sample image and stored. Next, a description will be given of the virtual space construction means 101. The virtual space construction means 101 generates a virtual space in which an event situation is reproduced by a subject model, based on the event information inputted by the information input means 100.

[0037] FIG. 11 is a flowchart for constructing a virtual space, FIG. 12 is a diagram showing an example of coordinate transformation of a venue layout, and FIG. 13 is a diagram showing an image of a subject model. The flow of generating a virtual space will be explained using FIGS. First, in step S1101, the virtual space construction means 101 converts the two-dimensional venue layout input to the information input means 100 into a three-dimensional coordinate space (xyz coordinate system) as shown in FIG.

[0038] Next, in step S1102, the virtual space construction means 101 places a subject model in three-dimensional coordinate space based on the event program information input to the information input means 100. An example of a subject model to be placed is shown in Fig. 13. In Fig. 13, reference numeral 1301 denotes a subject model of a student who will be the subject of the athletic meet, and is represented by a polyhedron (polygon) in three-dimensional coordinate space. Reference numeral 1302 in the figure is a forward mark that indicates the direction of the student's face. A plurality of subject models in Fig. 13 are placed in the three-dimensional coordinate space according to the subject position information.

[0039] 14 is a diagram showing an example of subject model placement in a virtual space, and shows the virtual space near the start line at time T0. The positions and orientations of subject models 1401, 1402, 1403, and 1404 change with time based on subject position information. Furthermore, the virtual space construction means 101 can make the subject model pose based on the contents of the subject position information, and can also place spectators such as students not participating in the competition and parents in the virtual space.

[0040] The method of making a polygonal subject model assume a desired pose is similar to that used in 3D games and the like, and is already widely known, so a detailed explanation will be omitted here. Next, the simulation image generating means 102 will be described with reference to Fig. 15. The simulation image generating means 102 generates simulation images that are obtained when photographing is performed under various photographing conditions in the virtual space.

[0041] FIG. 15 is a flowchart of the generation of a simulation image. First, in step S1501, the simulation image generating means 102 determines the shooting conditions (shooting position, shooting direction, angle of view, aperture value, shutter speed, ISO sensitivity, lighting conditions, etc.) when shooting within the three-dimensional coordinate space in which the subject model is placed. The shooting position is selected from P0 to P15 in FIG. 8 input to the information input means 100.

[0042] The shooting direction is selected from D0 to D15 in FIG. 6 inputted to the information input means 100. The angle of view is selected from F0 to F12 in FIG. 9(A) inputted to the information input means 100. The aperture value is selected from AV0 to AV10 in FIG. 9(B) inputted to the information input means 100. The shutter speed is selected from TV0 to TV13 in FIG. 9(B) inputted to the information input means 100. The ISO sensitivity is selected from SV5 to AV14 in FIG. 9(D) inputted to the information input means 100.

[0043] The lighting conditions are selected from the light source information EV1 to EV14 of FIG. Next, in step S1502, the simulation image generating means 102 performs a projection process to obtain a two-dimensional image (projected image) obtained when the virtual space is photographed at the photographing position, direction, and angle of view selected in step S1501. The projection process can be performed by a generally known technique such as perspective projection, and therefore a detailed description thereof will be omitted.

[0044] 16 is a diagram showing an example of a projection image, which is obtained when shooting is performed at time T1 from shooting position P14 in the direction of shooting orientation D4 with a field angle of F6. An image is obtained in which subject models 1601, 1602, and 1603 representing athletes are arranged on a screen 1600. The brightness of the subject models and virtual space in the projection image is generated with appropriate brightness.

[0045] Next, in step S1503, the simulation image generation means 102 adds a background image to the projection image generated in step S1502. The background image is generated by synthesizing image data from multiple viewpoints according to the shooting position, direction, and angle of view set in step S1501 based on the background information of the venue input to the information input means 100. That is, the simulation image generating means is configured to be able to add a background image to the simulation image based on image data of the background of the event venue.

[0046] The generation of image data using a plurality of viewpoints is known as an arbitrary viewpoint image synthesis technique and is therefore not described in detail here. FIG. 17 is a diagram showing an example of a simulation image in which a background is added to a projected image. Background objects 1701 and 1702 corresponding to trees 204 and 205 planted in the schoolyard in Fig. 2 are added to the projected image. Since the brightness of the images from multiple viewpoints before synthesis is adjusted to an appropriate brightness in advance, the brightness of the image after synthesis is also appropriate.

[0047] Next, in step S1504, the simulation image generating means 102 adjusts the brightness of the projected image to which the background was added in step S1503. The brightness of an image is expressed as appropriate when the entire image looks natural. Images that have been intentionally photographed brightly are expressed as over (N steps). The value of N is expressed as over 1 step and over 2 steps when the brightness (luminance value) is doubled and quadrupled based on the image brightness being appropriate.

[0048] In addition, images that are intentionally taken dark are expressed as underexposed (M steps). The M value is expressed as 1 step underexposed and 2 steps underexposed when the brightness (luminance value) is 1 / 2 or 1 / 4 of the standard brightness of the image. Image brightness (L) is calculated using the following formula 4 when the lighting conditions (EV), aperture value (AV), shutter speed (TV), and ISO sensitivity (SV) are given. L = (EV + SV) - (AV + TV) (Equation 4)

[0049] When L is 0, the captured image will have the appropriate brightness. Images with a background added will have the appropriate brightness for both the subject model and the background object, so when L is 0, no brightness adjustment is performed. When L is a value greater than 0, the captured image will be over L levels brighter. In this case, the brightness value Y(x, y) of each pixel in the image with the background added is converted to a brightness value Y'(x, y) using the following Equation 5. Y'(x,y)=2 L *Y(x,y) (Eq. 5)

[0050] When L is a negative value, the captured image will be L stops under-bright. In this case, the brightness value Y(x,y) of each pixel in the image with the background added is converted to a brightness value Y'(x,y) using the following Equation 6. Y'(x,y)=(1 / 2 |L| )*Y(x,y) (Eq. 6) As described above, the simulation image generating means can adjust the brightness of the simulation image based on the input shooting conditions, and outputs an image with the brightness adjusted in accordance with the shooting conditions as a simulation image.

[0051] FIG. 18 is a diagram showing an example of a simulation image whose brightness has been adjusted. FIG. 18 shows an example of a simulation image in which the brightness of the image is adjusted under the lighting conditions of outdoor cloudy weather (EV12), aperture value F4.0 (AV4), shutter speed 1 / 500 (TV9), and ISO sensitivity 100 (SV0). Since the brightness L of the image is −1 (one step under), the brightness (luminance value) of the entire screen is reduced by half compared to the image in FIG.

[0052] In this embodiment, the brightness is adjusted uniformly for all images, but the brightness may be adjusted taking into account the reflectance of each subject. In this case, the reflectance of the subject and background objects may be linked to the subject model and the background information of the venue input to the information input unit 100.

[0053] In addition, when the sun is the light source, such as outdoors, the angle of illumination may vary depending on the time of day. In such cases, by inputting not only the lighting conditions (EV value) but also the angle of illumination for each time of day as light source information, it is possible to precisely reproduce the shadows of the subject and the brightness of the face. Regarding a method of adjusting brightness taking into account the reflectance of each subject and the illumination angle of the light source, known techniques such as ray tracing and photon mapping may be used, and details will not be mentioned here.

[0054] Next, in step S1505, blur is added to the image whose brightness has been adjusted in step S1504. If the pixel value of a point P(X,Y) in the image before blurring is P(X,Y) and the value after blurring is P'(X,Y), the relationship between P' and P is expressed by the following equation 7 using a point spread function PSF that indicates the optical transfer characteristics of the lens attached to the camera that took the image. P'(X,Y)=P(X,Y)*PSF(F,d,D,AV)...(Formula 7)

[0055] Here, F is the focal length (photographing angle of view), d is the distance from the shooting position to the main subject (the subject to be focused on), D is the distance from the shooting position to the background object, and AV is the aperture value. Once the shooting time and shooting position are determined, the distances to the main subject and background objects can be uniquely calculated from the venue layout in Figure 2 and the event program information in Figure 5. A simulation image with background blur added can be obtained by performing the calculation of Equation 7 above for all pixels in the simulation image.

[0056] In this manner, in this embodiment, the simulation image generating means can add blur to at least one of the main subject and background image of the simulation image based on the input photographing conditions. Generally, if the focal length and aperture value of the lens attached to the camera are the same, the greater the difference between the distance to the subject and the distance to the background object, the greater the amount of blur. Also, if the distance to the subject and the distance to the background object are the same, the longer the focal length of the lens attached to the camera and the smaller the aperture value, the greater the amount of blur.

[0057] FIG. 19 is a diagram showing an example of a simulation image to which background blur has been added. 19, blur is added to objects 1902 and 1903 running behind a main subject 1901, which is the focus position. In terms of the amount of blur, more blur is added to 1903, which is farther away from the main subject 1901, than to 1902. Similarly, blur is added to background objects 1904 and 1905 according to their distance from the main subject 1901.

[0058] Next, a description will be given of the evaluation means 103. The evaluation means 103 evaluates the similarity between the simulation image generated by the simulation image generation means 102 and the desired photographed image input and selected by the information input means 100. FIG. 20 is a flowchart showing the calculation of the similarity. The evaluation of the similarity by the evaluation means 103 will be described with reference to the flowchart of FIG. First, in step S2001, the evaluation means 103 calculates the number of people similarity (Mn) and face position similarity (Mp) for the person area in the simulation image, as well as the face size similarity (Ms), face direction similarity (Md), face brightness similarity (Ml), and blur amount similarity (Mb).

[0059] The similarity (Mn) of the number of subjects is defined as the absolute value of mn, where n is the number of subjects in the simulation image and m is the number of subjects associated with the desired photographed image. The similarity is calculated from this difference in the number of subjects. Fig. 21 shows an example of a graph for calculating similarity, and Fig. 21(A) shows a graph with the difference in the number of people on the horizontal axis and the similarity on the vertical axis. Fig. 21(A) shows a graph in which the larger the difference in the number of people is, the smaller the similarity becomes linearly, but this is just an example and a curved line is also acceptable.

[0060] The similarity (Mp) of the face position is calculated by calculating the Euclidean distance between the center position (Xc, Yc) of the face of the subject in the simulation image and the center position (Xs, Ys) of the face associated with the desired photographed image, and calculating the similarity from this Euclidean distance. Figure 21 (B) shows a graph with the Euclidean distance on the horizontal axis and the similarity on the vertical axis.

[0061] The curve shown in the graph of Fig. 21(B) is an example, and may have a different shape. When the simulation image and the desired captured image contain multiple faces, pairing is performed with a combination that minimizes the Euclidean distance, and the similarity of each face position is calculated. The average value of the calculated similarity of each face position is set as the similarity of the face position of the entire image (Mp).

[0062] The face size similarity (Ms) is calculated using the face size indicated by the radius from the center of the face. If the face size of the subject in the simulation image is Rc and the face size associated with the desired captured image is Rs, the absolute value of Rs-Rc is defined as the face size difference Rd. Figure 21(C) shows a graph with the face size difference Rd on the horizontal axis and the similarity on the vertical axis.

[0063] The curve shown in the graph of Fig. 21(C) is an example, and may have a different shape. When multiple faces are included in the simulation image and the desired captured image, the similarity of the size of each face is calculated for each paired combination when the similarity of the face position is calculated. The average value of the calculated similarity of the size of each face is set as the similarity of the size of the face in the entire image (Ms).

[0064] The similarity (Md) of the face direction is evaluated by considering the face direction of the subject in the simulation image and the face direction associated with the desired captured image as directional vectors in three-dimensional space. Fig. 22 is a diagram for explaining an example of a method for calculating similarity regarding face orientation. For example, as shown in Fig. 22, an XY plane 900 including the center coordinates of a face of interest is extracted, a vector indicating the face orientation is projected onto the plane, and the direction in which the projected vector faces is obtained. The face orientation of the subject in the simulation image is superimposed on the face orientation associated with the desired captured image, and the smaller angle between the two vectors is obtained.

[0065] The absolute value of the angle difference is between 0° and 360°, but if it exceeds 180°, the smaller angle is found by subtracting the angle from 360°. The smaller angle value of 0° to 180° corresponds to 100% to 0% to determine the similarity on the XY plane 900. The similarity is similarly found for the XZ plane 901, and the similarity on the XY plane 900 and the similarity on the XZ plane 901 are multiplied together to determine the similarity between the orientations of the two faces.

[0066] When multiple faces are included in the simulation image and the desired captured image, the similarity of the orientation of each face is calculated for each pair of faces that were paired when the similarity of the face positions was calculated. The average value of the calculated similarities of the orientation of each face is set as the similarity (Md) of the orientation of the face for the entire image. A graph in which the angle between the vectors of the face direction is plotted on the horizontal axis and the similarity is plotted on the vertical axis is shown in Fig. 21(D). The curve shown in the graph in Fig. 21(D) is just an example, and other shapes may be used.

[0067] The similarity in facial brightness (Ml) is calculated from the difference between the average facial brightness Ls of the subject in the simulation image and the average facial brightness Lr associated with the desired captured image. The absolute value of Ls-Lr is defined as the facial brightness difference Ld. When brightness values ​​are expressed in 8 bits, the possible range of Ld is from 0 to 255. Figure 21(E) shows a graph with the facial brightness difference Ld on the horizontal axis and the similarity on the vertical axis.

[0068] It should be noted that the curve shown in the graph of FIG. 21(E) is merely an example, and a different shape may also be used. If the simulation image and the desired captured image contain multiple faces, the brightness similarity of each face is calculated for each pair of faces that were paired when the face position similarity was calculated. The average value of the calculated brightness similarity of each face is set as the brightness similarity (Ml) of the face in the entire image.

[0069] The degree of similarity (Mb) of the amount of blur is calculated from the difference between the amount of blur Bs of the subject's face in the simulation image and the amount of blur Br of the face associated with the desired photographed image. FIG. 23 is a diagram showing an example of a two-dimensional filter characteristic for detecting the amount of blur. The amount of blur is defined as the inverse of the value obtained by applying a two-dimensional high-frequency extraction BPF filter process with weighting characteristics as shown in Figure 23 to the pixel values ​​P(i, j) of the facial contour, and dividing the extracted high-frequency components by the number of pixels (N) of the reference facial contour.

[0070] For example, it can be expressed as in Equation 7 below. Br=N / ΣP(i, j)*BPF (Equation 7) The absolute value of Bs-Br is defined as the difference Bd in the amount of blur in the face. Figure 21(F) shows a graph in which the horizontal axis represents the difference Bd in the amount of blur in the face and the vertical axis represents the similarity.

[0071] The curve shown in the graph of Fig. 21(F) is an example, and may have a different shape. When multiple faces are included in the simulation image and the desired captured image, the similarity of the amount of blur of each face is calculated for each paired combination when the similarity of the face position is calculated. The average value of the calculated similarity of the amount of blur of each face is set as the similarity (Mb) of the amount of blur of the face in the entire image.

[0072] Next, in step S2002, the evaluation means 103 calculates the number similarity (M'n), position similarity (M'p), size similarity (M's), orientation similarity (M'd), brightness similarity (M'l), and blur amount similarity (M'b) for the background objects in the simulation image. The method of calculating the similarity is the same as that for the person area similarity calculation in step S2001. When there are multiple background objects, the similarity of each background object is calculated as in step S2001, and then the average value is calculated, and this value is taken as the similarity of the background objects.

[0073] Next, in step S2003, the evaluation means 103 calculates a similarity Mtotal of the entire image using the similarity of each item of the person region and background region calculated in steps S2001 and S2002 according to the following formula 8. Mtotal = Kn*Mn+Kp*Mp+Ks*Ms +Kd*Md+Kl*Ml+Kb*Mb +K'n*M'n+K'p*M'p+K's*M's +K'd*M'd+K'l*M'l+K'b*M'b...(Formula 8)

[0074] Here, Kn, Kp, Ks, Kd, ​​Kl, and Kb are coefficients for adjusting the weighting for the similarity of each item in the person area, and K'n, K'p, K's, K'd, K'l, and K'b are coefficients for adjusting the weighting for the similarity of each item in the background area.

[0075] Fig. 24 is a diagram showing an example of weighting coefficients for similarity calculation, and similarity evaluation can be performed with more emphasis on composition by making the coefficients Kn, Kp, Ks, and Kd of the number of subjects, position, size, and orientation larger than the brightness Kb and the amount of blur Kb as in the example of Fig. 24. Also, by making the coefficient of the person area larger than the coefficient of the background object, similarity evaluation can be performed with more emphasis on people.

[0076] By linking and setting these coefficients according to the characteristics of each desired captured image, it becomes possible to perform an appropriate similarity evaluation for each image. In this embodiment, the similarity is calculated focusing on the face area of ​​the subject, but similarity may be calculated for the size, position, orientation, and pose of the entire body of the subject in a similar manner. Furthermore, it is possible to evaluate the similarity including the color tone by applying a method similar to the brightness similarity calculation to each color component of each pixel.

[0077] In this way, by linking face color information, color temperature information, and the like to a desired photographed image and storing it, the similarity of face color and color temperature information may also be included in the similarity evaluation. That is, the evaluation means may calculate the similarity between the simulation image and the desired captured image for at least two of the following: the number of subjects, the position of the subjects, the size of the subjects, the orientation of the subjects, the brightness of the subjects, the background image, the amount of background blur, and color.

[0078] Next, the process of the notification means 104 will be described in detail. The notification means 104 compares the similarity (Mtotal) calculated by the evaluation means 103 with a preset threshold value (Th), and if Mtotal is greater than Th, notifies the user of the simulation image and the shooting conditions used to generate it. If Mtotal is not greater than Th, the user may be notified that the similarity is low or that no image with a high similarity exists in the simulation image. Notification methods include output to a display device of the user terminal, output by printing, data output to an external medium or a device connected to a network, etc.

[0079] Next, a detailed description will be given of the photographing condition evaluation process performed by the control means 105. The control means 105 sequentially evaluates simulation images obtained when photographing the virtual space under various conditions based on each piece of information, such as event program information, photographing information, desired photographed image, background information of the venue, and light source information, input to the information input means 100. Then, the control means 105 notifies the user of the evaluation results.

[0080] FIG. 25 is a flow chart showing an example of a flow for the imaging condition evaluation process of the simulation device, and an example of sequential execution of evaluation will be described with reference to FIG. Steps S2501 to S2502 are a loop for varying (scanning) various shooting conditions by assigning them in predetermined steps. In Fig. 25, the shooting conditions to be assigned are time, position, direction, and focal length, and are indicated by T for time, P for position, D for direction, and F for focal length, respectively. In this example, they are assigned in order in a brute force manner, but the order in which the shooting conditions are assigned is not limited. It is also possible to assign only the necessary conditions efficiently using some algorithm.

[0081] Step S2501 is a loop for comprehensively evaluating T (time) in the range of T0 to TN, and step S2502 is a loop for comprehensively evaluating P (position) in the range of P0 to PN. Step S2503 is a loop for comprehensively evaluating D (direction) in the range of D0 to DN, and step S2504 is a loop for comprehensively evaluating F (focal length) in the range of F0 to FN. Step S2505 is a loop for comprehensively evaluating AV (aperture value) in the range of AV0 to AVN.

[0082] Step S2506 is a loop for comprehensively evaluating TV (shutter speed) in the range of TV0 to TVN, and step S2507 is a loop for comprehensively evaluating SV (ISO sensitivity) in the range of SV0 to SVN. Furthermore, step S2508 is processing in which the virtual space construction means 101 constructs a virtual space under conditions based on the loop variable T. Furthermore, step S2509 is processing in which a simulation image is generated when shooting under shooting conditions based on the loop variables P, D, F, AV, TV, and SV.

[0083] In step S2510, the evaluation means 103 calculates an evaluation value between the simulation image generated in step S1709 and the desired captured image. Steps S2511 and S2512 are processes in which the notification means 104 notifies the user of a simulation image and shooting conditions based on the evaluation result calculated in step S2510. That is, if the similarity is equal to or greater than a predetermined value, the simulation image and the corresponding photographing conditions are notified to the user.

[0084] Steps S2513 to S2519 represent a determination as to whether or not to continue the loop processing for time T, position P, direction D, focal length F, aperture value AV, shutter speed TV, and ISO sensitivity SV, and steps S2508 to S2512 are repeated until all conditions are covered.

[0085] As described above, according to this embodiment, the shooting conditions for obtaining a desired photographic image can be automatically assigned, evaluated, and extracted using simulation, so that the shooting conditions for obtaining a desired photographic image can be extracted more accurately without having to bring a large amount of photographic equipment to the site in advance. In this way, depending on the desired captured image, the similarity is calculated for at least two of the number of subjects, the position of the subject, the size of the subject, the orientation of the subject, the brightness of the subject, the background image, the amount of background blur, and the color of the subject.

[0086] The similarity is evaluated using different weights for the person region and the background region. That is, the evaluation means calculates an evaluation value by weighting and adding at least two calculated similarities, making it possible to extract shooting conditions for more effectively obtaining an image close to a desired shot image. Therefore, since it is possible to know in advance what position and under what shooting conditions the image should be taken, the efficiency of the image taking work can be improved. Also, since the image taking equipment required to obtain the desired image is known, it is not necessary to bring in unnecessary image taking equipment. EXAMPLES

[0087] Next, a second embodiment of the present invention will be described. In the second embodiment, by inputting information about the photographic equipment into the simulation device, it becomes possible to extract in advance the optimum combination of photographic equipment in addition to the photographic conditions for obtaining a desired photographic image.

[0088] The basic configuration is the same as in the first embodiment, but the information input means 100 allows additional input of information relating to photographic equipment. The information about the photographic equipment includes the camera's sensor size (the size of the light receiving surface of the image sensor), shutter speed, ISO sensitivity, lens focal length, aperture value, and the like.

[0089] The information on the photographic equipment is inputted with respect to a plurality of cameras and a plurality of lenses. The information on the photographic equipment may be inputted with respect to cameras and lenses owned by the user, or with respect to cameras and lenses available for rental through a rental service or the like. The information on cameras and lenses that the user is considering purchasing may also be inputted. The information on the photographic equipment may be selected from among presets prepared in advance in the information input means for information on a plurality of cameras and lenses.

[0090] Depending on the combination of the camera and lens in the input information on the photographing equipment, the possible ranges of the photographing conditions (photographing angle of view, aperture value, shutter speed, ISO sensitivity) described in FIG. 9 of the first embodiment are determined. For example, Camera 1: Sensor size: Full size, Shutter speed: 1 second to 1 / 8000 second, ISO100 to ISO51200, Camera 2: Sensor size: APS-C size, Shutter speed: 1 second to 1 / 4000 second, ISO100 to ISO12800,

[0091] Lens 1: focal length 300 mm, F value 4.0, Lens 2: Focal length 135mm, F value 2.0, When the above information on the photographing equipment is given, photographing conditions are obtained by combining the photographing equipment as shown in, for example, FIGS.

[0092] Figure 26 is a diagram showing an example of shooting information (combination 1), Figure 27 is a diagram showing an example of shooting information (combination 2), Figure 28 is a diagram showing an example of shooting information (combination 3), and Figure 29 is a diagram showing an example of shooting information (combination 4). Here, combination 1 is the combination of camera 1 and lens 1, combination 2 is the combination of camera 1 and lens 2, combination 3 is the combination of camera 2 and lens 1, and combination 4 is the combination of camera 2 and lens 2.

[0093] The shooting angle of view is determined by the combination of the lens focal length and the camera's sensor size. If the sensor size is full size, the lens focal length is used as is as the focal length (mm) when converted to a 35mm sensor size. If the sensor size is APS-C, the lens focal length is multiplied by 1.6 to convert it to the focal length of a 35mm sensor size.

[0094] For example, as shown in Figure 28, when the sensor size is APS-C, the focal length of lens 1, 300 mm, is multiplied by 1.6 to obtain 480 mm, which is the focal length for the shooting angle of view. Note that 1.6 times is just an example, and a magnification appropriate to the sensor and lens should be used. Fig. 30 is a diagram showing an example of a simulation image (full size) with a background added, and Fig. 31 is a diagram showing an example of a simulation image (APS-C) with a background added. The shooting conditions for Fig. 30 and Fig. 31 are the same except for the sensor size. When using the same lens (focal length), the subject appears larger with an APS-C sensor size compared to a full size sensor.

[0095] The amount of background blur also differs depending on the sensor size. Generally, if the camera sensor size is the same, the longer the focal length of the lens, the more the background will be blurred. Also, if the focal length of the lens is the same, the larger the camera sensor size, the more the background will be blurred. Such differences in background blur due to combinations of sensor size and focal length can be expressed in a simulation image by changing the PSF in Equation 7 for adding background blur, explained in Example 1, according to the combination of camera and lens.

[0096] The flow of simulation image generation and evaluation in FIG. 25 described in the first embodiment is performed for the combinations 1 to 4, and the evaluation results calculated for each combination are compiled and compared to extract the optimal combination of equipment for obtaining the desired captured image. FIG. 32 is a flowchart showing an example of a flow for calculating an optimal equipment combination. The flow for extracting an optimal combination of photographing equipment based on input information on the photographing equipment will be described with reference to FIG.

[0097] First, in step S3201, the control means 105 generates shooting conditions corresponding to the combination of camera and lens as shown in Figs. 26 to 29 from information related to the shooting equipment. Next, in step S3202, the control means 105 executes a series of processes for generating and evaluating a simulation image, which has been described with reference to FIG. 25 in the first embodiment, for each of the generated combinations. For example, a case where the flow of FIG. 25 is performed for combination 1 will be described. Steps S2501 and S2502 form a loop for setting various shooting conditions.

[0098] In Fig. 25, the shooting conditions to be assigned (scanned) are time, position, direction, and focal length, where T is time, P is position, D is direction, and F is focal length. In this example, they are assigned in order in a brute force manner, but the order in which the shooting conditions are assigned is not limited to this. It is also possible to assign only the necessary conditions efficiently using some algorithm. Step S2501 is a loop for comprehensively evaluating T (time) in the range of T0 to TN, step S2502 is a loop for comprehensively evaluating P (position) in the range of P0 to PN, and step S2503 is a loop for comprehensively evaluating D (direction) in the range of D0 to DN.

[0099] Step S2504 is a loop for comprehensively evaluating F (focal length) in the range of F0 to FN, but since the F-value in the case of combination 1 is only F8 (300 mm), this loop is executed only once. Step S2505 is a loop for comprehensively evaluating AV (aperture value) in the range of AV0 to AVN, but since the F-value of combination 1 is only AV4 (F4.0), this loop is executed only once.

[0100] Step S2506 is a loop for comprehensively evaluating TV (shutter speed) in the range of TV0 to TVN, and step S2507 is a loop for comprehensively evaluating SV (ISO sensitivity) in the range of SV0 to SVN. Step S2508 is a process in which virtual space construction means 101 constructs a virtual space under conditions based on loop variable T.

[0101] Step S2509 is a process for generating a simulation image when shooting under shooting conditions based on the loop variables P, D, F, AV, TV, and SV. In step S2510, the evaluation means 103 calculates an evaluation value between the simulation image generated in step S1709 and the desired captured image. Steps S2511 and S2512 are processes in which the notification means 104 notifies the user of a simulation image and shooting conditions based on the evaluation result calculated in step S2510.

[0102] Steps S2510 to S2513 represent a determination of whether to continue loop processing for time T, position P, direction D, focal length F, aperture value AV, shutter speed TV, and ISO sensitivity SV, and steps S2508 to S2512 are repeated until all conditions are covered. Next, in step S3203, the control unit 105 determines whether or not the generation and evaluation of simulation images have been performed for all combinations. If the generation and evaluation of simulation images has been completed for all combinations, the process proceeds to step S3204.

[0103] Next, in step S3204, the control means 105 compiles the evaluation results of the simulation images performed for all combinations. FIG. 33 is a diagram showing an example of score calculation for each equipment combination. As shown in FIG. 33, for camera-lens combinations 1 to 4, a score for each desired captured image and a total score, which is the sum of the scores, are calculated.

[0104] Here, the score is calculated by integrating the evaluation values ​​(similarity) when the similarity (Mtotal) is equal to or greater than the threshold (Th) for one desired photographed image for each combination. The total score is calculated by integrating all the scores for each desired photographed image. The equipment combination with the highest total score is notified to the user via the notification means 104.

[0105] As described above, by using this embodiment, it is possible to notify the user of the optimum combination of photographic equipment for obtaining the desired photographic image, so that the user can more accurately select the photographic equipment for obtaining the desired photographic image without having to bring a large amount of photographic equipment to the site in advance. In addition, since the simulation can be used to automatically extract the shooting conditions for obtaining the desired photographic image, it is possible to extract the shooting conditions for obtaining the desired photographic image more accurately without having to bring a large amount of photographic equipment to the site in advance.

[0106] Furthermore, depending on the desired captured image, the system evaluates the similarity of the number of subjects, the position and size of the face, the direction, brightness, blur, etc., using different weights for the person area and the background area, making it possible to more effectively extract the shooting conditions for obtaining an image close to the desired captured image. EXAMPLES

[0107] Next, a third embodiment of the present invention will be described. In the third embodiment, a method for extracting a combination of photographic equipment that can obtain optimal photographing results within the total weight of the photographic equipment desired by the user by taking the weight of the photographic equipment into consideration will be described. The basic configuration is the same as in the first embodiment, but the information input means 100 is capable of inputting desired weight information as information on the photographic equipment.

[0108] That is, in this embodiment, the information about the photographic equipment includes the camera sensor size, shutter speed, ISO sensitivity, camera weight, lens focal length, aperture value, lens weight, etc. The desired weight information is the total weight of the photographic equipment desired by the user. Here, for example, 1,500 g is input.

[0109] The photographic equipment information is information about multiple cameras and multiple lenses. The photographic equipment information may be information about cameras and lenses that the user owns, or information about cameras and lenses that can be rented from a rental service, etc. Also, information about cameras and lenses that the user is considering purchasing may be input. Information regarding the photographic equipment may be prepared in advance as presets for a plurality of cameras and lenses in the information input means, and the information may be selected from the presets.

[0110] The possible ranges of the shooting conditions (shooting angle of view, aperture value, shutter speed, ISO sensitivity) described in FIG. 9 of the first embodiment are determined according to the combination of camera and lens in the input information about the shooting equipment. for example,

[0111] Camera 1: Sensor size: Full size, Shutter speed: 1 second to 1 / 8000 second, ISO 100 to ISO 51200, Weight: 800g Camera 2: Sensor size: APS-C, Shutter speed: 1 sec. to 1 / 4000 sec., ISO 100 to ISO 12800, 400g Lens 1: Focal length 300mm, F value 4.0, weight: 1100g Lens 2: Focal length 135mm, F value 2.0, weight: 700g Given the following information about the equipment:

[0112] The shooting conditions for combination 1 (camera 1 + lens 1) are as shown in Figure 26. The shooting conditions for combination 2 (camera 1 + lens 2) are as shown in Figure 27. The shooting conditions for combination 3 (camera 2 + lens 1) are as shown in Figure 28. The shooting conditions for combination 4 (camera 2 + lens 2) are as shown in FIG.

[0113] The total weight for each combination is then calculated. The total weight of combination 1 (1 camera + 1 lens) is 1,900g. The total weight of combination 2 (camera 1 and lens 2) is 1,500g. The total weight of combination 3 (2 cameras and 1 lens) is 1,500g. The total weight of combination 4 (2 cameras and 2 lenses) is 1,100g. It becomes.

[0114] Next, the total weight of each combination is compared with the desired weight information (1,500g). Since the total weight of combination 1 is heavier than the desired weight information, it is decided to exclude combination 1 from the simulation image generation. Since the total weight of combinations 2 to 4 is equal to or less than the desired weight information, it is decided to generate a simulation image.

[0115] The flow of generating and evaluating a simulation image (FIG. 25) explained in the first embodiment is carried out for the combinations 2 to 4, and the evaluation results calculated for each combination are collected and compared to extract the optimum combination of equipment for obtaining a desired photographed image. The rest of the process is the same as in the second embodiment, so the explanation is omitted.

[0116] In this embodiment, the total weight of each combination is simply compared with the desired weight information, but the difference Mg (=Mg-Tg) between the desired weight information Ig and the total weight Tg may be multiplied by a weighting coefficient Kg and added to the similarity of the entire image. In this case, the similarity Mtotal2 is calculated using Equation 9.

[0117] Mtotal2=Kn*Mn+Kp*Mp+Ks*Ms +Kd*Md+Kl*Ml+Kb*Mb +K'n*M'n+K'p*M'p+K's*M's +K'd*M'd+K'l*M'l+K'b*M'b+Kg*Mg··(Formula 9)

[0118] Since the contents other than Kg and Mg are the same as those in the first embodiment, the explanation will be omitted. If the total weight is heavier than the desired weight, Mg will be a negative number and Mtotal2 will be lower; if the total weight is lighter than the desired weight, Mg will be a positive number and Mtotal2 will be higher.

[0119] Mtotal2 may be compared with a preset threshold value (Th), and only when Mtotal2 is greater than Th, the user may be notified of a simulation image and information about the photographing equipment and photographing conditions used when generating the simulation image. In other words, the notifying means can notify the user of information about the camera and lens combination when the similarity is equal to or greater than a predetermined value. The notification method may be output to a display device of the user terminal, output by printing, or data output to an external medium or a device connected to a network.

[0120] As described above, according to this embodiment, the user can be notified of the optimum combination of photographic equipment taking into consideration the weight required to obtain the desired photographic image, and therefore the user can more accurately select the photographic equipment required to obtain the desired photographic image without having to bring a large amount of photographic equipment to the site in advance. In addition, since the shooting conditions for obtaining the desired photographic image can be automatically extracted using simulation, the shooting conditions for obtaining the desired photographic image can be extracted in advance more accurately without having to bring a large amount of photographic equipment to the site in advance.

[0121] Furthermore, the number of subjects, the position and size of the face, the direction, brightness, and the degree of similarity in the blur are evaluated using different weights for the person area and the background area according to the desired captured image, making it possible to more effectively extract in advance the shooting conditions for obtaining an image close to the desired captured image.

[0122] The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the gist of the present invention, and these modifications are not excluded from the scope of the present invention. A computer program for implementing all or part of the control in this embodiment may be provided to the simulation device via a network or various storage media. Then, a computer (or a CPU, MPU, etc.) in the simulation device may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention. [Explanation of symbols]

[0123] 100 Information input means 101 Virtual space construction means 102 Simulation image generation means 103 Evaluation methods 104 Means of Notification 105 Control Means

Claims

1. an information input means for inputting information on the event to be photographed, photographing information, and a desired photographed image; a virtual space constructing means for constructing a virtual space that imagines the event using a subject model based on the information input by the information inputting means; a simulation image generating means for generating a simulation image using the virtual space from an image obtained by photographing the event; an evaluation means for evaluating a similarity between the simulation image and a desired captured image in terms of at least one of brightness and blur amount; a notification means for notifying a user of a combination of photographing equipment required for taking the simulation image having a similarity evaluated by the evaluation means that is equal to or greater than a predetermined value, and of photographing conditions corresponding to the combination.

2. 2. The simulation device according to claim 1, wherein the event information includes subject position information representing the content of the event, the order of the event, and movement information of subjects participating in the event.

3. 3. The simulation device according to claim 2, wherein the subject position information includes information on at least one of the position of the subject and the orientation of the subject at each time.

4. 4. The simulation device according to claim 2, wherein the subject position information is selectable from a plurality of preset data.

5. The simulation device according to any one of claims 1 to 4, characterized in that the shooting information includes a shooting position, a shooting direction, and a shooting angle of view, as well as information regarding at least one of an aperture value, a shutter speed, an ISO sensitivity, and a white balance data.

6. The simulation device according to any one of claims 1 to 5, characterized in that the desired captured image includes image information and is associated with at least one of the number of subjects, the position of their faces, the size of their faces, and the direction of their faces.

7. 7. The simulation device according to claim 1, wherein the desired photographed image can be selected from a plurality of sample images stored as preset data.

8. 8. The simulation device according to claim 1, wherein the information input means is capable of inputting at least one of information regarding the brightness of a venue where the event is held and a background image of the event venue.

9. 9. The simulation device according to claim 1, wherein the simulation image generating means adds a background image to the simulation image based on image data of the background of a venue where the event is held.

10. 10. The simulation device according to claim 1, wherein the simulation image generating means adjusts brightness of the simulation image based on input photographing conditions.

11. 11. The simulation device according to claim 1, wherein the simulation image generating means adds blur to at least a background image of the simulation image based on input photographing conditions.

12. The simulation device according to any one of claims 1 to 11, characterized in that the evaluation means calculates a similarity between the simulation image and a desired captured image for at least two of the following: number of subjects, position of the subjects, size of the subjects, orientation of the subjects, brightness of the subjects, background image, amount of background blur, and color of the subjects.

13. 13. The simulation device according to claim 12, wherein the evaluation means calculates an evaluation value by weighting and adding the at least two calculated similarities.

14. 14. The simulation device according to claim 1, wherein the notification means notifies a user of the simulation image and the corresponding shooting conditions when the similarity is equal to or greater than a predetermined value.

15. 15. The simulation device according to claim 1, wherein the information input means is capable of inputting information relating to photographing equipment.

16. 16. The simulation device according to claim 15, wherein the information about the photographic equipment includes at least one of information about a sensor size of a camera, a shutter speed, an ISO sensitivity, and a weight of the camera.

17. 17. The simulation device according to claim 16, wherein the information about the photographic equipment includes information about a focal length of a lens, an aperture value, and a weight of a lens.

18. 18. The simulation device according to claim 17, wherein said notification means notifies a user of information regarding the combination of said camera and lens when said similarity is equal to or greater than a predetermined value.

19. A computer program for controlling each means of the simulation device according to any one of claims 1 to 18 by a computer.

20. 20. A computer readable storage medium storing the computer program according to claim 19.

Images