TRANSMISSION PROGRAM, TERMINAL DEVICE, AND TRANSMISSION METHOD

Abstract

Promote the provision of services using multi-view images. [Solution] The transmission program of the present application causes a terminal device to execute an acquisition procedure for acquiring a multi-perspective image of a transaction object taken from different viewpoints, a transmission procedure for transmitting the multi-perspective image together with object-related information relating to the transaction object to a service providing device that provides a service for distributing information about the transaction object, and a display procedure for displaying a guide for taking photographs of the multi-perspective image depending on the distance between the position where the photographs are taken and the transaction object.

Description

[Technical field]

[0001] The present invention relates to a transmission program, a terminal device, and a transmission method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there has been provided a technique for generating multi-viewpoint images (free-viewpoint images) that are a plurality of images obtained by capturing a subject (a subject to be photographed) from different viewpoints (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-119513 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional technology may not promote the provision of services using multi-view images. For example, the above-mentioned conventional technology allows users to utilize multi-view images, but does not take into consideration the provision of services using multi-view images.

[0005] The present application has been made in consideration of the above, and aims to promote the provision of services using multi-view images. [Means for solving the problem]

[0006] The transmission program of the present application causes a terminal device to execute an acquisition procedure for acquiring a multi-viewpoint image captured from different viewpoints of a transaction object, a transmission procedure for transmitting the multi-viewpoint image together with object-related information relating to the transaction object to a service providing device that provides a service for distributing information about the transaction object, and a display procedure for displaying a guide for taking photographs of the multi-viewpoint image depending on the distance between the position where the photographs are taken and the transaction object. Effect of the Invention

[0007] According to one aspect of the embodiment, it is possible to promote the provision of services using multi-view images. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of processing by an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of photographing using the terminal device according to the embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a gaze point according to the embodiment. [Figure 4] FIG. 4 is a diagram showing an example of shooting for generating a multi-view image according to the embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of a display during shooting according to the embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of the arrangement of the 3D model according to the embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a display of a 3D model according to the embodiment. [Figure 8] FIG. 8 is a diagram illustrating an example of a functional configuration of a terminal device according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating an example of a functional configuration realized by the first application according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a functional configuration realized by the second application according to the embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of a transaction target information storage unit according to the embodiment. [Figure 12] FIG. 12 is a diagram illustrating an example of the configuration of a service providing device according to the embodiment. [Figure 13] FIG. 13 is a flowchart showing an example of the flow of the transmission process according to the embodiment. [Figure 14] FIG. 14 is a flowchart showing an example of the flow of information processing according to the embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of processing by the information processing system according to the embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of a list display of trading objects according to the embodiment. [Figure 17] FIG. 17 is a diagram showing an example of display of an image group including multi-viewpoint images according to the embodiment. [Figure 18] FIG. 18 is a flowchart showing an example of the flow of the display process according to the embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, a detailed description will be given of a transmission program, a terminal device, and a transmission method according to the present application (hereinafter, referred to as "embodiments") with reference to the drawings. Note that the transmission program, the terminal device, and the transmission method according to the present application are not limited to the embodiments. In addition, the same components in the following embodiments are given the same reference numerals, and duplicated descriptions will be omitted.

[0010] [Embodiment] [1. Information Processing] An example of information processing according to the embodiment will be described below with reference to FIG. 1. FIG. 1 is a diagram showing an example of processing by the information processing system according to the embodiment. In the information processing system 1, a terminal device 10 captures multiple viewpoint images (free viewpoint images), which are multiple images of a subject (hereinafter also referred to as a "photographed object") captured from different viewpoints, and the terminal device 10 displays the multiple viewpoint images. In the following, a trading object traded in an electronic commerce service provided by the information processing system 1 will be described as an example of the photographed object. The trading object here refers to the use of a product or a service (services), etc. The trading object may be any object that is traded in the information processing system 1.

[0011] [1-1. Configuration of information processing system] First, the configuration of the information processing system 1 will be described. As shown in Fig. 1, the information processing system 1 includes a terminal device 10 and a service providing device 100. The terminal device 10 and the service providing device 100 are connected to each other via a predetermined communication network (such as network N in Fig. 8) so as to be able to communicate with each other by wire or wirelessly. Note that the information processing system 1 shown in Fig. 1 may include a plurality of terminal devices 10 and a plurality of service providing devices 100.

[0012] The service providing device 100 is a server device (information processing device) that accepts submission of information based on images captured by the terminal device 10. For example, the service providing device 100 accepts submission of multi-viewpoint images captured by the terminal device 10. The service providing device 100 provides a service of distributing information. For example, the service providing device 100 provides a service of distributing information on a trading object. For example, the service providing device 100 provides a service related to buying and selling of a trading object, such as putting the trading object up for sale.

[0013] For example, the service providing device 100 is an information processing device that provides an electronic commerce service to a user who uses the terminal device 10. The service providing device 100 distributes (provides) various information related to the electronic commerce service to the terminal device 10. In the following example, a case will be shown in which the service providing device 100 provides an auction service, which is an example of an electronic commerce service. For example, the service providing device 100 provides a service for listing trade objects and accepts listings of trade objects from users (sellers). The service providing device 100 also accepts bids from users for the listed trade objects. Note that the service providing device 100 may provide various services for trading trade objects, such as a shopping service, a flea market service, and the like, in addition to the auction service.

[0014] Moreover, since the electronic commerce services such as an auction service provided by the service providing device 100 are similar to normal electronic commerce services except for the registration and distribution of multi-viewpoint images, the description of the electronic commerce services will be omitted as appropriate. Note that the services provided by the service providing device 100 are not limited to electronic commerce services, and may be services that distribute various information, such as a navigation service, an electronic payment service, a search service, a news distribution service, a weather forecast service, a finance service, a route search service, a map providing service, a travel service, a restaurant introduction service, a facility reservation service, a SNS (Social Networking Service) service, and a web blog service.

[0015] The terminal device 10 is a device (computer) used by a user. In the example of FIG. 1, the terminal device 10 is a smartphone used by a user U1. The terminal device 10 accepts operations by the user. The terminal device 10 has an image sensor (hereinafter also referred to as a "camera") that captures an image. For example, the terminal device 10 has a camera 70 that is an outer camera that functions as an image sensor. The terminal device 10 captures an image using the camera 70. Note that the terminal device 10 is not limited to a smartphone, and may be various types of terminal devices such as a tablet terminal, a mobile phone, a PDA (Personal Digital Assistant), and a notebook PC (Personal Computer) as long as they are applicable to the processes described below. In the following description, the terminal device 10 may be regarded as the same as the user. That is, in the following description, the terminal device 10 may be read as the user, and the user may be read as the terminal device 10, as necessary.

[0016] [1-2. An example of information processing] From here, the process shown in FIG. 1 will be described. For example, the terminal device 10 starts an application (for example, the first application 81, etc.) for transmitting information on the transaction object to the service providing device 100, and executes the following process. First, the terminal device 10 acquires a multi-view image FV1 from nine images in chronological order, RP11, RP12, RP13, RP14, RP15, RP16, RP17, RP18, and RP19 (step S11). In the example of FIG. 1, the terminal device 10 acquires a multi-view image FV1 in which sneakers (hereinafter referred to as "shoes A"), which are the transaction object, are captured from nine different viewpoints. Note that, in FIG. 1, only three images, RP11, which is the first image, RP15, which is the middle image, and RP19, which is the last image, are illustrated among the nine images RP11 to RP19 constituting the multi-view image FV1. The multi-view image FV1 is a multi-view image taken from the left front toe, through the left side, to the left rear heel.

[0017] In the example of Fig. 1, the terminal device 10 captures an image of the shoe A (image subject SB1) that is the object of the transaction, and generates a multi-view image FV1, which will be described later. The terminal device 10 captures an image of the shoe A and generates the multi-view image FV1, thereby acquiring the multi-view image FV1.

[0018] Then, the terminal device 10 transmits the multi-view image FV1 of the shoes A together with information related to the shoes A (hereinafter also referred to as "object related information") to the service providing device 100 (step S12). In the example of FIG. 1, the terminal device 10 transmits listing information for putting the shoes A up for auction as the object related information INF1 to the service providing device 100 together with the multi-view image FV1 of the shoes A. For example, the object related information INF1 includes text information explaining the contents of the shoes A. The object related information INF1 also includes information related to the auction, such as the starting price of the shoes A, the minimum bid price, and the bidding period.

[0019] The service providing device 100 receives the multi-view image FV1 of shoe A and the object related information INF1 of shoe A from the terminal device 10, and registers the multi-view image FV1 and the object related information INF1 in association with shoe A (step S13). In the example of Fig. 1, the service providing device 100 adds shoe A as an item to be auctioned, and manages the multi-view image FV1 and the object related information INF1 in association with shoe A.

[0020] By the terminal device 10 transmitting a multi-view image of the trading object together with object-related information related to the trading object to the service providing device 100, the service providing device 100 can distribute the multi-view image of the trading object as soon as the auction of the trading object is registered from the terminal device 10. Therefore, the terminal device 10 can promote the provision of services using the multi-view image. This allows the information processing system 1 to display the multi-view image as an image of the trading object in an auction, a flea market, or the like.

[0021] [1-3. Photographing the subject] From here, a case where a user takes a photograph to generate a multi-view image FV1 will be described. First, photographing by a user will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of photographing using a terminal device according to an embodiment. In the following description, a case where a user U1 takes a photograph of a shoe A as a photographing subject SB1 will be described as an example.

[0022] As shown in Fig. 2, a user U1 aims a camera 70 provided in a terminal device 10 at the subject SB1 to capture an image of the subject SB1. In Fig. 2, the terminal device 10 captures the subject SB1 by directing the line of sight (referred to as "line of sight VL") of the camera 70 to the subject SB1 from a viewpoint position (referred to as "viewpoint VP") on the left side of the subject SB1.

[0023] Furthermore, the terminal device 10 determines the point of gaze based on the image captured by the camera 70. The terminal device 10 determines the point of gaze based on the line of sight VL from the viewpoint VP. The terminal device 10 determines the point of gaze based on line of sight information from two viewpoints. This point will be described with reference to FIG. 3. FIG. 3 is a diagram showing an example of the point of gaze according to the embodiment.

[0024] In FIG. 3, the terminal device 10 used by the user U1 is classified as “terminal device 10” according to the difference in the photographing position. a ", "Terminal device 10 b However, when there is no need to distinguish between them, the camera 70 will be referred to as the terminal device 10. Similarly, depending on the shooting position, the camera 70 will be referred to as the camera 70 a ", "Camera 70 b However, when no distinction is required, it will be referred to as camera 70.

[0025] In FIG. 3, a user U1 moves from a first location to a terminal device 10. a Camera 70 a The terminal device 10 then points the camera 10 toward the subject SB1 to capture the subject SB1. a From Camera 70 a Gaze of VL a is aimed at the subject SB1 to photograph the subject SB1.

[0026] Also, the user U1 accesses the terminal device 10 from the second location. b Camera 70 b The terminal device 10 then points the camera 10 at the left side of the subject SB1 to capture the subject SB1. b From Camera 70 b Gaze of VL b is aimed at the subject SB1 to photograph the subject SB1.

[0027] The terminal device 10 determines the gaze point using information acquired by shooting from two viewpoints, the first position and the second position. The terminal device 10 calculates the three-dimensional position (hereinafter also simply referred to as "position") of the gaze point using information acquired by shooting from two viewpoints, the first position and the second position. The terminal device 10 estimates the distance between the subject SB1 to be shot and the camera 70 using information acquired by shooting from the two viewpoints. The terminal device 10 calculates the gaze point from the viewpoint VP a View from VL a , and viewpoint VP b View from VL b The gaze point is determined based on the above.

[0028] For example, the terminal device 10 has a line of sight VL a and gaze VL b In this manner, the terminal device 10 determines the position (three-dimensional point) where the line of sight VL intersects with the line of sight VL as the gaze point GP1. a and gaze VL b The above is merely an example, and the terminal device 10 calculates the gaze point GP1 based on, for example, the line of sight VL a and gaze VL b If VL does not intersect with VL, a and gaze VL b For example, in FIG. 3, a planar view is shown for the sake of explanation, but in a three-dimensional space, the line of sight VL a and gaze VL b If the lines of sight VL and VL are offset in the height direction and do not intersect, a and gaze VL b In this manner, the terminal device 10 may determine the gaze point GP1 by any method as long as the method is capable of determining the gaze point.

[0029] Furthermore, the determination of the gaze point is not limited to the case where the gaze point is determined by shooting from two viewpoints as described above. For example, the terminal device 10 may automatically set the gaze point by appropriately using techniques such as identification of the image capture target and identification of depth. Furthermore, the terminal device 10 may determine the gaze point by shooting from, for example, one viewpoint. For example, the terminal device 10 may recognize the ground surface of the image capture target and determine (calculate) the gaze point (position). For example, the terminal device 10 may determine the gaze point using a gaze point determination logic provided by an external device. The line of sight VL may be a straight line along the optical axis of the camera 70.

[0030] The terminal device 10 determines a plurality of positions (viewpoints) for capturing images related to a multi-viewpoint image based on the gaze point GP1. This point will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of capturing images for generating a multi-viewpoint image according to an embodiment.

[0031] For example, the terminal device 10 determines (calculates) the viewpoints VP1 to VP9 based on the set shooting range and the number of shooting positions. For example, the terminal device 10 calculates nine viewpoints VP1 to VP9 using information on the estimated distance between the shooting target SB1 and the camera 70. In the example of FIG. 4, the terminal device 10 determines the viewpoints VP1 to VP9 based on the set shooting range (120°) and the number of shooting positions (9). For example, the terminal device 10 determines nine viewpoints VP1 to VP9 that are arranged at 15° intervals with the viewpoint VP5 at the center and where each of the lines of sight VL1 to VL9 passes through the gaze point GP1. The terminal device 10 performs the above-mentioned processing of determining the gaze point and viewpoints by appropriately using existing three-dimensional space information processing technology and image processing technology.

[0032] The above is merely an example, and the terminal device 10 may determine a position (viewpoint) by any method as long as the terminal device 10 can determine a position (viewpoint) that satisfies the set shooting range and the number of shooting positions based on the gaze point GP1. The shooting range (120°) and the number of shooting positions (9) shown in FIG. 4 are merely an example, and the shooting range (angle) and the number of shooting positions (number of cuts) can be set arbitrarily. For example, the shooting range may be 90°, and the number of shooting positions may be five. The positions of the viewpoints VP1 to VP9 may be changed according to a change in the positional relationship between the gaze point GP1 and the terminal device 10 before shooting starts. The terminal device 10 may also be able to switch parameters such as parallax angle by setting.

[0033] Here, the display of the terminal device 10 relating to the position (viewpoint) for determining the above-mentioned gaze point GP1 and for photographing will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of a display during photographing according to the embodiment. Display screens DP1 to DP5 shown in Fig. 5 show displays on the screen (output unit 50) of the terminal device 10. Note that the points described in Figs. 1 to 4 will be omitted as appropriate.

[0034] First, the display screen DP1 shows a case where shooting is performed from the first viewpoint in Fig. 3. As shown on the display screen DP1, the user U1 is shooting the subject SB1 from a viewpoint VP a From the camera 70 of the terminal device 10 a The user U1 points the camera 70 at the subject SB1 and taps the center of the subject SB1 on the display screen DP1 to capture the subject SB1. For example, the terminal device 10 detects the position where the user U1 tapped and the camera 70 a (Viewpoint VP a ) and the line of sight VL a As shown on the display screen DP1, by displaying a message "Please tap the center of the product", it is possible to capture images suitable for generating multi-viewpoint images.

[0035] The display screen DP2 shows a case where shooting is performed from the second viewpoint in Fig. 3. As shown on the display screen DP2, the user U1 is looking at the viewpoint VPb From the camera 70 of the terminal device 10 b The user U1 points the camera 70 at the subject SB1 and taps the center of the subject SB1 on the display screen DP2 to capture the subject SB1. For example, the terminal device 10 detects the position where the user U1 tapped and the camera 70 b (Viewpoint VP b ) and the line of sight VL b In this way, the user U1 photographs the subject SB1 from the first position, and photographs the subject SB1 from a different second position. As a result, the terminal device 10 determines the gaze point GP1 using information acquired by photographing from two viewpoints, the first position and the second position, and displays a guide according to the distance. For example, the terminal device 10 estimates the depth from the image, and determines the three-dimensional gaze point GP1 from the difference in tap positions.

[0036] Then, as shown on the display screen DP3, the terminal device 10 displays a pin MK1 for specifying an angle for shooting and a transparent shooting position guide GD1 in a superimposed manner. For example, the terminal device 10 appropriately uses existing AR (Augmented Reality) technology or image processing technology to display an object (virtual object) such as the shooting position guide GD1 superimposed on the display of the shooting area of ​​the camera 70. The shooting position guide GD1 is a fan-shaped guide with nine circles corresponding to each of the viewpoints VP1 to VP9 shown in FIG. 4 arranged on an arc centered on the gaze point GP1. The user U1 places the terminal device 10 at a desired position and taps the pin MK1 to specify the angle for shooting.

[0037] Display screen DP4 shows a case where the shooting angle is specified and the display of pin MK1 is changed to pin MK2. This shows a case where shooting is performed from the second viewpoint in Fig. 3. When the shooting angle is specified, the terminal device 10 displays the shooting position guide GD2 with a fixed orientation, as shown on display screen DP4.

[0038] Thereafter, the terminal device 10 starts shooting at each of the positions corresponding to the viewpoints VP1 to VP9, ​​as shown on the display screen DP5. The user U1 places the terminal device 10 at each viewpoint (position) and shoots at each viewpoint (position). In this way, before specifying the reference point of the gaze point, the terminal device 10 changes the orientation of the shooting position guide GD1 in accordance with the movement of the terminal device 10, and after the specification, fixes the orientation of the shooting position guide GD2. Then, when the terminal device 10 completes shooting at the corresponding viewpoint, the white circle turns yellow. The display below the camera button also changes.

[0039] In the example of FIG. 5, the terminal device 10 displays an object OB1 indicating the center of the shooting area. In addition, the terminal device 10 changes the display of the button BT1 to a button BT2 in response to the start of shooting. In the example of FIG. 5, the terminal device 10 displays a gauge GG1 that distinguishes between a shooting completion position where shooting is completed and a shooting incomplete position where shooting is incomplete among a plurality of positions. In the example of FIG. 5, it is shown that shooting at three positions (corresponding to viewpoints VP7 to VP9) on the heel side of the shooting target SB1 is completed. In addition, the terminal device 10 changes the color of the "circle" corresponding to the three positions (corresponding to viewpoints VP7 to VP9) on the heel side of the shooting target SB1 in the shooting position guide GD3 to a different color from the other "circle". In addition, the terminal device 10 may increase the transparency of the shooting position guide GD3 to a higher value than the shooting position guide GD2 in response to the start of shooting. The terminal device 10 performs the above-mentioned process of displaying the guide by appropriately using existing three-dimensional space information processing technology and image processing technology.

[0040] [1-4. Editing images] The captured image can be edited as desired. The terminal device 10 may accept a designation of a range to be cropped from the captured image and generate an image of only the designated range. This allows the user U1 to extract the crop range (the range including the product, the range of the gaze point) after capturing the image, and to create a smooth multi-view image.

[0041] The terminal device 10 may display the captured multi-view images in a line in a carousel format. The terminal device 10 may switch the display of images and rotate the multi-view images by operating a bar displayed at the bottom of the screen. The terminal device 10 may also accept editing for each image. The terminal device 10 displays the edited image in a manner that makes it distinguishable from other images. For example, the terminal device 10 displays the edited image with an arbitrary mark, such as a white circle icon, to make it distinguishable from other images.

[0042] Furthermore, the terminal device 10 may be capable of manually or automatically correcting colors, etc. The terminal device 10 may accept image correction for each image, or may accept correction for all images. This allows the image to be appropriately corrected and a multi-view image to be generated, even if the color, etc. changes due to backlighting, etc., during shooting. The terminal device 10 may automatically correct images using an averaging technique. The terminal device 10 may provide manual or automatic zoom.

[0043] [1-5. Multi-viewpoint image generation] Through the above-mentioned processing, the terminal device 10 captures nine captured images, namely, image P11, image P12, image P13, image P14, image P15, image P16, image P17, image P18, and image P19. The terminal device 10 captures image P11 at a position corresponding to viewpoint VP1. The terminal device 10 captures image P12 at a position corresponding to viewpoint VP2. Similarly, the terminal device 10 captures images P13 to P19 at positions corresponding to viewpoints VP3 to VP9, ​​respectively.

[0044] As a result, the terminal device 10 acquires images P11 to P19 that are the basis for generating a free-viewpoint image (multiple-viewpoint image). Then, the terminal device 10 generates a multi-viewpoint image based on the acquired images P11 to P19. Specifically, the terminal device 10 acquires appropriate parameters (for example, a projective transformation matrix) for converting the acquired nine images P11 to P19 into images for constituting the multi-viewpoint image by performing a predetermined calibration (camera calibration). Then, the terminal device 10 performs projective transformation on the image data (images P11 to P19) using the acquired parameters. Then, the terminal device 10 generates a free-viewpoint image (multiple-viewpoint image) that can smoothly transition the display of the multiple images (images RP11 to RP19) by connecting the projectively transformed images (images RP11 to RP19). However, the generation of the free-viewpoint image (multiple-viewpoint image) is similar to that of Patent Document 1, and therefore a detailed description thereof will be omitted. In this way, the terminal device 10 allows the user U1 to take an image, and then generates a free viewpoint image (multiple viewpoint image) whose viewpoint is controllable.

[0045] [1-6. Ease of shooting] Although a free viewpoint image (multiple viewpoint image) can be generated by the above-mentioned shooting, shooting is time-consuming. For example, it is difficult to shoot a plurality of images at equal intervals, and it is desired to facilitate shooting of images for generating free viewpoint images (multiple viewpoint images). Therefore, the terminal device 10 provides a model for guiding shooting as follows. This point will be described with reference to Figs. 6 and 7. Fig. 6 is a diagram showing an example of the arrangement of a 3D model according to the embodiment. Fig. 7 is a diagram showing an example of the display of a 3D model according to the embodiment. Note that the description of the same points as Figs. 1 to 5 will be omitted as appropriate.

[0046] As shown in Fig. 6, the terminal device 10 displays a model MD, which is a 3D model (three-dimensional model) indicating the shooting position and shooting direction. For example, the terminal device 10 appropriately uses existing AR technology or image processing technology to display an object (virtual object) such as the model MD by superimposing it on the display of the shooting area of ​​the camera 70. Although details will be described later, the terminal device 10 displays the model MD placed between the shooting target SB1 and the camera 70. For example, the terminal device 10 displays the model MD placed such that the angle and distance from the gaze point satisfy predetermined conditions.

[0047] Fig. 7 shows an example of the display of the model MD on the terminal device 10. Note that the display mode of the model MD is not limited to the mode shown in Fig. 7, and any display mode may be used as long as the user can recognize the shooting position and shooting direction at a specific viewpoint. In the example of Fig. 7, "1 / 13" is displayed on the terminal device 10, which indicates that shooting at one of the 13 viewpoints has been completed and shooting at the remaining 12 viewpoints has not been completed.

[0048] In FIG. 7, it is assumed that, among the circles (five circles are shown in FIG. 7) on the arc of the shooting position guide GD displayed on the terminal device 10, shooting has been completed at the viewpoint corresponding to the rightmost circle (◯). The terminal device 10 displays a model MD (black frame in FIG. 7) indicating the shooting position and shooting direction at the viewpoint corresponding to the second circle (◯) from the right end out of the 12 viewpoints for which shooting has not been completed. For example, the terminal device 10 displays the model MD so as to overlap with the shooting position guide GD, which is a guide icon corresponding to the gaze point GP1. Note that the display shown in FIG. 7 is merely an example, and the terminal device 10 performs processing such as displaying the model MD described above by appropriately using existing three-dimensional space information processing technology and image processing technology.

[0049] The user U1 photographs the object OB (white frame in FIG. 7) at a viewpoint corresponding to the circle at the right end of the photographing position guide GD by aligning the object OB with the model MD. For example, the terminal device 10 photographs the object OB according to the overlapping of the object OB with the model MD. When the overlapping range of the object OB and the model MD is equal to or greater than a predetermined threshold, the terminal device 10 determines that the condition is satisfied and performs photographing. For example, the terminal device 10 photographs the object OB or the like when the 2D guide and the 3D model of the model MD or the like overlap at a threshold or greater within a predetermined time such as a designated time. This allows the terminal device 10 to suppress camera shake during photographing. Note that, when the terminal device 10 uses the range where the 2D guide and the 3D model of the model MD or the like do not overlap for the judgment, the terminal device 10 may photograph the object OB or the like when the 2D guide and the 3D model of the model MD or the like overlap at a threshold or less. For example, the terminal device 10 may photograph the object OB or the like when the range where the 2D guide and the 3D model of the model MD or the like do not overlap is equal to or less than a threshold. In this way, the threshold value may be appropriately set according to the information used for the judgment. For example, when the object OB and the model MD match, the terminal device 10 may display or notify by vibration. Note that the terminal device 10 may allow some margin (error) in the position where the model MD is displayed, or may use a loose threshold value. Then, the terminal device 10 (automatically) corrects the captured image after photographing. For example, the terminal device 10 corrects the captured image according to the margin (error) in the display of the model MD.

[0050] As described above, the terminal device 10 can suppress camera shake during shooting by capturing an image according to the overlap of a 2D guide such as an object OB and a 3D model such as a model MD. Furthermore, the terminal device 10 can easily capture an image for generating a free viewpoint image (multiple viewpoint image) by displaying a guide such as the above-mentioned model. In this way, the terminal device 10 displays the model MD, which is a guide, in the form of AR (augmented reality) floating in space. As a result, the terminal device 10 can suppress camera shake of the user U1 by displaying the model MD so that the user U1 can recognize the shooting position, distance, etc.

[0051] [1-7. About the subject of photography] In the example of FIG. 1, a case is shown in which the shoe A, which is the object of trade, is photographed, but the subject to be photographed is not limited to the object of trade and may be, for example, a person, a landscape, or the like.

[0052] [1-8. About the generation target] In the above example, a case where a multi-viewpoint image is generated has been described, but the object to be generated is not limited to a multi-viewpoint image, and the terminal device 10 may generate a three-dimensional model (3D model) of a photographed object. For example, the terminal device 10 may generate a three-dimensional model (3D model) of the shoe A, which is the photographed object SB1, based on the photographed image.

[0053] [2. Example of functional configuration] An example of the functional configuration of the terminal device 10 and the service providing device 100 for executing the above-mentioned processes will be described below.

[0054] [2-1. An example of the functional configuration of a terminal device] First, an example of a functional configuration of the terminal device 10 will be described with reference to Fig. 8. Fig. 8 is a diagram for explaining an example of a functional configuration of the terminal device according to the embodiment. As shown in Fig. 8, the terminal device 10 has a communication unit 20, a storage unit 30, an input unit 40, an output unit 50, a physical sensor 60, a camera 70, and a control unit 80.

[0055] (Communication Department 20) The communication unit 20 is realized by, for example, a network interface card (NIC) or the like. The communication unit 20 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the service providing device 100. The communication unit 20 may be capable of communicating with a distribution server that distributes various contents.

[0056] (Storage unit 30) The storage unit 30 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 30 has a transaction object information storage unit 31. The storage unit 30 stores various information in addition to the transaction object information storage unit 31. The storage unit 30 stores information used to display a shooting position guide, a 3D model (model MD), etc. The storage unit 30 stores various information related to the transaction object to be displayed in the second application. For example, the storage unit 30 stores information on the transaction object provided by the service providing device 100.

[0057] (Transaction target information storage unit 31) The transaction object information storage unit 31 stores information related to a transaction object. Fig. 11 is a diagram showing an example of the transaction object information storage unit according to the embodiment. In the example shown in Fig. 11, the transaction object information storage unit 31 has items such as "transaction object ID", "transaction object", "object related information", "multi-view image ID", and "multi-view image information".

[0058] The "transaction object ID" indicates identification information for identifying the transaction object. The "transaction object" indicates information for identifying the transaction object, such as a name, for example, a product name or a service name (service name). The transaction object information storage unit 31 may store information indicating a category of the transaction object in association with the transaction object ID. For example, the transaction object information storage unit 31 may store information indicating a category of the transaction object, such as a product or service, such as "shoes" or "haircut."

[0059] "Object related information" is information relating to the description of a trading object associated with a "trading object ID." For example, when a trading object is put up for sale, the object related information includes various information (listing information) required to put up for sale the trading object. The object related information includes various information related to the trading object, such as text information explaining the contents of the trading object, price, shipping fee, and inventory quantity. Note that information such as the name and object related information may be input by the user, or may be acquired by the terminal device 10 from an external device.

[0060] The "multi-view image ID" indicates the identification information of a multi-view image (free viewpoint image) captured with the corresponding transaction object as the shooting target (subject). The "multi-view image information" has items such as "captured image data," "calibration data," and "converted image data," and stores information related to the multi-view image.

[0061] "Captured image data" refers to image data from which a multi-view image is generated. For example, the "captured image data" stores a plurality of pieces of image data captured by the terminal device 10 by the user following a guide from the first application. Since a multi-view image is generated from a plurality of pieces of image data, a plurality of pieces of captured image data are stored for one multi-view image.

[0062] "Calibration data" refers to predetermined calibration data (camera calibration) for generating a multi-view image. In FIG. 11, the calibration data is conceptually shown as "R11". The calibration data includes various data for appropriately projecting and transforming captured image data so that the terminal device 10 can acquire images that constitute the multi-view image. For example, the calibration data includes camera parameters of the camera 70 (imaging device) of the terminal device 10 that captured the images. For example, the calibration data includes various camera parameters required for generating a multi-view image, such as the focal length of the camera 70 and the aspect ratio and skew of the image generated by the camera 70.

[0063] The "converted image data" refers to image data constituting a multi-viewpoint image. The converted image data refers to image data that has been subjected to a predetermined calibration and projection transformation on the captured image data, for example.

[0064] In FIG. 11, the transaction object (transaction object SB1) identified by the transaction object ID "SB1" is shoes A. The object related information of the transaction object SB1 is object related information INF1. For example, the object related information INF1 includes various information related to the transaction object SB1, such as text information explaining the contents of the transaction object SB1, which is shoes A, the price, shipping fee, and inventory quantity.

[0065] Also, it is shown that the multi-view image of shoe A, which is the trade object SB1, is a multi-view image (multi-view image FV1) identified by a multi-view image ID "FV1."

[0066] Multi-viewpoint image FV1 indicates that the captured image data is nine images, "P11" to "P19." In order to generate multi-viewpoint image FV1, nine captured images, image P11, image P12, image P13, image P14, image P15, image P16, image P17, image P18, and image P19, are used.

[0067] Also, in the multi-viewpoint image FV1, the calibration data for each of the nine images P11 to P19 is indicated as “R11” to “R19.” For example, the calibration data for image P11 is indicated as calibration data R11.

[0068] The multi-viewpoint image FV1 indicates that the converted image data is nine images, "RP11" to "RP19." For example, the converted image RP11 indicates that the converted image is an image obtained by converting image P11 using calibration data R11. The multi-viewpoint image FV1 indicates that it is composed of nine converted images, image RP11, image RP12, image RP13, image RP14, image RP15, image RP16, image RP17, image RP18, and image RP19.

[0069] That is, the multi-view image FV1 is composed of nine images in chronological order, RP11, RP12, RP13, RP14, RP15, RP16, RP17, RP18, and RP19. For example, in the multi-view image FV1, the images are numbered 1 to 9 in the order RP11, RP12, RP13, RP14, RP15, RP16, RP17, RP18, and RP19. For example, the multi-view image FV1 is composed of nine images RP11 to RP19 in a format such as JPEG (Joint Photographic Experts Group). Note that the multi-view image FV1 may be in a format such as APNG (Animated Portable Network Graphics) as long as it is possible to provide a free viewpoint image (number of multi-view images) using the images RP11 to RP19.

[0070] The transaction object information storage unit 31 may store various information according to the purpose, not limited to the above. The transaction object information storage unit 31 may store an image (initial display image) corresponding to the initial display of the multi-view image. For example, the transaction object information storage unit 31 stores an image RP11 as the initial display image of the multi-view image FV1.

[0071] The transaction object information storage unit 31 may store various information used to generate each multi-view image in association with the corresponding multi-view image ID. For example, the transaction object information storage unit 31 stores information on the gaze point used when generating the multi-view image in association with the multi-view image ID. For example, the transaction object information storage unit 31 stores information indicating the gaze point GP1 in association with the multi-view image FV1.

[0072] Also, for example, the transaction object information storage unit 31 stores information indicating a plurality of positions for capturing the multi-view image in association with the multi-view image ID. For example, the transaction object information storage unit 31 stores information indicating the positions of each of the viewpoints VP1 to VP9 in association with the multi-view image FV1.

[0073] (Input unit 40) The input unit 40 is an input device that accepts various operations from a user. The input unit 40 accepts various operations from a user via a display surface using a touch panel function. For example, the input unit 40 functions as a reception unit that accepts user operations using a touch panel function. The input unit 40 may also accept various operations from buttons provided on the terminal device 10 or a keyboard or mouse connected to the terminal device 10.

[0074] For example, the input unit 40 accepts operations such as user specification operations via the screen (output unit 50) of the terminal device 10. Note that, although a capacitance method is mainly adopted as a method for detecting user operations by the input unit 40 in tablet terminals, any other detection method such as a resistive film method, a surface acoustic wave method, an infrared method, or an electromagnetic induction method may be adopted as long as it is possible to detect user operations and realize the function of a touch panel.

[0075] (Output unit 50) The output unit 50 is a display device for displaying various information, that is, a screen. For example, the output unit 50 is realized by a liquid crystal display or the like. When a touch panel is adopted in the terminal device 10, the input unit 40 and the output unit 50 are integrated. The output unit 50 may be referred to as a screen.

[0076] (Physical Sensor 60) The physical sensor 60 is a sensor that detects the physical state of the terminal device 10. For example, the physical sensor 60 is a gyro sensor that measures the inclination of the terminal device 10 in three axial directions. Note that the physical sensor 60 is not limited to a gyro sensor, and any sensor such as an acceleration sensor, a temperature sensor, a volume sensor, or a brightness sensor can be used.

[0077] (Camera 70) The camera 70 is an imaging device for capturing images (moving images or still images). The camera 70 is configured with imaging elements such as a CCD (charged-coupled devices) sensor or a CMOS (complementary metal-oxide-semiconductor) sensor. The camera 70 may be configured with a main camera installed on the rear side of the terminal device 10 (the surface opposite to the surface on which the screen is installed) and a sub-camera installed on the rear side of the terminal device 10 (the surface on which the screen is installed).

[0078] (Control unit 80) The control unit 80 is, for example, a controller, and is realized by a CPU (Central Processing Unit), MPU (Micro Processing Unit), etc., executing various programs (corresponding to examples of a transmission program, a provision program, a display program, etc.) stored in a storage device inside the terminal device 10 using a RAM as a working area. The control unit 80 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0079] In the example shown in FIG. 8, the control unit 80 has a functional configuration for implementing the above-mentioned first display process and second display process by executing the first application 81 and the second application 82. The control unit 80 may also execute any other application, such as an application that serves as the OS (Operating System) of the terminal device 10 and an application for implementing reception of various operations. The first application 81 and the second application 82 may be integrated into one application. For example, the first application 81 and the second application 82 may be applications incorporated into an application used for electronic commerce, such as an auction application.

[0080] (First Application 81) The first application 81 causes the terminal device 10 to execute the above-mentioned transmission process and information process. The first application 81 is an application used for generating and transmitting a multi-view image. For example, FIG. 9 is a diagram showing an example of a functional configuration realized by the first application according to the embodiment. As shown in FIG. 9, the first application 81 realizes a first acquisition unit 811, a first display unit 812, a determination unit 813, an image capture unit 814, a generation unit 815, and a transmission unit 816.

[0081] The first acquisition unit 811 acquires various information from the storage unit 30. The first acquisition unit 811 acquires various information from the transaction target information storage unit 31. The first acquisition unit 811 receives various information from an external device such as the service providing device 100. The first acquisition unit 811 acquires photographing target information related to the photographing target.

[0082] The first acquisition unit 811 acquires multi-view images of a transaction object captured from different viewpoints. The first acquisition unit 811 acquires multi-view images of a transaction object, which is an auction item, captured from different viewpoints. The first acquisition unit 811 acquires multi-view images from the transaction object information storage unit 31. The first acquisition unit 811 acquires images P11 to P19 that are used as a basis for generating a free viewpoint image (multi-view image).

[0083] The first display unit 812 displays various information on the output unit 50 (such as a liquid crystal display). The first display unit 812 displays various information on the output unit 50. The first display unit 812 controls the output unit 50 and executes a process of displaying information.

[0084] The first display unit 812 functions as a providing unit that provides a 3D model indicating the shooting position and shooting direction for shooting a multi-view image of a target. When the 3D model is provided by transmitting it to another device, the transmitting unit 816 may function as the providing unit.

[0085] The first display unit 812 displays a guide for taking pictures of a multi-view image. The first display unit 812 displays a guide including a 3D model indicating a shooting position and a shooting direction. The first display unit 812 displays the 3D model on the screen of the terminal device 10. The first display unit 812 displays the 3D model superimposed on the screen of the terminal device displaying a shooting area including a shooting object. The first display unit 812 displays the 3D model superimposed on the screen of the terminal device displaying a shooting area including a trading object.

[0086] The first display unit 812 displays an object indicating the center of the shooting area. The first display unit 812 displays a 3D model arranged between a viewpoint for shooting the shooting target and a gaze point on the shooting target. The first display unit 812 displays a 3D model arranged between the gaze point determined based on a line of sight from the viewpoint and the terminal device. The first display unit 812 displays a 3D model arranged between the gaze point determined based on a first line of sight from the first viewpoint and a second line of sight from the second viewpoint and the terminal device.

[0087] The first display unit 812 displays a shooting position guide showing a plurality of positions for shooting a multi-viewpoint image based on a gaze point of a transaction object. The first display unit 812 displays a shooting position guide in the shape of a fan centered on the gaze point. The first display unit 812 displays a shooting position guide in which each of the plurality of positions is arranged on an arc of the fan. The first display unit 812 displays a shooting position guide in the shape of a fan centered on an angle formed by a first radius connecting a first position on one end side of the plurality of positions and the gaze point, and a second radius connecting a second position on the other end side of the plurality of positions and the gaze point. For example, the first display unit 812 displays a shooting position guide GD1 in the shape of a downward arc centered on the gaze point GP1.

[0088] The first display unit 812 displays a transparent photographing position guide superimposed on the screen of the terminal device that displays the photographing area including the transaction object. The first display unit 812 displays, among the multiple positions, photographing completion positions where photographing has been completed and photographing incomplete positions where photographing has not been completed in a distinguishable manner.

[0089] The first display unit 812 performs various calculation processes necessary for the above-mentioned display. For example, the first display unit 812 determines a gaze point. The first display unit 812 calculates the gaze point using various information. The first display unit 812 determines the gaze point based on the line of sight from the viewpoint.

[0090] The first display unit 812 determines the point of gaze based on line-of-sight information from a plurality of viewpoints. The first display unit 812 determines the point of gaze based on line-of-sight information from two viewpoints. For example, the first display unit 812 determines the point of gaze based on a first line of sight from a first viewpoint and a second line of sight from a second viewpoint.

[0091] The first display unit 812 may calculate the gaze point using various information, not limited to the above. For example, the first display unit 812 may determine the gaze point based on line-of-sight information from one viewpoint and plane information. The first display unit 812 may determine the gaze point based on line-of-sight information from one viewpoint and plane information on which a subject to be photographed, such as a transaction object, is placed.

[0092] The first display unit 812 determines a plurality of positions (viewpoints) for capturing images related to a multi-viewpoint image based on the gaze point GP1. The first display unit 812 determines (calculates) the viewpoints VP1 to VP9 based on the set range for capturing images and the number of positions for capturing images. The first display unit 812 determines the viewpoints VP1 to VP9 based on the set range for capturing images (120°) and the number of positions for capturing images (9).

[0093] For example, the first display unit 812 calculates nine viewpoints VP1 to VP9 using information on the estimated distance between the subject SB1 and the camera 70. b The distance between the gaze point GP1 and the viewpoints VP1 to VP9 (hereinafter referred to as "distance Db") is set as the distance between the gaze point GP1 and the viewpoints VP1 to VP9. The first display unit 812 places the viewpoints VP1 to VP9 at positions that are a distance Db away from the gaze point GP1. Note that the above is merely an example, and the distance between the gaze point GP1 and the viewpoints VP1 to VP9 may be appropriately determined using various information. For example, the first display unit 812 displays the gaze point GP1 and the viewpoints VP1 to VP9 at positions that are a distance Db away from the gaze point GP1. aThe distance between the gaze point GP1 and the viewpoints VP1 to VP9 may be determined using the distance between them.

[0094] Also, for example, the first display unit 812 has a viewpoint VP b The first display unit 812 determines the position of the viewpoint VP5 as the position of the viewpoint VP5. The first display unit 812 determines nine viewpoints VP1 to VP9 that are arranged at 15° intervals along a predetermined plane with the viewpoint VP5 at the center, and each of the lines of sight VL1 to VL9 passes through the gaze point GP1.

[0095] For example, the first display unit 812 may include the camera 70 b On a plane corresponding to the posture of the user, nine viewpoints VP1 to VP9 may be arranged at intervals of 15° with the viewpoint VP5 at the center. b Nine viewpoints VP1 to VP9 may be arranged at 15° intervals with the viewpoint VP5 at the center on a plane (along the ground plane) that passes through and intersects in the height direction. Note that the above is merely an example, and the arrangement of the nine viewpoints VP1 to VP9 may be appropriately determined using various information. For example, the first display unit 812 may display the viewpoint VP a and VP b Nine viewpoints VP1 to VP9 may be placed on a plane passing through the intersections.

[0096] The first display unit 812 uses information on the position of the fixation point GP1 and the positions of the nine viewpoints VP1 to VP9 to display a sector-shaped shooting position guide GD1 with the fixation point GP1 at its center and the viewpoint VP5 at the center of the arc. The first display unit 812 displays a sector-shaped shooting position guide GD1 with the fixation point GP1 at its center and the radii of the line segment connecting the fixation point GP1 and the viewpoint VP1 and the line segment connecting the fixation point GP1 and the viewpoint VP2 as its radii.

[0097] The first display unit 812 displays a model MD, which is a 3D model indicating the shooting position and shooting direction. The first display unit 812 is disposed between the gaze point GP1 and the camera 70, and displays the model MD indicating the shooting position and shooting direction from any one of nine viewpoints VP1 to VP9.

[0098] The first display unit 812 displays a model MD indicating the shooting position and shooting direction of a viewpoint for which shooting has not been completed among the nine viewpoints VP1 to VP9. The first display unit 812 manages information on whether shooting has been completed for each of the nine viewpoints VP1 to VP9, ​​and displays a model MD indicating the shooting position and shooting direction of a viewpoint for which shooting has not been completed.

[0099] 4 as an example, the first display unit 812 displays a model MD indicating the shooting position and shooting direction at a viewpoint VP9 where shooting is incomplete, for example. The first display unit 812 places the model MD at a position between the gaze point GP1 and the viewpoint VP9.

[0100] When shooting at the viewpoint VP9 is completed, the first display unit 812 changes the display of the model MD to indicate the shooting position and shooting direction at a viewpoint other than the viewpoint VP9 for which shooting has not been completed. For example, when shooting at the viewpoint VP9 is completed, the first display unit 812 changes the display of the model MD to indicate the shooting position and shooting direction at the viewpoint VP8, which is a viewpoint adjacent to the viewpoint VP9 and for which shooting has not been completed. In this case, the first display unit 812 displays the model MD indicating the shooting position and shooting direction at the viewpoint VP8. The first display unit 812 places the model MD at a position between the gaze point GP1 and the viewpoint VP8.

[0101] Furthermore, when shooting at the viewpoint VP8 is completed, the first display unit 812 changes the display of the model MD so as to indicate the shooting position and shooting direction at a viewpoint other than the viewpoint VP8 where shooting has not been completed. For example, when shooting at the viewpoint VP8 is completed, the first display unit 812 changes the display of the model MD so as to indicate the shooting position and shooting direction at a viewpoint VP7 adjacent to the viewpoint VP8 where shooting has not been completed. In this case, the first display unit 812 displays the model MD indicating the shooting position and shooting direction at the viewpoint VP7. The first display unit 812 places the model MD at a position between the gaze point GP1 and the viewpoint VP7. The first display unit 812 displays the model MD so that a shadow indicates the shooting position and shooting direction at the viewpoint where shooting has not been completed until shooting is completed for all of the nine viewpoints VP1 to VP9.

[0102] Note that the above is merely an example, and the first display unit 812 may display the model MD indicating the shooting position and shooting direction in a desired manner.

[0103] The determination unit 813 performs various determination processes. The determination unit 813 determines whether or not to perform imaging. The determination unit 813 determines whether or not to perform imaging using information indicating the position of each object. The determination unit 813 determines whether or not to perform imaging using a threshold value for determining whether or not to perform imaging.

[0104] The determination unit 813 determines whether the center of the shooting area of ​​the terminal device 10 overlaps with the 3D model. The determination unit 813 determines whether the center indicated by the object overlaps with the 3D model. If the range in which the center and the 3D model overlap is equal to or greater than a predetermined threshold, the determination unit 813 determines that a predetermined condition is satisfied. For example, the determination unit 813 performs the determination using an image processing technique such as back projection. For example, the determination unit 813 uses back projection technology to identify where the 3D model is located and determines whether the center indicated by the object overlaps with the 3D model.

[0105] The photographing unit 814 controls the camera 70 and executes a process of photographing an image. The photographing unit 814 photographs when the determination unit 813 determines that the overlap between the center portion and the 3D model satisfies a predetermined condition. For example, the photographing unit 814 photographs when the 2D guide such as the object OB and the 3D model such as the model MD overlap at a certain threshold value or more within a predetermined time such as a specified time. Furthermore, when the photographing unit 814 uses a range in which the 2D guide such as the object OB and the 3D model such as the model MD do not overlap for the determination, the photographing unit 814 may photograph only when the 2D guide such as the object OB and the 3D model such as the model MD overlap at a certain threshold value or less.

[0106] The generating unit 815 generates a free viewpoint image (multiple viewpoint image). The generating unit 815 generates the free viewpoint image (multiple viewpoint image) by performing a predetermined conversion process on the captured image (original image data). The generating unit 815 derives a projective transformation matrix in order to convert a plurality of images captured from multiple viewpoints into a free viewpoint image (multiple viewpoint image) that is an image captured in a state where the optical axes of the imaging devices (cameras) overlap at a certain point in a three-dimensional position. In this case, the generating unit 815 refers to camera parameters including the position and orientation of the camera 70 as parameters of the imaging device that captures each image.

[0107] The generating unit 815 generates a free viewpoint image (multiple viewpoint image) based on the images P11 to P19 acquired by the first acquiring unit 811. The generating unit 815 generates appropriate parameters (for example, a projective transformation matrix) for converting the nine images P11 to P19 into images for forming the multi-viewpoint image by performing a predetermined calibration (camera calibration) on the nine images P11 to P19. For example, the generating unit 815 performs projective transformation on the images P11 to P19 using the parameters (camera parameters) of the camera 70. For example, the generating unit 815 performs projective transformation on the images P11 to P19 using the calibration data R11 to R19. Then, the generating unit 815 generates a free viewpoint image (multiple viewpoint image) that allows a smooth transition in display for the multiple images RP11 to RP19 by connecting the projectively transformed images RP11 to RP19.

[0108] The terminal device 10 may acquire free viewpoint images (multiple viewpoint images) generated by an external device. For example, the terminal device 10 may transmit images P11-P19 and calibration data R11-R19 to an external device (generation device) that generates free viewpoint images (multiple viewpoint images) via the transmission unit 816, and receive images RP11-RP19 generated by the generation device via the first acquisition unit 811. In this case, the first application 81 may not include the generation unit 815.

[0109] The transmission unit 816 transmits various information to an external device such as the service providing device 100. The transmission unit 816 transmits the multi-view image together with object-related information related to the transaction object to the service providing device 100 that provides a service of distributing information on the transaction object. The transmission unit 816 transmits the multi-view image together with object-related information of the transaction object to the service providing device that provides a service related to the buying and selling of the transaction object.

[0110] The transmission unit 816 transmits the multi-view image together with the object related information of the transaction object to a service providing device that provides a service related to the listing of the transaction object. The transmission unit 816 transmits the multi-view image together with the object related information of the transaction object to a service providing device that provides a service related to the listing of the transaction object. The transmission unit 816 transmits the multi-view image together with the object related information, which is the listing information of the transaction object, which is the item being sold, to a service providing device that provides an listing service of the transaction object. For example, when the transmission unit 816 functions as a providing unit, the transmission unit 816 transmits the 3D model to an external device such as the service providing device 100.

[0111] (Second Application 82) The second application 82 causes the terminal device 10 to execute the above-mentioned display process. The second application 82 is an application used to display content such as multi-view images. For example, FIG. 10 is a diagram showing an example of a functional configuration realized by the second application according to the embodiment. As shown in FIG. 10, the second application 82 realizes a second acquisition unit 821, a second display unit 822, and a reception unit 823.

[0112] The second acquisition unit 821 acquires various information from the storage unit 30. The second acquisition unit 821 acquires various information from the transaction object information storage unit 31. The second acquisition unit 821 receives various information from an external device such as the service providing device 100. The second acquisition unit 821 receives information provided (distributed) from the service providing device 100. For example, the second acquisition unit 821 receives a group of images including multi-view images as information on the transaction object.

[0113] The second acquisition unit 821 receives information on the transaction object provided from the service providing device 100, and stores the information in the storage unit 30. When the second application 82 does not store the information on the transaction object in the storage unit 30, the second application 82 may manage the information on the transaction object within the second application 82 so that the information can be referenced.

[0114] The second acquisition unit 821 acquires an image group including a first image, which is a multi-view image obtained by capturing a subject from different viewpoints, and a second image different from the first image. The second acquisition unit 821 acquires the image group from the storage unit 30.

[0115] The second display unit 822 displays various information on the output unit 50 (such as a liquid crystal display). The second display unit 822 displays various information on the output unit 50. The second display unit 822 controls the output unit 50 and executes a process of displaying information.

[0116] The second display unit 822 performs various calculation processes and the like necessary for display, which will be described later. The second display unit 822 switches between images included in the first image and displays them on the screen of the terminal device 10 in response to an operation by a user of the terminal device 10 to switch the display of the image group on the screen of the terminal device 10. The second display unit 822 switches between images included in the first image and displays them on the screen of the terminal device 10 in response to an operation during display in the first display mode in which each image in the image group is displayed.

[0117] In response to an operation, the second display unit 822 displays each image included in the first image in the order in which it was captured. In response to an operation, the second display unit 822 displays each image in the chronological order in which the first images were captured. The second display unit 822 displays the group of images in a second display mode in which the first images and the second images are displayed side by side in a list, and when an image displayed in the second display mode is designated, the second display mode is switched to the first display mode in which the image is displayed.

[0118] In the second display mode, the second display unit 822 displays a predetermined image of the first images as an image representing the first image. In the second display mode, the second display unit 822 displays the first image of the first images.

[0119] The second display unit 822 switches and displays images included in the first image in response to a user operation accepted by the accepting unit 823. The second display unit 822 switches and displays images included in the first image on the screen of the terminal device 10 in response to movement of the contact position. The second display unit 822 displays each image in the chronological order in which the first image was captured in response to movement of the contact position in a predetermined direction.

[0120] When the last image among the first images is displayed, second display unit 822 displays the second image next in the order of the first image in response to movement of the contact position in a predetermined direction. Second display unit 822 displays each image in reverse order to the chronological order in which the first images were captured in response to movement of the contact position in the opposite direction to the predetermined direction. When the first image among the first images is displayed, second display unit 822 displays the second image before the first image in response to movement of the contact position in the opposite direction.

[0121] The second display unit 822 displays a multi-view image FV1 of shoe A and an image group IGP including images IM1-IM2, IM4-IM6, etc. of other shoes. The second display unit 822 displays content CT1 that displays a list of shoe images, with the images IM1-IM2 of other shoes and the multi-view image FV1 of shoe A arranged in the top row and the images IM4-IM6 of other shoes arranged in the second row.

[0122] In response to an operation by user U1, second display unit 822 changes the display from a list display mode of images (second display mode) to a display mode of each image (first display mode). In a state in which only the image IM1 of other shoes is displayed, second display unit 822 switches the display from a state in which only the image IM1 of other shoes is displayed to a state in which only the image IM2 of other shoes is displayed in response to a forward swipe operation by user U1. In a state in which only the image IM2 of other shoes is displayed, second display unit 822 switches the display from a state in which only the image IM2 of other shoes is displayed to a state in which only the image IM1 of other shoes is displayed to a state in which only the image IM1 of other shoes is displayed in response to a backward swipe operation by user U1.

[0123] In response to a forward swipe operation by user U1, the second display unit 822, in a state where only the image IM2 of other shoes is displayed, switches the display from a state where only the image IM2 of other shoes is displayed to a state where only the multi-view image FV1 of shoe A is displayed. In response to a forward swipe operation by user U1, the second display unit 822 switches the display from a state where only the image IM2 of other shoes is displayed to a state where the first image RP11 of the multi-view image FV1 of shoe A is displayed.

[0124] In response to the user U1's swipe operation moving forward, the second display unit 822 displays the multi-view image FV1 in chronological order from image RP11, to image RP12, image RP13, image RP14, image RP15, image RP16, image RP17, image RP18, and image RP19.

[0125] In response to a forward swipe by the user U1 while the last image RP19 is being displayed, the second display unit 822 displays the next image IM4 of the multi-view image FV1. In response to a forward swipe by the user U1, the second display unit 822 switches the display from a state in which only the multi-view image FV1 of shoe A is displayed to a state in which only the image IM4 of another shoe is displayed.

[0126] The reception unit 823 receives a user operation. The reception unit 823 receives a user operation via the input unit 40. For example, the reception unit 823 receives a touch by the user to the screen (output unit 50) of the terminal device 10 as a user operation. The reception unit 823 receives a swipe operation by the user. In addition, the reception unit 823 receives various operations such as a flick, a tap, a double tap, a long tap, a pinch in, and a pinch out.

[0127] The accepting unit 823 accepts an operation by a user in the first display mode. The accepting unit 823 accepts an operation to move the contact position in a predetermined direction when the user is in contact with the screen of the terminal device 10. The accepting unit 823 accepts an operation to move the contact position in a direction opposite to the predetermined direction when the user is in contact with the screen of the terminal device 10.

[0128] The reception unit 823 receives a request for information from a user in response to a user operation. The reception unit 823 transmits the received information request to the service providing device 100 via the communication unit 20. The reception unit 823 receives a request for transaction information of the category “shoes” from the user and transmits the request for transaction information of the category “shoes” to the service providing device 100.

[0129] In addition, each process by the control unit 80 described above is not limited to using dedicated applications such as the first application 81 and the second application 82, but may be realized using, for example, JavaScript (registered trademark) or HTML (HyperText Markup Language).

[0130] [2-2. An example of the functional configuration of the service providing device] Next, an example of the functional configuration of the service providing device 100 will be described with reference to Fig. 12. Fig. 12 is a diagram showing an example of the configuration of the service providing device according to the embodiment. As shown in Fig. 12, the service providing device 100 has a communication unit 110, a storage unit 120, and a control unit 130. Note that the service providing device 100 may have an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator of the service providing device 100, and an output unit (e.g., a liquid crystal display, etc.) that displays various information.

[0131] (Communication unit 110) The communication unit 110 is realized by, for example, a NIC etc. The communication unit 110 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the terminal device 10.

[0132] (Storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 stores various information necessary for providing the service. For example, the storage unit 120 stores various information necessary for providing the auction service.

[0133] The storage unit 120 stores various information related to the transaction object. The storage unit 120 stores various information related to the transaction object that is provided in the provision of the service. For example, the storage unit 120 stores a multi-view image of the transaction object and object-related information in association with identification information that identifies the provider that provides the transaction object.

[0134] (Control unit 130) The control unit 130 is, for example, a controller, and is realized by a CPU, an MPU, or the like, executing various programs (corresponding to an example of a notification program) stored in a storage device inside the service providing device 100 using a RAM as a working area. The control unit 130 is also a controller, and is realized, for example, by an integrated circuit such as an ASIC or an FPGA.

[0135] As shown in FIG. 12, the control unit 130 has a receiving unit 131, a processing unit 132, and a distribution unit 133, and realizes or executes the functions and actions of information processing described below.

[0136] (Receiving unit 131) The receiving unit 131 receives various information. The receiving unit 131 receives various information from an external device such as the terminal device 10. The receiving unit 131 receives a multi-view image from the terminal device 10. The receiving unit 131 receives a multi-view image from the terminal device 10.

[0137] The receiving unit 131 accepts the listing of a transaction object from the terminal device 10. The receiving unit 131 receives a multi-view image of the transaction object together with object-related information related to the transaction object. The receiving unit 131 receives the multi-view image together with object-related information, which is listing information of the transaction object that is an item for sale. For example, the receiving unit 131 receives a 3D model indicating a shooting position and a shooting direction for shooting the multi-view image of the shooting object.

[0138] The receiving unit 131 receives an information request from the terminal device 10. The receiving unit 131 receives information indicating a category of a trading object as the information request. The receiving unit 131 accepts bids from the terminal device 10 for the trading object put up for sale.

[0139] (Processing section 132) The processing unit 132 executes various processes related to the provision of services. The processing unit 132 registers the information received by the receiving unit 131 in the storage unit 120. The processing unit 132 registers various information related to the object of transaction in the storage unit 120.

[0140] The processing unit 132 selects various information. The processing unit 132 selects various information based on various information stored in the storage unit 120. The processing unit 132 also selects information from the storage unit 120. The processing unit 132 selects information on a transaction object corresponding to a user request from among the transaction objects registered in the storage unit 120. The processing unit 132 selects information on a transaction object corresponding to a category requested by the user from among the transaction objects registered in the storage unit 120.

[0141] (Distribution Section 133) The distribution unit 133 distributes various types of information. The distribution unit 133 transmits various types of information to the terminal device 10. The distribution unit 133 distributes information on a transaction object requested by the terminal device 10. The distribution unit 133 transmits information on a transaction object selected by the processing unit 132 to the terminal device 10. The distribution unit 133 transmits information on the transaction object including multi-view images to the terminal device 10. The distribution unit 133 transmits a group of images including multi-view images to the terminal device 10.

[0142] [3. Information processing flow] Next, an example of the procedure of the transmission process according to the embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the flow of the transmission process according to the embodiment. For example, the terminal device 10 acquires a multi-view image in which a transaction object is captured from different viewpoints (step S101). Then, the terminal device 10 transmits the multi-view image together with object-related information related to the transaction object to the service providing device 100 that provides a service of distributing information on the transaction object (step S102).

[0143] Next, an example of the procedure of information processing according to the embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart showing an example of the flow of information processing according to the embodiment. For example, the terminal device 10 acquires shooting target information related to a shooting target (step S201). Then, the terminal device 10 provides a model indicating the shooting position and shooting direction for shooting a multi-view image of the shooting target (step S202). For example, the terminal device 10 provides a 3D model indicating the shooting position and shooting direction for shooting a multi-view image of the shooting target.

[0144] [4. An example of display processing] The free viewpoint image (multi-viewpoint image) generated by the above-described process may be used for information distribution, and the distributed multi-viewpoint image is displayed on the distribution destination terminal device 10. This point will be described with reference to Figs. 15 to 17.

[0145] First, the flow of the distribution process will be described with reference to FIG. 15. FIG. 15 is a diagram showing an example of the process by the information processing system according to the embodiment. Note that the description of the same points as in FIG. 1 etc. will be omitted as appropriate. For example, the terminal device 10 starts an application (e.g., the second application 82 etc.) for displaying information on the transaction object, and executes the following process. First, the terminal device 10 requests information on the transaction object from the service providing device 100 (step S21). In the example of FIG. 15, the terminal device 10 transmits a request for information on the transaction object corresponding to the category "shoes."

[0146] Then, the service providing device 100, which has received the request from the terminal device 10, selects information on the transaction object corresponding to the category "shoes" from among the transaction objects whose information is stored in the storage unit 120 (see FIG. 12) (step S22).

[0147] Then, the service providing device 100 distributes information on the transaction object corresponding to the selected category "shoes" to the terminal device 10 (step S23). In the example of FIG. 15, the service providing device 100 distributes an image group related to shoes (hereinafter referred to as "image group IGP") including a multi-view image FV1 of shoe A, etc. to the terminal device 10. For example, the service providing device 100 distributes an image group IGP including a multi-view image FV1 of shoe A and images IM1-IM2, IM4-IM6, etc. of other shoes. For example, it is assumed that the image group IGP is arranged in the following order: images IM1-IM2 of other shoes, multi-view image FV1, images IM4-IM6 of other shoes.

[0148] Then, the terminal device 10 that has received the image group IGP displays information about the image group IGP (step S24). For example, the terminal device 10 displays a list of each image included in the image group IGP as shown in FIG. 16. FIG. 16 is a diagram showing an example of a list display of a trading object according to the embodiment. The terminal device 10 displays the content CT1 that displays a list of shoe images, with the images IM1 to IM2 of other shoes and the multi-view image FV1 of the shoe A arranged in the top row and the images IM4 to IM6 of other shoes arranged in the second row. The content CT1 may be generated based on the image group IGP by the terminal device 10 that has received the image group IGP, or may be generated and distributed by the service providing device 100. Note that in the list display, the multi-view image may be displayed so as to be distinguishable from a normal image. For example, the terminal device 10 may indicate that the multi-view image FV1 is a multi-view image different from the other images IM1 to IM2 and IM4 to IM6 by adding an icon or the like indicating that the multi-view image FV1 is a multi-view image.

[0149] Then, the terminal device 10 changes the display from a list display mode (second display mode) of images to a display mode (first display mode) for each image in response to an operation by the user U1. For example, when the user U1 designates an image IM1 of another shoe, the display is changed to the first display mode in which only the image IM1 of the other shoe is displayed. Hereinafter, the switching of the display in the first display mode will be described with reference to FIG. 17. FIG. 17 is a diagram showing an example of the display of an image group including a multi-view image according to the embodiment. In FIG. 17, the terminal devices 10 are shown as terminal devices 10-1 to 10-6 in a distinguishable manner in response to a change in the display content of the terminal device 10, but when there is no particular distinction to be made, the terminal devices 10 will be described as terminal devices 10.

[0150] In FIG. 17, the terminal device 10-1 shows a state in which only the image IM1 of other shoes is displayed. The terminal device 10 displays the images in order in response to a user's operation (swipe operation) of moving a finger touching the screen of the terminal device 10 from right to left (forward direction). For example, in a state in which only the image IM1 of other shoes is displayed, the terminal device 10 switches the display from the terminal device 10-1 displaying only the image IM1 of other shoes to the terminal device 10-2 displaying only the image IM2 of other shoes in response to a swipe operation in the forward direction by the user U1. That is, the terminal device 10 displays the next image in response to a swipe operation in the forward direction by the user U1.

[0151] The terminal device 10 displays the images in reverse order in response to a user's operation (swipe operation) of moving a finger touching the screen of the terminal device 10 from left to right (reverse direction). For example, in a state in which only the image IM2 of another shoe is displayed, the terminal device 10 switches the display from the terminal device 10-2 displaying only the image IM2 of another shoe to the terminal device 10-1 displaying only the image IM1 of another shoe in response to a swipe operation in the reverse direction by the user U1. That is, the terminal device 10 displays the previous image in response to a swipe operation in the reverse direction by the user U1.

[0152] 17, the terminal device 10-2 shows a state in which only the image IM2 of the other shoe is displayed. For example, in a state in which only the image IM2 of the other shoe is displayed, the terminal device 10 switches the display from the terminal device 10-2 displaying only the image IM2 of the other shoe to the terminal device 10-3 displaying only the multi-view image FV1 of the shoe A in response to a forward swipe operation by the user U1 in the terminal device 10 displaying only the image IM2 of the other shoe to the terminal device 10-3 displaying the first image RP11 of the multi-view image FV1 of the shoe A in response to a forward swipe operation by the user U1 in the state in which only the image IM2 of the other shoe is displayed.

[0153] Then, the terminal device 10 displays the display of the multi-view image FV1 in the order of image RP11, image RP12, image RP13, image RP14, image RP15, image RP16, image RP17, image RP18, and image RP19 in time sequence in response to the movement of the user U1's swipe operation in the forward direction. In FIG. 17, the terminal device 10-4 shows a state in which the image RP15 of the multi-view image FV1 is displayed, and the terminal device 10-5 shows a state in which the last image RP19 of the multi-view image FV1 is displayed. Note that, for the purpose of explanation, FIG. 17 illustrates a state in which only three images of the multi-view image FV1, the first image RP11, the middle image RP15, and the last image RP19, are displayed, but the other images RP12, RP13, RP14, RP16, RP17, and RP18 are also displayed in order in response to the movement of the user U1's swipe operation in the forward direction.

[0154] Furthermore, in response to a forward swipe operation by the user U1 in the state of the terminal device 10-6 displaying the last image RP19, the terminal device 10 displays the next image IM4 of the multi-view image FV1. Specifically, in the state of the terminal device 10-5 displaying the first image RP19 of the multi-view image FV1 of the shoe A, the terminal device 10 switches the display from the terminal device 10-5 displaying only the multi-view image FV1 of the shoe A to the terminal device 10-6 displaying only the image IM4 of another shoe in response to a forward swipe operation by the user U1 in the state of the terminal device 10-5 displaying the first image RP19 of the multi-view image FV1 of the shoe A. The terminal device 10 appropriately uses existing image processing technology to perform processing such as switching and displaying the image group including the above-mentioned multi-view image.

[0155] As described above, the terminal device 10 can easily switch the display of a group of images including a multi-view image by switching and displaying images included in the multi-view image in response to an operation (swipe operation) for switching the display of the group of images.

[0156] In this way, the terminal device 10 displays the images of the image group in a carousel in the first display mode. The terminal device 10 displays images from different viewpoints for the multi-view image in response to a swipe operation, a slide operation, or the like by the user. Note that, when the image being displayed is not a multi-view image, the terminal device 10 performs normal image switching.

[0157] For example, in the second display mode, the terminal device 10 displays a list of only the thumbnail of the first image among the multi-view images, or the thumbnail of the last image among the multi-view images. When thumbnails are displayed on the entire screen (such as the first display mode), the terminal device 10 switches and displays the multi-view images by switching the display of the image group. Furthermore, the terminal device 10 seamlessly displays another image by the same operation. For example, the terminal device 10 displays the last image and the last image among the multi-view images in a carousel, and during the switching, the other images of the multi-view images are displayed in sequence. As a result, the terminal device 10 can display seamlessly without having to perform the operation of switching the carousel many times. Therefore, the terminal device 10 can seamlessly display the multi-view images by the same operation as other images in various services that display images in the form of a carousel.

[0158] [4-1. Other display examples] Note that the above-described display is merely an example, and any other display mode may be used as long as the images included in the multi-view image can be switched and displayed in response to an operation of the terminal device 10 to switch the display of the image group.

[0159] The terminal device 10 may display the multi-view image by overlaying it. For example, the terminal device 10 may display images other than the first and last images of the multi-view image by overlaying them. In addition, when displaying the multi-view image, the terminal device 10 may display a button or the like that cancels the switching display of the images included in the multi-view image and transitions to the display of the previous and next images. When displaying the multi-view image, the terminal device 10 may arrange arrow marks on both the left and right ends, and when the left arrow mark is selected, the next image of the multi-view image may be displayed, and when the right arrow mark is selected, the previous image of the multi-view image may be displayed. Note that the operation may be set inversely, and the terminal device 10 may display the previous image of the multi-view image when the left arrow mark is selected, and the next image of the multi-view image may be displayed when the right arrow mark is selected.

[0160] Furthermore, the terminal device 10 may display the first image (thumbnail) and the last image (thumbnail) of the multi-view images, and may display the multi-view images by switching the display from the image selected by the user to the other image in sequence. In this case, the terminal device 10 displays the first image (thumbnail) and the last image (thumbnail) of the multi-view images in a list display area arranged in a part (for example, the lower side) of the screen. Then, when the user selects one of the first image (thumbnail) and the last image (thumbnail) displayed in the list display area, the terminal device 10 may display the multi-view images by switching the display from the image selected by the user to the other image in sequence in a selection target display area arranged in a part (for example, the upper side) of the screen. When the user selects the first image, the terminal device 10 may display the multi-view images from the first image to the last image in chronological order in the selection target display area. When the user selects the last image, the terminal device 10 may display the multi-view images from the last image to the first image in reverse chronological order in the selection target display area.

[0161] The terminal device 10 may display the first image and the last image in the list display area, and may switch the display of the multi-view images in the selection target display area according to the display state of the selection target display area and the user's designation for the list display area. For example, when the first image is displayed in the selection target display area, if the user selects the last image in the list display area, the terminal device 10 may display the multi-view images in chronological order from the first image to the last image in the selection target display area. Also, when the last image is displayed in the selection target display area, if the user selects the first image in the list display area, the terminal device 10 may display the multi-view images in chronological order from the last image to the first image in the selection target display area.

[0162] [4-2. Information processing flow] Next, an example of the procedure of the display process according to the embodiment will be described with reference to Fig. 18. Fig. 18 is a flowchart showing an example of the flow of the display process according to the embodiment. For example, the terminal device 10 acquires an image group including a first image, which is a multi-view image obtained by capturing an object from different viewpoints, and a second image different from the first image (step S301). Then, the terminal device 10 switches the images included in the first image and displays them on the screen of the terminal device in response to an operation by the user of the terminal device 10 to switch the display of the image group on the screen of the terminal device (step S302).

[0163] [5. Other] In addition, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, and conversely, all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0164] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0165] Furthermore, the above-described embodiments can be appropriately combined as long as the processing contents are not contradictory.

[0166] [6. Program] Moreover, the terminal device 10 and the service providing device 100 according to the above-described embodiment are realized by a computer 1000 having a configuration as shown in Fig. 19, for example. Fig. 19 is a diagram showing an example of a hardware configuration. The computer 1000 is connected to an output device 1010 and an input device 1020, and has a configuration in which a calculation device 1030, a primary storage device 1040, a secondary storage device 1050, an output IF (Interface) 1060, an input IF 1070, and a network IF 1080 are connected by a bus 1090.

[0167] The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050 and programs read from the input device 1020, and executes various processes. The primary storage device 1040 is a memory device, such as a RAM, that temporarily stores data used by the arithmetic device 1030 for various calculations. The secondary storage device 1050 is a storage device in which data used by the arithmetic device 1030 for various calculations and various databases are registered, and is realized by a ROM (Read Only Memory), a HDD (Hard Disk Drive), a flash memory, or the like.

[0168] The output IF 1060 is an interface for transmitting information to be output to an output device 1010 that outputs various types of information, such as a monitor or a printer, and is realized by a connector conforming to a standard such as USB (Universal Serial Bus), DVI (Digital Visual Interface), or HDMI (registered trademark) (High Definition Multimedia Interface). The input IF 1070 is an interface for receiving information from various input devices 1020, such as a mouse, keyboard, and scanner, and is realized by a USB, for example.

[0169] The input device 1020 may be a device that reads information from, for example, an optical recording medium such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory. The input device 1020 may also be an external storage medium such as a USB memory.

[0170] The network IF 1080 receives data from other devices via the network N and sends it to the arithmetic device 1030, and also transmits data generated by the arithmetic device 1030 to other devices via the network N.

[0171] The arithmetic unit 1030 controls the output device 1010 and the input device 1020 via the output IF 1060 and the input IF 1070. For example, the arithmetic unit 1030 loads a program from the input device 1020 or the secondary storage device 1050 onto the primary storage device 1040 and executes the loaded program.

[0172] For example, when the computer 1000 functions as the terminal device 10 , the arithmetic unit 1030 of the computer 1000 executes a program loaded onto the primary storage device 1040 to realize the functions of the control unit 80 .

[0173] 7. Effects As described above, the terminal device according to the embodiment (terminal device 10 in the embodiment) has an acquisition unit (first acquisition unit 811 in the embodiment) and a transmission unit (transmission unit 816 in the embodiment). The first acquisition unit 811 acquires multi-view images of a transaction object captured from different viewpoints. The transmission unit 816 transmits the multi-view images together with object-related information related to the transaction object to a service providing device (service providing device 100 in the embodiment) that provides a service of distributing information on the transaction object.

[0174] In this manner, the terminal device 10 according to the embodiment transmits a multi-view image of a transaction object together with information related to the transaction object to a service providing device that provides a service of distributing information on the transaction object. This allows the terminal device 10 to facilitate the distribution of information including the multi-view image by the service providing device, thereby facilitating the provision of services using the multi-view image.

[0175] In addition, in the terminal device 10 according to the embodiment, the transmission unit 816 transmits the multi-view image together with the object-related information of the transaction object to the service providing device that provides the service related to the buying and selling of the transaction object. This allows the terminal device 10 to facilitate the distribution of information including the multi-view image by the service providing device that provides the service related to the buying and selling of the transaction object, thereby facilitating the provision of the service using the multi-view image.

[0176] In addition, in the terminal device 10 according to the embodiment, the transmission unit 816 transmits the multi-view image together with the object-related information of the trading object to a service providing device that provides a service related to the listing of the trading object, for example, an auction, and the like, and thus the terminal device 10 can facilitate the distribution of information including the multi-view image by the service providing device that provides a service related to the listing of the trading object, for example, an auction, and can promote the provision of services using the multi-view image.

[0177] In addition, in the terminal device 10 according to the embodiment, the transmission unit 816 transmits the multi-view image together with the object-related information of the trading object to a service providing device that provides a service related to the listing of the trading object, for example, an auction, and the like, and thus the terminal device 10 can facilitate the distribution of information including the multi-view image by the service providing device that provides a service related to the listing of the trading object, for example, an auction, and can promote the provision of services using the multi-view image.

[0178] In addition, in the terminal device 10 according to the embodiment, the first acquisition unit 811 acquires multi-view images of a trading object, which is an item for sale, captured from different viewpoints. The transmission unit 816 transmits the multi-view images together with object-related information, which is listing information of the trading object, to a service providing device that provides a listing service for the trading object. This allows the terminal device 10 to facilitate users' listings in, for example, an auction, and thus promote the provision of services using multi-view images.

[0179] Moreover, the terminal device 10 according to the embodiment has a display unit (first display unit 812 in the embodiment). The first display unit 812 displays a guide for taking pictures related to multi-viewpoint images. This allows the terminal device 10 to facilitate the user's taking pictures related to multi-viewpoint images, thereby promoting the provision of services using multi-viewpoint images.

[0180] Furthermore, in the terminal device 10 according to the embodiment, the first display unit 812 displays a guide including a 3D model indicating the shooting position and shooting direction. This allows the user of the terminal device 10 to recognize the shooting position and shooting direction, and makes it easy for the user to shoot multi-viewpoint images, thereby promoting the provision of services using multi-viewpoint images.

[0181] Furthermore, in the terminal device 10 according to the embodiment, the first display unit 812 superimposes a 3D model on the screen (the output unit 50 in the embodiment) of the terminal device 10 displaying a shooting area including the transaction object. This allows the terminal device 10 to superimpose a guide on the shooting area, making it easy for the user to shoot multi-viewpoint images, and thus promoting the provision of services using multi-viewpoint images.

[0182] Furthermore, in the terminal device 10 according to the embodiment, the first display unit 812 displays a shooting position guide indicating a plurality of positions for shooting a multi-view image based on a gaze point of a transaction object. As a result, the terminal device 10 can easily shoot a multi-view image by a user by displaying a shooting position guide indicating a plurality of positions for shooting a multi-view image, and therefore can promote the provision of services using multi-view images.

[0183] Furthermore, in the terminal device 10 according to the embodiment, the first display unit 812 displays a shooting position guide in the shape of a fan centered on the gaze point. This allows the terminal device 10 to easily shoot multi-viewpoint images by the user by displaying the shooting position guide in the shape of a fan, thereby promoting the provision of services using multi-viewpoint images.

[0184] Furthermore, in the terminal device 10 according to the embodiment, the first display unit 812 displays a shooting position guide in which each of a plurality of positions is arranged on an arc of a fan. As a result, the terminal device 10 can easily shoot multi-viewpoint images by a user by displaying each of a plurality of positions arranged on an arc of a fan, and therefore can promote the provision of services using multi-viewpoint images.

[0185] In addition, in the terminal device 10 according to the embodiment, the first display unit 812 displays a transparent shooting position guide superimposed on the screen of the terminal device 10 displaying the shooting area including the transaction object. This allows the terminal device 10 to facilitate the user's shooting of multi-viewpoint images, thereby facilitating the provision of services using multi-viewpoint images.

[0186] Furthermore, in the terminal device 10 according to the embodiment, the first display unit 812 displays, among the multiple positions, a shooting completion position where shooting has been completed and a shooting incomplete position where shooting has not been completed in a distinguishable manner. This allows the terminal device 10 to allow the user to recognize whether shooting at each position has been completed or not, and makes it easy for the user to shoot multi-viewpoint images, thereby promoting the provision of services using multi-viewpoint images.

[0187] The above describes the embodiment of the present application in detail based on the drawings, but this is merely an example, and the present invention can be implemented in other forms that include various modifications and improvements based on the knowledge of those skilled in the art, including the forms described in the Disclosure of the Invention section.

[0188] Moreover, the above-mentioned "section, module, unit" can be read as "means" or "circuit", etc. For example, the reception section can be read as reception means or reception circuit. [Explanation of symbols]

[0189] 1. Information Processing Systems 10 Terminal Equipment 20 Communications Department 30 Storage section 31 Transaction target information storage unit 40 Input section 50 Output section 60 Physical Sensors 70 Camera 80 Control section 81 First Application 811 First Acquisition Department 812 1st display section (provider section) 813 Judgment section 814 Photography Department 815 Generation part 816 Transmitter 82 Second Application 821 Second Acquisition Department 822 2nd display section 823 Reception Department 100 Service providing device 110 Communications Department 120 Storage section 130 Control section 131 Receiving unit 132 Processing section 133 Distribution Department

Claims

1. An acquisition step of acquiring multi-viewpoint images of a transaction target from different viewpoints; a transmission step of transmitting the multi-view image together with object-related information related to the transaction object to a service providing device that provides a service of distributing information on the transaction object; a display step of displaying a guide for taking pictures of the multi-view image together with a pin for a user to specify an angle for taking pictures in accordance with a distance between a position for taking pictures and the object of transaction; A transmission program that causes a terminal device to execute the above.

2. The transmission step includes: The multi-view image is transmitted to the service providing device that provides a service related to the buying and selling of the trading object together with the object-related information of the trading object.

2. The transmission program according to claim 1 .

3. The transmission step includes: The multi-view image is transmitted to the service providing device that provides a service related to the auction of the trading object together with the object-related information of the trading object.

3. The transmission program according to claim 1 or 2.

4. The acquisition step includes: Acquire the multi-viewpoint images of the transaction object, which is the auction item, captured from different viewpoints; The transmission step includes: The multi-view image is transmitted to the service providing device that provides a listing service for the transaction object together with the object-related information that is listing information for the transaction object that is an item for sale.

4. The transmission program according to claim 3.

5. The display procedure includes: Displaying the guide including a 3D model showing the shooting position and shooting direction 5. The transmission program according to claim 1, wherein the transmission program is a program for transmitting a signal to a communication device.

6. The display procedure includes: The 3D model is superimposed on the screen of the terminal device that displays a photographing area including the transaction object.

6. The transmission program according to claim 5.

7. The display procedure includes: Displaying a shooting position guide showing a plurality of positions for shooting the multi-viewpoint image based on a gaze point of the transaction object 7. The transmission program according to claim 4, wherein the transmission program is a program for transmitting a signal to a communication device.

8. The display procedure includes: The shooting position guide having a fan shape with the gaze point as a center is displayed.

8. The transmission program according to claim 7.

9. The display procedure includes: Displaying the shooting position guide, each of the plurality of positions being positioned on an arc of the fan.

9. The transmission program according to claim 8.

10. The display procedure includes: The photographing position guide having transparency is superimposed on the screen of the terminal device which displays the photographing area including the transaction object.

10. The transmission program according to claim 7, wherein the transmission program is a program for transmitting a signal to a communication device.

11. The display procedure includes: Among the plurality of positions, a photographing completion position where photographing has been completed and a photographing incomplete position where photographing has not been completed are displayed in a distinguishable manner. The transmission program according to any one of claims 7 to 10.

12. an acquisition unit for acquiring multi-viewpoint images of a transaction target captured from different viewpoints; a transmission unit that transmits the multi-view image together with object-related information related to the transaction object to a service providing device that provides a service of distributing information on the transaction object; a display unit that displays a guide for taking the multi-view image together with a pin for a user to specify an angle for taking the image in accordance with a distance between a position for taking the image and the object of transaction; A terminal device comprising:

13. An acquisition step of acquiring multi-viewpoint images of a transaction object captured from different viewpoints; a transmission step of transmitting the multi-view image together with object-related information related to the transaction object to a service providing device that provides a service of distributing information on the transaction object; a display step of displaying a guide for taking pictures of the multi-view image together with a pin for a user to specify an angle for taking pictures in accordance with a distance between a position for taking pictures and the object of transaction; A transmission method comprising:

Images