Content providing device, content providing method, program, and recording medium

Abstract

A content providing device, a content providing method, a program, and a recording medium are provided that are capable of appropriately displaying content provided by reading printouts of images acquired via a network. [Solution] In this invention, at least a part of a target image provided by a first user is set as a first image, content displayed on the display screen of a second user's terminal is saved in association with the first image, printing data is sent to the terminal, and when a printout generated by printing the target image and the second image on a specified medium in accordance with the printing data is read by the terminal, the read image is received from the terminal, a storage destination is identified from the part of the read image that corresponds to the second image, and display data is sent to the terminal to display the content saved in the storage destination at a position on the display screen that corresponds to the part of the read image that corresponds to the first image.

Description

[Technical Field]

[0001] The present invention relates to a content providing device, a content providing method, a program, and a recording medium that provide content when an image printed on a medium is read. [Background technology]

[0002] The widespread use of network communication technology has made it possible for users to exchange images, and also for one user to print an image provided by another user (see, for example, Patent Document 1).

[0003] Furthermore, in order to encourage printing of provided images, content may be provided when a printed image is scanned by a scanning device (see, for example, Patent Document 2). Patent Document 2 describes a technique in which a printout or the like is photographed (captured) with an information terminal such as a smartphone, and predetermined video content is synthesized and displayed on the image of the printout displayed on the information terminal. In other words, the technology described in Patent Document 2 provides content for augmented reality (AR). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-192016 [Patent Document 2] Japanese Patent Application Laid-Open No. 2016-58024 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology of Patent Document 2, the content provided when the printout is scanned is displayed at a position determined based on a part of the image (a so-called marker) printed on the printout. In other words, the content is displayed at a position corresponding to the position of the marker on the display screen of the information terminal that captures the printout.

[0006] On the other hand, the position of the marker relative to the print (more specifically, the relative position between the print and the marker) may change depending on, for example, the size of the medium on which the image is printed. Therefore, if the size of the medium used to print the image is not appropriate, the content displayed by photographing (capturing) the print may not be displayed appropriately on the display screen.

[0007] The present invention has been made in consideration of the above circumstances, and specifically, it is an object of the present invention to provide a content providing device, a content providing method, a program, and a recording medium that can appropriately display content provided by reading an image obtained via a network when the image is printed. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the content providing device of the present invention is a content providing device equipped with a processor, which performs the following processes: setting at least a part of a target image provided by a first user as a first image; saving content displayed on the display screen of a terminal of a second user in association with the first image; sending print data to the terminal for printing the target image and a second image corresponding to the destination of the content on a designated medium specified by the first user; when a printout generated by printing the target image and the second image on the designated medium according to the print data is read by the terminal, receiving a read image of the read printout from the terminal; identifying a destination of storage from a portion of the read image corresponding to the second image; and sending display data to the terminal for displaying the content saved in the destination at a position on the display screen corresponding to the portion of the read image corresponding to the first image.

[0009] In addition, in the process of transmitting the display data, the processor may transmit to the terminal display data that displays the content in a state in which the content is combined with the scanned image displayed on the display screen.

[0010] The processor may also execute a process of receiving setting information indicating settings related to content display from the first terminal, and in this case, the processor may transmit display data to the terminal for displaying the content according to the settings.

[0011] Furthermore, in the process of transmitting the display data, the processor may transmit, to the terminal, display data for displaying content that is saved in the save destination and that has been set by the first user.

[0012] In addition, in the process of sending printing data, the above processor may send printing data to the terminal for printing the target image and the second image on a specified medium whose size is specified by the first user.

[0013] Furthermore, the number of times that display data can be transmitted may be stored in the storage device for each user. In this case, the processor may execute a process of reducing the number of times that the second user can transmit display data depending on the number of times that the process of transmitting display data to the terminal has been executed. Furthermore, when the remaining number of times that the second user can transmit display data is 0, transmission of display data to the terminal may be restricted.

[0014] Furthermore, whether or not the fee for the display data has been paid may be stored in the storage device. In this case, the processor may transmit the display data to the terminal of the second user who has paid the fee.

[0015] The processor may also execute a process for remitting at least a portion of the fee paid by the second user to a predetermined remittance destination.

[0016] In addition, the processor may perform a process of updating the content stored in the storage location in association with the first image, and in the process of transmitting display data to the terminal after the content has been updated, transmit display data that displays the updated content at the above-mentioned position.

[0017] Alternatively, a first user may use a first terminal, a second user may use a second terminal, and the first terminal may acquire printer information regarding a printer to be used by the second user when printing an image. In this case, the processor may transmit, to the second terminal, print data for printing the target image and the second image on a specified medium according to the printer information acquired by the first terminal.

[0018] In addition, in order to solve the above-mentioned problem, the content provision method of the present invention is characterized in that the following steps are carried out by a processor: setting at least a part of a target image provided by a first user as a first image; saving the content displayed on the display screen of a terminal of a second user in association with the first image; sending print data to the terminal for printing the target image and a second image corresponding to the destination of the content on a designated medium specified by the first user; when a printout generated by printing the target image and the second image on the designated medium according to the print data is read by the terminal, receiving a read image from the terminal, which is an image of the read printout; identifying a destination of storage from a portion of the read image corresponding to the second image; and sending display data to the terminal for displaying the content saved in the destination at a position on the display screen corresponding to the portion of the read image corresponding to the first image.

[0019] In addition, in the step of transmitting the display data, the processor may transmit to the terminal display data that displays the content in a state in which the content is combined with the scanned image displayed on the display screen.

[0020] The processor may also perform the step of acquiring setting information indicating settings related to the display of the content. In this case, in the step of transmitting display data, the processor may transmit, to the terminal, display data for displaying the content according to the settings.

[0021] The processor may also update the content stored in the storage location in association with the first image. In this case, in the step of transmitting display data to the terminal after updating the content, the processor may transmit display data for displaying the updated content at the position.

[0022] According to the present invention, it is also possible to realize a program for causing a computer to execute each step included in the content providing method described above. Furthermore, according to the present invention, it is also possible to realize a computer-readable recording medium on which a program for causing a computer to execute each step included in the content providing method described above is recorded. [Effects of the Invention]

[0023] According to the present invention, when an image acquired via a network is printed, the content provided by reading the printed matter can be displayed appropriately. [Brief explanation of the drawings]

[0024] [Figure 1A] FIG. 10 is a diagram showing a first example of content. [Figure 1B] FIG. 10 is a diagram showing a second example of content. [Figure 1C] FIG. 10 is a diagram showing a third example of content. [Figure 1D] FIG. 10 is a diagram showing a fourth example of content. [Figure 2A] FIG. 1 is a diagram showing the procedure for providing content (part 1). [Figure 2B] FIG. 10 is a diagram showing the procedure for providing content (part 2). [Figure 3] FIG. [Figure 4] FIG. 1 is a diagram illustrating a configuration of a content providing system. [Figure 5] 1 is a diagram illustrating a hardware configuration of a content providing device according to an embodiment of the present invention. [Figure 6] FIG. 2 is a diagram illustrating an example of management data stored in a storage device. [Figure 7] FIG. 2 is an explanatory diagram illustrating functions of a content providing device according to an embodiment of the present invention. [Figure 8] FIG. 1 is a diagram showing a content provision flow according to an embodiment of the present invention (part 1). [Figure 9]FIG. 2 is a diagram showing a content provision flow according to an embodiment of the present invention (part 2). [Figure 10] FIG. 10 is a diagram showing a content update flow. DETAILED DESCRIPTION OF THE INVENTION

[0025] One embodiment of the present invention (hereinafter referred to as the present embodiment) will be described with reference to the drawings. However, the embodiment described below is merely an example given to facilitate understanding of the present invention and is not intended to limit the present invention. Furthermore, the present invention may be modified or improved from the following embodiment without departing from the spirit of the present invention. Furthermore, the present invention includes equivalents thereof.

[0026] In this specification, the concept of "device" includes a single device that performs a specific function by itself, as well as multiple devices that exist independently and in a distributed manner but cooperate (link) to perform a specific function.

[0027] In this specification, unless otherwise specified, "image" refers to image data, including, for example, lossy compressed image data such as JPEG (Joint Photographic Experts Group) format, and lossless compressed image data such as GIF (Graphics Interchange Format) or PNG (Portable Network Graphics) format.

[0028] In this specification, a "user" refers to a user who uses the content providing device of the present invention. Using the content providing device means using the functions of the content providing device, and includes not only directly operating the content providing device but also using the functions of the content providing device from other devices (for example, a terminal owned by the user). Furthermore, a user may be an individual, a group including multiple people such as a family, a legal entity such as a company, or an institution or other group.

[0029] <Content provision service> A service (hereinafter referred to as a content providing service) realized by the content providing device and content providing method according to this embodiment will be described. A user (first user) can provide content to another user (second user) through the content providing service. The second user can receive and use content provided in response to a request from the first user through the content providing service.

[0030] The content is digital content, and in particular, content that can be displayed on a display device such as a display. In this embodiment, content for augmented reality (hereinafter, AR content) is provided. That is, on the side of the second user to whom the AR content is provided, when a real-world image (more specifically, a real-time image) captured by a camera or the like is displayed on the display, the AR content is composited with the image and displayed.

[0031] The AR content may be, for example, content A1 consisting of text (message) as shown in Fig. 1A, or content A2 consisting of video or images as shown in Fig. 1B. Furthermore, the AR content may be content A3 consisting of illustrations drawn by computer graphics as shown in Fig. 1C. Furthermore, the AR content may be content A4, as shown in Fig. 1D, which is synthesized with other images to add a background effect (a background effect of falling rain in Fig. 1D). The content may be displayed in a static state, or in a moving state like a video, and may also be displayed with audio output.

[0032] The procedure for providing and using AR content will be outlined with reference to Figures 2A and 2B. First, the first user transmits an image required for providing the content (hereinafter, "target image"). Here, "the first user transmits the target image" means that the target image is transmitted from the information processing terminal used by the first user (hereinafter, "first terminal 11"), specifically, the target image is uploaded. The target image is provided by the first user and is one or more images designated from a group of images owned by the first user, such as an image purchased by the second user from the first user.

[0033] 2A, the first user sets the AR content to be provided and also sets the conditions (display conditions) for displaying the AR content on the second user side. Setting the AR content means creating new AR content and selecting one of multiple types of AR content prepared in advance.

[0034] The display conditions of the AR content are settings related to the display of the AR content, such as the display size, display position, and orientation of the AR content when the AR content is displayed on the second user side. Here, the display position is determined using the position of the first image (described later) as a reference position.

[0035] The content providing service generates print data based on the target image sent by the first user and information about the AR content set by the first user (specifically, the storage destination of the AR content, etc.). The print data is data for printing the image for obtaining the content. The generated print data is sent to the second user as shown in FIG. 2A. Here, "sending print data to the second user" means sending data to the information processing terminal (corresponding to a terminal, hereinafter referred to as second terminal 12) used by the second user.

[0036] When the second user (more specifically, the second terminal 12) receives the print data, the second user prints the content acquisition image in accordance with the print data. The device used to print the content acquisition image is, for example, a printer owned by the second user (more specifically, the portable printer 13 described below). However, the present invention is not limited to this, and the second user may use a printing device that is not owned by the second user but that can be used by logging in by entering a PIN or account information, such as a device installed in a store.

[0037] By printing the content acquisition image, a print P shown in Fig. 3 is generated. The print P is a medium (for example, a photosensitive film F described below) on which a target image and an identification image are printed.

[0038] 3, the image forming surface of the print P includes a target image area Pa and a second image area Pb. The target image area Pa contains the printed target image (i.e., the visualized image of the target image) on the image forming surface.

[0039] Furthermore, the target image area Pa includes an area corresponding to a first image set in the target image (hereinafter referred to as the first image area). The first image is a template image when displaying AR content, and is composed of a part or all of the target image. The template image functions as a marker to be tracked when the AR content is displayed. In the case shown in FIG. 3, for example, the face of the subject in the target image is set as the template image.

[0040] The second image area Pb is an area where an optically readable second image, such as a QR code (registered trademark), is printed. The second image is an image generated according to the storage destination of the AR content set by the first user, and more specifically, is an image obtained by encoding (barcoding) the storage destination. Note that the second image is not limited to an encoded image such as a QR barcode (registered trademark), and may be any readable image, such as letters, symbols, numbers, or encrypted characters. The second image may also include encryption information, which is encrypted information for identifying the first image in the target image.

[0041] 3, the second image area Pb may be provided in a position overlapping with the target image area Pa, or may be provided in a position separated from the target image area Pa. In this case, the second image area Pb is preferably disposed near the target image area Pa.

[0042] Then, the second user reads the printout P using a second terminal 12 equipped with a reading function, as shown in Fig. 3. Reading the printout P means obtaining a read image of the printout P, ​​and the read image represents the density (more specifically, gradation value) of the printout P for each pixel, and is obtained by reading the printout P. To put it simply, photographing (capturing) the printout P with an imaging device corresponds to reading the printout P, ​​and the captured image (real-time image) corresponds to the read image.

[0043] When the print P is scanned, the scanned image is sent from the second user as shown in Fig. 2B. At this stage, the scanned image, i.e., a real-time image of the print P, is displayed on the display screen of the second terminal 12.

[0044] The content providing service identifies a storage destination of the AR content set by the first user based on a portion of the scanned image sent from the second user that includes the second image area Pb (i.e., a portion corresponding to the second image). Then, display data for displaying the AR content stored in the identified storage destination is transmitted (i.e., downloaded) to the second user. The display data is received by the second terminal 12. As a result, the AR content is provided to the second user.

[0045] As shown in FIGS. 1A to 1D, the AR content is displayed on the display screen of the second terminal 12 that has received the display data, superimposed on the scanned image. At this time, the AR content is displayed according to the display conditions set by the first user, specifically, at the display size, display position, and orientation specified by the first user. In particular, the display position of the AR content is determined according to the portion of the display screen in which the first image area appears (i.e., the portion corresponding to the first image in the scanned image). As a result, the AR content is displayed on the display screen with its positional relationship with the portion in which the first image area appears maintained constant.

[0046] According to the content providing service described above, for example, a first user sells an image, and a second user purchases an image (target image) from the first user and prints the purchased image to obtain a printout P. Then, when the second user reads the printout P using the second terminal 12, the AR content set by the first user is displayed on the display screen of the second terminal 12, superimposed on the read image. This allows the second user to use the AR content provided through the content providing service.

[0047] Regarding the content providing service, the use of the AR content providing service may be free or may be subject to a fee. In this embodiment, the service is subject to a fee, and the second user can use the AR content (strictly speaking, the display data of the AR content) only if the second user pays the fee, i.e., the second user can display the AR content on the second terminal 12.

[0048] Note that a part or all of the fee for the AR content paid by the second user may be remitted to the first user who sold the target image, or to a predetermined remittance destination other than the first user. The predetermined remittance destination other than the first user may be, for example, a business operator that operates a content provision service. With the above configuration, the fee for the AR content paid by the second user can be shared between the first user and the service provider.

[0049] Furthermore, certain restrictions may be placed on the provision of AR content to the second user, i.e., the transmission of display data of the AR content. For example, the period during which AR content can be provided to the second user may be limited to a certain period after the second user purchases an image (target image) from one user. Furthermore, the number of times the display data of the AR content can be transmitted may be determined, and the AR content may be provided to the second user until the remaining number of transmissions set for the second user reaches 0. Conversely, when the remaining number reaches 0, the provision of the AR content to the second user may be restricted (terminated).

[0050] Furthermore, the content provided to the second user when the second user reads the printed matter P may be the same content or may change depending on the time of reading. In other words, the content provided when the same printed matter P is read may be updated depending on the time, etc.

[0051] <Contents provision system> Next, with reference to FIG. 4, the configuration of a communication system constructed to realize a content providing service, that is, a content providing system S, will be described. As shown in FIG. 4, the content providing system S is made up of a server computer 10, a first terminal 11, a second terminal 12, a portable printer 13, and the like. Each device will be described below.

[0052] (server computer) The server computer 10 is a computer that forms a platform for a content providing service, and constitutes the content providing device 20 of the present invention. The server computer 10 may be a server for a cloud service, specifically a server computer for an ASP (Application Service Provider), SaaS (Software as a Service), PaaS (Platform as a Service), or IaaS (Infrastructure as a Service).

[0053] The server computer 10 can communicate with the first terminal 11 and the second terminal 12 via an external network N such as the Internet or a mobile communication network, and acquires various information from these devices. The acquired information is stored in a storage device 14, and the server computer 10 can read the information stored in the storage device 14. The storage device 14 is configured, for example, by a storage device built into or externally attached to the server computer 10, or a NAS (Network Attached Storage), etc. However, without being limited thereto, the storage device 14 may also be a third computer capable of communicating with the server computer 10, such as a so-called online storage.

[0054] The server computer 10 then executes various information processes related to the content providing service. Specifically, the server computer 10 generates print data for the content acquisition image and transmits the data to the second user (see FIG. 2A). The server computer 10 also generates display data for the AR content and transmits the data to the second user (see FIG. 2B).

[0055] (1st terminal) The first terminal 11 is a device operated by the first user, and is configured as a PC (Personal Computer), a workstation, a smartphone, a tablet terminal, or the like. The first terminal 11 accepts operations performed by the first user to use the content providing service. The operations accepted by the first terminal 11 include, for example, an image designation operation, a print condition designation operation, an AR content setting operation, and a display condition setting operation.

[0056] The image designation operation is an operation for designating a target image to be sent (transferred) to a second user from among a group of images owned by a first user. The image designation operation may also include an editing operation for applying image processing such as trimming or image quality adjustment to the designated target image. When the first terminal 11 accepts the image designation operation, it transmits the target image designated by that operation to the server computer 10. At this time, the target image or the edited image is transmitted in association with the ID information of the first user (strictly speaking, the first terminal 11).

[0057] The print condition specification operation is an operation for setting print conditions when a content acquisition image including a target image is printed by a second user. Specifically, the first user specifies, as print conditions, the second user who will perform the printing and the medium (hereinafter, designated medium) on which the content acquisition image is to be printed. The print conditions may also include, for example, the layout of each of the target image and the second image in the content acquisition image, as well as the print size of each image. In this embodiment, the first user first checks the ID information of the second user (strictly speaking, the second terminal 12) and the type of portable printer 13 used by the second user, and sets the print conditions after understanding this information. When the first terminal 11 accepts a print condition specification operation, it transmits information on the print conditions specified by the first user (hereinafter, "specified information") to the server computer 10. At this time, the specified information is transmitted in association with the ID information of the first user (strictly speaking, the first terminal 11). Here, when the first user specifies the printing conditions, this includes not only the case where the first user specifies the printer conditions by operating the first terminal 11, but also the case where the printer conditions are automatically specified by the first terminal 11 used by the first user, as described below.

[0058] The method for acquiring information regarding the type of portable printer 13 of the second user, print size, etc. (hereinafter referred to as printer information) is not limited to the above method. For example, the printer information may be stored in the second terminal 12, or may be stored in the server computer 10 in association with the second user's ID information. In this case, the first terminal 11 may acquire the printer information by communicating with the second terminal 12 or the server computer 10. The first terminal 11 may also automatically set print conditions, including the specified medium, based on the acquired printer information.

[0059] The AR content setting operation is an operation for setting AR content to be provided to the second user, and for example, if the AR content is a message, it is an operation for creating (inputting) the message. Also, if multiple candidates for AR content that can be provided are prepared in advance, the first user may perform the AR content setting operation by selecting one or more candidates. When the first terminal 11 accepts the AR content setting operation, it transmits the AR content set (specifically, created or selected) by the first user to the server computer 10. At this time, the AR content is transmitted in association with ID information of the first user (strictly speaking, the first terminal 11).

[0060] The display condition setting operation is an operation for setting display conditions when the AR content is displayed on the second user side, and specifically, sets the display size, display position, orientation, etc. of the AR content. When the first terminal 11 accepts the display condition setting operation, it transmits information on the display conditions set by the first user (hereinafter, "setting information") to the server computer 10. At this time, the setting information is transmitted in association with ID information of the first user (strictly speaking, the first terminal 11).

[0061] (Second terminal) The second terminal 12 is a device operated by the second user and equipped with a reading function, and more specifically, is configured as a smartphone, tablet terminal, or the like equipped with a photographing device such as a camera. The second terminal 12 accepts operations performed by the second user to use the content providing service. Examples of operations accepted by the second terminal 12 include a purchase operation, a payment operation, and a reading instruction operation.

[0062] The purchase operation is an operation performed by a second user when purchasing an image from a first user. When the second terminal 12 accepts the purchase operation, it transmits information for identifying the image to be purchased to the server computer 10. The information for identifying the image to be purchased is associated with ID information of the second user (strictly speaking, the second terminal 12) and transmitted from the server computer 10 to the first terminal 11.

[0063] The payment operation is an operation performed by the second user when paying for the AR content (more specifically, the fee for the display data of the AR content), i.e., a settlement operation. When the second terminal 12 accepts the purchase operation, it transmits information regarding the payment of the fee to the server computer 10.

[0064] The read instruction operation is an operation performed by the second user when instructing the second user to read the print P. When the second terminal 12 accepts the read instruction operation, the reading device (i.e., the photographing device) of the second terminal 12 is started. Then, with the photographing unit (specifically, the photographing lens) of the reading device facing the print P, reading (photographing) of the print P is started. As a result, a live video of the read image is displayed on the display screen of the second terminal 12. In addition, the second terminal 12 transmits the read image to the server computer 10. At this time, the read image is transmitted in association with the ID information of the second user (strictly speaking, the second terminal 12).

[0065] Furthermore, when the second terminal 12 receives the printing data from the server computer 10, it controls the portable printer 13 in accordance with the printing data and causes the portable printer 13 to print the image for obtaining the content. When printing the image, the second terminal 12 may inquire of the second user whether or not printing is necessary. In this case, the second terminal 12 may cause the portable printer 13 to print the image for obtaining the content, provided that the second user replies that printing is necessary.

[0066] Furthermore, when the second terminal 12 receives the display data of the AR content from the server computer 10, it unpacks the display data and displays the AR content. At this time, the AR content is displayed on the display screen of the second terminal 12 according to the display conditions set by the first user. Specifically, the AR content is displayed by being combined with the scanned image of the printout P displayed on the display screen. At this time, the AR content is displayed at the display position, display size, and orientation set by the first user. In particular, the display position of the AR content on the display screen is determined according to the portion of the scanned image that shows the first image area.

[0067] (Portable printer) The portable printer 13 is a device used by the second user to print images for obtaining content, and is wirelessly connected to the second terminal 12 via Wi-Fi (registered trademark) or Bluetooth (registered trademark). However, the present invention is not limited to this, and the portable printer 13 may also be connected to the second terminal 12 via a wired connection. The portable printer 13 then prints the image under the control of the second terminal 12.

[0068] The portable printer 13 of this embodiment prints images using an instant photography method. That is, the portable printer 13 stores photosensitive film inside and exposes the image-forming surface of the photosensitive film to light to form a latent image. Next, the portable printer 13 breaks a developer pot located at the end of the film, spreading developer on the film surface and making the latent image visible. After this process, the photosensitive film is ejected from the printer, and development progresses, ultimately forming a complete image on the image-forming surface of the photosensitive film. This results in a print P. The image printing method is not limited to the instant photography method, but may be an inkjet method, a dye-sublimation thermal transfer method, or an electrophotographic method using toner.

[0069] The photosensitive film is a medium on which an image is printed, and is a film dedicated to the portable printer 13. There are multiple sizes of photosensitive film that can be used with the portable printer 13, and the type of film to be used is determined depending on the type of portable printer 13. Specifically, in this embodiment, there are multiple types of portable printers 13, and the size of the photosensitive film is determined to be one of the sizes depending on the type of portable printer 13 that the second user will use.

[0070] In this embodiment, even when printing the same image, the size of the printed image changes depending on the size of the photosensitive film. Therefore, when a content acquisition image is printed on photosensitive film, the size of the target image area Pa, in other words, the position of the first image, changes depending on the size of the photosensitive film. Therefore, when the second user requests the content acquisition image to be printed, the second user must specify the type (size) of photosensitive film to be used. Therefore, in this embodiment, printing data is generated according to the specified type of photosensitive film (corresponding to the specified medium), and the generated data is sent to the second user, specifically, to the second terminal 12.

[0071] <Configuration example of content providing device> Next, a configuration example of the content providing device 20 according to this embodiment will be described. As described above, the content providing device 20 according to this embodiment is configured by the server computer 10. The number of computers configuring the content providing device 20 may be one, or two or more. In other words, the content providing device 20 is realized by a processor and a program executable by the processor, and is configured by, for example, a general-purpose computer.

[0072] As shown in FIG. 5, the server computer 10 constituting the content providing device includes a processor 10A, a memory 10B, a communication interface 10C, and the like.

[0073] The processor 10A is configured by, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a tensor processing unit (TPU). The memory 10B is configured by, for example, semiconductor memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The communication interface 10C is configured by, for example, a network interface card or a communication interface board.

[0074] In addition, a program (hereinafter, "content providing program") for functioning as the content providing device of the present invention is installed in the server computer 10. The content providing program is a program for causing a computer to execute each step included in the content providing method of the present invention. That is, the processor 10A reads the content providing program and executes a series of processes related to the content providing service described above.

[0075] The content providing program may be acquired by reading it from a computer-readable recording medium, or may be acquired by receiving (downloading) it via a communication line such as the Internet or an intranet.

[0076] The content providing device 20 is connected to the storage device 14 and can read out information stored in the storage device 14. The storage device 14 stores information necessary for the content providing service, such as information or images acquired by the content providing device 20 from the first terminal 11 and the second terminal 12. The storage device 14 also stores management data for each user for managing the usage history of the content providing service, etc.

[0077] The management data includes information about users of the content providing service, information about the AR content to be provided, and information about the history of content provision. Specifically, as shown in FIG. 6, the management data includes ID information about the first terminal 11 of the first user, a storage destination for the AR content set by the first user, and ID information about the second terminal 12 of the second user to which the AR content is provided. The management data also includes printer information indicating the type of portable printer 13 used by the second user. Furthermore, the management data includes, for the second user, whether or not the fee for the AR content has been paid, the amount paid, the period during which the AR content can be provided, the number of times it can be provided, and the like.

[0078] The configuration of the content providing device 20 will be explained again from a functional perspective with reference to Fig. 7. The content providing device 20 has a first receiving unit 21, a setting unit 22, a second receiving unit 23, a storage unit 24, a third receiving unit 25, a fourth receiving unit 26, a first transmitting unit 27, a fifth receiving unit 28, an identifying unit 29, and a second transmitting unit 30. The content providing device 20 also has a determining unit 31, a restriction setting unit 32, an updating unit 33, and a remittance processing unit 34. These functional units are realized by cooperation between hardware devices included in the server computer 10 and software including the content providing program described above.

[0079] The first receiving unit 21 receives the target image transmitted from the first terminal 11. The received target image is stored in the storage device 14, for example.

[0080] The setting unit 22 sets a first image for the target image acquired by the first receiving unit 21. At this time, the setting unit 22 sets a part or all of the target image as the first image. For example, if a person is shown in the target image, the person's face may be set as the first image. Note that the first image may be set based on a part of the target image designated by the first user, or may be set automatically by the setting unit 22.

[0081] The second receiving unit 23 receives AR content set by the first user and provided to the second user (i.e., content displayed on the display screen of the second terminal 12). When the first user selects one or two candidates from multiple candidates to set AR content, the second receiving unit 23 receives the selection result.

[0082] The storage unit 24 stores the AR content received by the second receiving unit 23 in association with the first image set by the setting unit 22. The storage destination of the AR content is not particularly limited, and may be, for example, the storage device 14. From the viewpoint of reducing the data capacity in the content providing device 20, the storage destination of the AR content is preferably the storage device 14 other than the content providing device 20. Furthermore, the storage unit 24 generates a second image (for example, a code image such as a QR code (registered trademark)) according to the storage destination of the AR content.

[0083] The third receiving unit 25 receives and acquires setting information indicating the details (setting details) set by the first user regarding the display conditions of the AR content from the first terminal 11. The display conditions indicated by the setting information include the display size, display position, orientation, etc. of the AR content. The display position is a position determined based on the above-mentioned first image.

[0084] The fourth receiving unit 26 receives, from the first terminal 11, designation information indicating print conditions designated by the first user for the content acquisition images including the target image and the second image. The print conditions indicated by the designation information include the size of the photosensitive film on which the content acquisition images are printed (in other words, the type of portable printer 13), the layout of each of the target image and the second image, and the print size of each image. The designation information also includes ID information of the second user who is to have the content acquisition images printed.

[0085] The first transmitting unit 27 generates printing data and transmits it to the second terminal 12. The printing data is data for printing the content acquisition image in accordance with the printing conditions indicated by the specification information received by the fourth receiving unit 26. By transmitting the printing data from the first transmitting unit 27, the content acquisition image is printed on a photosensitive film of the type (size) specified by the first user at the second user side. The print data cannot be changed (overwritten) on the second terminal 12. In other words, the second user can print the content acquisition image only if he / she uses photosensitive film of the size specified by the first user.

[0086] The fifth receiving unit 28 receives a scanned image of the printed matter P from the second terminal 12. The printed matter P is generated by printing an image for obtaining content in accordance with the above-mentioned printing data, and the scanned image is acquired by the second user scanning the printed matter P with the second terminal 12. When receiving the scanned image from the second terminal 12, the fifth receiving unit 28 receives the scanned image in association with the ID information of the second terminal 12.

[0087] The identification unit 29 analyzes the read image received by the fifth receiving unit 28, and identifies the storage destination of the AR content from the part of the read image that shows the second image area Pb (the part that corresponds to the second image).

[0088] The second transmission unit 30 creates display data of the AR content stored in the storage destination identified by the identification unit 29 and transmits the data to the second terminal 12. At this time, the second terminal 12 displays the scanned image of the printout P on its display screen. Then, when the display data is received by the second terminal 12, the AR content, i.e., the AR content set by the first user, is displayed on the display screen in a state where it is combined with the read image. Specifically, the AR content is displayed on the display screen at a position corresponding to the portion of the read image in which the first image area appears (the portion corresponding to the first image). The AR content is displayed according to the display conditions (settings) indicated by the setting information received by the third receiving unit 25.

[0089] The determination unit 31 determines whether the second user to whom the AR content is provided has paid the fee for the AR content (more specifically, the fee for the display data of the AR content). Specifically, the determination unit 31 identifies the ID information of the second terminal 12 associated with the scanned image received by the fifth receiving unit 28, and refers to the management data corresponding to the ID information. Then, the determination unit 31 determines whether the second user using the second terminal 12 has paid the fee.

[0090] If the determination unit 31 determines that the second user has paid the fee, the second transmission unit 30 transmits display data of the AR content to the second terminal 12 of the second user. Conversely, if the second user has not paid the fee, the AR content is not provided to the second user, that is, the display data of the AR content is not transmitted to the second terminal 12 of the second user. In this way, in this embodiment, the AR content can be displayed only on the second terminal 12 of the second user who purchased the target image.

[0091] The restriction setting unit 32 sets a restriction on providing the AR content to the second user (i.e., transmitting display data of the AR content to the second terminal 12). Specifically, the restriction setting unit 32 sets a period during which the AR content is available to the second user. The second transmission unit 30 transmits the display data of the AR content to the second terminal 12 of the second user only within the period during which the AR content is available to the second user.

[0092] The restriction setting unit 32 also sets the number of times that AR content can be provided, i.e., the number of times that display data of the AR content can be transmitted, for the second user. Furthermore, the restriction setting unit 32 reduces the number of times that transmissions can be made for the second user in accordance with the number of times that the second transmission unit 30 executes the process of transmitting display data to the second terminal 12. When the number of times that transmissions can be made for the second user reaches 0, transmission of display data from the second user to the second terminal 12 is subsequently restricted.

[0093] The number of times that the AR content can be provided (the number of times that the display data can be transmitted) may be any number of times, for example, only once. In other words, after the AR content has been provided to the second user once, provision of the AR content (transmission of the display data) may be limited. The method for restricting the provision of AR content is not particularly limited, and the provision of the AR content (transmission of display data) may be prohibited, or the AR content itself may be deleted from the storage location. Furthermore, when a restriction is set, a notification prompting the second user to pay a fee to remove the restriction may be sent to the second user.

[0094] When a predetermined condition (hereinafter, referred to as an update condition) is satisfied, the update unit 33 updates the AR content to be provided when a second user reads the same printout P. That is, the target of update by the update unit 33 is the AR content that is associated with the same first image and saved in a save location specified by the second image.

[0095] The updated AR content may be set by, for example, the first user. That is, when an update condition is satisfied, the first user sets new AR content and display conditions, and the second receiving unit 23 receives and acquires the new AR content, and the third receiving unit 25 receives and acquires setting information indicating the new display conditions. The update unit 33 saves the new AR content as the updated AR content in the storage location of the AR content before the update. At this time, the updated AR content is associated with the first image that was associated with the AR content before the update. When the AR content is updated, it is preferable to notify the second user of this.

[0096] When the second transmission unit 30 transmits the display data of the AR content to the second terminal 12 after updating the content, the display data of the updated AR content is transmitted. As a result, the updated AR content is displayed on the display screen of the second terminal 12 at a position corresponding to the portion of the first image area in the read image.

[0097] The update condition is not particularly limited and may be any condition, but may be set in association with the second user, for example. Specifically, the update condition may be that a predetermined period of time has passed since the second user first scanned the printout P, ​​or that the second user's birthday has arrived.

[0098] The remittance processing unit 34 executes a process (hereinafter referred to as the remittance process) for remitting at least a portion of the fee for the AR content paid by the second user to a predetermined remittance destination. The remittance process is, for example, a process for remitting money to an account of the first user or an operator of the content providing service, and the process content and procedure are similar to those of known remittance processes in electronic commerce.

[0099] <Information processing flow for providing content according to one embodiment of the present invention> An information processing flow (hereinafter referred to as the "content provision flow") by a content provision device according to one embodiment of the present invention will be described. The content provision flow employs the content provision method of the present invention. In other words, each step in the content provision flow corresponds to a component of the content provision method of the present invention.

[0100] The content provision flow proceeds as shown in Figures 8 and 9. Each process (step) in the content provision flow is executed by the processor 10A of the server computer 10 constituting the content provision device 20, and the first terminal 11 and the second terminal 12, in cooperation with each other.

[0101] The content provision flow is triggered, for example, by a second user purchasing an image from a first user. In the content provision flow, first, the first terminal 11 transmits the image purchased by the second user, i.e., the target image, to the server computer 10 (S001). The processor 10A of the server computer 10 receives the target image from the first terminal 11 (S012). At this time, the processor 10A also acquires information about the second terminal 12 of the second user who purchased the target image.

[0102] Next, the processor 10A sets at least a part of the target image as the first image (S003). In S003, the processor 10A may set the first image based on an editing operation on the target image performed by the first user on the first terminal 11, or may set the first image automatically in accordance with a predetermined setting standard.

[0103] Next, the first terminal 11 sets, based on the operation by the first user, AR content to be provided to the second user and displayed on the display screen of the second terminal 12, and its display conditions (S004). Thereafter, the first terminal 11 transmits the set AR content and setting information indicating the set display conditions to the server computer 10 (S005). The processor 10A receives and acquires the AR content and the setting information from the first terminal 11 (S006).

[0104] Next, the processor 10A associates the AR content received in S006 with the first image set in S003 and saves it (S007). At this time, the setting information may also be associated with the first image and saved together with the AR content. Also, in S007, the processor 10A encodes information indicating the save destination of the AR content to generate a second image.

[0105] Next, first terminal 11, based on an operation by the first user, specifies print conditions for the content acquisition images including the target image and the second image (S008). Specifically, in S008, the type (size) of photosensitive film used to print the content acquisition images is specified as the print conditions. Thereafter, first terminal 11 transmits specification information indicating the specified print conditions to server computer 10 (S009). Processor 10A receives the specification information from first terminal 11 (S010).

[0106] Next, the processor 10A generates print data for printing the content acquisition image in accordance with the print conditions indicated by the designation information, and transmits the generated data to the second terminal 12 (S011). The print data is received by the second terminal 12 of the second user who purchased the target image (S012).

[0107] Next, the second terminal 12 controls the portable printer 13 in accordance with the print data to cause the portable printer 13 to print the image for obtaining the content (S013). As a result, the image for obtaining the content is printed on the photosensitive film of the type (size) specified by the first user, and as a result, the second user obtains the print P.

[0108] Next, the second terminal 12 uses its reading function to read the printed matter P, and the read image is transmitted to the server computer 10 (S014). At this time, the read image is transmitted in association with the ID information of the second terminal 12, which is the transmission source. The transmitted read image is received by the processor 10A (S015).

[0109] Next, the processor 10A analyzes the scanned image received in S015 and extracts a portion of the scanned image that includes the second image area Pb (a portion corresponding to the second image). The processor 10A then identifies, from the extracted portion, a storage destination for the AR content set by the first user and provided to the second user (S016). The processor 10A then reads the AR content from the identified storage destination.

[0110] Next, the processor 10A determines whether the second user, who is the recipient of the AR content, has paid the fee for the AR content (the fee for the display data of the AR content) (S017). If the second user has paid the fee, the process proceeds to S018, and if the fee is unpaid, the process ends.

[0111] Next, the processor 10A determines whether the remaining number of times that the display data can be transmitted, set for the second user, is 0 (S018). If the remaining number is not 0, the process proceeds to S019, and if the remaining number is 0, the process ends.

[0112] Next, processor 10A generates display data for the AR content stored in the storage destination identified in S016 and transmits the generated display data to second terminal 12 (S019). When second terminal 12 receives the display data (S020), it unpacks the display data and displays the AR content on the display screen (S021). At this time, the read image (i.e., a real-time image of printout P) is displayed on the display screen of second terminal 12, and the AR content is displayed in a state where it is composited with the read image. That is, in S019, display data for displaying the AR content in a state where it is composited with the read image displayed on the display screen is transmitted to second terminal 12.

[0113] Furthermore, in S021, the AR content is displayed according to the display conditions indicated by the setting information received in S006. More specifically, the AR content is displayed on the display screen at a position determined based on the portion of the read image in which the first image area appears (the portion corresponding to the first image). That is, in S019, display data for displaying the AR content according to the display conditions, more specifically, display data for displaying the AR content on the display screen at a position corresponding to the portion in which the first image area appears, is transmitted to the second terminal 12.

[0114] The content provision flow ends when the above series of steps are completed. According to the content provision flow of this embodiment, the content acquisition image acquired via the network is printed, and the AR content provided can be displayed appropriately by scanning the printout P.

[0115] Specifically, if the type (size) of photosensitive film used to print the content acquisition image changes, the size of the printed image changes. In this case, the position of the first image (i.e., template image) in the printed image changes. Therefore, if the correct type (size) of photosensitive film is not used, the AR content may not be displayed in the appropriate position. In such a situation, there is a risk that the intention of the first user who set the content may not be properly reflected when the AR content is displayed.

[0116] In contrast, in this embodiment, data to be printed on a type of photosensitive film (designated medium) designated by the first user or the first terminal is transmitted to the second terminal 12 as printing data for the content acquisition image. This printing data cannot be changed (overwritten) by the second terminal 12, so the content acquisition image is printed only on the type of photosensitive film designated by the first user. This allows the content acquisition image to be properly printed on the photosensitive film, and the first image is positioned in the correct position in the content acquisition image. As a result, when the printout P of the content acquisition image is read, the AR content is displayed properly on the display screen (more specifically, at the appropriate position on the display screen).

[0117] The above-described effect is particularly significant when AR content is synthesized and displayed on a scanned image of a printout P. In other words, in this embodiment, AR content can be appropriately displayed using the above-described procedure, which facilitates widespread provision of AR content through network printing. As a result, content provision services can be revitalized.

[0118] Furthermore, in this embodiment, restrictions are placed on the provision of AR content (in other words, transmission of display data of AR content) in terms of timing, number of times, etc. This makes it possible to avoid inappropriate use and duplication of AR content, and as a result, it is possible to ensure appropriate use and protection of AR content.

[0119] In this embodiment, a content update flow shown in Fig. 10 is also executed separately. The content update flow is started when an update condition is met (S031). When the flow starts, the first user sets new AR content and its display conditions, and the first terminal 11 transmits setting information indicating the new AR content and the new display conditions to the server computer 10 (S032, S033).

[0120] When the processor 10A receives the new AR content and the setting information (S034), the processor 10A updates the AR content that was stored in association with the first image in S007 to the new AR content (S035), thereby completing the content update flow.

[0121] Then, in step S019 of the content provision flow that is performed after the AR content is updated, processor 10A transmits display data for the updated AR content. This display data causes the updated AR content to be displayed on the display screen of second terminal 12 at a position corresponding to the portion of the first image area in the scanned image. By updating the AR content that can be displayed by scanning one printout P in this way, the second user's interest, specifically, interest in the AR content that is provided, can be sustained compared to when only a single AR content is displayed.

[0122] <Other embodiments> The embodiment described above is a specific example given to clearly explain the content providing device and content providing method of the present invention, and is merely an example, and other embodiments are also possible.

[0123] In the above embodiment, the print data of the content acquisition image is received by the second terminal 12, but this is not limited to this. For example, the portable printer 13 may directly receive the print data.

[0124] In the above embodiment, from the viewpoint of content protection, restrictions are imposed on the provision of AR content, specifically, a period during which the content can be provided and the number of times the content can be provided are set. However, this is not limited to this, and when the number of second users to whom a certain AR content is provided (specifically, the number of second users who have purchased the same target image) reaches a predetermined number or more, the restrictions on the AR content may be lifted.

[0125] Furthermore, in the above embodiment, even if the printed matter P is read by the second terminal 12, if the fee for the AR content has not been paid, the AR content will not be displayed. However, this is not limited to this, and for example, when a user who has not paid the fee for the AR content (for example, a family member of the second user who has paid the fee) reads the printed matter P with a camera-equipped smartphone or the like, the AR content may be displayed on the display screen.

[0126] Furthermore, in the above embodiment, the functions of the content providing device of the present invention are performed by the processor 10A provided in the server computer 10, but this is not limited to this. The processor provided in the content providing device of the present invention may be provided in the terminals used by the user (i.e., the first terminal 11 and the second terminal 12) and the portable printer 13. In other words, some or all of the above-mentioned functional units provided in the content providing device of the present invention may be provided in the first terminal 11, the second terminal 12, or the portable printer 13.

[0127] Furthermore, the content providing device of the present invention may be the first terminal 11, that is, the processor included in the content providing device of the present invention may be a processor included in the first terminal 11. In this case, the processor of the first terminal 11 will perform each step in the content providing flow described above. That is, the first terminal 11 may acquire printer information from the second terminal 12, generate print data based on the acquired printer information, and transmit the data to the second terminal 12. The first terminal 11 may also receive a scanned image from the second terminal 12, identify a storage destination for the content from a portion of the scanned image that corresponds to the second image, and transmit display data for displaying the content to the second terminal 12.

[0128] The processor included in the content providing device of the present invention includes various types of processors, including, for example, a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units. The various processors also include PLDs (Programmable Logic Devices), which are processors whose circuit configuration can be changed after manufacturing, such as FPGAs (Field Programmable Gate Arrays). Furthermore, various processors include dedicated electrical circuits such as ASICs (Application Specific Integrated Circuits), which are processors having circuit configurations designed specifically for performing specific processes.

[0129] Furthermore, one processing unit possessed by the content providing device of the present invention may be configured by one of the various processors described above, or may be configured by a combination of two or more processors of the same or different types, for example, a combination of multiple FPGAs, or a combination of an FPGA and a CPU, etc. Furthermore, the plurality of functional units included in the content providing device of the present invention may be configured by one of various processors, or two or more of the plurality of functional units may be collectively configured by one processor. Furthermore, as in the above-described embodiment, one processor may be configured by combining one or more CPUs and software, and this processor may function as multiple functional units.

[0130] Also, for example, a processor may be used that realizes the functions of the entire system including multiple functional units in the content providing device of the present invention on a single IC (Integrated Circuit) chip, as typified by an SoC (System on Chip), etc. Furthermore, the hardware configuration of the various processors described above may be an electric circuit (Circuitry) that combines circuit elements such as semiconductor elements. [Explanation of symbols]

[0131] 10 Server Computers 10A processor 10B memory 10C communication interface 11 Terminal 1 12 Terminal 2 13 Portable Printer 14 Storage device 20 Content providing device 21 First receiving unit 22 Setting section 23 Second receiving unit 24 Preservation Department 25 Third Receiving Unit 26 Fourth Receiving Unit 27 First Transmission Unit 28 5th Receiving Unit 29 Specific section 30 Second Transmission Unit 31 Judgment section 32 Restriction setting section 33 Update section 34 Remittance Processing Section A1, A2, A3, A4 content N External Network P Printed material Pa Target image area Pb Second image area S Contents provision system

Claims

1. A content providing device including a processor, The processor: setting at least a portion of a target image provided by a first user as a first image; a process of storing content displayed on a display screen of a terminal of a second user in association with the first image; a process of transmitting, to the terminal, print data for printing the target image and a second image corresponding to the storage destination of the content on a designated medium designated by the first user; a process of receiving, when a printout generated by printing the target image and the second image on the specified medium in accordance with the print data is read by the terminal, a read image of the printout from the terminal; a process of identifying the storage destination from a portion of the scanned image that corresponds to the second image; transmitting, to the terminal, display data for displaying the content stored in the storage location at a position on the display screen corresponding to a portion of the scanned image that corresponds to the first image; A content providing device that executes the above.

2. The content providing device according to claim 1, wherein in the process of transmitting the display data, the processor transmits to the terminal the display data that displays the content in a state in which the content is superimposed on the scanned image displayed on the display screen.

3. the processor executes a process of acquiring setting information indicating settings related to display of the content; The content providing device according to claim 1 , wherein the processor transmits the display data for displaying the content according to the setting to the terminal.

4. 4. A content providing device according to claim 1, wherein the processor, in the process of transmitting the display data, transmits to the terminal the display data that displays the content stored in the storage destination and set by the first user.

5. A content providing device described in any one of claims 1 to 4, wherein in the process of sending the printing data, the processor sends the printing data to the terminal for printing the target image and the second image on the specified medium whose size is specified by the first user.

6. the number of times the display data can be transmitted is stored in a storage device for each user; the processor executes a process of reducing the number of times the second user can transmit the display data to the terminal in accordance with the number of times the process of transmitting the display data to the terminal has been executed; The content providing device according to claim 1 , wherein when the remaining number of transmissions allowed for the second user is 0, transmission of the display data to the terminal is restricted.

7. whether or not the fee for the display data has been paid is stored in a storage device; The content providing device according to claim 1 , wherein the processor transmits the display data to the terminal of the second user who has paid the fee.

8. The content providing device according to claim 7 , wherein the processor executes a process for transferring at least a portion of the fee paid by the second user to a predetermined transfer destination.

9. The processor: performing a process of updating the content stored in the storage location in association with the first image; The content providing device according to claim 1 , wherein the process of transmitting the display data to the terminal after updating the content comprises transmitting the display data for displaying the updated content at the position.

10. The first user uses a first terminal, and the terminal used by the second user is a second terminal; A content providing device described in any one of claims 1 to 9, wherein when the first terminal acquires printer information regarding a printer to be used by the second user when printing an image, the processor transmits the printing data to the second terminal for printing the target image and the second image on the specified medium according to the printer information acquired by the first terminal.

11. setting at least a portion of a target image provided by a first user as a first image; storing content displayed on a display screen of a terminal of a second user in association with the first image; transmitting print data to the terminal for printing the target image and a second image corresponding to the storage destination of the content on a designated medium designated by the first user; a step of receiving, when a printout generated by printing the target image and the second image on the specified medium in accordance with the print data is read by the terminal, a read image of the printout that has been read from the terminal; specifying the storage destination from a portion of the scanned image that corresponds to the second image; transmitting, to the terminal, display data for displaying the content stored in the storage location at a position on the display screen corresponding to a portion of the scanned image that corresponds to the first image; A content providing method, wherein the above is performed by a processor.

12. 12. The content providing method according to claim 11, wherein in the step of transmitting the display data, the processor transmits to the terminal the display data that displays the content in a state in which the content is superimposed on the scanned image displayed on the display screen.

13. acquiring setting information indicating settings related to display of the content by the processor; 13. The content providing method according to claim 11, wherein in the step of transmitting the display data, the processor transmits the display data for displaying the content in accordance with the setting details to the terminal.

14. updating the content stored in the storage location in association with the first image, performed by the processor; 14. The content providing method according to claim 11, wherein in the step of transmitting the display data to the terminal after updating the content, the processor transmits the display data that causes the updated content to be displayed at the position.

15. A program for causing a computer to execute each step included in the content providing method according to any one of claims 11 to 14.

16. A computer-readable recording medium, A recording medium on which a program for causing a computer to execute each step included in the content providing method according to any one of claims 11 to 14 is recorded.

Images