Program product, content editing method, and information processing device

Abstract

The invention provides a program product, a content editing method, and an information processing apparatus, which make editing of content easier. The program product causes a computer to execute: a first image in which a contour of a first object is represented by a line on the basis of a captured image obtained by capturing the first object; displaying an editing image obtained by superimposing a content image representing the content on at least a part of the first image; receiving an input for changing at least one of a position, a shape, and a size of a content image in the editing image; and changing at least one of a position, a shape, and a size of the content image based on the input to change.

Description

Technical Field

[0001] The present invention relates to a program product, a content editing method and an information processing device. Background Art

[0002] In projection mapping, which projects an image onto a projection target as a three-dimensional object using a device such as a projector, it is usually necessary to adjust the position, shape and size of the image so as to match the position, shape and size of the projection target.

[0003] For example, Patent Document 1 describes the following: a captured image is generated by capturing at least a portion of a projection image projected by a projector onto a projection object, and at least one of a position and a size of the projection image in the captured image is set. Here, a guide image that prompts execution of an operation for setting the image is displayed on a display device in an overlapping manner with the captured image.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2021-158625

[0005] In the technology described in Patent Document 1, it is difficult to visually confirm the outline of the projection object in the captured image displayed on the display device, so it is sometimes difficult to set the projection image to the position and size intended by the user. Summary of the invention

[0006] A program product in one embodiment of the present disclosure causes a computer to perform the following processing: displaying a first image in which the outline of the first object is represented by lines based on a photographed image obtained by photographing the first object; displaying an editing image in which a content image representing content is superimposed on at least a portion of the first image; accepting input for changing at least one of the position, shape, and size of the content image in the editing image; and changing at least one of the position, shape, and size of the content image based on the input for making the change.

[0007] A program product in another aspect of the present disclosure causes a computer to perform the following processing: based on a photographed image obtained by photographing a first object, displaying a first image that represents the outline of the first object by making the inner and outer colors of the outline different from each other; displaying an editing image that is obtained by overlapping a content image representing content on at least a portion of the first image; accepting input for changing at least one of the position, shape, and size of the content image in the editing image; and changing at least one of the position, shape, and size of the content image based on the input for making the change.

[0008] A content editing method according to one embodiment of the present invention includes: displaying a first image in which the outline of the first object is represented by lines based on a captured image obtained by capturing the first object; displaying an editing image in which a content image representing content is overlapped on at least a portion of the first image; accepting input for changing at least one of the position, shape and size of the content image in the editing image; and displaying the content image in a state in which at least one of the position, shape and size of the content image is changed based on the input for making the change.

[0009] An information processing device according to one embodiment of the present invention includes an input device, a display device, and a processing device, wherein the processing device performs the following processing: based on a captured image obtained by capturing a first object, causing the display device to display a first image in which a contour of the first object is represented by lines; causing the display device to display an editing image in which a content image representing content is superimposed on at least a portion of the first image; accepting, via the input device, an input for changing at least one of a position, a shape, and a size of the content image in the editing image; and changing at least one of a position, a shape, and a size of the content image based on the input for making the change. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a diagram schematically showing a system used in the content editing method according to the first embodiment.

[0011] Figure 2 This is a block diagram of the information processing device according to the first embodiment.

[0012] Figure 3 This is a flowchart showing the flow of the content editing method according to the first embodiment.

[0013] Figure 4 This is a diagram showing a display example for accepting acquisition of a captured image.

[0014] Figure 5 This is a diagram showing a display example of the first image.

[0015] Figure 6 A diagram showing a display example of an editing image.

[0016] Figure 7 This is a diagram showing a display example of a first display mode of an edited content image.

[0017] Figure 8 This is a diagram showing a display example of the second display mode of the edited content image.

[0018] Fig. 9 It is a diagram for explaining projection using edited content images.

[0019] Fig.10 It is a diagram showing a display example of an editing image in the second embodiment.

[0020] Fig.11 This is a diagram showing a display example of an edited content image when the first image is selected.

[0021] Fig.12 This is a diagram showing a display example of the edited content image when the second image is selected.

[0022] Fig.13 A diagram showing another display example of the first image.

[0023] Description of symbols

[0024] 10: camera; 20: projector; 30: information processing device; 31: storage device; 32: processing device; 32a: acquisition unit; 32b: display control unit; 32c: image editing unit; 32d: projector control unit; 32e: measurement unit; 33: communication device; 34: display device; 35: input device; 100: system; B1: button; B2: button; B3: button; B4: button; B5: button; B6: button; CUR: cursor; DGa: captured image data; DGb: line graph data; DGc: content image data; DGd: image data for editing; DGe: image data for projection; DGf: conversion data; GU: user interface image; GU-1: user interface image; GU-2: user interface image; GU-3: user interface image; GU-4: user interface image; GU-5: user interface image Image; Ga: captured image; Gaa: image; Gab: image; Gac: image; Gb: line graph; Gba: image; Gba-1: image (1st image); Gba-2: image (2nd image); Gbb: image; Gbc: image; Gc: content image; Gd: image for editing; Ge: image; Gea: image; Gs1: label; Gs2: label; OJa: object (1st object); OJb: object; OJc: object; PR: program; R1: area; R2: area; R3: area; RC: shooting area; RP: projection area; S1: step; S10: step; S11: step; S12: step; S13: step; S14: step; S2: step; S3: step; S4: step; S5: step; S6: step; S7: step; S8: step; S9: step; W: wall. DETAILED DESCRIPTION

[0025] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. In addition, in the accompanying drawings, the dimensions and scales of each part are appropriately different from the actual ones, and there are also parts that are schematically shown for easy understanding. In addition, the scope of the present disclosure is not limited to these embodiments unless there is a record of the subject matter of the present disclosure that is particularly limited in the following description.

[0026] 1. First Implementation Method

[0027] 1-1. Overview of system used in content editing method

[0028] Figure 1 1 is a diagram schematically showing a system 100 used in the content editing method of the first embodiment. The system 100 is a projection mapping system that projects an image based on the shape of an object OJa as a projection target, etc. The object OJa is an example of a "first object".

[0029] exist Figure 1 In the example shown, the object OJa is a plain white T-shirt worn on the object OJb. The object OJb is, for example, a torso or a human body model. The shapes, sizes, and positions of the objects OJa and OJb are not limited to Figure 1 The examples shown are arbitrary. The object OJa is any object as long as it is a projection object of projection mapping, and is not limited to a T-shirt. The object OJb is not limited to a torso or a human body model, and is used as needed, and may be omitted.

[0030] like Figure 1 As shown, the system 100 includes a camera 10, a projector 20, and an information processing device 30. Figure 1 , briefly describing each part of system 100.

[0031] The camera 10 is a digital camera having an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 generates captured image data DGa, which is described later, representing a captured image Ga, which is described later, obtained by capturing the objects OJa and Ojb. Here, the objects OJa and OJb are included in a capture area RC, which is a captureable area of ​​the camera 10.

[0032] exist Figure 1 In the example shown, in addition to the objects OJa and OJb, the imaging region RC also includes an object OJc. The object OJc is a screen set along the wall W behind the objects OJa and OJb when viewed from the camera 10 and the projector 20. The object OJc is used as needed and may be omitted. The installation position and installation posture of the camera 10 are not limited to Figure 1 The examples shown are arbitrary. In addition, the camera 10 may also be a part of the information processing device 30.

[0033] The projector 20 is a display device that projects an image onto the object OJa under the control of the information processing device 30. Here, the objects OJa, OJb, and OJc are included in the projection region RP, which is an area that the projector 20 can project.

[0034] exist Figure 1 In the example shown, the projection area RP is included in the imaging area RC. In addition, the projection area RP only needs to include the objects OJa and OJb, and may also have a portion that is not included in the imaging area RC. In addition, the installation position and installation posture of the projector 20 are not limited to Figure 1 The examples shown are arbitrary.

[0035] Although not shown, the projector 20 has an image processing circuit, a light source, a light modulator, and a projection optical system. The image processing circuit of the projector 20 is a circuit that controls the driving of the light modulator of the projector 20 according to information from the information processing device 30. The light source of the projector 20 includes, for example, a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, which emits red, green, and blue light, respectively. The light modulator of the projector 20 includes three light modulators arranged corresponding to red, green, and blue. The three light modulators are, for example, display panels such as a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (digital micromirror device). The three light modulators modulate the red, green, and blue light, respectively, according to the signal from the image processing circuit of the projector 20, to generate image lights of various colors. Such image lights of various colors are synthesized by the color synthesis optical system to become full-color image lights. The projection optical system of the projector 20 is an optical system including a projection lens, etc., which images the full-color image lights on the projection object and projects them.

[0036] The information processing device 30 is a computer that executes a content editing method described in detail below, and has functions of acquiring a captured image from the camera 10 , editing an image for projection by the projector 20 using the captured image, and controlling the operation of the projector 20 .

[0037] exist Figure 1 In the example shown, the information processing device 30 is a notebook computer. In addition, the information processing device 30 is not limited to a notebook computer, and may be, for example, a desktop computer, a smartphone, or a tablet terminal. In addition, when the information processing device 30 has a shooting function, the information processing device 30 may also serve as the camera 10.

[0038] As will be described in detail later, the information processing device 30 has a display device 34 and an input device 35, and a user interface image GU as a GUI (graphical user interface) image required for executing the content editing method is displayed on the display device 34. In addition, in the drawings, "user interface" is sometimes marked as "UI". On this basis, the information processing device 30 performs editing of the image for projection by the projector 20 according to the instruction from the user via the input device 35. The user interface image GU can accept an editing operation from the user to make the shape of the image projected from the projector 20 to the projection object consistent with the shape of the projection object. Here, the user interface image GU includes a line graph Gb described later, which represents the outline of the projection object with a line based on the captured image Ga described later obtained from the camera 10, so that the shape of the image projected from the projector 20 to the projection object is easy to be consistent with the shape of the projection object.

[0039] 1-2. Information processing device

[0040] Figure 2 FIG. 1 is a block diagram of the information processing device 30 according to the first embodiment. Figure 2 As shown, the information processing device 30 includes a storage device 31, a processing device 32, a communication device 33, a display device 34, and an input device 35. These are connected to each other so as to be able to communicate with each other.

[0041] The storage device 31 is a storage device that stores programs such as an operating system and application programs executed by the processing device 32 and data processed by the processing device 32. The storage device 31 includes, for example, a hard disk drive or a semiconductor memory. In addition, part or all of the storage device 31 may be a storage device external to the information processing device 30, or may be provided in an external device such as a server connected to the information processing device 30 via a communication network such as the Internet.

[0042] The storage device 31 stores a program PR, captured image data DGa, line graph data DGb, content image data DGc, editing image data DGd, projection image data Dge, and conversion data DGf.

[0043] The program PR is a program for executing the content editing method described in detail later. The captured image data DGa is data representing a captured image obtained from the camera 10. The captured image data DGa may be a captured image obtained during the measurement of the measuring unit 32e described later, or may be a captured image different from the captured image obtained during the measurement of the measuring unit 32e described later. The line graph data DGb is data representing the captured image shown in the captured image data DGa by a line graph. The content image data DGc is data representing a content image Gc, which represents the content. The content image is an image that represents a photo, a pattern, a color, or a combination thereof as content. The content image may also be a dynamic image or a still image. The content image data DGc may include at least one data representing the content image Gc. In the present embodiment, the content image data DGc includes a plurality of data files representing the content image Gc. The method of obtaining the data file representing the content image is arbitrary. For example, at least one of the data files may be pre-stored in the information processing device 30, or may be obtained from outside the information processing device 30. The editing image data DGd is data representing an image obtained by superimposing the content image Gc represented by the content image data DGc on the line graph represented by the line graph data DGb. The projection image data DGe is data representing an image projected onto the object OJa by the projector 20. The conversion data DGf is data representing a conversion matrix for projective conversion between the coordinate system of the image captured by the camera 10 and the coordinate system of the display panel of the projector 20.

[0044] The processing device 32 is a processing device having the function of controlling each part of the information processing device 30 and the function of processing various data. The processing device 32 is configured to include a processor such as a CPU (Central Processing Unit). In addition, the processing device 32 can be composed of a single processor or a plurality of processors. In addition, a part or all of the functions of the processing device 32 can also be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).

[0045] The communication device 33 is a communication device that can communicate with the projector 20 and the like. For example, the communication device 33 is a wired communication device such as a wired LAN (Local Area Network), USB (Universal Serial Bus), HDMI (High Definition Multimedia Interface), a wireless communication device such as LPWA (Low Power Wide Area), a wireless LAN including Wi-Fi, and Bluetooth. "HDMI", "Wi-Fi", and "Bluetooth" are registered trademarks. In addition, the communication device 33 may be able to communicate with the camera 10, or may be able to communicate with an external server and the like.

[0046] The display device 34 displays various images under the control of the processing device 32. The display device 34 is a display device including various display panels such as a liquid crystal display panel and an organic EL (electro-luminescence) display panel.

[0047] The input device 35 is an input device that receives operations from the user. For example, the input device 35 is configured to include a touch pad, a touch panel, or a pointing device such as a mouse. Here, when the input device 35 is configured to include a touch panel, it can also serve as the display device 34.

[0048] In the above information processing device 30, the processing device 32 realizes various functions by referring to the above-mentioned various data stored in the storage device 31 and executing the program PR stored in the storage device 31. Specifically, the processing device 32 functions as an acquisition unit 32a, a display control unit 32b, an image editing unit 32c, a projector control unit 32d, and a measuring unit 32e by executing the program PR. Therefore, the processing device 32 includes the acquisition unit 32a, the display control unit 32b, the image editing unit 32c, the projector control unit 32d, and the measuring unit 32e.

[0049] The acquisition unit 32a acquires various information from various devices connected to the information processing device 30 by controlling the operation of the communication device 33, and stores the acquired information in the storage device 31. For example, the acquisition unit 32a acquires captured image data DGa from the camera 10, or acquires content image data DGc from a server not shown.

[0050] The display control unit 32b controls the operation of the display device 34 to display various information on the display device 34. Specifically, the display control unit 32b displays a user interface image GU required for executing the content editing method described later on the display device 34. Here, the display control unit 32b causes the display device 34 to display a line graph shown by the line graph data DGb, or causes the display device 34 to display an editing image shown by the editing image data DGd.

[0051] The image editing unit 32c performs various processes required for the execution of the content editing method described in detail later. Specifically, the image editing unit 32c generates line graph data DGb based on the captured image data DGa, or generates editing image data DGd based on the line graph data DGb and the content image data DGc, or edits the content image shown in the editing image data DGd according to the input result from the user. Here, the generation of the line graph data DGb is performed, for example, by processing the captured image shown in the captured image data DGa using a known line graph generation technology. Therefore, the objects OJa, OJb, and OJc can be detected as long as their boundaries can be detected from the captured image Ga. That is, the objects OJa, OJb, and OJc are not limited to independent objects, but may be continuous at least in part, or may be planes. The generation of the editing image data DGd is performed, for example, by synthesizing the line graph data DGb and the content image data DGc. The content image Gc shown in the editing image data DGd is edited, for example, by editing the content image data DGc based on the input result from the user and then synthesizing the line drawing data DGb and the content image data DGc. In addition, in this specification, generating data related to an image is sometimes referred to as "generating an image."

[0052] Furthermore, the image editing unit 32c generates projection image data DGe by conversion using the conversion data DGf based on the edited content image data DGc. The generation of the projection image data DGe may also be performed by the projector control unit 32d.

[0053] The measuring unit 32e measures the projection surface of the projector 20 using the camera 10 and the projector 20, thereby generating the conversion data DGf. Specifically, the measuring unit 32e causes the projector 20 to sequentially project a plurality of measurement patterns onto the projection object, and causes the camera 10 to capture each of the projected measurement patterns, thereby acquiring a plurality of captured images. Then, based on the plurality of captured images, the measuring unit 32e performs correspondence between the coordinates of the measurement pattern in the coordinate system of the captured image of the camera 10 and the coordinates of the measurement pattern in the coordinate system of the display panel of the projector 20, thereby generating a conversion matrix for performing projective conversion of these coordinate systems. Thus, the conversion data DGf is obtained.

[0054] The projector control unit 32d controls the operation of the projector 20 to cause the projector 20 to display various information. Specifically, the projector control unit 32d causes the projector 20 to project the image indicated by the projection image data DGe.

[0055] 1-3. Content editing method

[0056] Figure 3 1 is a flowchart showing the flow of the content editing method according to the first embodiment. The content editing method is executed by the information processing device 30 described above.

[0057] Specifically, if Figure 3 As shown, first, in step S1 , when the program PR is started, the display control unit 32 b causes the display device 34 to display the user interface image GU.

[0058] Next, in step S2, the acquisition unit 32a acquires the content image data DGc from a server (not shown) by controlling the operation of the communication device 33. In addition, step S2 can be performed before step S5 or after step S3. When using the content image data DGc already stored in the storage device 31, step S2 can be omitted.

[0059] Next, in step S3, the acquisition unit 32a acquires the captured image data DGa from the camera 10 by controlling the operation of the communication device 33. Also, in step S3, the measurement unit 32e generates the converted data DGf.

[0060] Next, in step S4, the image editing unit 32c generates line graph data DGb based on the captured image data DGa. Also, in step S4, the display control unit 32b causes the display device 34 to display an image represented by the line graph data DGb.

[0061] Next, in step S5, the image editing unit 32c determines whether a content image has been selected. This determination is performed until a content image is selected (step S5: No). Figure 6 As described above, the selection of the content image is performed using the user interface image GU.

[0062] When the content image is selected (step S5 : Yes), in step S6 , the image editing unit 32 c generates editing image data DGd based on the data representing the selected content image and the line graph data DGb.

[0063] Then, in step S7, the display control unit 32b controls the operation of the display device 34 to display the editing image indicated by the editing image data DGd on the display device 34. This display is performed in the user interface image GU.

[0064] Next, in step S8, the image editing unit 32c determines whether there is an instruction for cutting the content image. This determination is performed until there is an instruction for cutting the content image (step S8: No). Figure 6 As described above, the cutting shape is instructed using the user interface image GU.

[0065] When there is an instruction for the cutout shape of the content image (step S8 : Yes), in step S9 , the image editing unit 32 c edits the content image displayed on the display device 34 based on at least one of the position, shape, and size based on the instruction.

[0066] Next, in step S10, the image editing unit 32c determines whether or not the outline shape of the content image is adjusted. Figure 7 As described above, this adjustment is performed using the user interface image GU.

[0067] When there is adjustment of the outline shape of the content image (step S10 : Yes), in step S11 , the image editing unit 32 c edits the outline shape of the content image displayed on the display device 34 based on the adjustment.

[0068] Then, or when the outline shape of the content image is not adjusted (step S10: No), the image editing unit 32c determines in step S12 whether there is an instruction to project the projector 20. When there is no instruction to project the projector 20 (step S12: No), the image editing unit 32c returns to the above step S10.

[0069] When there is an instruction to project the projector 20 (step S12: Yes), in step S13, the image editing unit 32c generates projection image data DGe based on the information on the position, shape, and size of the edited content image. Here, the projection image data DGe is generated based on the edited content image data DGc by conversion using the conversion data DGf.

[0070] Then, in step S14 , the projector control unit 32 d causes the projector 20 to project an image based on the projection image data DGe.

[0071] The above is the process of the content editing method. Figures 4 to 8 An example of the user interface image GU used in the content editing method will be described. Figures 4 to 8 User interface images GU-1 to GU-5 are shown as user interface images GU that change according to the progress of content editing. In the following, sometimes the user interface images GU-1 to GU-5 are not distinguished and are referred to as user interface images GU. In addition, the user interface image GU is not limited to Figures 4 to 8 Example shown.

[0072] Figure 4 2 is a diagram showing a display example for accepting acquisition of a captured image Ga. Figure 4 , a user interface image GU-1 displayed on the display device 34 after executing the above step S2 and before executing step S3 is shown. The user interface image GU-1 includes regions R1, R2 and buttons B1, B2.

[0073] The progress of content editing is displayed in region R1. Figure 4 In the example shown, items “SETTING”, “SCAN”, “CONTENT CREATION”, “PROJECT”, and “DONE” are displayed in region R1, and items to be executed or executed are displayed in a distinguishable manner from items not executed.

[0074] Here, the items of "Settings", "Scanning", "Content Creation", "Projection", and "Completion" are executed in this order. The item of "Settings" is a display indicating that guidance on the settings of the camera 10 and the projector 20 is being performed. The item of "Scanning" is a display indicating that the shape measurement of the projection object is being performed. The item of "Content Creation" is a display indicating that the content image is being edited. The item of "Projection" is a display indicating that projection by the projector 20 is being performed. "Completion" is a display indicating that content editing is completed.

[0075] Region R1 of the user interface image GU-1 indicates that "Scan" is the execution target, and a text "Execute 5-minute scan" is displayed in region R1 to prompt the start of "Scan". When the execution target is transferred from "Setting" to "Scan", the camera 10 can capture the projection object. When the execution target is transferred from "Setting" to "Scan", the acquisition unit 32a acquires the captured image data DGa by controlling the operation of the camera 10 in the above step S3.

[0076] Although not shown, when "Settings" is the execution target, the display control unit 32b displays a user interface image GU including an image for guiding the setting of the projector 20 so that the light of the projector 20 is irradiated to the projection object, or guiding the setting of the camera 10 so that the projection object can be photographed. In addition, when "Settings" is the execution target, a button for starting to execute content editing is displayed in the user interface image GU. When the user operates the button for starting to execute content editing, the execution target is transferred to "Scanning".

[0077] In region R2, a desired image is displayed according to the progress of content editing. In region R2 of user interface image GU-1, a captured image Ga shown in captured image data Dga is displayed. The captured image Ga includes an image Gaa representing object OJa, an image Gab representing object OJb, and an image Gac representing object OJc. Thus, the user can visually confirm the projection object included in the capturing region RC of camera 10 in region R2. In addition, for example, if an image captured when a monochrome image is projected from projector 20 is used as captured image Ga, the user can visually confirm in region R2 that there are projection objects in the capturing region RC of camera 10 and the projection region RP of projector 20, respectively. In addition, in this case, the projected image may not be monochrome. By using a monochrome image such as white, captured image Ga can be simplified.

[0078] Button B1 is a display for accepting a request to return the execution target item to the previous item. Figure 4 In the example shown, when the button B1 is operated, the execution target item becomes “settings”.

[0079] Button B2 is a display for accepting a request to start processing of the target item. Figure 4 In the example shown, when the button B2 is operated, "scan" which is the execution target is started.

[0080] When the "scan" is started, the measuring unit 32e controls the operation of the projector 20 so that it sequentially projects a plurality of measurement patterns onto the projection object, and controls the operation of the camera 10 so that it captures each measurement pattern projected onto the projection object. Thus, a plurality of captured images are obtained by capturing the measurement patterns by the camera 10. Then, the measuring unit 32e measures the surface of the projection object, i.e., the projection surface, based on the plurality of captured images. The measurement of the projection surface refers to the generation of a transformation matrix for performing a projective transformation between the coordinate system of the image captured by the camera 10 and the coordinate system of the display device of the projector 20. The measuring unit 32e generates the transformation matrix by matching the coordinates of the measurement pattern in the coordinate system of the image captured by the camera 10 with the coordinates of the measurement pattern in the coordinate system of the display device of the projector 20 based on the plurality of captured images.

[0081] As the measurement pattern, for example, a binary code pattern is used. A binary code pattern refers to an image used to express the coordinates of a display device using binary code. Binary code refers to a technique for expressing the value of each bit when an arbitrary numerical value is expressed in a binary number by turning a switch on and off. When a binary code pattern is used as a measurement pattern, the image projected by the projector 20 is equivalent to the switch, and a measurement pattern of the number of bits of the binary number representing the coordinate value is required. In addition, different measurement patterns are required for the longitudinal coordinate and the lateral coordinate, respectively. For example, when the resolution of the display panel of the projector 20 is 120×90, 120 and 90 are respectively expressed in 7-bit binary numbers, so 7 images are required to express the longitudinal coordinate, and 7 images are required to express the lateral coordinate.

[0082] In addition, when a binary code pattern is used as a measurement pattern, the robustness of the measurement is generally reduced due to the influence of interference light such as lighting. Therefore, when a binary code pattern is used as a measurement pattern, from the perspective of suppressing the influence of interference light and improving the robustness of the measurement, it is preferred to use a complementary pattern. A complementary pattern refers to an image after black and white are reversed.

[0083] In addition, the measurement pattern is not limited to the binary code pattern, and may be other structured light such as a dot pattern, a rectangular pattern, a polygonal pattern, a checkered pattern, a Gray code pattern, a phase shift pattern, or a random dot pattern.

[0084] Figure 5 This is a diagram showing a display example of the first line graph. Figure 5 , a user interface image GU-2 displayed on the display device 34 after executing the above step S4 and before executing step S5 is shown. The user interface image GU-2 includes buttons B3 and B4 instead of the buttons B1 and B2 of the user interface image GU-1.

[0085] Region R1 of the user interface image GU-2 indicates that "scanning" is the execution object, and a text "scanning completed" is displayed in region R1 to indicate that "scanning" has been completed. By completing "scanning", conversion data DGf is obtained. Then, in step S4, the image editing unit 32c generates line graph data DGb based on the captured image data DGa.

[0086] A line graph Gb shown in the line graph data DGb is displayed in the region R2 of the user interface image GU-2. The line graph Gb includes an image Gba representing the object OJa, an image Gbb representing the object OJb, and an image Gbc representing the object OJc. This can improve the visual confirmation of the outline of the projected object. Here, the images Gba, Gbb, and Gbc are respectively represented as closed areas divided by the lines of the line graph Gb. The image Gba is an example of the "first image".

[0087] In this way, the program PR causes the computer to execute, in the above-mentioned step S4 , the image Gba showing the outline of the object OJa by lines based on the captured image Ga obtained by capturing the object OJa.

[0088] Button B3 is a display for accepting a request to return to the state before the execution of "Scan". Button B4 is a display for accepting a request to advance the execution target item to the next item. Figure 5 In the example shown, when the button B4 is operated, the execution target is transferred to "content creation".

[0089] Figure 6 It is a diagram showing a display example of the editing image Gd. Figure 6 The user interface image GU-3 displayed on the display device 34 during the execution of the above-mentioned step S8 is shown. The user interface image GU-3 includes buttons B5 and B6 instead of the buttons B3 and B4 of the user interface image GU-2, and includes a region R3.

[0090] In the region R3, an image Gcg including a plurality of content images indicated by the content image data DGc is displayed. Each of the plurality of content images can be selected. Figure 6 , four content images are arranged in a matrix, and the lower left content image among the four content images is selected. The number of content images displayed in region R3 is not limited to Figure 6 The examples shown are arbitrary and may be less than 3 or more than 5, or may be a single one.

[0091] Region R1 of the user interface image GU-3 indicates that "Content Creation" is an execution target, and in region R1, after the content image is selected, a text "Please click to select the shape to be cut" is displayed to urge the designation of the cutting shape of the content image. In addition, although not shown in the figure, in region R1, before the display urging the designation of the cutting shape of the content image is displayed, a display urging the selection of the content image is performed in step S5.

[0092] In the region R2 of the user interface image GU-3, an editing image Gd indicated by the editing image data DGd is displayed. The editing image Gd is an image in which the line graph Gb indicated by the line graph data DGb and the content image Gc indicated by the content image data DGc are superimposed. Figure 6In the example shown, the content image Gc shown in the content image data DGc is superimposed on the entire area of ​​the line graph Gb shown in the line graph data DGb. The editing image Gd is generated in the above-mentioned step S6 and then displayed in the above-mentioned step S7. In addition, although not shown in the figure, in the execution of step S5, in the area R1 before the selection of the content image, only the line graph Gb shown in the line graph data DGb may be displayed, or an arbitrary content image Gc may be superimposed on the line graph Gb.

[0093] Thus, the program PR causes the computer to execute the display of the editing image Gd obtained by superimposing the content image Gc representing the content on at least a part of the image Gba in the above step S7. In the present embodiment, the content image Gc is displayed in the entire region R2, but the position, shape, and size of the content image Gc are not limited thereto. For example, in the initial state, the content image Gc smaller than the region R2 may be displayed, and the content image Gc and the image Gba may not overlap partially or completely.

[0094] Here, in the above step S8, the editing image Gd can be selected by the closed area divided by the lines of the line graph Gb. When this selection is made (step S8: Yes), the shape of the selected closed area is designated as the cut shape, and the above step S9 is executed. Figure 6 In the example shown, the selection is made by clicking on the desired closed area with the cursor CUR. Figure 6 , the case where a closed area corresponding to the image Gba representing the object OJa is selected is illustrated. It is also possible to select a closed area other than the image Gba, such as an area corresponding to the background, by clicking. In this case, the image representing the selected closed area is equivalent to the "first image", and the object corresponding to the selected closed area becomes the projection object.

[0095] Thus, the program PR causes the computer to execute the input of accepting at least one of the position, shape and size of the content image Gc in the editing image Gd in the above step S8. For example, the position of the content image Gc is changed by selecting a closed area. The shape and size of the content image Gc are changed by specifying a cut shape. Depending on the position, shape and size of the content image Gc before the selection, there may be a case where one of the position, shape and size does not change as a result of the selection. Here, the input in the above step S8 includes the input of selecting the image Gba.

[0096] In addition, when this selection is made or the cursor CUR is overlapped and focused on a closed area divided by the lines of the line graph Gb, the area is highlighted and displayed. Figure 6In the example shown, the cursor CUR is superimposed on the closed area corresponding to the image Gba representing the object OJa, and the emphasis display is performed by thickening the line representing the outline of the image Gba. In addition, the method of this emphasis display is not limited to Figure 6 The example shown may be a method of making the outline, overall color, brightness, etc. of the region of the image Gba different from those of other regions.

[0097] In this way, the program PR described above causes the computer to execute highlighting and displaying the image Gba when the image Gba is selected or focused on during the execution of the step S8 described above.

[0098] In addition, in the region R2, labels Gs1 and Gs2 are displayed for accepting the operation of switching the display mode. When the label Gs1 is operated, as the first display mode, the editing image Gd is displayed in the region R2 as described above. On the other hand, when the label Gs2 is operated, instead of the editing image Gd or on the basis of the editing image Gd, as the second display mode, the captured image Ga is displayed in the region R2 as described later. In addition, the labels Gs1 and Gs2 are a type of button for accepting the switching operation, and are not limited to the so-called label shape.

[0099] Thus, the program PR causes the computer to execute the following processing in the above-mentioned step S8: displaying the user interface image GU for selecting one of the first display mode and the second display mode; and displaying the editing image Gd when the first display mode is selected.

[0100] Button B5 is a display for accepting a request to return the execution target item to the item before "Content Creation", that is, "Scan". Button B6 is a display for accepting a request to advance the execution target item to the next item. Figure 6 In the example shown, when the button B6 is operated, the execution object is transferred to "Project". In addition, when the button B6 is operated without selecting any closed area, the processing of steps S9 to S11 may be omitted and the process may be transferred to step S13, thereby transferring the execution object to "Project", or the execution object may not be transferred but a text or the like may be displayed to notify that no shape to be cut is specified.

[0101] Figure 7 1 is a diagram showing a display example of a first display mode of the edited content image Gc. Figure 7FIG. 4 shows a user interface image GU-4 displayed on the display device 34 during the execution of the above steps S10 and S11. The user interface image GU-4 is the same as the above user interface image GU-3 except that the shape of the content image Gc of the editing image Gd displayed in the region R2 is different and the shape of the content image Gc can be adjusted. Figure 7 , a first display mode which is a display mode when an operation is performed on the above-mentioned tag Gs1 is shown.

[0102] In this way, the program PR described above causes the computer to execute and display the editing image Gd when the first display mode is selected during the execution of the above-mentioned steps S10 and S11.

[0103] Region R1 of user interface image GU- 4 indicates that “content creation” is an execution target, and text “content shape can be adjusted” indicating that the shape of the content image can be adjusted is displayed in region R1 .

[0104] The editing image Gd is displayed in the region R2 of the user interface image GU-4. Here, in the above step S9, the shape of the content image Gc of the editing image Gd is trimmed to match the shape of the image Gba, and the content image Gc is displayed in the region R2 of the user interface image GU-4 in the trimmed shape. In addition, the content image Gc in the trimmed shape can be displayed by editing the content image data DGc in a manner of cutting out the unnecessary part of the content image Gc, or by displaying a mask image that hides the unnecessary part of the content image Gc without editing the content image data DGc.

[0105] Thus, the program PR causes the computer to display the content image Gc in a state changed based on the input in the step S8 in the step S9. Specifically, when the program PR receives the input of the selected image Gba in the step S8, the program causes the computer to display the content image Gc overlapping the image Gba in a shape close to the shape of the image Gba.

[0106] Here, the content image Gc can accept the adjustment of the outline shape. Figure 7 In the example shown, a plurality of points are arranged along the outline of the content image Gc. The outline shape of the content image Gc is adjusted by selecting any one of the plurality of points using the cursor CUR and then moving the selected point. In addition, the display for adjusting the outline shape of the content image Gc is not limited to the display using the cursor CUR. Figure 7The example of the point shown can be any display that can accept the operation of selecting and moving part or all of the outline. In the user interface image GU-4, a different content image Gc can be selected by accepting the operation on the area R3. In this case, the newly selected content image Gc can also be displayed in the area R2 after being trimmed according to the shape of the content image Gc when the area R3 is operated.

[0107] Thus, the program PR described above causes the computer to execute the process of accepting input of changing at least one of the position, shape, and size of the content image Gc in the editing image Gd in the above-mentioned step S10 .

[0108] Here, the input in step S10 includes input for deforming the outline of the content image Gc. When receiving the input in step S10, the program PR causes the computer to display the content image Gc superimposed on the image Gba while deforming the outline of the content image Gc.

[0109] Figure 8 2 is a diagram showing a display example of the second display mode of the edited content image Gc. Figure 8 FIG. 4 shows a user interface image GU-5 displayed on the display device 34 during the execution of the above steps S10 and S11. The user interface image GU-5 is the same as the above user interface image GU-4 except that the captured image Ga is displayed in the region R2. Figure 8 , a second display mode is shown which is a display mode when an operation is performed on the above-mentioned tag Gs2.

[0110] Region R1 of the user interface image GU- 5 indicates that “content creation” is an execution target, and text “Next, project the created content” indicating that the content image can be projected is displayed in the region R1 .

[0111] In the region R2 of the user interface image GU-5, the captured image Ga is displayed superimposed on the editing image Gd. Here, the content image Gc is displayed in the region R2 of the user interface image GU-5 in the trimmed shape. In this way, by superimposing the trimmed content image Gc on the captured image Ga, an image close to the actual projection state of the content image Gc can be visually provided through the display of the display device 34.

[0112] As described above, the program PR causes the computer to display the content image Gc superimposed on at least a portion of the captured image Ga when the second display mode is selected during the execution of the above steps S10 and S11.

[0113] Fig. 9is a diagram for explaining the projection of the edited content image Gc. Fig. 9 As shown, the edited content image Gc is projected onto the object OJa by the projector 20.

[0114] As described above, the content editing method includes: displaying an image Gba in which the outline of the object OJa is represented by lines based on a captured image Ga obtained by capturing the object Oja; displaying an editing image Gd obtained by overlapping a content image Gc representing the content on at least a portion of the image Gba; accepting input for changing at least one of the position, shape and size of the content image Gc in the editing image Gd; and displaying the content image Gc in a state after the change based on the input for making the change.

[0115] In addition, as described above, the information processing device 30 includes the input device 35, the display device 34, and the processing device 32. Moreover, the processing device 32 causes the display device 34 to display the image Gba in which the outline of the object OJa is represented by lines, based on the captured image Ga obtained by capturing the object OJa. In addition, the processing device 32 causes the display device 34 to display the editing image Gd in which the content image Gc representing the content is superimposed on at least a part of the image Gba. In addition, the processing device 32 causes the input of changing at least one of the position, shape, and size of the content image Gc in the editing image Gd to be received through the input device 35. In addition, the processing device 32 causes the display device 34 to display the content image Gc in a changed state based on the input for the change.

[0116] In the above-mentioned content editing method, information processing device 30 or program PR, an image Gba is displayed in which the outline of the object OJa is represented by a line, so that the user can easily visually confirm the outline of the object OJa as the projection object represented by the image Gba. In addition, an editing image Gd obtained by overlapping the content image Gc on at least a part of the image Gba is displayed, so that the user can easily visually confirm the relationship between the position, shape and size of the content image Gc relative to the image Gba. On this basis, an input for changing at least one of the position, shape and size of the content image Gc in the editing image Gd is accepted, so that at least one of the position, shape and size of the content image Gc can be adjusted with high precision to match the outline of the object OJa. As a result, the editing of the content becomes easy, and the projection image can be set to the position and size desired by the user.

[0117] In the present embodiment, as described above, the input for making the change includes the input of selecting the image Gba. The program PR further causes the computer to execute: when the input of selecting the image Gba is accepted, the content image Gc is displayed superimposed on the image Gba in a shape close to the shape of the image Gba. Therefore, the object OJa to be the projection object can be specified by a simple operation such as inputting the selection image Gba, and the content image Gc can be displayed in a shape corresponding to the shape of the object OJa. In addition, by superimposing the content image Gc on the image Gba in a shape close to the shape of the image Gba, it is easy for the user to visually confirm the relationship between the position, shape, and size of the content image Gc relative to the image Gba.

[0118] In addition, as described above, the input for making the change includes the input for deforming the outline of the content image Gc. The program PR further causes the computer to execute: when the input for making the deformation is received, the content image Gc is displayed superimposed on the image Gba in a state where the outline of the content image Gc is deformed. Therefore, the content image Gc can be edited according to the user's intention.

[0119] Furthermore, as described above, the program PR causes the computer to execute the following processing: displaying a user interface image GU for selecting one of the first display mode and the second display mode; displaying an editing image Gd when the first display mode is selected; and displaying a content image Gc in a manner overlapping at least a portion of the captured image Ga when the second display mode is selected. Therefore, it is possible to select a state in which the content image Gc is displayed in a manner overlapping at least a portion of the image Gba and a state in which the content image Gc is displayed in a manner overlapping at least a portion of the captured image Ga according to the user's intention.

[0120] In addition, as described above, the program PR causes the computer to execute: when the image Gba is selected or focused, the image Gba is displayed with emphasis. Therefore, the visibility of the outline of the object OJa, which is the projection target, shown in the image Gba can be improved according to the user's intention.

[0121] 2. Second Implementation Method

[0122] In the following exemplary embodiments, elements having the same functions and effects as those of the first embodiment are assigned the same reference numerals as those used in the description of the first embodiment, and detailed descriptions thereof are appropriately omitted.

[0123] Fig.10 It is a diagram showing a display example of the editing image Gd in the second embodiment. Fig.10The user interface image GU-3 displayed on the display device 34 during the execution of the above-mentioned step S8 is shown. This embodiment is the same as the above-mentioned first embodiment except that there are two objects OJa that can be projection objects. Therefore, in this embodiment, two objects OJa are captured in the captured image Ga. Here, one object OJa of the two objects OJa is an example of the "first object", and the other object OJa is an example of the "second object".

[0124] In this embodiment, if Fig.10 As shown, the line graph Gb includes an image Gba-2 representing the object OJa corresponding to the second object in addition to the image Gba-1 representing the object OJa corresponding to the first object. The image Gba-1 is an example of the "first image", and the image Gba-2 is an example of the "second image". In this way, the displayed image Gba-1 includes the image Gba-2 representing the outline of the object OJa corresponding to the second object by lines in addition to the displayed image Gba-1.

[0125] Here, image Gba-1 and image Gba-2 are similar to image Gba in the first embodiment, and can accept operations such as designation of a cut shape. Thus, the input in step S8 of this embodiment includes input for selecting image Gba-1 or image Gba-2. Fig.10 In the example shown, the cursor CUR is superimposed on the closed region corresponding to the image Gba-1, and this emphasized display is performed by thickening the line indicating the outline of the image Gba-1.

[0126] Fig.11 3 is a diagram showing a display example of the content image Gc after editing when the image Gba-1 is selected. Fig.11 The user interface image GU- 6 displayed on the display device 34 during the execution of the above-mentioned steps S10 , S11 is shown.

[0127] In this embodiment, if Fig.11 As shown, when receiving an input of selecting the image Gba-1, the program PR causes the computer to display the content image Gc in a shape close to the shape of the image Gba-1 superimposed on the image Gba-1.

[0128] Fig.12 3 is a diagram showing a display example of the content image Gc after editing when the image Gba-2 is selected. Fig.12 The user interface image GU- 7 displayed on the display device 34 during the execution of the above-mentioned steps S10 , S11 is shown.

[0129] In this embodiment, if Fig.12As shown, the program PR causes the computer to execute, when receiving an input of selecting the image Gba-2, displaying the content image Gc superimposed on the image Gba-2 in a shape close to the shape of the image Gba-2.

[0130] According to the second embodiment described above, it is also possible to facilitate the editing of the content. In this embodiment, as described above, it is possible to select the input of the image Gba-1 or the image Gba-2, so even if there are multiple objects that can be the projection object, it is possible to edit the content image Gc based on the object that can be the projection object selected according to the user's intention.

[0131] 3. Modifications

[0132] The above-exemplified embodiments can be variously modified. The following examples illustrate specific modified embodiments that can be applied to the above-exemplified embodiments. Two or more embodiments arbitrarily selected from the following examples can be appropriately combined within the scope of no contradiction.

[0133] 3-1. Modification 1

[0134] In the above-described embodiment, an example in which the first image is a line drawing is described, but the present invention is not limited to this example, and the first image may be any image as long as the outline of the first object is easier to observe than the captured image.

[0135] Fig.13 is a diagram showing another display example of the first image. Fig.13 , the user interface image GU-2 displayed on the display device 34 after executing step S4 and before executing step S5 is shown. Fig.13 The user interface image GU-2 shown in FIG. 1 is the same as the user interface image GU-2 of the first embodiment except that the image Ge is displayed in the region R2 instead of the line graph Gb. The image Ge is an example of the “first image”. Fig.13 In the figure, for the sake of convenience, the display of images corresponding to objects OJb and OJc in the image Ge is omitted.

[0136] The image Ge includes an image Gea representing the object OJa. Here, the image Gea represents the outline of the object OJa by making the inside and outside colors of the outline different from each other. That is, by separately applying the inside color and the outside color of the outline of the object OJa shown in the image Gea, the outline is represented without using a thick or thin line. With such an image Gea, the visual confirmation of the outline of the object OJa as the projection object can also be improved. In addition, in Fig.13In order to facilitate drawing, the inside of the outline of the object OJa is shown in white and the outside is shown in black, but the colors of the inside and outside of the outline of the object OJa are not particularly limited as long as they are different colors. In addition, the image Ge can also be displayed overlapping with the above-mentioned line graph Gb. In addition, the number of colors and closed areas is not limited to two. For example, a third color can also be used to display an image representing the object OJb.

[0137] 3-2. Modification 2

[0138] In the above-described embodiment, an example in which the number of objects OJa that can be projection targets is one or two is described, but the present invention is not limited to this example, and the number may be three or more.

[0139] 3-3. Modification 3

[0140] In the above embodiment, a method of selecting one closed region as the first image is exemplified, but the present invention is not limited to this method, and two or more closed regions may be selected. For example, in the second embodiment, the image Gba-1 and the image Gba-2 may be selected.

[0141] 4. Notes

[0142] The following is a summary of the present disclosure.

[0143] (Supplementary Note 1) A program that causes a computer to perform the following processing: displaying a first image in which the outline of the first object is represented by lines based on a photographed image obtained by photographing the first object; displaying an editing image in which a content image representing content is superimposed on at least a portion of the first image; accepting input for changing at least one of the position, shape and size of the content image in the editing image; and changing at least one of the position, shape and size of the content image based on the input for making the change.

[0144] In the above-mentioned method of supplementary note 1, the first image in which the outline of the first object is represented by a line is displayed, so that it is easy for the user to visually confirm the outline of the projection object, that is, the first object, shown in the first image. In addition, an editing image obtained by overlapping the content image on at least a part of the first image is displayed, so that it is easy for the user to visually confirm the relationship between the position, shape and size of the content image relative to the first image. On this basis, an input for changing at least one of the position, shape and size of the content image in the editing image is accepted, so that at least one of the position, shape and size of the content image can be adjusted with high precision to match the outline of the first object. As a result, the editing of the content becomes easy, and the projection image can be set to the position and size desired by the user.

[0145] (Note 2) A program according to Note 1, wherein the input for making the change includes an input for selecting the first image, and the change includes: when the input for selecting the first image is accepted, displaying the content image overlapping the first image in a shape close to a shape of the first image.

[0146] In the above-mentioned method of Supplement 2, an object to be projected can be designated by a simple operation of inputting the selection of the first image, and the content image can be displayed in a shape corresponding to the shape of the object. In addition, by overlapping and displaying the content image on the first image in a shape close to the shape of the first image, it is easy for the user to visually confirm the relationship between the position, shape and size of the content image with respect to the first image.

[0147] (Note 3) According to the program described in Note 1 or Note 2, the captured image captures the first object and the second object, displaying the first image includes displaying the first image and the second image in which the outline of the second object is represented by a line, and the input for making the change includes an input for selecting the first image or the second image, and making the change includes: when the input for selecting the first image is accepted, the content image is displayed on the first image in a shape close to the shape of the first image; and when the input for selecting the second image is accepted, the content image is displayed on the second image in a shape close to the shape of the second image.

[0148] In the above-described embodiment of Supplement 3, even when there are a plurality of objects that can be projection targets, it is possible to edit the content image after selecting an object to be projection according to the user's intention.

[0149] (Note 4) A program according to any one of Notes 1 to 3, wherein the input for making the change includes an input for deforming the outline of the content image, and the change includes: deforming the outline of the content image when the input for deforming the change is accepted; and displaying the content image after the deformed outline overlapping the first image.

[0150] In the above-described embodiment of Supplement 4, the content image can be edited according to the user's intention.

[0151] (Note 5) A program according to any one of Notes 1 to 4, wherein the program further causes the computer to execute: displaying a user interface image for selecting either a first display mode or a second display mode; displaying the editing image when the first display mode is selected; and displaying the content image in a manner overlapping at least a portion of the captured image when the second display mode is selected.

[0152] In the above aspect of Supplementary Note 5, the state in which the content image is displayed superimposed on at least a portion of the first image and the state in which the content image is displayed superimposed on at least a portion of the captured image can be selected according to the user's intention.

[0153] (Supplementary Note 6) The program according to any one of Supplementary Notes 1 to 5, further causing a computer to execute: when the first image is selected or when the first image is focused, highlighting and displaying the first image.

[0154] In the aspect of Supplementary Note 6 described above, the visibility of the outline of the first object, which is the projection target shown in the first image, can be improved according to the user's intention.

[0155] (Supplementary Note 7) A program that causes a computer to execute the following operations: based on a photographed image obtained by photographing a first object, displaying a first image that represents the outline of the first object by making the inner and outer colors of the outline of the first object different from each other; displaying an editing image that is obtained by overlapping a content image that represents content on at least a portion of the first image; accepting input for changing at least one of the position, shape, and size of the content image in the editing image; and changing at least one of the position, shape, and size of the content image based on the input for making the change.

[0156] In the above-mentioned method of supplementary note 7, the first image is displayed to represent the outline of the first object by making the colors of the inside and outside of the outline different from each other, so that it is easy for the user to visually confirm the outline of the projection object, that is, the first object, shown in the first image. In addition, an editing image is displayed in which the content image is superimposed on at least a part of the first image, so that it is easy for the user to visually confirm the relationship between the position, shape and size of the content image relative to the first image. On this basis, an input for changing at least one of the position, shape and size of the content image in the editing image is accepted, so that at least one of the position, shape and size of the content image can be adjusted with high precision to match the outline of the first object. As a result, the editing of the content becomes easy, and the projection image can be set to the position and size desired by the user.

[0157] (Supplementary Note 8) A content editing method, comprising: displaying a first image in which the outline of the first object is represented by lines based on a photographed image obtained by photographing the first object; displaying an editing image in which a content image representing content is overlapped on at least a portion of the first image; accepting an input for changing at least one of the position, shape and size of the content image in the editing image; and displaying the content image in a state in which at least one of the position, shape and size of the content image is changed based on the input for making the change.

[0158] In the above-mentioned method of supplementary note 8, the first image in which the outline of the first object is represented by a line is displayed, so that it is easy for the user to visually confirm the outline of the projection object, that is, the first object, shown in the first image. In addition, an editing image obtained by overlapping the content image on at least a part of the first image is displayed, so that it is easy for the user to visually confirm the relationship between the position, shape and size of the content image relative to the first image. On this basis, an input for changing at least one of the position, shape and size of the content image in the editing image is accepted, so that at least one of the position, shape and size of the content image can be adjusted with high precision to match the outline of the first object. As a result, the editing of the content becomes easy, and the projection image can be set to the position and size desired by the user.

[0159] (Supplementary Note 9) An information processing device, comprising: an input device; a display device; and a processing device, wherein the processing device performs the following processing: based on a captured image obtained by capturing a first object, causing the display device to display a first image in which the outline of the first object is represented by lines; causing the display device to display an editing image in which a content image representing content is superimposed on at least a portion of the first image; accepting, via the input device, an input for changing at least one of the position, shape, and size of the content image in the editing image; and changing at least one of the position, shape, and size of the content image based on the input for making the change.

[0160] In the above-mentioned method of supplementary note 9, the first image in which the outline of the first object is represented by a line is displayed, so that it is easy for the user to visually confirm the outline of the projection object, that is, the first object, shown in the first image. In addition, an editing image obtained by overlapping the content image on at least a part of the first image is displayed, so that it is easy for the user to visually confirm the relationship between the position, shape and size of the content image relative to the first image. On this basis, an input for changing at least one of the position, shape and size of the content image in the editing image is accepted, so that at least one of the position, shape and size of the content image can be adjusted with high precision to match the outline of the first object. As a result, the editing of the content becomes easy, and the projection image can be set to the position and size desired by the user.

Claims

1. A program product that causes a computer to execute the following processing: Based on a captured image obtained by capturing a first object, displaying a first image in which a contour of the first object is represented by a line; displaying an editing image in which a content image representing content is superimposed on at least a portion of the first image; receiving an input for changing at least one of a position, a shape, and a size of the content image in the editing image; as well as At least one of the position, shape, and size of the content image is changed based on the input for making the change.

2. The program product according to claim 1, wherein The input for making the change includes an input for selecting the first image, The changing includes, when an input selecting the first image is accepted, displaying the content image superimposed on the first image in a shape close to a shape of the first image.

3. The program product according to claim 1, wherein: The captured image captures the first object and the second object, Displaying the first image includes displaying the first image and a second image in which a contour of the second object is represented by a line, The input for making the change includes an input for selecting the first image or the second image, Making the changes includes: When an input selecting the first image is received, the content image is displayed superimposed on the first image in a shape close to a shape of the first image; as well as When an input selecting the second image is accepted, the content image is displayed superimposed on the second image in a shape close to a shape of the second image.

4. The program product according to claim 1, wherein: The input for making the change includes an input for deforming the outline of the content image, Making the changes includes: When receiving the input for performing the deformation, deforming the outline of the content image; and The content image after deforming the outline is displayed superimposed on the first image.

5. The program product according to any one of claims 1 to 4, wherein: The program product also causes the computer to perform the following processing: displaying a user interface image for selecting either the first display mode or the second display mode; When the first display mode is selected, displaying the editing image; as well as When the second display mode is selected, the content image is displayed superimposed on at least a portion of the captured image.

6. The program product according to claim 1, wherein: The program product further causes the computer to execute a process of emphasizing and displaying the first image when the first image is selected or when the first image is focused.

7. A program product that causes a computer to execute the following processing: Based on a captured image obtained by capturing a first object, displaying a first image that represents the outline of the first object by making the inner and outer colors of the outline of the first object different from each other; displaying an editing image in which a content image representing content is superimposed on at least a portion of the first image; receiving an input for changing at least one of a position, a shape, and a size of the content image in the editing image; as well as At least one of the position, shape, and size of the content image is changed based on the input for making the change.

8. A content editing method, comprising: Based on a captured image obtained by capturing a first object, displaying a first image in which a contour of the first object is represented by a line; displaying an editing image in which a content image representing content is superimposed on at least a portion of the first image; receiving an input for changing at least one of a position, a shape, and a size of the content image in the editing image; as well as The content image is displayed in a state where at least one of the position, shape, and size of the content image is changed based on the input for making the change.

9. An information processing device, comprising: Input device; Display device; as well as processing device, The processing device performs the following processing: Based on a captured image obtained by capturing a first object, causing the display device to display a first image in which a contour of the first object is represented by lines; causing the display device to display an editing image obtained by superimposing a content image representing content on at least a portion of the first image; receiving, via the input device, an input for changing at least one of a position, a shape, and a size of the content image in the editing image; as well as At least one of the position, shape, and size of the content image is changed based on the input for making the change.

Images