Program, content editing method, and information processing device
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- SEIKO EPSON CORP
- Filing Date
- 2023-08-08
- Publication Date
- 2026-06-02
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a program, a content editing method, and an information processing device. [Background technology]
[0002] In projection mapping, which uses equipment such as a projector to project an image onto a three-dimensional object, it is generally necessary to adjust the position, shape, and size of the image to match those of the object to be projected.
[0003] For example, Patent Document 1 describes a method of generating a captured image by capturing at least a part of a projection image projected by a projector onto a projection target, and setting at least one of the position and size of the projection image in the captured image. Here, a guide image that prompts the user to perform an operation for the setting is displayed on a display device superimposed on the captured image. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Patent Publication No. 2021-158625 Summary of the Invention [Problem to be solved by the invention]
[0005] In the technology described in Patent Document 1, since the contour of the projection target object is difficult to visually recognize in the captured image displayed on the display device, it may be difficult for the user to set the projection image at the position and size intended by the user. [Means for solving the problem]
[0006] A program according to one embodiment of the present disclosure causes a computer to perform the following operations: display a first image showing the outline of a first object based on an image captured of the first object; display an editing image in which a content image showing content is superimposed on at least a portion of the first image; accept input to change at least one of a position, shape, and size of the content image in the editing image; and change at least one of the position, shape, and size of the content image based on the input to change the first image.
[0007] A program according to another embodiment of the present disclosure causes a computer to perform the following operations: display a first image showing a contour of a first object by making the inside and outside of the contour different colors based on an image of the first object; display an editing image in which a content image showing content is superimposed on at least a portion of the first image; accept input to change at least one of a position, shape, and size of the content image in the editing image; and change at least one of the position, shape, and size of the content image based on the input to change the contour.
[0008] A content editing method according to one embodiment of the present disclosure includes displaying a first image showing an outline of a first object based on an image captured of the first object, displaying an editing image in which a content image showing content is superimposed on at least a portion of the first image, accepting input to change at least one of a position, shape, and size of the content image in the editing image, and displaying at least one of the position, shape, and size of the content image in a changed state based on the input to change the content image.
[0009] An information processing device according to one embodiment of the present disclosure includes an input device, a display device, and a processing device, and the processing device performs the following operations: displaying a first image on the display device, the first image showing the outline of a first object with lines based on an captured image of the first object; displaying an editing image on the display device in which a content image showing content is superimposed on at least a portion of the first image; accepting input via the input device for changing at least one of a position, shape, and size of the content image in the editing image; and changing at least one of the position, shape, and size of the content image based on the input for change. [Brief description of the drawings]
[0010] [Figure 1] 1 is a diagram showing an overview of a system used in a content editing method according to a first embodiment. [Diagram 2] 1 is a block diagram of an information processing device according to a first embodiment. [Diagram 3] 4 is a flowchart showing a flow of a content editing method according to the first embodiment. [Figure 4] FIG. 13 is a diagram showing an example of a display for accepting acquisition of a captured image. [Diagram 5] FIG. 11 is a diagram showing a display example of a first image. [Figure 6] FIG. 13 is a diagram showing a display example of an image for editing. [Figure 7] FIG. 13 is a diagram showing a display example of a first display mode of a content image after editing. [Figure 8] 13 is a diagram showing a display example of a second display mode of a content image after editing. FIG. [Figure 9] 13 is a diagram for explaining projection using an edited content image. FIG. [Figure 10] FIG. 11 is a diagram showing a display example of an image for editing in the second embodiment. [Figure 11] FIG. 13 is a diagram showing an example of how the edited content image is displayed when the first image is selected. [Figure 12]FIG. 13 is a diagram showing an example of how the edited content image is displayed when the second image is selected. [Figure 13] FIG. 13 is a diagram showing another display example of the first image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the dimensions and scale of each part in the drawings may differ from the actual dimensions, and some parts are shown diagrammatically to facilitate understanding. In addition, the scope of the present disclosure is not limited to these forms unless otherwise specified in the following description to the effect that the present disclosure is limited. 1. First embodiment 1-1. Overview of the system used for the content editing method 1 is a diagram showing an outline of a system 100 used in a content editing method according to the first embodiment. The system 100 is a projection mapping system that projects an image so as to match the shape of an object OJa, which is a projection target. The object OJa is an example of a "first object."
[0012] In the example shown in FIG. 1, the object OJa is a plain T-shirt, such as white, worn by the object OJb. The object OJb is, for example, a torso or a mannequin. The shapes, sizes, positions, and other aspects of the objects OJa and OJb are not limited to the example shown in FIG. 1 and are arbitrary. The object OJa is not limited to a T-shirt and is arbitrary as long as it is an object onto which projection mapping is performed. The object OJb is not limited to a torso or a mannequin and may be used or omitted as necessary.
[0013] As shown in Fig. 1, the system 100 includes a camera 10, a projector 20, and an information processing device 30. Each unit of the system 100 will be briefly described below with reference to Fig. 1.
[0014] The camera 10 is a digital camera having an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 generates captured image data DGa, which will be described later, indicating a captured image Ga, which will be described later, capturing an image of objects OJa and OJb. Here, the objects OJa and OJb are included in an imaging region RC, which is an area that the camera 10 can capture.
[0015] In the example shown in FIG. 1, the imaging region RC includes an object OJc in addition to the objects OJa and OJb. The object OJc is a screen installed along a wall W located behind the objects OJa and OJb when viewed from the camera 10 and the projector 20. The object OJc may be used as necessary and may be omitted. The installation position and installation posture of the camera 10 are not limited to the example shown in FIG. 1 and may be arbitrary. Furthermore, the camera 10 may be a part of the information processing device 30.
[0016] The projector 20 is a display device that projects an image onto an object OJa under the control of an information processing device 30. Here, a projection area RP, which is an area where the projector 20 can project, includes the objects OJa, OJb, and OJc.
[0017] In the example shown in Fig. 1, the projection area RP is included in the imaging area RC. The projection area RP only needs to include the objects OJa and OJb, and may have a portion that is not included in the imaging area RC. The installation position and installation posture of the projector 20 are not limited to the example shown in Fig. 1, and are arbitrary.
[0018] Although not shown, the projector 20 has an image processing circuit, a light source, a light modulation device, and a projection optical system. The image processing circuit of the projector 20 is a circuit that controls the driving of the light modulation device of the projector 20 based on information from the information processing device 30. The light source of the projector 20 includes, for example, a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, and emits red, green, and blue light, respectively. The light modulation device of the projector 20 includes three light modulation elements provided corresponding to red, green, and blue. Each of the three light modulation elements is, for example, a display panel such as a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (digital mirror device). The three light modulation elements modulate the red, green, and blue light, respectively, based on a signal from the image processing circuit of the projector 20 to generate image light of each color. Such image light of each color is synthesized by a color synthesis optical system to become full-color image light. The projection optical system of the projector 20 is an optical system including a projection lens and the like, and forms and projects the above-mentioned full-color image light onto a projection target.
[0019] The information processing device 30 is a computer that executes a content editing method described in detail later, and has the functions of acquiring captured images from the camera 10, using the captured images to edit images to be projected by the projector 20, and controlling the operation of the projector 20.
[0020] 1, the information processing device 30 is a notebook computer. Note that the information processing device 30 is not limited to a notebook computer, and may be, for example, a desktop computer, a smartphone, a tablet terminal, or the like. In addition, when the information processing device 30 has an imaging function, the information processing device 30 may also function as the camera 10.
[0021] As will be described later in detail, the information processing device 30 has a display device 34 and an input device 35, and displays a user interface image GU, which is a GUI (graphical user interface) image required to execute the content editing method, on the display device 34. In the drawings, the "user interface" may be written as "UI". Then, the information processing device 30 executes editing of an image to be used for projection by the projector 20 based on an instruction from a user via the input device 35. The user interface image GU can receive an editing operation from a user to match the shape of an image projected from the projector 20 onto a projection object to the shape of the projection object. Here, the user interface image GU includes a line drawing Gb, which is described later and shows the outline of the projection object with a line based on a captured image Ga, which is described later and is acquired from the camera 10, so that the shape of the image projected from the projector 20 onto the projection object can be easily matched to the shape of the projection object.
[0022] 1-2. Information processing device Fig. 2 is a block diagram of an information processing device 30 according to the first embodiment. As shown in Fig. 2, the information processing device 30 includes a storage device 31, a processing device 32, a communication device 33, a display device 34, and an input device 35. These are connected to each other so as to be able to communicate with each other.
[0023] The storage device 31 is a storage device that stores programs such as an operating system and application programs executed by the processing device 32, and data processed by the processing device 32. The storage device 31 includes, for example, a hard disk drive or a semiconductor memory. Note that a part or all of the storage device 31 may be an external storage device of the information processing device 30, or may be provided in an external device such as a server connected to the information processing device 30 via a communication network such as the Internet.
[0024] The storage device 31 stores a program PR, captured image data DGa, line drawing data DGb, content image data DGc, edit image data DGd, projection image data DGe, and conversion data DGf.
[0025] The program PR is a program for executing a content editing method described later in detail. The captured image data DGa is data indicating a captured image acquired from the camera 10. The captured image data DGa may be a captured image obtained during measurement by a measuring unit 32e described later, or may be a captured image separate from the captured image obtained during measurement by a measuring unit 32e described later. The line drawing data DGb is data expressing the captured image indicated by the captured image data DGa in line drawing. The content image data DGc is data indicating a content image Gc indicating a content. The content image is an image indicating a photograph, a pattern, a color, or a combination of these as a content. The content image may be a video or a still image. The content image data DGc may include at least one data indicating the content image Gc. In this embodiment, the content image data DGc includes a plurality of data files indicating the content image Gc. The method of acquiring the data file indicating the content image is arbitrary. For example, at least one of the data files may be stored in the information processing device 30 in advance, or may be acquired from outside the information processing device 30. The editing image data DGd is data representing an image obtained by superimposing the content image Gc represented by the content image data DGc on the line drawing represented by the line drawing data DGb. The projection image data DGe is data representing an image projected onto the object OJa by the projector 20. The conversion data DGf is data representing a conversion matrix for performing projective conversion between the coordinate system of the captured image of the camera 10 and the coordinate system of the display panel of the projector 20.
[0026] The processing device 32 is a processing device having a function of controlling each part of the information processing device 30 and a function of processing various data. The processing device 32 is configured to include a processor such as a CPU (Central Processing Unit). The processing device 32 may be configured with a single processor or multiple processors. In addition, some or all of the functions of the processing device 32 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).
[0027] The communication device 33 is a communication device capable of communicating with the projector 20 and the like. For example, the communication device 33 is a wired communication device such as a wired LAN (Local Area Network), a Universal Serial Bus (USB), or an HDMI (High Definition Multimedia Interface), or a wireless communication device such as a LPWA (Low Power Wide Area), a wireless LAN including Wi-Fi, or Bluetooth. "HDMI", "Wi-Fi", and "Bluetooth" are each registered trademarks. The communication device 33 may be capable of communicating with the camera 10, or may be capable of communicating with an external device such as a server.
[0028] The display device 34 displays various images under the control of the processing device 32. The display device 34 is a display device including various display panels such as a liquid crystal display panel and an organic EL (electro-luminescence) display panel.
[0029] The input device 35 is an input device that accepts operations from a user. For example, the input device 35 includes a touch pad, a touch panel, or a pointing device such as a mouse. Here, when the input device 35 includes a touch panel, it may also function as the display device 34.
[0030] In the above information processing device 30, the processing device 32 realizes various functions by referring to the above-mentioned various data stored in the storage device 31 and by executing the program PR stored in the storage device 31. Specifically, the processing device 32 functions as an acquisition unit 32a, a display control unit 32b, an image editing unit 32c, a projector control unit 32d, and a measurement unit 32e by executing the program PR. Therefore, the processing device 32 includes the acquisition unit 32a, the display control unit 32b, the image editing unit 32c, the projector control unit 32d, and the measurement unit 32e.
[0031] The acquisition unit 32a controls the operation of the communication device 33 to acquire various pieces of information from various devices connected to the information processing device 30, and stores the acquired information in the storage device 31. For example, the acquisition unit 32a acquires captured image data DGa from the camera 10, and acquires content image data DGc from a server (not shown) or the like.
[0032] The display control unit 32b controls the operation of the display device 34 to cause the display device 34 to display various information. Specifically, the display control unit 32b causes the display device 34 to display a user interface image GU required for executing a content editing method described below. Here, the display control unit 32b causes the display device 34 to display a line drawing indicated by the line drawing data DGb, and causes the display device 34 to display an editing image indicated by the editing image data DGd.
[0033] The image editing unit 32c executes various processes required for executing the content editing method described later in detail. Specifically, the image editing unit 32c generates line drawing data DGb based on the captured image data DGa, generates editing image data DGd based on the line drawing data DGb and the content image data DGc, and edits the content image indicated by the editing image data DGd based on the input result from the user. Here, the line drawing data DGb is generated, for example, by processing the captured image indicated by the captured image data DGa using a known line drawing generation technique. Therefore, the objects OJa, OJb, and OJc may be objects whose boundaries can be detected from the captured image Ga. In other words, the objects OJa, OJb, and OJc are not limited to objects independent of each other, and may be at least partially continuous or flat. The editing image data DGd is generated, for example, by combining the line drawing data DGb and the content image data DGc. The content image Gc indicated by the editing image data DGd is edited, for example, by editing the content image data DGc based on the input from the user, and then synthesizing the line drawing data DGb and the content image data DGc. Note that in this specification, generating data related to an image may be referred to as "generating an image."
[0034] Furthermore, the image editing unit 32c generates the projection image data DGe by conversion using the conversion data DGf based on the edited content image data DGc. Note that the generation of the projection image data DGe may be performed by the projector control unit 32d.
[0035] The measurement unit 32e generates the conversion data DGf by measuring the projection surface of the projector 20 using the camera 10 and the projector 20. Specifically, the measurement unit 32e acquires a plurality of captured images by having the projector 20 sequentially project a plurality of measurement patterns onto the projection target and having the camera 10 capture each of the projected measurement patterns. Then, based on the plurality of captured images, the measurement unit 32e associates the coordinates of the measurement patterns in the coordinate system of the captured image of the camera 10 with the coordinates of the measurement patterns in the coordinate system of the display panel of the projector 20, thereby generating a transformation matrix for performing projective transformation of these coordinate systems. As a result, the conversion data DGf is obtained.
[0036] The projector control unit 32d controls the operation of the projector 20 to cause the projector 20 to display various information. Specifically, the projector control unit 32d causes the projector 20 to project an image indicated by the projection image data DGe.
[0037] 1-3. How to edit content 3 is a flowchart showing the flow of the content editing method according to the first embodiment. The content editing method is executed by the information processing device 30 described above.
[0038] More specifically, as shown in FIG. 3, first, in step S1, the display control unit 32b causes the display device 34 to display the user interface image GU when the program PR is started.
[0039] Next, in step S2, the acquisition unit 32a acquires the content image data DGc from a server (not shown) or the like by controlling the operation of the communication device 33. Note that step S2 may be executed before step S5, or after step S3. When using the content image data DGc already stored in the storage device 31, step S2 may be omitted.
[0040] Next, in step S3, the acquisition unit 32a acquires the captured image data DGa from the camera 10 by controlling the operation of the communication device 33. Also in step S3, the measurement unit 32e generates the converted data DGf.
[0041] Next, in step S4, the image editing unit 32c generates line drawing data DGb based on the captured image data DGa. Also in step S4, the display control unit 32b causes the display device 34 to display an image represented by the line drawing data DGb.
[0042] Next, in step S5, the image editing unit 32c judges whether or not a content image is selected. This judgment is repeated until a content image is selected (step S5: NO). Here, the content image is selected using a user interface image GU, as will be described later with reference to FIG.
[0043] If a content image has been selected (step S5: YES), then in step S6, the image editing unit 32c generates editing image data DGd based on data indicating the selected content image and the line drawing data DGb.
[0044] After that, in step S7, the display control section 32b controls the operation of the display device 34 to cause the display device 34 to display the image for editing indicated by the image data for editing DGd. This display is performed within the user interface image GU.
[0045] Next, in step S8, the image editing unit 32c judges whether or not a cut shape for the content image is instructed. This judgment is executed until a cut shape for the content image is instructed (step S8: NO). Here, the cut shape is instructed using a user interface image GU, as will be described later with reference to FIG.
[0046] If an instruction for the cut shape of the content image is given (step S8: YES), in step S9, the image editing unit 32c edits the content image displayed on the display device 34 according to at least one of the position, shape, and size based on the instruction.
[0047] Next, in step S10, the image editing unit 32c determines whether or not to adjust the contour shape of the content image. This adjustment is performed using a user interface image GU, as will be described later with reference to FIG.
[0048] If the contour shape of the content image has been adjusted (step S10: YES), in step S11, the image editing unit 32c edits the contour shape of the content image displayed on the display device 34 in accordance with the adjustment.
[0049] After that, or if there is no adjustment to the contour shape of the content image (step S10: NO), the image editing unit 32c determines in step S12 whether or not there is an instruction to project by the projector 20. If there is no instruction to project by the projector 20 (step S12: NO), the image editing unit 32c returns to the above-mentioned step S10.
[0050] If a command to project using the projector 20 has been issued (step S12: YES), in step S13, the image editing unit 32c generates projection image data DGe based on information on the position, shape, and size of the edited content image. Here, the projection image data DGe is generated by conversion using the conversion data DGf based on the edited content image data DGc.
[0051] After that, in step S14, the projector control unit 32d causes the projector 20 to project an image based on the projection image data DGe.
[0052] The above is the flow of the content editing method. An example of a user interface image GU used in the content editing method will be described below with reference to Figs. 4 to 8. Note that Figs. 4 to 8 show user interface images GU-1 to GU-5 as user interface images GU that change according to the progress of content editing. Hereinafter, the user interface images GU-1 to GU-5 may be referred to as user interface images GU without distinguishing between them. Also, the user interface image GU is not limited to the examples shown in Figs. 4 to 8.
[0053] Fig. 4 is a diagram showing an example of a display for accepting acquisition of a captured image Ga. Fig. 4 shows a user interface image GU-1 that is displayed on the display device 34 after execution of the above-mentioned step S2 and before execution of step S3. The user interface image GU-1 includes regions R1 and R2 and buttons B1 and B2.
[0054] The progress of content editing is displayed in area R1. In the example shown in Fig. 4, the items "Installation", "Scan", "Content Creation", "Projection", and "Complete" are displayed in area R1, and items to be executed or executed are displayed in a distinguishable manner from items that have not been executed.
[0055] Here, the items "Installation," "Scan," "Content Creation," "Projection," and "Completion" are executed in this order. The "Installation" item is a display indicating that guidance regarding the installation of the camera 10 and projector 20 will be provided. The "Scan" item is a display indicating that shape measurement of the projection object will be performed. The "Content Creation" item is a display indicating that editing of a content image will be performed. The "Projection" item is a display indicating that projection will be performed by the projector 20. "Completion" is a display indicating that content editing has been completed.
[0056] Area R1 of user interface image GU-1 indicates that "scan" is the execution target, and area R1 displays the text "Scan will run for 5 minutes" to prompt the user to start "scan." When the execution target shifts from "installation" to "scan," the camera 10 can capture an image of the projection target. When the execution target shifts from "installation" to "scan," the acquisition unit 32a acquires captured image data DGa by controlling the operation of the camera 10 in the above-mentioned step S3.
[0057] Although not shown, when "Installation" is the execution target, the display control unit 32b displays a user interface image GU including an image for guiding the installation of the projector 20 so as to shine light from the projector 20 on the projection target, or for guiding the installation of the camera 10 so as to capture the projection target. When "Installation" is the execution target, a button for starting content editing is displayed in the user interface image GU. When the user operates the button for starting content editing, the execution target transitions to "Scan".
[0058] In the region R2, a necessary image is displayed according to the progress of the content editing. In the region R2 of the user interface image GU-1, a captured image Ga indicated by the captured image data DGa is displayed. The captured image Ga includes an image Gaa indicating an object OJa, an image Gab indicating an object OJb, and an image Gac indicating an object OJc. This allows the user to visually recognize the projection object included in the imaging region RC of the camera 10 in the region R2. Also, for example, if an image captured when a monochromatic image is projected from the projector 20 over the entire surface is used as the captured image Ga, the user can visually recognize in the region R2 that the projection object exists in each of the imaging region RC of the camera 10 and the projection region RP of the projector 20. Note that the image to be projected in this case does not have to be monochromatic. By using a monochromatic image such as white, the captured image Ga can be simplified.
[0059] The button B1 is a display for accepting a command to return the item to be executed to the previous item. In the example shown in FIG. 4, when the button B1 is operated, the item to be executed becomes "Install."
[0060] The button B1 is a display for accepting the start of processing of the item to be executed. In the example shown in Fig. 4, when the button B2 is operated, the "scan" to be executed is started.
[0061] When the execution of "scan" is started, the measurement unit 32e controls the operation of the projector 20 to sequentially project a plurality of measurement patterns onto the projection object, and controls the operation of the camera 10 to capture each measurement pattern projected onto the projection object. This results in a plurality of captured images of the measurement patterns captured by the camera 10. The measurement unit 32e then measures the projection surface, which is the surface of the projection object, based on the plurality of captured images. Measuring the projection surface means generating a transformation matrix for performing projective transformation between the coordinate system of the captured image of the camera 10 and the coordinate system of the display device of the projector 20. The measurement unit 32e generates the transformation matrix by associating the coordinates of the measurement pattern in the coordinate system of the captured image of the camera 10 with the coordinates of the measurement pattern in the coordinate system of the display device of the projector 20 based on the plurality of captured images.
[0062] For example, a binary code pattern is used as the measurement pattern. The binary code pattern is an image for expressing the coordinates of a display device using a binary code. The binary code is a technique for expressing the value of each digit when an arbitrary numerical value is expressed in binary by turning a switch on and off. When a binary code pattern is used as the measurement pattern, the image projected by the projector 20 corresponds to the switch, and measurement patterns are required for the number of digits of the binary number expressing the coordinate value. In addition, separate measurement patterns are required for each of the vertical coordinate and the horizontal coordinate. For example, when the resolution of the display panel of the projector 20 is 120×90, 120 and 90 are each expressed as a 7-digit binary number, so 7 images are required to express the vertical coordinate and 7 images are required to express the horizontal coordinate.
[0063] Furthermore, when a binary code pattern is used as a measurement pattern, the robustness of the measurement generally decreases due to the influence of disturbance light such as lighting. Therefore, when a binary code pattern is used as a measurement pattern, it is preferable to use a complementary pattern in addition to the binary code pattern in order to reduce the influence of disturbance light and improve the robustness of the measurement. A complementary pattern is an image in which black and white are inverted.
[0064] The measurement pattern is not limited to a binary code pattern, and may be other structured light patterns such as a dot pattern, a rectangular pattern, a polygonal pattern, a checkered pattern, a gray code pattern, a phase shift pattern, or a random dot pattern.
[0065] Fig. 5 is a diagram showing a display example of the first line drawing. Fig. 5 shows a user interface image GU-2 displayed on the display device 34 after the execution of the above-mentioned step S4 and before the execution of step S5. User interface image GU-2 includes buttons B3 and B4 instead of buttons B1 and B2 of user interface image GU-1.
[0066] Area R1 of user interface image GU-2 indicates that "scan" is to be executed, and area R1 displays the text "Scan Completed" indicating that "scan" has been completed. Upon completion of "scan", converted data DGf is obtained. Then, in step S4, image editing unit 32c generates line drawing data DGb based on captured image data DGa.
[0067] A line drawing Gb represented by the line drawing data DGb is displayed in region R2 of user interface image GU-2. The line drawing Gb includes an image Gba representing object OJa, an image Gbb representing object OJb, and an image Gbc representing object OJc. This can improve the visibility of the contours of the projection target. Here, each of the images Gba, Gbb, and Gbc is represented as a closed area partitioned by the lines of the line drawing Gb. The image Gba is an example of a "first image."
[0068] In this way, the above-mentioned program PR causes the computer to execute, in the above-mentioned step S4, displaying an image Gba showing the contour of the object OJa with lines, based on a captured image Ga obtained by capturing an image of the object OJa.
[0069] Button B3 is an indication for accepting a return to the state before "Scan" was executed. Button B4 is an indication for accepting a move from the item to be executed to the next item. In the example shown in FIG. 5, when button B4 is operated, the item to be executed shifts to "Content Creation."
[0070] Fig. 6 is a diagram showing a display example of the editing image Gd. Fig. 6 shows a user interface image GU-3 that is displayed on the display device 34 during execution of the above-mentioned step S8. User interface image GU-3 includes buttons B5 and B6 instead of buttons B3 and B4 of user interface image GU-2, and also includes an area R3.
[0071] In the region R3, an image Gcg including a plurality of content images indicated by the content image data DGc is displayed. Each of the plurality of content images is capable of accepting selection. In FIG. 6, an example is shown in which four content images are arranged in a matrix, and the state in which the lower left content image of the four content images is selected is shown. Note that the number of content images displayed in the region R3 is not limited to the example shown in FIG. 6, and is arbitrary, and may be three or less, five or more, or may be a single content image.
[0072] Area R1 of user interface image GU-3 indicates that "content creation" is to be executed, and area R1 displays the text "Tap to select the shape to cut", which indicates that the cut shape of the content image is to be specified after the content image is selected. Note that, although not shown, area R1 displays a display prompting the selection of a content image in step S5, before the display prompting the specification of the cut shape of the content image.
[0073] In the region R2 of the user interface image GU-3, an editing image Gd indicated by the editing image data DGd is displayed. The editing image Gd is an image in which the line drawing Gb indicated by the line drawing data DGb and the content image Gc indicated by the content image data DGc are superimposed. In the example shown in FIG. 6, the content image Gc indicated by the content image data DGc is superimposed over the entire area of the line drawing Gb indicated by the line drawing data DGb. The editing image Gd is generated in the above-mentioned step S6 and then displayed in the above-mentioned step S7. Although not shown, in the region R1 before the selection of the content image during the execution of step S5, only the line drawing Gb indicated by the line drawing data DGb may be displayed, or any content image Gc may be superimposed on the line drawing Gb.
[0074] In this way, the aforementioned program PR causes the computer to execute the above-mentioned step S7 to display an editing image Gd in which a content image Gc showing the content is superimposed on at least a part of the image Gba. In this embodiment, the content image Gc is displayed in the entire area R2, but the position, shape, and size of the content image Gc are not limited to this. For example, in the initial state, a content image Gc smaller than the area R2 may be displayed, and the content image Gc and the image Gba may not overlap in part or in whole.
[0075] Here, in the above-mentioned step S8, the image for editing Gd can receive the selection of a closed area partitioned by the lines of the line drawing Gb. When this selection is made (step S8: YES), the shape of the selected closed area is designated as the cut shape, and the above-mentioned step S9 is executed. In the example shown in FIG. 6, the desired closed area is selected by tapping with the cursor CUR. FIG. 6 illustrates the case where a closed area corresponding to an image Gba showing an object OJa is selected. It is also possible to select a closed area other than the image Gba, for example, an area corresponding to the background, by tapping. In this case, the image representing the selected closed area corresponds to the "first image", and the object corresponding to the selected closed area becomes the projection target.
[0076] Thus, the aforementioned program PR causes the computer to execute, in the aforementioned step S8, receiving an input for changing at least one of the position, shape, and size of the content image Gc in the editing image Gd. For example, the position of the content image Gc is changed by selecting a closed region. The shape and size of the content image Gc are changed by specifying a cut shape. Depending on the position, shape, and size of the content image Gc before the selection, it is possible that any of the position, shape, and size may not change as a result of the selection. Here, the input in the aforementioned step S8 includes an input for selecting the image Gba.
[0077] Also, when the selection is performed or when the cursor CUR is placed over a closed region partitioned by the lines of the line drawing Gb and focused, the region is highlighted. In the example shown in Fig. 6, the cursor CUR is placed over a closed region corresponding to the image Gba showing the object OJa, and the highlighting is performed by thickening the line showing the outline of the image Gba. Note that the method of highlighting is not limited to the example shown in Fig. 6, and may be, for example, a method of making the outline or the overall color or brightness of the region of the image Gba different from other regions.
[0078] Thus, the aforementioned program PR causes the computer to highlight the image Gba if the image Gba is selected or focused during execution of the aforementioned step S8.
[0079] Also, in the region R2, tabs Gs1 and Gs2 that accept an operation to switch the display mode are displayed. When an operation is performed on the tab Gs1, the image for editing Gd is displayed in the region R2 as the first display mode, as described above. On the other hand, when an operation is performed on the tab Gs2, instead of or in addition to the image for editing Gd, the captured image Ga is displayed in the region R2 as the second display mode, as described below. Note that the tabs Gs1 and Gs2 are a type of button that accepts a switching operation, and are not limited to a so-called tab shape.
[0080] In this way, the aforementioned program PR causes the computer to execute, in the aforementioned step S8, displaying a user interface image GU for selecting either the first display mode or the second display mode, and, if the first display mode is selected, displaying an editing image Gd.
[0081] Button B5 is a display for accepting that the item to be executed is to be returned to "scan", which was the item before "content editing". Button B6 is a display for accepting that the item to be executed is to be advanced to the next item. In the example shown in FIG. 6, when button B6 is operated, the execution target shifts to "projection". Note that, when button B6 is operated when no closed area is selected, the processing from steps S9 to S11 may be omitted and the process may shift to step S13, thereby shifting the execution target to "projection", or text or the like may be displayed to notify that the shape to be cut has not been specified without shifting the execution target.
[0082] Fig. 7 is a diagram showing a display example of the first display mode of the edited content image Gc. Fig. 7 shows a user interface image GU-4 displayed on the display device 34 during execution of the above-mentioned steps S10 and S11. The user interface image GU-4 is similar to the above-mentioned user interface image GU-3, except that the shape of the content image Gc of the editing image Gd displayed in the region R2 is different and the shape of the content image Gc can be adjusted. Note that Fig. 7 shows the first display mode, which is the display mode when an operation is performed on the above-mentioned tab Gs1.
[0083] In this way, the above-mentioned program PR causes the computer to display the editing image Gd when the first display mode is selected, even during the execution of the above-mentioned steps S10 and S11.
[0084] Area R1 of user interface image GU-4 indicates that "content creation" is the target to be executed, and area R1 displays the text "You can adjust the shape of the content," indicating that the shape of the content image can be adjusted.
[0085] The editing image Gd is displayed in the region R2 of the user interface image GU-4. Here, the shape of the content image Gc of the editing image Gd is trimmed to fit the shape of the image Gba in the above-mentioned step S9, and the content image Gc is displayed in this trimmed shape in the region R2 of the user interface image GU-4. Note that the content image Gc in this trimmed shape may be displayed by editing the content image data DGc so as to cut out unnecessary parts of the content image Gc, or may be displayed by displaying a mask image that hides unnecessary parts of the content image Gc without editing the content image data DGc.
[0086] In this way, the aforementioned program PR causes the computer to display the content image Gc in a state changed based on the input in the aforementioned step S9. Specifically, when the aforementioned program PR receives an input to select the image Gba in the aforementioned step S8, it causes the computer to display the content image Gc superimposed on the image Gba in a shape that approximates the shape of the image Gba.
[0087] Here, the content image Gc can accept an adjustment of the contour shape. In the example shown in FIG. 7, a plurality of dots are arranged along the contour of the content image Gc. The contour shape of the content image Gc is adjusted by selecting any one dot from the plurality of dots using the cursor CUR and then moving the selected dot. Note that the display for adjusting the contour shape of the content image Gc is not limited to the example using dots shown in FIG. 7, and may be any display that can accept an operation to select and move a part or all of the contour. Note that a different content image Gc may be selected by accepting an operation on the region R3 in the user interface image GU-4. In this case, the newly selected content image Gc may be trimmed to match the shape of the content image Gc when the operation on the region R3 is performed, and then displayed in the region R2.
[0088] In this way, the aforementioned program PR also causes the computer to execute, in the aforementioned step S10, receiving an input for changing at least one of the position, shape, and size of the content image Gc in the image for editing Gd.
[0089] Here, the input in step S10 includes an input for deforming the contour of the content image Gc. When the program PR receives the input in step S10, the program PR causes the computer to execute the process of displaying the content image Gc superimposed on the image Gba with the contour of the content image Gc deformed.
[0090] Fig. 8 is a diagram showing a display example of the second display mode of the edited content image Gc. Fig. 8 shows a user interface image GU-5 displayed on the display device 34 during execution of the above-mentioned steps S10 and S11. The user interface image GU-5 is similar to the above-mentioned user interface image GU-4, except that the captured image Ga is displayed in the area R2. Fig. 8 shows the second display mode, which is the display mode when an operation is performed on the above-mentioned tab Gs2.
[0091] Area R1 of user interface image GU-5 indicates that "Content creation" is the target to be executed, and area R1 displays the words "Next, project the created content," indicating that it is possible to project a content image.
[0092] In the region R2 of the user interface image GU-5, the captured image Ga is displayed superimposed on the image for editing Gd. Here, in the region R2 of the user interface image GU-5, the content image Gc is displayed in this trimmed shape. In this way, by superimposing the trimmed content image Gc on the captured image Ga, an image close to the actual projection state of the content image Gc can be visually provided by the display device 34.
[0093] In this way, the above-mentioned program PR causes the computer to display the content image Gc superimposed on at least a part of the captured image Ga when the second display mode is selected during execution of the above-mentioned steps S10 and S11.
[0094] 9 is a diagram for explaining projection using the edited content image Gc. In the above-mentioned step S13, the edited content image Gc is projected onto the object OJa by the projector 20, as shown in FIG.
[0095] As described above, the content editing method includes displaying an image Gba showing the contour of the object OJa with lines based on an captured image Ga of the object OJa, displaying an editing image Gd in which a content image Gc showing the content is superimposed on at least a portion of the image Gba, accepting input to change at least one of the position, shape and size of the content image Gc in the editing image Gd, and displaying the content image Gc in a state changed based on the input to change it.
[0096] As described above, the information processing device 30 includes the input device 35, the display device 34, and the processing device 32. The processing device 32 causes the display device 34 to display an image Gba showing the contour of the object OJa with a line based on an image Ga obtained by capturing an image of the object OJa. The processing device 32 also causes the display device 34 to display an image for editing Gd in which a content image Gc showing a content is superimposed on at least a part of the image Gba. The processing device 32 also receives an input via the input device 35 to change at least one of the position, shape, and size of the content image Gc in the image for editing Gd. The processing device 32 also causes the display device 34 to display the content image Gc in a state changed based on the input for change.
[0097] In the above content editing method, information processing device 30, or program PR, an image Gba showing the contour of the object OJa by a line is displayed, so that the contour of the object OJa, which is the projection target shown in the image Gba, is easily visible to the user. Also, an editing image Gd in which the content image Gc is superimposed on at least a part of the image Gba is displayed, so that the relationship of the position, shape, and size of the content image Gc with respect to the image Gba is easily visible to the user. Furthermore, an input for changing at least one of the position, shape, and size of the content image Gc in the editing image Gd is accepted, so that at least one of the position, shape, and size of the content image Gc can be adjusted with high precision so as to match the contour of the object OJa. As a result, content editing becomes easy, and the projection image can be set to the position and size intended by the user.
[0098] In this embodiment, as described above, the input to be changed includes an input to select the image Gba. When the program PR receives an input to select the image Gba, the program PR causes the computer to further execute the following: displaying the content image Gc on the image Gba in a shape that is close to the shape of the image Gba and superimposing the content image Gc on the image Gba. Therefore, by a simple operation of inputting the input to select the image Gba, the object OJa to be projected can be specified and the content image Gc can be displayed in a shape corresponding to the shape of the object OJa. In addition, by displaying the content image Gc on the image Gba in a shape that is close to the shape of the image Gba and superimposing the content image Gc on the image Gba, the user can easily visually recognize the relationship of the position, shape, and size of the content image Gc with respect to the image Gba.
[0099] As described above, the input for modification includes an input for modifying the contour of the content image Gc. When the program PR receives the input for modification, the program PR further causes the computer to display the content image Gc with the contour of the content image Gc modified, superimposed on the image Gba. This allows the content image Gc to be edited according to the user's intention.
[0100] Furthermore, as described above, the program PR further causes the computer to execute the following operations: displaying a user interface image GU for selecting either the first display mode or the second display mode, displaying an image for editing Gd when the first display mode is selected, and displaying the content image Gc superimposed on at least a part of the captured image Ga when the second display mode is selected. Therefore, according to the user's intention, it is possible to select a state in which the content image Gc is displayed superimposed on at least a part of the image Gba, or a state in which the content image Gc is displayed superimposed on at least a part of the captured image Ga.
[0101] As described above, the program PR further causes the computer to highlight the image Gba when the image Gba is selected or focused. This makes it possible to enhance the visibility of the outline of the object OJa, which is the projection target shown in the image Gba, according to the user's intention.
[0102] 2. Second embodiment A second embodiment of the present disclosure will be described below. In the following exemplary embodiment, for elements whose actions and functions are similar to those of the first embodiment, the reference symbols used in the description of the first embodiment will be used and detailed descriptions of each will be omitted as appropriate.
[0103] FIG. 10 is a diagram showing a display example of an editing image Gd in the second embodiment. FIG. 10 shows a user interface image GU-3 displayed on the display device 34 during execution of the above-mentioned step S8. This embodiment is similar to the above-mentioned first embodiment except that there are two objects OJa that can be projection targets. Therefore, in this embodiment, two objects OJa are captured in the captured image Ga. Here, of the two objects OJa, one object OJa is an example of a "first object" and the other object OJa is an example of a "second object."
[0104] In this embodiment, as shown in Fig. 10, the line drawing Gb includes an image Gba-1 showing an object OJa corresponding to the first object, as well as an image Gba-2 showing an object OJa corresponding to the second object. The image Gba-1 is an example of a "first image", and the image Gba-2 is an example of a "second image". In this manner, displaying the image Gba-1 includes displaying, in addition to the image Gba-1, an image Gba-2 showing the outline of the object OJa corresponding to the second object in lines.
[0105] Here, each of the images Gba-1 and Gba-2 can accept operations such as specifying the cut shape, similar to the image Gba in the first embodiment described above. Thus, the input in step S8 in this embodiment includes an input to select the image Gba-1 or the image Gba-2. In the example shown in Fig. 10, the cursor CUR is placed over the closed region corresponding to the image Gba-1, and the image Gba-1 is highlighted by thickening the line showing the outline of the image Gba-1.
[0106] Fig. 11 is a diagram showing an example of how the edited content image Gc is displayed when the image Gba-1 is selected. Fig. 11 shows a user interface image GU-4 that is displayed on the display device 34 during the execution of the above-mentioned steps S10 and S11.
[0107] In this embodiment, when the program PR receives input to select image Gba-1, as shown in FIG. 11, it causes the computer to display the content image Gc superimposed on image Gba-1 in a shape that approximates the shape of image Gba-1.
[0108] Fig. 12 is a diagram showing an example of how the edited content image Gc is displayed when the image Gba-2 is selected. Fig. 12 shows a user interface image GU-4 that is displayed on the display device 34 during the execution of the above-mentioned steps S10 and S11.
[0109] In this embodiment, when the program PR receives input to select image Gba-2, as shown in FIG. 12, it causes the computer to display the content image Gc superimposed on image Gba-2 in a shape that approximates the shape of image Gba-2.
[0110] The second embodiment described above also makes it easy to edit content. In this embodiment, as described above, since it is possible to input the selection of image Gba-1 or image Gba-2, even if there are multiple objects that can be projection targets, the content image Gc can be edited after selecting the object to be the projection target according to the user's intention.
[0111] 3. Variations Each of the above-mentioned exemplary embodiments may be modified in various ways. Specific modified embodiments that may be applied to each of the above-mentioned embodiments are illustrated below. Two or more embodiments selected from the following examples may be appropriately combined as long as they are not mutually contradictory.
[0112] 3-1. Variation 1 In the above-described embodiment, an example is given in which the first image is a line drawing, but this is not limited to this example, and the first image may be in any example in which the contour of the first object is easier to see than in the captured image.
[0113] Fig. 13 is a diagram showing another display example of the first image. Fig. 13 shows a user interface image GU-2 displayed on the display device 34 after the execution of the above-mentioned step S4 and before the execution of step S5. The user interface image GU-2 shown in Fig. 13 is similar to the user interface image GU-2 of the first embodiment, except that an image Ge is displayed in the region R2 instead of the line drawing Gb. The image Ge is an example of a "first image." Note that, for convenience of explanation, the display of images corresponding to the objects OJb and OJc in the image Ge is omitted in Fig. 13.
[0114] The image Ge includes an image Gea showing an object OJa. Here, the image Gea shows the contour of the object OJa by making the inside and outside colors of the contour different from each other. That is, the contour is shown without using a line having a thickness by painting the inside and outside colors of the contour of the object OJa shown in the image Gea differently. Even with such an image Gea, the visibility of the contour of the object OJa, which is the projection target, can be improved. Note that in FIG. 13, for convenience of drawing, an embodiment is illustrated in which the inside of the contour of the object OJa is displayed in white and the outside in black, but the colors of the inside and outside of the contour of the object OJa are not particularly limited and are arbitrary as long as they are different colors. Note that the image Ge may be displayed overlapping the above-mentioned line drawing Gb. Also, the number of colors and closed regions is not limited to two. For example, an image showing the object OJb may be displayed using a third color.
[0115] 3-2. Variation 2 In the above-described embodiment, the number of objects OJa that can be projection targets is one or two, but the present invention is not limited to this, and the number may be three or more.
[0116] 3-3. Variation 3 In the above embodiment, one closed region is selected as the first image, but the present invention is not limited to this embodiment, and two or more closed regions may be selected. For example, in the second embodiment, an image Gba-1 and an image Gba-2 may be selected.
[0117] 4. Notes The following is a summary of this disclosure.
[0118] (Note 1) Displaying a first image showing a contour of a first object by a line based on a captured image of the first object; displaying an image for editing in which a content image showing a content is superimposed on at least a part of the first image; receiving an input for changing at least one of a position, a shape, and a size of the content image in the editing image; modifying at least one of a position, a shape, and a size of the content image based on the modification input; A program that causes a computer to execute the following.
[0119] In the above aspect of Supplementary Note 1, since the first image showing the outline of the first object by a line is displayed, the outline of the first object, which is the projection target shown in the first image, is easily visible to the user. Also, since the editing image in which the content image is superimposed on at least a part of the first image is displayed, the relationship of the position, shape, and size of the content image relative to the first image is easily visible to the user. Furthermore, since an input for changing at least one of the position, shape, and size of the content image in the editing image is accepted, at least one of the position, shape, and size of the content image can be adjusted with high precision to match the outline of the first object. As a result, editing of the content becomes easy, and the projection image can be set to the position and size intended by the user.
[0120] (Supplementary Note 2) The input for changing includes an input for selecting the first image, The program described in Appendix 1, wherein the modifying includes, when an input for selecting the first image is received, displaying the content image superimposed on the first image in a shape approximating the shape of the first image.
[0121] In the above aspect of Supplementary Note 2, the object to be projected can be specified by a simple operation of inputting the selection of the first image, and the content image can be displayed in a shape corresponding to the shape of the object. Also, by displaying the content image superimposed on the first image in a shape close to the shape of the first image, the user can easily visually recognize the relationship of the position, shape, and size of the content image relative to the first image.
[0122] (Note 3) The captured image includes an image of the first object and a second object, Displaying the first image includes displaying the first image and a second image showing an outline of the second object; the input for changing includes an input for selecting the first image or the second image; The modifying step comprises: When an input for selecting the first image is received, the content image is displayed in a shape that approximates a shape of the first image so as to be superimposed on the first image; When an input for selecting the second image is received, the content image is displayed superimposed on the second image in a shape approximating the shape of the second image; (a) a program as set forth in Appendix 1 or Appendix 2,
[0123] In the aspect of Supplementary Note 3 above, even if there are a plurality of objects that can be projection targets, the user can select an object to be the projection target according to his or her intention, and then edit the content image.
[0124] (Supplementary Note 4) The input for changing includes an input for deforming a contour of the content image, The modifying step comprises: When receiving the input to deform, deforming a contour of the content image; displaying the content image with the contour deformed so as to be superimposed on the first image; Any one of the programs in Appendix 1 to Appendix 3, including:
[0125] In the above aspect of Supplementary Note 4, the content image can be edited according to the user's intention.
[0126] (Appendix 5) Displaying a user interface image for selecting either the first display mode or the second display mode; displaying the image for editing when the first display manner is selected; When the second display mode is selected, the content image is displayed so as to be superimposed on at least a part of the captured image; 5. The program according to claim 1, further comprising:
[0127] In the aspect of Supplementary Note 5 above, depending on the user's intention, it is possible to select a state in which the content image is displayed superimposed on at least a portion of the first image, and a state in which the content image is displayed superimposed on at least a portion of the captured image.
[0128] (Supplementary Note 6) When the first image is selected or when the first image is focused, the first image is highlighted; The programs of Appendix 1 to Appendix 5 are further executed.
[0129] In the above aspect of Supplementary Note 6, the visibility of the contour of the first object, which is the projection target shown in the first image, can be increased according to the user's intention.
[0130] (Note 7) Displaying a first image showing a contour of a first object by making the inside and outside of the contour of the first object have different colors based on a captured image of the first object; displaying an image for editing in which a content image showing a content is superimposed on at least a part of the first image; receiving an input for changing at least one of a position, a shape, and a size of the content image in the editing image; modifying at least one of a position, a shape, and a size of the content image based on the modification input; A program that causes a computer to execute the following.
[0131] In the above aspect of Supplementary Note 7, the first image showing the contour of the first object is displayed by making the colors inside and outside the contour of the first object different from each other, so that the contour of the first object, which is the projection target shown in the first image, is easily visible to the user. Also, since the editing image in which the content image is superimposed on at least a part of the first image is displayed, the relationship of the position, shape, and size of the content image relative to the first image is easily visible to the user. Furthermore, since an input for changing at least one of the position, shape, and size of the content image in the editing image is accepted, at least one of the position, shape, and size of the content image can be adjusted with high precision to match the contour of the first object. As a result, editing of the content becomes easy, and the projection image can be set to the position and size intended by the user.
[0132] (Supplementary Note 8) Displaying a first image showing a contour of a first object by a line based on a captured image of the first object; displaying an image for editing in which a content image showing a content is superimposed on at least a part of the first image; receiving an input for changing at least one of a position, a shape, and a size of the content image in the editing image; displaying the content image in a state where at least one of a position, a shape, and a size is changed based on the change input; How to edit content, including:
[0133] In the above aspect of Supplementary Note 8, since the first image showing the outline of the first object by a line is displayed, the outline of the first object, which is the projection target shown in the first image, is easily visible to the user. Also, since the editing image in which the content image is superimposed on at least a part of the first image is displayed, the relationship of the position, shape, and size of the content image relative to the first image is easily visible to the user. Furthermore, since an input for changing at least one of the position, shape, and size of the content image in the editing image is accepted, at least one of the position, shape, and size of the content image can be adjusted with high precision to match the outline of the first object. As a result, editing of the content becomes easy, and the projection image can be set to the position and size intended by the user.
[0134] (Appendix 9) An input device; A display device; a processing device; The processing device includes: displaying, on the display device, a first image showing a contour of the first object by a line based on a captured image of the first object; displaying, on the display device, an image for editing in which a content image showing a content is superimposed on at least a part of the first image; receiving, via the input device, an input for changing at least one of a position, a shape, and a size of the content image in the editing image; modifying at least one of a position, a shape, and a size of the content image based on the modification input; An information processing device that executes the above.
[0135] In the above aspect of Supplementary Note 9, since the first image showing the outline of the first object by a line is displayed, the outline of the first object, which is the projection target shown in the first image, is easily visible to the user. Also, since the editing image in which the content image is superimposed on at least a part of the first image is displayed, the relationship of the position, shape, and size of the content image relative to the first image is easily visible to the user. Furthermore, since an input for changing at least one of the position, shape, and size of the content image in the editing image is accepted, at least one of the position, shape, and size of the content image can be adjusted with high precision to match the outline of the first object. As a result, editing of the content becomes easy, and the projection image can be set to the position and size intended by the user. [Explanation of symbols]
[0136] 10...camera, 20...projector, 30...information processing device, 31...storage device, 32...processing device, 32a...acquisition unit, 32b...display control unit, 32c...image editing unit, 32d...projector control unit, 32e...measurement unit, 33...communication device, 34...display device, 35...input device, 100...system, B1...button, B2...button, B3...button, B4...button, B5...button, B6...button, CUR...cursor, DGa...captured image data, DGb...line drawing data, DGc...content image data, DGd...image data for editing, DGe...image data for projection, DGf...conversion data, GU...user interface image, GU-1...user interface image, GU-2...user interface image, GU-3...user interface image, GU-4...user interface Image, GU-5...user interface image, Ga...captured image, Gaa...image, Gab...image, Gac...image, Gb...line drawing, Gba...image, Gba-1...image (first image), Gba-2...image (second image), Gbb...image, Gbc...image, Gc...content image, Gd...image for editing, Ge...image, Gea...image, Gs1...tab, Gs2...tab, OJa...object (first object), OJb...object, OJc...object, PR...program, R1...area, R2...area, R3...area, RC...captured area, RP...projection area, S1...step, S10...step, S11...step, S12...step, S13...step, S14...step, S2...step, S3...step, S4...step, S5...step, S6...step, S7...step, S8...step, S9...step, W...wall surface.
Claims
1. displaying a first image showing a contour of the first object by a line based on a captured image of the first object; displaying an image for editing in which a content image showing a content is superimposed on at least a part of the first image; receiving an input for changing at least one of a position, a shape, and a size of the content image in the editing image; modifying at least one of a position, a shape, and a size of the content image based on the modification input; Have the computer execute program.
2. the input for modifying includes an input for selecting the first image; the changing includes, when an input for selecting the first image is received, displaying the content image in a shape that is close to a shape of the first image so as to be superimposed on the first image; The program according to claim 1.
3. The captured image includes an image of the first object and a second object, Displaying the first image includes displaying the first image and a second image showing an outline of the second object; the input for changing includes an input for selecting the first image or the second image; The modifying step comprises: When an input for selecting the first image is received, the content image is displayed in a shape that is close to a shape of the first image so as to be superimposed on the first image; When an input for selecting the second image is received, the content image is displayed in a shape that is close to a shape of the second image so as to be superimposed on the second image; Including, The program according to claim 1.
4. the modifying input includes an input for modifying a contour of the content image; The modifying step comprises: When receiving the input to deform, deforming a contour of the content image; displaying the content image with the contour deformed so as to be superimposed on the first image; Including, The program according to claim 1.
5. displaying a user interface image for selecting one of the first display mode and the second display mode; displaying the image for editing when the first display mode is selected; When the second display mode is selected, the content image is displayed so as to be superimposed on at least a part of the captured image; Further execute The program according to any one of claims 1 to 4.
6. highlighting the first image when the first image is selected or when the first image is focused; Further execute The program according to claim 1.
7. displaying a first image showing a contour of the first object by making the inside and outside of the contour of the first object have different colors based on a captured image of the first object; displaying an image for editing in which a content image showing a content is superimposed on at least a part of the first image; receiving an input for changing at least one of a position, a shape, and a size of the content image in the editing image; modifying at least one of a position, a shape, and a size of the content image based on the modification input; Have the computer execute program.
8. displaying a first image showing a contour of the first object by a line based on a captured image of the first object; displaying an image for editing in which a content image showing a content is superimposed on at least a part of the first image; receiving an input for changing at least one of a position, a shape, and a size of the content image in the editing image; displaying the content image in a state where at least one of a position, a shape, and a size is changed based on the change input; Including, How to edit content.
9. An input device; A display device; a processing device; The processing device includes: displaying, on the display device, a first image showing a contour of the first object by a line based on a captured image of the first object; displaying, on the display device, an image for editing in which a content image showing a content is superimposed on at least a part of the first image; receiving, via the input device, an input for changing at least one of a position, a shape, and a size of the content image in the editing image; modifying at least one of a position, a shape, and a size of the content image based on the modification input; Execute Information processing device.