Cost calculation method, program, and information processing apparatus

Abstract

To estimate the cost for creating content.SOLUTION: A cost calculation method includes: a computer analyzing a picked-up image obtained by picking up at least one object to estimate the shape of the at least one object; and on the basis of the shape of the at least one object, calculating the cost related to creating an image to be projected on the at least one object by using a projector and notifying of the costs.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a cost calculation method, a program, and an information processing device. [Background technology]

[0002] In projection mapping, which uses equipment such as a projector to project an image onto a three-dimensional object, it is generally necessary to adjust the position, shape, and size of the image to match those of the object.

[0003] For example, Patent Document 1 describes a method of acquiring a projection device viewpoint image of a three-dimensional projection target, preparing a three-dimensional model corresponding to the projection target as projection content, converting the three-dimensional model into a two-dimensional image that matches the projection device viewpoint image, and projecting the two-dimensional image onto the projection target. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-192040 A Summary of the Invention [Problem to be solved by the invention]

[0005] When creating image content tailored to a projection target, as in Patent Document 1, content is created for each projection target individually, and the cost required to create the content varies depending on the size and shape of the projection target, etc. Conventionally, it has been difficult to estimate the costs in such cases. [Means for solving the problem]

[0006] A cost calculation method according to one embodiment of the present disclosure includes a computer estimating a shape of at least one object by analyzing an image obtained by imaging the at least one object, calculating a cost associated with creating an image for projection onto the at least one object using a projector based on the shape of the at least one object, and notifying the cost.

[0007] A program according to one embodiment of the present disclosure causes a computer to estimate a shape of at least one object by analyzing an image obtained by imaging the at least one object, calculate a cost associated with creating an image to be projected onto the at least one object using a projector based on the shape of the at least one object, and notify the cost.

[0008] An information processing device according to one embodiment of the present disclosure includes a processing device that performs the following operations: estimating a shape of at least one object by analyzing an image obtained by imaging the at least one object; calculating a cost associated with creating an image to be projected onto the at least one object using a projector based on the shape of the at least one object; and notifying the cost. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram of a system used in a cost calculation method according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a camera and a projector. [Diagram 3] 1 is a block diagram of an information processing apparatus according to an embodiment. [Figure 4] 1 is a flowchart showing a flow of a cost calculation method according to an embodiment. [Diagram 5] FIG. 2 is a diagram for explaining installation of a projector. [Figure 6] FIG. 13 is a diagram for explaining the installation of a camera. [Figure 7] 11 is a diagram for explaining an instruction to start acquiring a captured image used for measuring a projection surface. FIG. [Figure 8] FIG. 13 is a diagram showing an example of a display of calculated costs. [Figure 9] FIG. 13 is a diagram illustrating an example of a unit price table used for calculating costs. [Figure 10] FIG. 13 is a diagram showing a display example for changing the estimate area. [Figure 11] FIG. 13 is a diagram for explaining an operation for changing an estimation area. [Figure 12] FIG. 13 is a diagram showing an example of a display after the estimate area is changed. [Figure 13] FIG. 13 is a diagram showing an example of a display for requesting the creation of content. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the dimensions and scale of each part in the drawings may differ from the actual dimensions, and some parts are shown diagrammatically to facilitate understanding. In addition, the scope of the present disclosure is not limited to these forms unless otherwise specified in the following description to the effect that the present disclosure is limited. 1. Embodiment 1-1. Overview of the system used for cost calculation method 1 is a schematic diagram of a system 100 used in a cost calculation method according to an embodiment. The system 100 is a projection mapping system, and has a function of calculating the cost required to create an image to be used in projection mapping. In the following, the image may be referred to as content. The content may be a still image or a video.

[0011] 1, the system 100 includes a camera 10, a projector 20, an information processing device 30, and an external device 40. These are connected to each other so as to be able to communicate with each other via a network NW including the Internet. Note that the camera 10, the projector 20, the information processing device 30, and the external device 40 may be connected to each other so as to be able to communicate with each other wirelessly or via a wired connection without using the network NW.

[0012] The camera 10 is a digital camera having an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and captures an image of the projection region RP of the projector 20.

[0013] The projector 20 is a display device that projects an image onto a projection object under the control of an information processing device 30.

[0014] Although not shown, the projector 20 has an image processing circuit, a light source, a light modulation device, and a projection optical system. The image processing circuit of the projector 20 is a circuit that controls the driving of the light modulation device of the projector 20 based on information from the information processing device 30. The light source of the projector 20 includes, for example, a halogen lamp, a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), or a laser light source, and emits red, green, and blue light, respectively. The light modulation device of the projector 20 includes three light modulation elements provided corresponding to red, green, and blue. Each of the three light modulation elements is, for example, a display panel such as a transmissive liquid crystal panel, a reflective liquid crystal panel, or a DMD (digital mirror device). The three light modulation elements modulate the red, green, and blue light, respectively, based on a signal from the image processing circuit of the projector 20 to generate image light of each color. Such image light of each color is synthesized by a color synthesis optical system to become full-color image light. The projection optical system of the projector 20 is an optical system including a projection lens and the like, and forms and projects the above-mentioned full-color image light onto a projection target.

[0015] The information processing device 30 is a computer such as a notebook computer, a desktop computer, a smartphone, or a tablet terminal that executes the cost calculation method. The information processing device 30 has a function of controlling the operations of the camera 10 and the projector 20, and a function of calculating the cost related to the image used for projection by the projector 20 using the captured image acquired by the camera 10.

[0016] The external device 40 is a computer owned or managed by a company or the like that creates images to be used for projection by the projector 20. The external device 40 may be a computer owned or managed by a company or the like, or a server device or the like connected to the information processing device 30.

[0017] 1-2. Camera and projector Fig. 2 is a diagram for explaining the camera 10 and the projector 20. Fig. 2 illustrates a case where an image is projected so as to match the shape of an object OJa that is a projection target.

[0018] In the example shown in FIG. 2, the object OJa is a plain kimono, such as white, worn by an object OJb in front of the wall W. The object OJb is, for example, a torso or a mannequin. The shapes, sizes, positions, and other aspects of the objects OJa and OJb are not limited to the example shown in FIG. 2 and are arbitrary. The object OJa is not limited to a kimono and is arbitrary as long as it is an object onto which projection mapping is performed. The object OJb is not limited to a torso or a mannequin and may be used or omitted as necessary.

[0019] The camera 10 generates imaging data DGa, which will be described later, indicating a captured image of the objects OJa and OJb. Here, the objects OJa and OJb are included in an imaging region RC, which is an area that the camera 10 can capture.

[0020] The projector 20 projects an image onto the object OJa under the control of the information processing device 30. Here, the objects OJa and OJb are included in a projection area RP, which is an area where the projector 20 can project.

[0021] In the example shown in Fig. 2, the projection area RP is included in the imaging area RC. The projection area RP only needs to include the objects OJa and OJb, and may have a portion that is not included in the imaging area RC. The installation position and installation posture of the projector 20 are not limited to the example shown in Fig. 2, and are arbitrary.

[0022] 1-3. Information processing device Fig. 3 is a block diagram of an information processing device 30 according to an embodiment. As shown in Fig. 3, the information processing device 30 includes a storage device 31, a processing device 32, a communication device 33, a display device 34, and an input device 35. These are connected to each other so as to be able to communicate with each other.

[0023] The storage device 31 is a storage device that stores programs such as an operating system and application programs executed by the processing device 32, and data processed by the processing device 32. The storage device 31 includes, for example, a hard disk drive or a semiconductor memory. Note that a part or all of the storage device 31 may be an external storage device of the information processing device 30, or may be provided in an external device such as a server connected to the information processing device 30 via a communication network such as the Internet.

[0024] The storage device 31 stores a program PR, imaging data DGa, analysis data DGb, condition data DGc, request information DGd, and a unit price table TB.

[0025] The program PR is a program for executing a cost calculation method, which will be described in detail later. The image data DGa is data representing an image captured by the camera 10.

[0026] The imaging data DGa is data indicating a plurality of captured images obtained during measurement by the acquisition unit 32a described later. The plurality of captured images include an initial captured image, which is an image obtained by capturing an image of the projection area RP when a black image or a white image included in a plurality of pattern images described later is projected onto the projection area RP. The imaging data DGa may include data indicating a captured image separate from the captured image obtained during measurement by the acquisition unit 32a described later. The initial captured image may also be an image obtained by capturing an image of the projection area RP in a state where the projector 20 does not project.

[0027] The analysis data DGb includes data indicating the shape of the projection surface in the imaging region RC and data indicating a contrast ratio, which is an index value related to the brightness of the space in which the camera 10 has taken an image, and is obtained by the analysis of the image analysis unit 32c described later. The data indicating the shape of the projection surface in the imaging region RC indicates, for example, regions in which the projection surface is divided by object. The data indicating the shape of the projection surface indicates whether the shape of each region of the projection surface is a three-dimensional shape or a two-dimensional shape. As a result, the data indicating the shape of the projection surface indicates whether the shape of the object OJa is a three-dimensional shape or a two-dimensional shape. The data indicating the shape of the projection surface may further be data in which the region of the projection surface is divided by shape type so that it belongs to any of a plane, a cylindrical surface, a spherical surface, and a complex surface. The types of shapes are only an example, and the shape of the projection surface may be classified into fewer types or more types. A complex surface is a region of a shape that is not classified into any other shape type. In addition, the two-dimensional shape region may also be classified according to the type of shape in two dimensions, such as a polygon, a circle, a complex shape, etc.

[0028] The condition data DGc is information indicating conditions necessary for calculating the cost related to an image used for projection by the projector 20. The condition includes, for example, at least one of the duration of the image used for projection by the projector 20, the area of ​​the projection object, and the number of projection objects. Note that the duration of an image is, for example, the length of time required to play the image once when the image is a single video. In the case of a still image, the duration may be a fixed value and the cost may be calculated. The area of ​​the projection object is, for example, "1.5 m2". 2 Even if it is a single number such as "1.5m 2 More than 2.0m 2 It may be a range of values ​​such as "less than."

[0029] The unit price table TB is data showing the correspondence between the cost per unit area of ​​the image used for projection by the projector 20 and the shape of the projection target.

[0030] The request information DGd includes information indicating the result of an estimate of the cost required to create an image to be used for projection by the projector 20, and information indicating at least one captured image among the multiple captured images indicated by the imaging data DGa.

[0031] The processing device 32 is a processing device having a function of controlling each part of the information processing device 30 and a function of processing various data. The processing device 32 is configured to include a processor such as a CPU (Central Processing Unit). The processing device 32 may be configured with a single processor or multiple processors. In addition, some or all of the functions of the processing device 32 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array).

[0032] The communication device 33 is a communication device capable of communicating with the projector 20 and the like. For example, the communication device 33 is a wired communication device such as a wired LAN (Local Area Network), a USB (Universal Serial Bus), or an HDMI (High Definition Multimedia Interface), or a wireless communication device such as an LPWA (Low Power Wide Area), a wireless LAN including Wi-Fi, or Bluetooth. "HDMI", "Wi-Fi", and "Bluetooth" are each registered trademarks. The communication device 33 may be capable of communicating with the camera 10, or may be capable of communicating with an external device such as a server.

[0033] The display device 34 displays various images under the control of the processing device 32. The display device 34 is a display device including various display panels such as a liquid crystal display panel, an organic EL (electro-luminescence) display panel, etc. The display device 34 of this embodiment is a touch panel as described later, and also serves as the input device 35.

[0034] The input device 35 is an input device that accepts operations from a user. Note that the input device 35 is not limited to a touch panel, and may be, for example, a touch pad provided separately from the display device 34 or a pointing device such as a mouse.

[0035] In the above information processing device 30, the processing device 32 executes the program PR stored in the storage device 31 to realize various functions. Specifically, the processing device 32 functions as an acquisition unit 32a, an image analysis unit 32c, a display control unit 32b, a cost calculation unit 32e, and a projector control unit 32f by executing the program PR. Therefore, the processing device 32 includes the acquisition unit 32a, the display control unit 32b, the image analysis unit 32c, the condition setting unit 32d, the cost calculation unit 32e, and the projector control unit 32f.

[0036] The acquisition unit 32a controls the operation of the communication device 33 to acquire various information from various devices connected to the information processing device 30, and stores the acquired information in the storage device 31. The acquisition unit 32a acquires imaging data DGa from the camera 10. Specifically, the acquisition unit 32a acquires a plurality of captured images by having the projector 20 sequentially project a plurality of pattern images onto a projection target and having the camera 10 capture each of the projected pattern images. As a result, the imaging data DGa is obtained.

[0037] The display control unit 32b controls the operation of the display device 34 to cause the display device 34 to display various information. Specifically, the display control unit 32b causes the display device 34 to display a user interface image GU, which will be described later, that is necessary for executing a cost calculation method, which will be described later. The user interface image GU is also capable of accepting an input of information necessary for generating the condition data DG. Although not shown in the figure, the user interface image GU for accepting an input of at least one of the time length of an image, the area of ​​a projection object, and the number of projection objects, for example.

[0038] The image analysis unit 32c estimates the shape of the object OJa by analyzing the imaging data DGa. As a result, the image analysis unit 32c generates analysis data DGb. Specifically, the image analysis unit 32c generates analysis data DGb based on the multiple captured images indicated by the imaging data DGa, using the correspondence between the coordinates of the pattern image in the coordinate system of the captured image of the camera 10 and the coordinates of the pattern image in the coordinate system of the display panel of the projector 20.

[0039] The condition setting unit 32d generates the condition data DGc based on the input using the user interface image GU described later. Here, the condition setting unit 32d may generate the preset condition data DGc before the input using the user interface image GU described later or when some items of the condition data DGc have not been input.

[0040] The cost calculation unit 32e generates request information DGd based on the unit price table TB, the analysis data DGb, and the condition data DGc. The cost calculation unit 32e also transmits the request information DGd to the external device 40 via the communication device 33 based on an instruction from the user.

[0041] The projector control unit 32f controls the operation of the projector 20 to cause the projector 20 to display various images.

[0042] 1-4. Cost calculation method 4 is a flowchart showing the flow of a cost calculation method according to an embodiment. The cost calculation method is executed by the information processing device 30 described above.

[0043] More specifically, as shown in FIG. 4, first, in step S101, the display control unit 32b causes the display device 34 to display a user interface image GU, which will be described later.

[0044] Next, in step S102, the acquisition unit 32a judges whether the installation of the camera 10 and the projector 20 is complete. This judgment is performed until it is judged that the installation of the camera 10 and the projector 20 is complete based on an operation on the user interface image GU (step S102: NO). Here, if an input to the button B2 is received in a user interface image GU-2 shown in FIG. 6 described later after an input to the button B1 is received in a user interface image GU-1 shown in FIG. 5 described later, it is judged that the installation of the camera 10 and the projector 20 is complete (step S102: YES). On the other hand, after an input to the button B1 is received in a user interface image GU-1 shown in FIG. 5 described later, step S102 is repeated until an input to the button B2 is received in a user interface image GU-2 shown in FIG. 6 described later (step S102: NO).

[0045] When it is determined that the installation of the camera 10 and the projector 20 is complete (step S102: YES), in step S103, the acquisition unit 32a determines whether or not there is an instruction to start scanning for measuring the projection surface of the projector 20. This determination is performed based on an operation on the user interface image GU until there is an instruction to start scanning (step S103: NO). Here, when an input to button B3 is received in the user interface image GU-3 shown in FIG. 7 described later, it is determined that there is an instruction to start scanning (step S103: YES), and step S103 is repeated until an input to button B3 is received (step S103: NO).

[0046] If an instruction to start scanning has been given (step S103: YES), in step S104, the acquisition unit 32a executes scanning. In this specification, measuring the projection surface may be referred to as scanning. In this embodiment, imaging data DGa is acquired by scanning. Furthermore, using the imaging data DGa, information is generated that indicates the correspondence between the coordinates of each pixel in the coordinate system of the captured image of the camera 10 and the coordinates of each pixel in the coordinate system of the display panel of the projector 20.

[0047] Next, in step S105, the image analysis unit 32c estimates the shape of the object OJa by analyzing the projection surface based on the imaging data DGa, thereby generating analysis data DGb.

[0048] Next, in step S106, the cost calculation unit 32e generates request information DGd based on the analysis data DGb and the condition data DGc.

[0049] Next, in step S107, the display control unit 32b causes the display device 34 to display the cost indicated by the request information DGd.

[0050] Next, in step S108, the cost calculation unit 32e judges whether or not the condition indicated by the condition data DGc has been modified. Here, if an input to button B7 is received in a user interface image GU-6 shown in FIG. 11 after an input to button B5 is received in a user interface image GU-5 shown in FIG. 10 described later, it is judged that the condition has been modified (step S108: YES), and step S106 is executed. On the other hand, if an input to button B6 is received in a user interface image GU-5 shown in FIG. 10 described later, it is judged that the condition has not been modified (step S108: NO).

[0051] If the conditions indicated by the condition data DGc have been modified (step S108: YES), the cost calculation unit 32e returns to the execution of the above-mentioned step S106.

[0052] On the other hand, if the condition indicated by the condition data DGc has not been modified (step S108: NO), in step S109, the cost calculation unit 32e judges whether or not there has been an input requesting the creation of an image based on the request information DGd. Here, if an input to button B8 in a user interface image GU-7 shown in FIG. 12 described later is accepted, it is judged that there has been an input requesting the creation of an image (step S109: YES), and step S110 is executed. On the other hand, steps S108 and S109 are repeated (step S109: NO) until an input to button B8 in a user interface image GU-7 shown in FIG. 12 described later is accepted.

[0053] If there is no request instruction based on the request information DGd (step S109: NO), the cost calculation unit 32e returns to the execution of the above-mentioned step S108.

[0054] On the other hand, if a request based on the request information DGd has been issued (step S109: YES), the cost calculation unit 32e transmits the request information DGd to the external device 40 via the communication device 33 in step S110.

[0055] The above is the flow of the cost calculation method. An example of a user interface image GU used in the content editing method will be described below with reference to Figs. 5 to 13. Figs. 4 to 8 and Figs. 10 to 13 show user interface images GU-1 to GU-8 that change according to the progress of the cost calculation method. Hereinafter, the user interface images GU-1 to GU-9 may be referred to as user interface images GU without distinction. Note that the user interface images GU are not limited to the examples shown in Figs. 4 to 8 and Figs. 10 to 13.

[0056] Fig. 5 is a diagram for explaining the installation of projector 20. Fig. 5 shows a user interface image GU-1 that is displayed on display device 34 when the above-mentioned step S102 is executed. User interface image GU-1 includes regions R1 and R2 and a button B1.

[0057] Necessary images are displayed in the region R1 according to the progress of the cost calculation method. A real-time captured image G0 by the camera 10 is displayed in the region R1 of the user interface image GU-1. The captured image G0 includes an image G0a showing the object OJa. This allows the user to visually confirm in the region R1 that the projection object exists in both the captured region RC of the camera 10 and the projection region RP of the projector 20.

[0058] In area R2, an explanation according to the progress of the cost calculation method is displayed. In area R2 of the user interface image GU-1, an explanation regarding the installation method of the projector 20 is displayed. In the example shown in Fig. 5, the text "Adjust the position of the projector so that the projector's light covers the target object" is displayed in area R2.

[0059] Button B1 is a display for accepting that the installation of projector 20 has been completed, and is used for the judgment in step S102 described above. When button B1 is operated, a display for installing camera 10 is displayed as shown in the following FIG.

[0060] Fig. 6 is a diagram for explaining the installation of the camera 10. Fig. 6 shows a user interface image GU-2 that is displayed on the display device 34 when the above-mentioned step S102 is executed. The user interface image GU-2 is similar to the above-mentioned user interface image GU-1, except that it includes buttons B2 and BB instead of button B1.

[0061] However, an area R2 of the user interface image GU-2 displays an explanation on how to install the camera 10. In the example shown in Fig. 6, the text "Adjust the camera position so that the target object is included in this screen" is displayed in the area R2.

[0062] Button B2 is a display for accepting that the installation of camera 10 has been completed, and is used in the determination of step S102 described above. When button B2 is operated, step S103 described above is executed.

[0063] The button BB is a display for returning the display of the user interface image GU to the previous state. When the button BB is operated in the user interface image GU-2, the above-mentioned user interface image GU-1 is displayed. Note that the button BB is also displayed in each of the user interface images GU-3 to GU-8 described below, and when the button BB is operated, the user interface image GU is returned to the previous state.

[0064] Fig. 7 is a diagram for explaining an instruction to start acquiring a captured image used for measuring the projection surface. Fig. 7 shows a user interface image GU-3 that is displayed on the display device 34 when the above-mentioned step S103 is executed. The user interface image GU-3 is similar to the above-mentioned user interface image GU-2, except that it includes a button B3 instead of the button B2.

[0065] However, in the region R2 of the user interface image GU-3, an explanation regarding the start of measurement of the projection surface is displayed. In the example shown in Fig. 7, the text "Scanning the object will start. Please wait about 5 minutes until the process is completed" is displayed in the region R2.

[0066] Button B3 is a display for accepting the start of measurement of the projection surface, and is used in the judgment of step S103 described above. When button B3 is operated, step S104 described above is executed.

[0067] In step S104, the acquisition unit 32a controls the operation of the projector 20 so as to sequentially project a plurality of pattern images onto the projection object, and controls the operation of the camera 10 so as to capture each of the pattern images projected onto the projection object. As a result, imaging data DGa indicating a plurality of captured images of the pattern images captured by the camera 10 is obtained.

[0068] As the pattern image, for example, a binary code pattern is used. The binary code pattern is an image for expressing the coordinates of a display device using a binary code. The binary code is a technique for expressing the value of each digit when an arbitrary numerical value is expressed in binary by turning a switch on and off. When a binary code pattern is used as the pattern image, the image projected by the projector 20 corresponds to the switch, and pattern images are required for the number of digits of the binary number expressing the coordinate value. In addition, separate pattern images are required for each of the vertical coordinate and the horizontal coordinate. For example, when the resolution of the display panel of the projector 20 is 120×90, 120 and 90 are each expressed by a 7-digit binary number, so seven images are required to express the vertical coordinate and seven images are required to express the horizontal coordinate. Here, the multiple pattern images include one or both of a white image that displays white color over the entire projection area RP and a black image that displays black color over the entire projection area RP.

[0069] Furthermore, when a binary code pattern is used as a pattern image, the robustness of the measurement generally decreases due to the influence of disturbance light such as lighting. Therefore, when a binary code pattern is used as a pattern image, it is preferable to use a complementary pattern in combination with the binary code pattern in order to reduce the influence of disturbance light and improve the robustness of the measurement. A complementary pattern is an image in which black and white are inverted.

[0070] It should be noted that the pattern image is not limited to a binary code pattern, and may be other structured light such as a dot pattern, a rectangular pattern, a polygonal pattern, a checkered pattern, a gray code pattern, a phase shift pattern, or a random dot pattern.

[0071] As described above, a plurality of captured images of the pattern image are obtained by the camera 10. In the above-mentioned step S105, the image analysis unit 32c estimates the shape of the projection surface based on the plurality of captured images and determines an index value related to the brightness of the space where the image is captured, thereby generating the analysis data DGb. The shape estimation includes, for example, estimating the shape of the projection surface included in the imaging region RC by analyzing the captured images.

[0072] Specifically, the image analysis unit 32c, for example, associates the coordinates of the pattern image in the coordinate system of the captured image of the camera 10 with the coordinates of the pattern image in the coordinate system of the display device of the projector 20 based on the multiple captured images, and divides the projection surface into areas where the difference in the positions of the associated coordinates is equal to or greater than a predetermined threshold. This allows the projection surface to be divided into an area of ​​the object OJa and other areas.

[0073] The image analysis unit 32c may measure the distance between the projector 20 and the projection surface for each pixel of the projector 20 based on the multiple captured images, extract pixels in which the change rate of the distance between adjacent pixels is equal to or greater than a certain value, and divide the projection surface into areas surrounded by the extracted pixels. This also allows the projection surface to be divided into an area of ​​the object OJa and other areas. The image analysis unit 32c may divide the projection surface into areas using Watershed of OpenCV, or may divide the projection surface into areas using segmentation using deep learning.

[0074] Moreover, the image analysis unit 32c divides the projection surface into regions as described above, and then determines the shape of each divided region. The determination of whether the shape of the target region is planar or other shapes is performed, for example, by plane detection using Unity AR Foundation, plane detection using a 3D Hough transform, or plane detection using python RANSAC. If the shape of the target region is other than planar, the determination of the shape of the target region is performed, for example, by estimating the shape using deep learning. "python" is a registered trademark.

[0075] The image analysis unit 32c calculates an index value related to the brightness of the space where the image was captured based on the captured image. Specifically, for example, the image analysis unit 32c calculates a contrast ratio between the brightness of the projection surface in the initial captured image when no image is projected from the projector 20 or when a black image is projected, and the brightness of the projection surface in the initial captured image when a white image is projected. Note that the smaller the contrast ratio, the brighter the environment is and the harder it is to see the image. Therefore, when the contrast ratio is smaller than a predetermined value, a notification indicating that the environment is too bright and not suitable for projection may be displayed on the user interface image GU. The brightness index may also be obtained by other known methods.

[0076] As described above, the cost calculation method includes estimating the shape of the object OJa by analyzing the captured image obtained by capturing an image of the object OJa using a computer in step S105.

[0077] Here, estimating the shape includes determining whether the shape of the object OJa is a three-dimensional shape or a two-dimensional shape.

[0078] Moreover, estimating the shape includes dividing the object OJa into regions according to the type of shape. In this embodiment, the types of shapes used are a complex surface, a spherical surface, a cylindrical surface, and a flat surface.

[0079] The cost calculation method also includes determining an index value related to the brightness of the space in which the image was captured by analyzing the captured image.

[0080] Fig. 8 is a diagram showing an example of how the calculated cost is displayed. Fig. 8 shows a user interface image GU-4 that is displayed on the display device 34 by executing the above-mentioned step S107. User interface image GU-4 is similar to the above-mentioned user interface image GU-3, except that it includes button B4 instead of button B3.

[0081] However, an image G1 is displayed in the region R1 of the user interface image GU-4. The image G1 includes the captured image Ga. The captured image Ga is the above-mentioned initial captured image among the multiple captured images shown in the imaging data DGa, and includes an image Gaa showing the object OJa. In the image G1, each of the above-mentioned divided regions is displayed in a different manner according to its shape.

[0082] The estimated cost of creating the image is displayed in area R2 of user interface image GU-4. In the example shown in Fig. 8, the following text is displayed in area R2: "Space estimate completed", indicating that the estimate has been completed, the estimated amount "29,200 yen", and the breakdown of the estimated amount "Complex (20%): 20,000 yen, Cylindrical (3%): 1,500 yen, Planar (77%): 7,700 yen".

[0083] Button B4 is a display for accepting confirmation of the estimated amount. When button B4 is operated, the display shown in Fig. 10 described below is displayed. Note that the display of user interface image GU-4 in Fig. 8 may be omitted, and the display shown in Fig. 10 may be displayed after the display of user interface image GU-3.

[0084] Fig. 9 is a diagram showing an example of a unit price table TB used for calculating the cost. The cost estimate result as shown in Fig. 8 is calculated by the cost calculation unit 32e in step S106 using the unit price table TB as shown in Fig. 9. Here, the cost calculation unit 32e generates the request information DGd by calculating the cost based on the analysis data DGb and the condition data DGc in addition to the unit price table TB as described above.

[0085] As shown in Fig. 9, the unit price table TB indicates the relationship between the shape of the projection surface and the unit price. In the example shown in Fig. 9, the shapes are classified into complex surface, spherical surface, cylindrical surface, and flat surface, and a unit price is set for each type of shape. Note that the unit price shown in Fig. 9 is an example and is not limited to this. Also, the number of classifications of shape types in the unit price table TB is not limited to the example shown in Fig. 9 and is arbitrary.

[0086] The cost calculation unit 32e calculates the cost based on the following formula, for example. (Cost) = (unit price of complex surface × area ratio of complex surface + unit price of spherical surface × area ratio of spherical surface + unit price of cylindrical surface × area ratio of cylindrical surface + unit price of flat surface × area ratio of flat surface) × (duration of content) × (number of divided areas) × (contrast index) Here, the area ratio refers to the ratio of the area of ​​each shape, for which the cost is to be calculated, to the total area of ​​all areas in the captured image.

[0087] As described above, the cost of creating content can be estimated so that the larger the projection area, the more complex the shape of the projection object, the brighter the projection environment, and the longer the content, the more expensive it is.

[0088] A lower limit of the cost may be set in advance, in which case a certain cost is calculated even if the duration of the content is extremely short.

[0089] As described above, the cost calculation method includes using a computer to calculate the cost of creating an image to be projected onto the object OJa using the projector 20, based on the shape of the object OJa.

[0090] Here, calculating the cost includes calculating the cost based on a unit price set for each type of shape and the ratio of the area of ​​the object OJa to the captured image Ga.

[0091] Moreover, calculating the cost includes calculating the cost based on the shape of the object OJa and an index value related to the brightness of the space in which the image was captured.

[0092] Fig. 10 is a diagram showing a display example for changing the estimate area. Fig. 10 shows a user interface image GU-5 that is displayed on the display device 34 after an operation is performed on the button B4 shown in Fig. 8 described above. The user interface image GU-5 is similar to the user interface image GU-4 described above, except that it includes buttons B5 and B6 instead of the button B4.

[0093] Button B5 is a display for accepting a transition to processing for correcting the target area of ​​the estimate. When button B5 is operated, the display shown in FIG. 11 described later is displayed.

[0094] The button B6 is a display for accepting a transition to processing for a content creation request. When the button B6 is operated, the display shown in FIG.

[0095] Fig. 11 is a diagram for explaining the operation of changing the estimate area. Fig. 11 shows a user interface image GU-6 that is displayed on the display device 34 after an operation is performed on the button B5 shown in Fig. 10 described above. The user interface image GU-6 is similar to the user interface image GU-4 described above, except that it includes a button B7 instead of the buttons B5 and B6.

[0096] However, image G1 of area R1 of user interface image GU-6 can accept input to select an area divided according to the above-mentioned shape type. The area tapped by finger F, indicated by a two-dot chain line in Fig. 11, is selected. Tapping the same area again cancels the selection.

[0097] In this manner, the cost calculation method further includes accepting input of conditions related to the cost.

[0098] Button B7 is a display for accepting the determination of the target area for the estimate. When button B7 is operated, step S106 is executed. As a result, the estimated cost for the selected area is calculated.

[0099] Fig. 12 is a diagram showing an example of a display after the estimate area has been changed. Fig. 12 shows a user interface image GU-7 that is displayed on the display device 34 after an operation is performed on button B7 shown in Fig. 11 described above. User interface image GU-7 is similar to the user interface image GU-6 described above, except that it includes button B8 instead of button B7.

[0100] However, image G1 in area R1 of user interface image GU-7 displays the selection result in the above-mentioned Fig. 11. Also, area R2 of user interface image GU-7 displays the estimated price according to the selection result in the above-mentioned Fig. 11.

[0101] In this way, calculating the cost includes calculating the cost based on the shape of the object OJa and the conditions indicated by the condition data DGc.

[0102] Here, the condition indicated by the condition data DGc includes at least one of the time length of the content, the area of ​​the object OJa, and the number of the objects OJa.

[0103] Furthermore, accepting an input of a condition indicated by the condition data DGc includes accepting an input of selecting at least one object OJa from a plurality of objects.

[0104] The button B8 is a display for accepting a transition to processing for a content creation request. When the button B8 is operated, the display shown in FIG.

[0105] Fig. 13 is a diagram showing a display example for requesting the creation of content. Fig. 13 shows a user interface image GU-8 displayed on the display device 34 after an operation is performed on button B6 shown in Fig. 10 described above, or after an operation is performed on button B8 shown in Fig. 12 described above. Fig. 13 shows a display example when the order details are modified in user interface image GU-7. User interface image GU-8 is similar to the user interface image GU-7 described above, except that it includes button B11 instead of button B8.

[0106] However, area R2 of user interface image GU-7 displays the order details and the name of the supplier. Area R2 of user interface image GU-8 also includes a button B9 for accepting changes to the order details and a button B10 for accepting changes to the supplier. When button B9 is operated, a display for accepting changes to the order details appears (not shown). When button B10 is operated, a display for accepting changes to the supplier appears (not shown).

[0107] The button B11 is a display for receiving an input for requesting the creation of content. When the button B11 is operated (step S109: YES), in step S110, the request information DGd is transmitted to the external device 40. That is, the request information DGd is transmitted to the supplier. The supplier is the owner or manager of the external device 40. If the external device 40 is a server device or the like, the supplier may obtain the request information DGd by receiving a notification via the server device or the like or by accessing the server device.

[0108] In this way, the cost calculation method further includes accepting input requesting the creation of an image, and, when the input requesting the creation of the image is accepted, transmitting request information DGd indicating the cost and the captured image Ga to external device 40.

[0109] As described above, the cost calculation method includes estimating the shape of at least one object OJa by analyzing an image obtained by imaging the at least one object OJa using a computer, and calculating the cost of creating an image to be projected onto the at least one object OJa using the projector 20 based on the shape of the at least one object OJa.

[0110] In the above-described cost calculation method, it is possible to estimate the cost of creating an image to be projected onto the object OJa using the projector 20, based on a captured image obtained by capturing an image of the object OJa. Therefore, it is possible to forecast the cost required for creating the image.

[0111] In this embodiment, as described above, estimating the shape includes determining whether the shape of the at least one object OJa is a three-dimensional shape or a two-dimensional shape. Therefore, it is possible to calculate different costs when the shape of the object OJa is a three-dimensional shape and when it is a two-dimensional shape. As a result, it is possible to improve the validity of the calculated cost.

[0112] Also, as described above, estimating the shape includes dividing the at least one object OJa into regions for each type of shape. Therefore, it is possible to calculate different costs according to the type of shape of the object OJa. As a result, it is possible to improve the validity of the calculated costs.

[0113] Furthermore, as described above, calculating the cost includes calculating the cost based on the unit price set for each type of shape and the ratio of the area of ​​the at least one object OJa in the captured image Ga. Therefore, it is possible to calculate different costs according to the ratio of the area of ​​the object OJa in the captured image Ga. As a result, it is possible to improve the validity of the calculated cost.

[0114] As described above, the cost calculation method further includes determining an index value related to the brightness of the space where the image is captured by analyzing the captured image. Then, calculating the cost includes calculating the cost based on the shape of the at least one object OJa and the index value. Therefore, it is possible to calculate different costs according to the brightness of the space where the image is captured. As a result, it is possible to improve the validity of the calculated cost.

[0115] Furthermore, as described above, the cost calculation method further includes receiving input of a condition related to the cost. Then, calculating the cost includes calculating the cost based on the shape of the at least one object OJa and the condition. Therefore, it is possible to calculate different costs according to the condition related to the cost. As a result, it is possible to improve the validity of the calculated cost.

[0116] Also, as described above, the condition includes at least one of the time length of the image, the area of ​​the at least one object OJa, and the number of the at least one object OJa. Therefore, it is possible to calculate different costs according to at least one of the time length of the image, the area of ​​the object OJa, and the number of the object OJa. As a result, it is possible to improve the validity of the calculated cost.

[0117] Furthermore, as described above, the at least one object OJa is a plurality of objects OJa. And, accepting the input of the condition includes accepting an input of selecting at least one object OJa from the plurality of objects OJa. Therefore, for the object OJa specified by the user, the cost of creating an image to be projected using the projector 20 can be calculated.

[0118] As described above, the cost calculation method further includes receiving an input requesting the creation of the image, and, when the input requesting the creation of the image is received, transmitting request information DGd indicating the cost and the captured image Ga to the external device 40. Therefore, after checking the calculated cost, it is possible to request the creation of the image from an external device.

[0119] The above-mentioned cost calculation method is performed by the processing device 32 included in the information processing device 30 executing the program PR. In this way, the program PR causes the computer to execute the following: estimating the shape of at least one object OJa by analyzing a captured image obtained by capturing an image of the at least one object OJa; and calculating the cost of creating an image to be projected onto the at least one object OJa using the projector 20 based on the shape of the at least one object OJa.

[0120] 2. Variations Each of the above-mentioned exemplary embodiments may be modified in various ways. Specific modified embodiments that may be applied to each of the above-mentioned embodiments are illustrated below. Two or more embodiments selected from the following examples may be appropriately combined as long as they are not mutually contradictory.

[0121] 2-1. Variation 1 In the above embodiment, the number of objects that can be the projection target is one, but the present invention is not limited to this, and the number may be two or more. When the number of objects is two or more, an input for selecting one or more of the objects may be received. For example, by tapping an object in the user interface image GU-6 in FIG. 11, the object can be selected or deselected.

[0122] 2-2. Variation 2 In the above embodiment, a mode in which correction of the conditions indicated by the condition data DGc is accepted is exemplified, but the present invention is not limited to this mode, and a mode in which no correction is made may also be used. In this mode, step S108 in FIG. 4 is omitted. In this case, the cost may be calculated for all regions. An input indicating whether or not to calculate the cost for a region of a specified shape may be accepted before the cost calculation, and the cost may be calculated for only the specified shape. For example, if "cylinder" and "complex" are selected and "plane" is not selected, the cost may be calculated in S107 for only "cylinder" and "complex".

[0123] 3. Notes The following is a summary of this disclosure.

[0124] (Supplementary Note 1) A cost calculation method including: estimating a shape of at least one object by analyzing an image obtained by imaging the at least one object by a computer; calculating a cost associated with creating an image to be projected onto the at least one object using a projector based on the shape of the at least one object; and notifying the user of the cost.

[0125] In the above aspect of Supplementary Note 1, it is possible to estimate the cost of creating an image to be projected onto an object using a projector based on a captured image obtained by capturing an image of the object. Therefore, it is possible to forecast the cost required to create the image.

[0126] (Supplementary Note 2) The cost calculation method of Supplementary Note 1, wherein estimating the shape of the at least one object includes determining whether the shape of the at least one object is a three-dimensional shape or a two-dimensional shape. In the above aspect of Supplementary Note 2, different costs can be calculated when the shape of the object is a three-dimensional shape and when it is a two-dimensional shape. As a result, the validity of the calculated cost can be improved.

[0127] (Supplementary Note 3) The cost calculation method of Supplementary Note 1 or Supplementary Note 2, wherein estimating the shape of the at least one object includes dividing the at least one object into regions for each type of shape. In the above aspect of Supplementary Note 3, different costs can be calculated according to the type of shape of the object. As a result, the validity of the calculated cost can be improved.

[0128] (Supplementary Note 4) The cost calculation method according to any one of Supplementary Note 1 to Supplementary Note 3, wherein calculating the cost includes calculating the cost based on a unit price set for each type of shape and a ratio of the area of ​​the at least one object in the captured image. In the above aspect of Supplementary Note 4, different costs can be calculated according to the ratio of the area of ​​the object in the captured image. As a result, the validity of the calculated cost can be improved.

[0129] (Supplementary Note 5) The cost calculation method according to any one of Supplementary Note 1 to Supplementary Note 4, further comprising: determining an index value related to the brightness of the space where the image was captured by analyzing the captured image; and calculating the cost includes calculating the cost based on the shape of the at least one object and the index value. In the above aspect of Supplementary Note 5, different costs can be calculated depending on the brightness of the space where the image was captured. As a result, the validity of the calculated cost can be improved.

[0130] (Supplementary Note 6) The cost calculation method according to any one of Supplementary Note 1 to Supplementary Note 5, further comprising accepting input of cost-related conditions, and calculating the cost comprising calculating the cost based on the shape of the at least one object and the conditions. In the above aspect of Supplementary Note 6, different costs can be calculated according to cost-related conditions. As a result, the validity of the calculated cost can be improved.

[0131] (Supplementary Note 7) The cost calculation method according to Supplementary Note 6, wherein the conditions include at least one of the duration of the image, the area of ​​the at least one object, and the number of the at least one object. In the above aspect of Supplementary Note 7, different costs can be calculated according to at least one of the duration of the image, the area of ​​the object, and the number of objects. As a result, the validity of the calculated cost can be improved.

[0132] (Supplementary Note 8) The cost calculation method according to any one of Supplementary Note 1 to Supplementary Note 7, wherein the at least one object is a plurality of objects, and accepting the input of the condition includes accepting an input of selecting one or more objects from the plurality of objects. In the aspect of Supplementary Note 8 above, it is possible to calculate the cost of creating an image for projection using a projector for an object specified by a user.

[0133] (Supplementary Note 9) The cost calculation method according to any one of Supplementary Note 1 to Supplementary Note 8, further comprising: receiving an input for requesting the creation of the image; and, when the input for requesting the creation of the image is received, transmitting request information indicating the cost and the captured image to an external device. In the above aspect of Supplementary Note 9, the creation of the image can be requested from an external device after the calculated cost is confirmed.

[0134] (Appendix 10) A program that causes a computer to execute the following steps: estimating a shape of at least one object by analyzing an image obtained by imaging the at least one object; calculating a cost associated with creating an image to be projected onto the at least one object using a projector based on the shape of the at least one object; and notifying the user of the cost.

[0135] In the above aspect of Supplementary Note 10, it is possible to estimate the cost of creating an image to be projected onto an object using a projector based on a captured image obtained by capturing an image of the object. Therefore, it is possible to forecast the cost required for creating the image.

[0136] (Appendix 11) An information processing device including a processing device that executes the following operations: estimating a shape of at least one object by analyzing an image obtained by imaging the at least one object; calculating a cost associated with creating an image to be projected onto the at least one object using a projector based on the shape of the at least one object; and notifying the cost.

[0137] In the above aspect of Supplementary Note 11, it is possible to estimate the cost of creating an image to be projected onto an object using a projector based on a captured image obtained by capturing an image of the object. Therefore, it is possible to forecast the cost required for creating the image. [Explanation of symbols]

[0138] 10...camera, 20...projector, 30...information processing device, 31...storage device, 32...processing device, 32a...acquisition unit, 32b...display control unit, 32c...image analysis unit, 32d...condition setting unit, 32e...cost calculation unit, 32f...projector control unit, 33...communication device, 34...display device, 35...input device, 40...external device, 100...system, B1...button, B2...button, B3...button, B4...button, B5...button, B6...button, B7...button, B8...button, B9...button, B10...button, B11...button, DG...condition data, DGa...imaging data, DGb...analysis data, DGc...condition data, DGd...request information, F...finger, G0...imaging image, G0a...image, G1...image, GU...user interface image, GU-1...user interface image, GU-2...user interface image, GU-3...user interface image, GU-4...user interface image, GU-5...user interface image, GU-6...user interface image, GU-7...user interface image, GU-8...user interface image, Ga...captured image, Gaa...image, NW...network, OJa...object, OJb...object, PR...program, R1...area, R2...area, RC...captured area, RP...projection area, S101...step, S102...step, S103...step, S104...step, S105...step, S106...step, S107...step, S108...step, S109...step, S110...step, TB...unit price table, W...wall.

Claims

1. The computer estimating a shape of at least one object by analyzing a captured image obtained by capturing an image of the at least one object; calculating a cost associated with creating an image for projection onto the at least one object using a projector based on a shape of the at least one object; notifying said costs; Including, Cost calculation method.

2. estimating the shape of the at least one object includes determining whether the shape of the at least one object is a three-dimensional shape or a two-dimensional shape. The cost calculation method according to claim 1 .

3. estimating the shape of the at least one object includes dividing the at least one object into regions of different shape types; The cost calculation method according to claim 1 .

4. calculating the cost includes calculating the cost based on a unit price set for each type of shape and a ratio of an area of ​​the at least one object in the captured image. The cost calculation method according to claim 3.

5. The method further includes determining an index value related to the brightness of the space in which the image was captured by analyzing the captured image; Calculating the cost includes calculating the cost based on a shape of the at least one object and the index value. The cost calculation method according to any one of claims 1 to 3.

6. The method further includes accepting input of terms related to costs; calculating the cost includes calculating the cost based on a shape of the at least one object and the condition. The cost calculation method according to any one of claims 1 to 3.

7. the condition includes at least one of a time length of the image, an area of ​​the at least one object, and a number of the at least one object; The cost calculation method according to claim 6.

8. the at least one object is a plurality of objects; Receiving the input of the condition includes receiving an input of selecting one or more objects from the plurality of objects. The cost calculation method according to any one of claims 1 to 3.

9. accepting an input requesting creation of the image; when receiving an input for requesting creation of the image, transmitting request information indicating the cost and the captured image to an external device; Further comprising: The cost calculation method according to any one of claims 1 to 3.

10. estimating a shape of at least one object by analyzing a captured image obtained by capturing an image of the at least one object; calculating a cost associated with creating an image for projection onto the at least one object using a projector based on a shape of the at least one object; notifying said costs; Have the computer execute program.

11. estimating a shape of at least one object by analyzing a captured image obtained by capturing an image of the at least one object; calculating a cost associated with creating an image for projection onto the at least one object using a projector based on a shape of the at least one object; notifying said costs; a processor for executing Information processing device.

Images