Display control device, control method, computer program, computer-readable medium, and manufacturing method

Abstract

To allow for effectively supporting production in the real space.SOLUTION: A display device is provided, comprising detection means for detecting feature points of a product being produced by a user from a real image, posture adjustment means configured to adjust the posture of a model so as to match the detected feature points, and control means configured to display an image obtained by superimposing the posture-adjusted model on the real image.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a display control device, a control method, a computer program, a computer-readable medium, and a manufacturing method. [Background technology]

[0002] In recent years, technologies such as Augmented Reality (AR) and Mixed Reality (MR) have been developing to provide user experiences in environments that combine reality and virtuality. For example, technologies such as overlaying virtual objects on images captured with a smartphone, or using a head-mounted display (HMD) equipped with various sensors to capture user movements and display them in sync with the movements in the mixed world, can provide unprecedented user experiences.

[0003] AR and MR are becoming increasingly popular at the individual level, and are being used in a variety of areas, including everyday life, learning, and creative work. For example, Patent Document 1 shows a system that supports the creation of illustrations by overlaying a model on the smartphone screen when drawing. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] US Patent Application Publication No. 2020 / 0363931 Summary of the Invention [Problem to be solved by the invention]

[0005] Patent Document 1 detects the drawing area (such as the edge of a piece of paper) in a captured image of the user's hand, and then superimposes a model image that has been set in advance so that it follows the drawing area. Detecting the drawing area has the advantage of eliminating the need for extra work such as marking the position.

[0006] However, due to the nature of detecting feature points from a two-dimensional production area, if the production area is lost due to a change in the work environment, the expected superimposed display may not be possible.

[0007] An object of the present invention is to provide a technique for effectively supporting production in real space. [Means for solving the problem]

[0008] A first aspect of the present invention is a search means for searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; A detection means for detecting feature points of a creation being created by a user from a real image; a posture adjustment means for adjusting the posture of the model so that it matches the detected feature points; a control means for displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with 、 the posture adjustment means adjusts the posture of the model by rotating, moving, or enlarging or reducing the model; When there are a plurality of posture adjustment methods for matching one example data with the detected feature points, the control means generates a screen for presenting the example data after each adjustment to the user as an example candidate. table It is a display control device. A second aspect of the present invention is a search means for searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; A detection means for detecting feature points of a creation being created by a user from a real image; a posture adjustment means for adjusting the posture of the model so that it matches the detected feature points; a control means for displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with The control means generates an image for presenting the plurality of models searched by the search means as model candidates to the user, and sets the model candidate selected by the user as the model to be superimposed on the real image. A display control device. A third aspect of the present invention is a search means for searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; A detection means for detecting feature points of a creation being created by a user from a real image; a posture adjustment means for adjusting the posture of the model so that it matches the detected feature points; a control means for displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with the model data is stored in the storage means in association with a drawing start position as an attribute; the detection means detects a start position of the creation by the user; the posture adjustment means adjusts the posture of the model so that a start position of the creation by the user coincides with a start position of the drawing stored as an attribute of the model data. A display control device.

[0009] The present invention four The present invention provides a control method performed by a computer, comprising: a search step of searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; a feature point detection step of detecting feature points of an object being created by a user from a real image; a posture adjustment step of adjusting the posture of the model so that it matches the detected feature points; a display screen generating step of generating an image in which the posture-adjusted model is superimposed on the real image; Equipped with 、 In the posture adjustment step, the posture of the model is adjusted by performing at least one of rotation, movement, and enlargement / reduction on the model; When there are a plurality of posture adjustment methods for matching one example data with the detected feature points, the control step generates a screen for presenting the example data after each adjustment to the user as an example candidate. system This is the method. A fifth aspect of the present invention is a control method performed by a computer, comprising: a search step of searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; a detection step of detecting feature points of the creation being made by the user from the real image; a posture adjustment step of adjusting the posture of the model so that it matches the detected feature points; a control step of displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with The control step generates an image for presenting the plurality of models searched for in the search step as model candidates to the user, and sets the model candidate selected by the user as the model to be superimposed on the real image. It is a control method. A sixth aspect of the present invention is a control method performed by a computer, comprising: a search step of searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; a detection step of detecting feature points of the creation being made by the user from the real image; a posture adjustment step of adjusting the posture of the model so that it matches the detected feature points; a control step of displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with the model data is stored in the storage means in association with a drawing start position as an attribute; In the detecting step, a start position of the creation by the user is detected; In the posture adjustment step, the posture of the model is adjusted so that a start position of the creation by the user coincides with a start position of the drawing stored as an attribute of the model data. It is a control method. [Effects of the Invention]

[0010] According to the present invention, production in real space can be effectively supported. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a hardware configuration of a production support system. [Figure 2] FIG. 2 is a diagram showing an example of a software logical configuration of the production support system. [Figure 3] FIG. 10 is a diagram illustrating an example of a table of user data. [Figure 4] FIG. 10 is a diagram illustrating an example of a table of model data. [Figure 5] FIG. 10 is a diagram illustrating an example of a table of model candidates. [Figure 6] 1 is a diagram illustrating an overview of a production support system according to a first embodiment. [Figure 7] FIG. 10 is a diagram illustrating an example of a method for presenting candidates for presentation frames in the first embodiment. [Figure 8] 10 is a flowchart of a model candidate search in the first embodiment. [Figure 9] 10 is a flowchart of model determination in the first embodiment. [Figure 10] FIG. 11 is a diagram illustrating an overview of a production support system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the following description is merely an example of an embodiment of the present invention and is not intended to limit the present invention. The present invention can be implemented in various ways within the scope of its technical concept.

[0013] This embodiment is a production support system that supports a user in creating a work. The production support system according to this embodiment supports the user's creation by superimposing a model image of the work the user is trying to create on a real image. The work may be a two-dimensional image (such as an illustration) drawn on paper, a whiteboard, or a display, or a three-dimensional object produced by processing materials such as wood, clay, or ice.

[0014] The production support system includes a display (display device), a camera (image capture device), an input / output device, and a computer (display control device) that controls what is displayed on the display.

[0015] FIG. 1 is a diagram showing an example of the hardware configuration of a computer included in a production support system 100. The production support system 100 of this embodiment has the same configuration as a typical computer, and includes a CPU 101, a ROM 102, a RAM 103, and an I / F 104. These components are connected via a bus 105. The production support system 100 is realized by loading a computer program for implementing the functions described below from a computer-readable medium into the RAM 103 and having the CPU 101 execute the program. The RAM 103 is also used as a work memory for storing temporary data for processing by the CPU 101. The I / F 104 is an interface for communicating with the outside world, and receives input of image data from the real world and data for determining the user's situation, and outputs image data to be displayed.

[0016] Although only one CPU is shown in FIG. 1, the image processing device may be realized by multiple processors. Also, an auxiliary component such as a graphics processing unit (GPU) may be included. Furthermore, while only RAM 103 is shown as a component for holding temporary work memory, secondary and tertiary storage areas may be provided on the same or different media. Other possible media include a hard disk drive (HDD) and a solid-state drive (SSD). The configuration of bus 105 is not limited to this, and components may be connected in multiple stages.

[0017] 2 is a diagram showing an example of the software logical configuration of the production support system 100. This software logical configuration is common to all embodiments.

[0018] User data 221 stores information such as user attributes and past operation history. Model data 222 stores multiple models to support user creation. The data format of model data 222 is not particularly limited, and may be two-dimensional data or three-dimensional data.

[0019] The model search unit 213 searches the model data 222 for one or more models that are candidates for being superimposed on real-world video for production support. The model search unit 213 performs a search before production begins, for example, in accordance with user attribute information registered by the user and input information about what to produce. The model candidate 223 represents one or more models selected by the model search unit 213.

[0020] The feature point detection unit 211 acquires a real image of the user's field of view from the imaging unit 201, and detects feature points of the work being created by the user from the real image. The feature point detection unit 211 may be realized using any existing technology such as SIFT or HOG.

[0021] The model posture adjustment unit 212 adjusts the model posture registered in the model candidate 223 so that it matches the detected feature points. Specifically, the model posture adjustment unit 212 adjusts the model posture in various ways by adjusting at least one of the position, rotation, and scaling of the model, and identifies the posture that best matches the detected feature points. The posture adjustment method is not particularly limited. For example, the model posture adjustment unit 212 may determine the adjustment method based on the detected feature points and the model posture, or may adjust the model posture multiple times using predetermined methods and select the adjustment method that best matches the detected feature points. The model posture adjustment unit 212 may also determine the adjustment method using a model learned by machine learning. The identified posture is stored in the model candidate 223 as the posture to be superimposed on the real image. Furthermore, if there are multiple posture adjustment methods for one model to match the detected feature points, the model after each adjustment may be stored in the model candidate 223.

[0022] The model superimposition unit 214 generates an image in which the model candidate 223, whose posture has been adjusted, is superimposed on the real image captured by the imaging unit 201. The model superimposition unit 214 uses the posture information determined by the model posture adjustment unit 212 when superimposing the model candidate 223 on the real image.

[0023] The display screen generation unit 215 generates a screen that displays the image generated by the model superimposition unit 214 as a presentation image of a model candidate, and outputs the screen to the display unit 202. The model superimposition unit 214 and the display screen generation unit 215 can be considered as a display control means that controls the display on the display unit 202.

[0024] The model selection unit 216 sets the model selected by the user from the presented models using the operation unit 203 as a model candidate 223 to be continuously superimposed and displayed.

[0025] Thereafter, the model posture adjustment unit 212 and the model superimposition unit 214 adjust the model posture and superimpose it with the real image only on the selected model. In addition, the display screen generation unit 215 generates a screen that continuously displays the image generated by the model superimposition unit 214 as the final model across the entire field of view of the HMD, and outputs it to the display unit 202. The user data 221 and model data 222 described in this logical configuration may be configured to be stored on an external storage device via a network.

[0026] FIG. 3 shows an example of a table of user data stored in the user data 221. "ID" indicates a unique ID for each user. "User name" indicates a name that identifies the user. A user name may be an actual name, or an account name such as an email address. "Attributes" indicates attribute information that classifies users. Attributes may include gender, age, and occupation. "History" indicates information about model data that the user has used in the past. The history may be only the most recently used data, or a list of data used within a certain period of time.

[0027] 4 shows an example of a table of model data stored in the model data 222. As shown in the figure, the model data is stored in the storage unit in association with an ID, a thumbnail, an attribute, and a start position.

[0028] "ID" indicates a unique ID assigned to each sample data.

[0029] "Model" refers to the actual model data. The data format of the model data is not particularly limited. For example, when supporting two-dimensional production such as drawing an illustration, the model may be two-dimensional data or three-dimensional data. Furthermore, when supporting three-dimensional production such as model creation, it is preferable to use three-dimensional data, but two-dimensional data is also acceptable. In this way, the data format of the model can be determined according to the intended use.

[0030] A "thumbnail" refers to a snapshot image of a model in any pose. Thumbnails are used, for example, to display in the corner of a presentation frame so that it is easy to identify which model is being displayed in the presentation frame. The data format of the thumbnail can be determined according to the intended use.

[0031] "Attributes" indicates attribute information for classifying the model. The attribute information may include classification in sub-categories, such as "type" indicating the characteristics of the model itself and "scene" indicating the use case in which the model can be used.

[0032] The "start position" indicates the starting position when drawing the model. Assuming that the model data is drawn in a two-dimensional drawing area of ​​a specific size, the coordinate system is expressed with the upper left corner as the starting point, the horizontal axis as the x-axis, and the vertical axis as the y-axis. The start position is used to determine the position of the model to be superimposed depending on where the user places the pen when drawing. Only one start position may be defined for each model, or multiple start positions (e.g., 2 to 4) may be defined. When there are two start positions, the coordinate information to be used may be selected depending on whether the drawing area is divided into two halves (left and right) and the user starts drawing from the upper left or upper right. When there are four start positions, the coordinate information to be used may be selected depending on whether the drawing area is divided into four halves (top left, top right, bottom left, or bottom right) and the user starts drawing from the upper left, top right, bottom left, or bottom right. Specific usage of the "start position" will be explained in Example 3.

[0033] Figure 5 shows an example of a table of example data stored in the example candidate 223. "ID" indicates a unique ID for each example candidate. "Example" indicates a unique ID for each example data listed in the example data table. "Posture" indicates the posture of the example that matches the current feature points. The posture is composed of position, rotation, and scaling. Position indicates the amount of movement from the example's initial coordinates for each (x, y, z) axis component. Rotation indicates the amount of rotation from the example's initial position for each (x, y, z) axis component. Scaling indicates the magnification of the scale from the example's initial size for each (x, y, z) axis component. If there is no posture that matches the feature points, this field is left blank. "Selected" indicates that the example is the one that will continue to be displayed as the final example across the entire field of view of the HMD.

[0034] Example 1 FIG. 6 shows an image displayed on the HMD worn by the user by the production support system in this embodiment. FIG. 10 is a diagram illustrating an example of a surface.

[0035] Screen 601 shows a screen displayed across the entire field of view of the HMD. An image representing real space and an image generated by the production support system 100 are superimposed on screen 601. In Fig. 6, the paper, pen, and lines drawn on the paper are the image of real space, and a model image 604 is superimposed on this image of real space. The model image displayed superimposed on the real production will also be referred to as a support image below.

[0036] Presentation frames 602 and 603 are presentation frames that display candidate model images to be superimposed. The images displayed in presentation frames 602 and 603 are referred to as presentation images. In this embodiment, the user first registers in advance with the production system that they will draw a "dog." Then, only model data containing the attribute information of a "dog" is picked up as candidate model images. An image of the user's hands is displayed on screen 601. When the user begins drawing, the system detects feature points of the drawing result from the image and selects a model of a "dog" that matches the feature points by adjusting its posture (movement, rotation, scaling). The selected model is superimposed on the real image to match the feature points and displayed in presentation frames 602 and 603. While two model images are displayed here, three or more model images may also be displayed. When multiple candidates are presented, they are presented in order of highest degree of match. If there are more model images than the number of presentation frames, new model candidates may be displayed one after another in response to user operations such as gestures. When the user selects the model image they wish to continue to superimpose from the presented model images, model image 604 is displayed superimposed on the real image on screen 601. Thereafter, the selected model image is continuously displayed in a posture that follows the feature points of the drawing result.

[0037] In this embodiment, the model data may be two-dimensional data, but the model may be three-dimensional data. Images displayed in the presentation frame may also be treated as the same presentation candidate when the same model has different postures with the same coordinates of some of its feature points. In this case, the user can freely change the posture of the model while maintaining the coordinates of some of its feature points by using gestures, as shown in FIG. 7.

[0038] Fig. 8 is a flowchart of the initial setting in Example 1. The process in Fig. 8 is performed as the initial setting before the user starts actual production.

[0039] In step S801, the production support system 100 registers user data based on input from the user. In this embodiment, the production support system 100 receives a designation of the type of product to be produced from the user, and registers the received type of product as user data 221. For example, if the user wants to draw a picture of a "dog," the type of product to be produced is registered in the user data 221 as "dog."

[0040] Step S802 is processing by the model search unit 213, which searches for model candidate data from all of the model data. If the type of work is specified, the model search unit 213 picks out models having attributes that match the specified type from the model data 222 as model candidates. For example, if it is registered that a picture of a "dog" is to be drawn, the model search unit 213 picks out models having the attribute "dog" from the model data as model candidates.

[0041] FIG. 9 is a flowchart showing the process up to model determination in the first embodiment.

[0042] In step S901, the feature point detection unit 211 detects feature points from the rendering result of the product captured as the user's field of view video.

[0043] In step S902, the model posture adjustment unit 212 adjusts the model posture of the model data picked up as a model candidate so that it matches the feature points detected in step S901. The calculated posture is reflected in the posture attribute of the model candidate.

[0044] In step S903, the model superimposition unit 214 generates an image for presenting the multiple models picked up as model candidates to the user as model candidates. More specifically, the model superimposition unit 214 adjusts the posture of the model data by referring to the posture attributes reflected in each model candidate, and generates an image in which the posture-adjusted model is superimposed on the real image.

[0045] In step S904, the display screen generation unit 215 generates a presentation screen to be displayed in the presentation frame using the image generated in step S903.

[0046] In step S905, when the user selects a model to be displayed as a support image from the model candidates presented in step S904, the model selection unit 216 registers the model as a support image in the system.

[0047] In step S906, the display screen generation unit 215 generates a screen for displaying the model support image set in the system in step S905 as a main image on the display unit using the superimposed image generated by the model superimposition unit 214.

[0048] Although not described in the flowchart from here on, the detection of feature points by the feature point detection unit 211, the adjustment of the model posture by the model posture adjustment unit 212, and the superimposition of the model by the model superimposition unit will continue for the model selected by the user in step S905.

[0049] According to this embodiment, the user can receive support in the creation process by simply specifying the type of work to be created, and an appropriate model is selected and superimposed on the real image. The model is selected based on the pre-specified type and the feature points of the work actually created by the user, so an appropriate model can be selected without the user having to specify a specific model in advance. In addition, the model's posture is adjusted to match the feature points of the work, so the model follows the real work. Furthermore, since the four corners of the drawing area are not used to determine the model's posture, the model can continue to be tracked and displayed even if the four corners of the drawing area are lost.

[0050] Example 2 In the first embodiment, the type of work to be created by the user is specified in advance, but in this embodiment, a model is selected without specifying the type of work. Specifically, in this embodiment, a model is selected based on the attribute information of the user.

[0051] The screen displayed by the production support system 100 is the same as in the first embodiment (FIG. 6). In this embodiment, in step S801 of the initial processing (FIG. 8), the user registers their own attribute information in advance in the production system as user data 221. In step S802, the model search unit 213 picks out models that match the user attributes as model candidates. For example, if the user has the attribute "elementary school student," models that have the related model attribute "education" are picked out. The correspondence between the user attributes and the model attributes may be determined in advance, for example. Furthermore, the order in which the models are presented in the presentation frames 902 and 903 may be determined based on the degree of match with the user attributes as well as the degree of match with the feature points.

[0052] This embodiment can also achieve the same effects as those of embodiment 1. Note that embodiment 1 and embodiment 2 may be combined to extract model candidates using both the type of product and the user attributes.

[0053] Example 3 In this embodiment, the display position of the model is determined taking into consideration the starting position of the drawing by the user.

[0054] 10 is a diagram showing an example of a screen displayed by the production support system in this embodiment. Screen 1001 shows a screen displayed across the entire field of view of the HMD. Presentation frames 1002 and 1003 are presentation frames showing candidate models to be superimposed. In this embodiment, it is assumed that the user has decided in advance on the model to be used for drawing.

[0055] In this embodiment, the system detects the start position of the user's drawing (the position where the pen is placed) as a feature point. The system aligns the detected start position of the drawing with the attribute information of the start position of the model, and superimposes the model image. Specifically, the model posture adjuster 212 adjusts the position of the model so that the start position included in the model data (see Figure 4) matches the start position of the user's drawing. The posture of the model (rotation, scaling) other than the position can be determined appropriately. For example, rotation may be the posture when the user selects the model. Scaling may be set to the maximum size that maintains an arbitrary margin for the drawing area, or to a predetermined size.

[0056] In this embodiment, the start position of the image may be determined depending on which area in the drawing area the pen is placed in. As described above, the model data 222 may store a plurality of associated drawing start positions. For example, the model data 222 may define start positions for when the drawing area is divided into two, left and right, and writing begins in each area. In this case, the model posture adjustment unit 212 uses the corresponding start position depending on which area of ​​the drawing area the user's drawing start position falls into. For example, if the user places the pen in the upper left of the drawing area, the upper left coordinate of the start position (Figure 4) is used. If the pen is placed in the upper right, the upper right coordinate of the start position (Figure 4) is used. The same applies to the bottom left and bottom right.

[0057] Thereafter, the model posture adjustment unit 212 adjusts the posture of the model so that the model follows the starting position of the user's drawing.

[0058] According to this embodiment, the drawing position of the model is determined according to the starting position of the drawing by the user, so that the model can be tracked and continued to be displayed even if the four corners of the drawing area are lost.

[0059] In the above description, it is assumed that the model is determined in advance, but the model may also be selected according to the start position of the drawing. The model selection based on the start position of the drawing may be performed based on model data that has been registered in advance, or based on the user's past history (the correspondence between the drawing start position and the type of work). If multiple candidates are selected, they may be presented in presentation frames 1001 and 1002, as in the first embodiment, so that the user can select which candidate to use. In this way, the effort of specifying the model in advance can be eliminated.

[0060] Example 4 In this embodiment, support is provided for three-dimensional production. In this embodiment, the system detects feature points from two-dimensional images of a three-dimensional object. The system detects feature points carved out by the user through machining such as cutting the material, and determines whether they match a specific pose of the model in two dimensions and superimposes them. The logic for superimposing is the same as for two-dimensional models, so a detailed explanation will be omitted.

[0061] (Other Examples) It should be noted that Examples 1 to 4 are merely examples, and configurations obtained by appropriately modifying or changing the configurations of Examples 1 to 4 within the scope of the gist of the present invention are also included in the present invention.

[0062] The present invention can also be implemented as a head-mounted display equipped with a display and a camera, or as a smartphone worn on the user's head like goggles.The present invention can also be realized as a computer (display control device) to which at least one of a display and a camera is externally attached, or as a computer program for causing the computer to function as a display control device.

[0063] The present invention can also be considered as a method for producing a work using the production support system. That is, the present invention can be considered as a production method including the steps of checking a model displayed by the production support system (display control device) and drawing an image or processing materials based on the displayed model. By producing a work with the support of the production support system in this way, the user can easily produce a work that matches the model.

[0064] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0065] 211: Feature point detection unit 212: Model posture adjustment unit 215: Display screen generation unit

Claims

1. a search means for searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; A detection means for detecting feature points of a creation being created by a user from a real image; a posture adjustment means for adjusting the posture of the model so that it matches the detected feature points; a control means for displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with the posture adjustment means adjusts the posture of the model by rotating, moving, or enlarging or reducing the model; When there are a plurality of posture adjustment methods for matching one example data with the detected feature points, the control means generates a screen for presenting the example data after each adjustment to the user as an example candidate. Display control device.

2. the model data is stored in the storage means in association with attributes, The search means searches for a model that matches the model attributes specified by the user. The display control device according to claim 1 .

3. the model data is stored in the storage means in association with attributes, the search means searches for a model having attributes that match the attributes of the user; The display control device according to claim 1 .

4. The control means generates an image for presenting the plurality of models searched by the search means as model candidates to the user, and sets the model candidate selected by the user as the model to be superimposed on the real image. The display control device according to claim 1 .

5. the model data is stored in the storage means in association with a drawing start position as an attribute; the detection means detects a start position of the creation by the user; the posture adjustment means adjusts the posture of the model so that a start position of the creation by the user coincides with a start position of the drawing stored as an attribute of the model data. The display control device according to claim 1 .

6. the model data is stored in the storage means in association with a plurality of start positions of the drawing; the posture adjustment means selects a start position of one of the drawings according to a start position of the creation by the user, and adjusts the posture of the model so that the start position of the creation by the user and the start position of the selected drawing coincide with each other. The display control device according to claim 5 .

7. The real image is an image captured of the user's field of view. The display control device according to claim 1 .

8. an imaging means for acquiring the real image; a display means for displaying an image generated by the control means; The display control device according to claim 1 , further comprising:

9. The work is a two-dimensional work. The display control device according to claim 1 .

10. The work is a three-dimensional work. The display control device according to claim 1 .

11. a search means for searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; A detection means for detecting feature points of a creation being created by a user from a real image; a posture adjustment means for adjusting the posture of the model so that it matches the detected feature points; a control means for displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with the model data is stored in the storage means in association with attributes, the search means searches for a model having attributes that match the attributes of the user; Display control device.

12. a search means for searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; A detection means for detecting feature points of a creation being created by a user from a real image; a posture adjustment means for adjusting the posture of the model so that it matches the detected feature points; a control means for displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with the model data is stored in the storage means in association with a drawing start position as an attribute; the detection means detects a start position of the creation by the user; the posture adjustment means adjusts the posture of the model so that a start position of the creation by the user coincides with a start position of the drawing stored as an attribute of the model data. Display control device.

13. A control method performed by a computer, comprising: a search step of searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; a feature point detection step of detecting feature points of an object being created by a user from a real image; a posture adjustment step of adjusting the posture of the model so that it matches the detected feature points; a display screen generating step of generating an image in which the posture-adjusted model is superimposed on the real image; Equipped with In the posture adjustment step, the posture of the model is adjusted by performing at least one of rotation, movement, and enlargement / reduction on the model; When there are a plurality of posture adjustment methods for matching one example data with the detected feature points, the display screen generation step generates a screen for presenting the example data after each adjustment to the user as an example candidate. Control method.

14. A control method performed by a computer, comprising: a search step of searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; a detection step of detecting feature points of the creation being made by the user from the real image; a posture adjustment step of adjusting the posture of the model so that it matches the detected feature points; a control step of displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with the model data is stored in the storage means in association with attributes, In the search step, a model having attributes matching the attributes of the user is searched for. Control method.

15. A control method performed by a computer, comprising: a search step of searching for a model to be superimposed on the real image and displayed from among a plurality of model data stored in a storage means; a detection step of detecting feature points of the creation being made by the user from the real image; a posture adjustment step of adjusting the posture of the model so that it matches the detected feature points; a control step of displaying an image in which the posture-adjusted model is superimposed on the real image; Equipped with the model data is stored in the storage means in association with a drawing start position as an attribute; In the detecting step, a start position of the creation by the user is detected; In the posture adjustment step, the posture of the model is adjusted so that a start position of the creation by the user coincides with a start position of the drawing stored as an attribute of the model data. Control method.

16. A computer program for causing a computer to function as each of the means of the display control device according to any one of claims 1 to 12.

17. A computer-readable medium storing a computer program for causing a computer to function as each of the means of the display control device according to any one of claims 1 to 12.

18. a step of checking a model using the display control device according to any one of claims 1 to 12; Drawing an image or processing a material based on the model; A method for manufacturing a product, comprising:

Images