Simulation device and simulation program

Abstract

This simulation device comprises: a storage unit that stores data of a plurality of three-dimensional models respectively corresponding to a plurality of objects serving as candidates to be placed in a simulation space and data of a plurality of two-dimensional models respectively corresponding to the plurality of objects; a display unit that displays the simulation space; a reception unit that receives a user operation for drawing a line drawing freehand in the simulation space; a two-dimensional model selection unit that selects, from the plurality of two-dimensional models, a two-dimensional model which is similar to the line drawing; a three-dimensional model selection unit that selects, from the plurality of three-dimensional models, a three-dimensional model of an object corresponding to the selected two-dimensional model; and a placement unit that places the selected three-dimensional model in the simulation space on the basis of the position of the line drawing.

Description

Simulation device and simulation program

[0001] The present disclosure relates to a simulation device and a simulation program.

[0002] A simulation device that uses simulation to teach a robot an operation program is known (see, for example, Patent Document 1). The simulation device displays a teaching screen for teaching the robot an operation program. The teaching screen displays a three-dimensional simulation space, and a user must arrange three-dimensional models of objects, such as a robot, peripheral devices, and structures, within the simulation space so that they correspond to their actual positions. For example, an object menu containing multiple types of objects with different shapes is displayed on the teaching screen so that the user can arrange the objects within the simulation space. The user selects an object from the object menu, places the selected object at a desired position within the simulation space, and adjusts its position and size after placement to complete the object arrangement within the simulation space. These user operations are performed using a mouse, keyboard, etc., but require various types of operations, such as selecting an object, placing the object, adjusting its position, and adjusting its size. Therefore, placing objects within the simulation space can be a very time-consuming task for users who are not familiar with the operation.

[0003] JP 2008-021092 A

[0004] There is a need for a technique that simplifies the process of placing objects in a simulation space.

[0005] The simulation device according to the present disclosure includes a memory unit that stores data on a plurality of three-dimensional models corresponding to a plurality of objects as candidates for placement in a simulation space, and data on a plurality of two-dimensional models corresponding to the plurality of objects, respectively; a display unit that displays the simulation space; a reception unit that receives user operations for drawing a line drawing freehand within the simulation space; a two-dimensional model selection unit that selects a two-dimensional model that approximates the line drawing from the plurality of two-dimensional models; a three-dimensional model selection unit that selects a three-dimensional model of an object corresponding to the selected two-dimensional model from the plurality of three-dimensional models; and a placement unit that places the selected three-dimensional model within the simulation space based on the position of the line drawing.

[0006] FIG. 1 is a diagram showing a simulation system including a simulation device according to this embodiment. FIG. 2 is a hardware configuration diagram of the simulation device according to this embodiment. FIG. 3 is a functional block diagram of the simulation device according to this embodiment. FIG. 4 is a diagram showing an example of an object information management table stored in the storage unit of FIG. 3. FIG. 5 is a diagram showing an example of a simulation screen displayed on the display unit of FIG. 3. FIG. 6 is a supplementary diagram for explaining the 2D model selection unit of FIG. 3. FIG. 7 is a supplementary diagram for explaining the 3D model placement unit of FIG. 3. FIG. 8 is a flowchart showing an example of the procedure for object selection and placement processing by the simulation device according to this embodiment. FIG. 9 is another supplementary diagram for explaining the 2D model selection unit of FIG. 3.

[0007] The simulation device according to this embodiment will be described below with reference to the drawings. In the following description, components having substantially the same functions and configurations are designated by the same reference numerals, and redundant description will be given only when necessary.

[0008] Fig. 1 shows a simulation system including a simulation device according to this embodiment. As shown in Fig. 1, in the simulation system 1, a simulation device 2 is connected to a control device 7 that controls a robot 6 via a network 90 such as the Internet. A control program for the robot 6 created by the simulation device 2 is sent from the simulation device 2 to the control device 7 and executed by the control device 7, causing the robot 6 to perform a predetermined operation.

[0009] The simulation device 2 is a computer device having the following functions: displaying a simulation space; receiving user operations for drawing a line drawing freehand in the simulation space; selecting a 2D model that approximates the line drawing from a plurality of 2D models; selecting a 3D model of an object corresponding to the 2D model selected by the selection function from a plurality of 3D models; and arranging the selected 3D model in the simulation space based on the position of the line drawing. The simulation device 2 is realized by a PC, a tablet, or a smartphone. The simulation device 2 may be standalone and not connected to any other device.

[0010] In this embodiment, the "line drawing" is drawn freehand by the user using only lines to form the general shape of the object desired to be placed. Here, "freehand" means that the user draws freely, and does not limit the tool used to draw the line drawing. The user may use a tool that allows the user to draw lines freely, or a tool that allows the user to draw predetermined shapes such as circles, rectangles, or ellipses, or may combine these tools to draw the line drawing. In this embodiment, the information obtained from the line drawing is the shape, size, and position of the line drawing. Therefore, as long as these information can be obtained, the information is not limited to a line drawing, and may be, for example, a picture.

[0011] 2 is a hardware configuration diagram of the simulation device 2. As shown in Fig. 2, a RAM 12, a ROM 13, a storage device 14, an input controller 15, a display controller 16, and a communication device 17 are connected to a processor 11 via a data / control bus 10. An optional input device 18 such as a keyboard, a mouse, or operation buttons is connected to the input controller 15. An optional display device 19 such as an organic EL display is connected to the display controller 16.

[0012] The processor 11 is realized by a CPU, a GPU, etc. The RAM 12 functions as the main memory, work area, etc. of the processor 11. The ROM 13 stores the BIOS, OS, etc. The storage device 14 stores a simulation program. The communication device 17 is realized by a communication module conforming to any communication standard, and transmits and receives various data to and from external devices such as the control device 7 under the control of the processor 11.

[0013] 3 is a functional block diagram of the simulation device 2. When the processor 11 executes the simulation program loaded from the storage device 14 to the RAM 12, the simulation device 2 functions as a reception unit 21, a display unit 22, a transmission / reception unit 23, a storage unit 24, a screen creation unit 25, a simulation space creation unit 26, a projection processing unit 27, a 2D model selection unit 28, a 3D model selection unit 29, and a 3D model placement unit 30.

[0014] The reception unit 21 receives a user operation via the input device 18. Specifically, the reception unit 21 receives a user operation for drawing a line drawing freehand in the simulation space displayed on the simulation screen.

[0015] The display unit 22 is realized by the function of the display device 19 shown in Fig. 2. The display unit 22 displays a simulation screen created by the screen creation unit 25.

[0016] The transmitting / receiving unit 23 is realized by the function of the communication device 17 shown in Fig. 2. The transmitting / receiving unit 23 transmits and receives data to and from the control device 7.

[0017] The storage unit 24 is realized by the functions of the storage device 14 shown in Fig. 2. The storage unit 24 stores an object information management table for managing object information, data on a plurality of two-dimensional models relating to a plurality of objects managed in the object information management table, and data on a plurality of three-dimensional models relating to a plurality of objects managed in the object information management table.

[0018] The screen creation unit 25 creates various screens related to the simulation program. Specifically, the screen creation unit 25 creates a simulation screen. The simulation screen displays the simulation space created by the simulation space creation unit 26. The simulation space displayed on the simulation screen is a two-dimensional simulation space image projected from an arbitrary direction from the three-dimensional simulation space.

[0019] The simulation space creation unit 26 creates a software simulation space (virtual space) that represents the operating space of the robot 6 in three dimensions.

[0020] The projection processing unit 27 generates data of a two-dimensional simulation space image by projecting the three-dimensional simulation space created by the simulation space creation unit 26 from an arbitrary direction, with an arbitrary position on the simulation space as the projection center position. The simulation space image is displayed on the simulation screen. The projection direction is also called the line of sight direction. The projection method may be parallel projection or far-sighted projection. The line of sight direction and the projection center position can be arbitrarily changed by a user operation on the simulation space image. The projection processing unit 27 repeatedly generates a simulation space image every time one of the line of sight direction and the projection center position is changed, and the simulation space image displayed on the simulation screen is repeatedly updated.

[0021] The two-dimensional model selection unit 28 selects, from a plurality of two-dimensional models, a two-dimensional model that approximates a two-dimensional line drawing drawn by the user on the simulation space image (displayed simulation space).

[0022] The three-dimensional model selection unit 29 selects one three-dimensional model from the plurality of three-dimensional models, which has an object in common with the two-dimensional model selected by the two-dimensional model selection unit 28 .

[0023] The three-dimensional model placement unit 30 places, within the simulation space, the three-dimensional model selected by the three-dimensional model selection unit 29. Specifically, the three-dimensional model placement unit 30 places the three-dimensional model within the simulation space based on the position of the line drawing drawn by the user on the simulation space image.

[0024] The object information management table stored in the storage unit 24 will be described below with reference to FIG. 4. FIG. 4 shows an example of the object information management table. As shown in FIG. 4, in the object information management table, an ID for identifying an object is associated with the name of the object, a two-dimensional model of the object, a three-dimensional model of the object, and placement conditions. For example, a two-dimensional model of an object is a three-dimensional model of the object projected from a specific line of sight direction. 2D models of the same object with different postures are distinguished from each other. For example, in FIG. 4, ID "001" is an ID for identifying a roller conveyor (high), and a two-dimensional model in which the roller conveyor (high) is arranged horizontally is distinguished from a two-dimensional model in which the roller conveyor (high) is arranged vertically, and IDs "001-1" and "001-2" are assigned to these, respectively.

[0025] The simulation screen displayed on the display unit 22 will be described below with reference to FIG. 5. FIG. 5 shows an example of the simulation screen. As shown in FIG. 5, a simulation space image 110 is displayed on the simulation screen 100, in which an arbitrary position in the three-dimensional simulation space is set as the projection center position and the three-dimensional simulation space is projected from the line of sight. Here, it is assumed that a three-dimensional model of the robot 6 is placed in the three-dimensional simulation space. Therefore, the simulation space image 110 displayed on the simulation screen 100 includes a robot model image 120 in which the three-dimensional model of the robot 6 is projected from the line of sight.

[0026] Clicking the start button 190 triggers the display of a simulation space image 110 projecting a three-dimensional simulation space from a predetermined viewing direction (hereinafter referred to as a specific viewing direction). The system accepts a user operation in which the user draws a line drawing 130 freehand on the simulation space image 110 corresponding to the specific viewing direction. Typically, the specific viewing direction is determined so as not to be parallel to each of the three axes (X-axis, Y-axis, and Z-axis) defined in the simulation space. In other words, the specific viewing direction is determined to be inclined relative to each of the three axes (X-axis, Y-axis, and Z-axis) defined in the simulation space. This allows the line drawing 130 drawn by the user on the simulation space image 110 corresponding to the specific viewing direction to take into account the length, width, and height (three-axis directions) of the object, thereby improving the accuracy of selecting an object that resembles the line drawing. The projection magnification (display magnification) and projection center position may be predetermined, similar to the specific viewing direction, or may be arbitrarily set by the user.

[0027] The processing by the 2D model selection unit 28 will be described in detail below with reference to FIG. 6 . FIG. 6 is a supplementary diagram for explaining the 2D model selection unit 28 of FIG. 3 . Here, it is assumed that a line drawing 130 is drawn on the simulation space image 110 corresponding to a specific line of sight direction. For example, the 2D model selection unit 28 calculates a score representing the degree of approximation of each of the multiple 2D models to the line drawing 130. For example, the score is calculated based on the proportion of the area of ​​the 2D model to the area defined by the line drawing 130. The 2D model selection unit 28 repeatedly calculates the score while enlarging, reducing, and translating the 2D model relative to the line drawing 130, and the maximum score is determined as the score of the 2D model. The 2D model selection unit 28 selects one 2D model 200 with the highest score from the multiple 2D models. In other words, the 2D model selection unit 28 selects the 2D model 200 of an object having a general shape closest to the frame shape represented by the line drawing 130.

[0028] The 2D model selection unit 28 may narrow down the 2D models to be presented to the user from among the multiple 2D models in order to ultimately allow the user to select one 2D model. For example, the 2D model selection unit 28 may select two or three 2D models from the multiple 2D models in descending order of score as candidates to be presented to the user, or may select all 2D models with scores higher than a predetermined value as candidates to be presented to the user. Naturally, some users are good at drawing line drawings, while others are not. In this way, by presenting the user with several candidates that are relatively close to the line drawing drawn by the user and accepting the user's selection of one 2D model, it is possible to address the variability in users' strengths and weaknesses in drawing line drawings.

[0029] The method of calculating the score is not limited to the above, as long as it represents the degree of approximation of the 2D model to the line drawing. For example, the score may be calculated based on the ratio of the area of ​​the region where the 2D model overlaps with the region defined by the line drawing 130 to the area of ​​the region where the two regions do not overlap. Alternatively, a selection model trained using the line drawing and the 2D model to be selected by the line drawing as training data may be used to select a 2D model that corresponds to the line drawing 130 drawn by the user.

[0030] The processing by the three-dimensional model placement unit 30 will be described in detail below with reference to FIG. 7 . FIG. 7 is a supplementary diagram for explaining the three-dimensional model placement unit 30 of FIG. 3 . Here, it is assumed that the three-dimensional model selection unit 29 has selected a three-dimensional model 300 that shares an object (ID "002" in FIG. 4 ) with the two-dimensional model 200 that most closely resembles the line drawing 130. The three-dimensional model placement unit 30 sets the orientation for placing the three-dimensional model 300 in the three-dimensional simulation space as follows. That is, the three-dimensional model placement unit 30 sets the posture (orientation) of the two-dimensional model 200 as the posture of the three-dimensional model 300. The three-dimensional model placement unit 30 sets the position for placing the three-dimensional model 300 in the three-dimensional simulation space as follows. For example, the three-dimensional model placement unit 30 sets the position of the three-dimensional model 300 in the three-dimensional simulation space such that the center of gravity 300g of the projection model obtained by projecting the three-dimensional model 300 from a specific line of sight direction overlaps with the center of gravity 130g of the area defined by the line drawing 130. Furthermore, the three-dimensional model placement unit 30 sets the position of the three-dimensional model 300 in the three-dimensional simulation space to a position where the projection model obtained by projecting the three-dimensional model 300 from a specific line of sight direction fits within the area defined by the line drawing 130 or overlaps with the area defined by the line drawing 130, and where the positioning condition of the corresponding object of the three-dimensional model 300 is met. The placement conditions are associated with the objects, as shown in FIG. 4 . As shown in FIG. 4 , the placement condition "placed on the floor" is associated with the object "conveyor," and the placement condition "placed on the conveyor or table" is associated with the object "workpiece (box)." Therefore, "position according to placement conditions" means, if the object is a conveyor, the position where the three-dimensional model of the conveyor touches the floor surface in the three-dimensional simulation space, and if the object is a workpiece, the position where the three-dimensional model of the workpiece touches the conveyor in the three-dimensional simulation space.

[0031] The position of the 3D model 210 placed in the 3D simulation space does not need to be precise, as it may be fine-tuned after placement. For example, the 3D model placement unit 30 may set the position of the 3D model in the 3D simulation space so that when the 3D model 300 is projected from a specific line of sight direction, the projected model falls within the area defined by the line drawing 130.

[0032] Hereinafter, the object selection and placement process performed by the simulation device 2 according to this embodiment will be described with reference to FIG. 8 . FIG. 8 is a flowchart illustrating an example of the object selection and placement process performed by the simulation device 2 according to this embodiment. The object setting process is initiated, for example, when the start button 190 displayed on the simulation screen 100 of FIG. 5 is operated. As shown in FIG. 5 , when the object setting process is initiated, the simulation device 2 displays a simulation screen including a simulation space image obtained by projecting a three-dimensional simulation space from a specific line-of-sight direction (S11), and accepts a user operation for drawing a line drawing freehand on the displayed simulation space image (S12). Information about the line drawing drawn by the user is input by the processing of step S12. The information about the line drawing includes the shape, size, and position of the line drawing on the simulation space image.

[0033] Next, the simulation device 2 selects a 2D model that approximates the line drawing from among the multiple 2D models based on the shape and size of the line drawing (S13). When two or more 2D models are selected in step S13 (S14; Yes), the simulation device 2 displays the two or more 2D models to accept a selection operation from the user (S15), and selects one of the two or more 2D models based on the user operation (S16). The simulation device 2 then selects a 3D model from the multiple 3D models that shares an object with the one 2D model (S17), and places the selected 3D model in the simulation space based on the position of the line drawing (S18).

[0034] According to the simulation device of this embodiment, a user can simply draw a line drawing in the simulation space displayed on the simulation screen, and a three-dimensional model of the object that approximates the line drawing can be placed in the three-dimensional simulation space, without having to go through a series of conventional tasks of selecting an object from multiple objects, placing the selected object in the simulation space, and correcting the orientation and position of the placed object. Because the simple task of drawing a line drawing is all that is required, the process of placing an object in the simulation space can be simplified. Furthermore, the user does not need to draw the line drawing in three dimensions, but can simply draw it in two dimensions while imagining the general shape of the object as viewed from a specific line direction. This reduces the effort required for the user to draw the line drawing and reduces variation in drawing skills among users.

[0035] Furthermore, in the object selection and placement process, there is no need to use the three-dimensional model data until the three-dimensional model is finally placed in the simulation space, so that the object selection and placement can be performed while reducing the processing load.

[0036] In this embodiment, the 2D model selection unit 28 selects a 2D model that is similar to the line drawing from among multiple 2D models. This selection process may take into account placement conditions associated with the object. For example, as shown in FIG. 9A , when a line drawing 140 is drawn on a 2D model of a conveyor (conveyor model) 400, the 2D model selection unit 28 references the object information management table in FIG. 4 and selects 2D models of two objects (IDs “005” and “006”) that are limited to being on the conveyor as placement conditions as placement candidates. The 2D model selection unit 28 then calculates a score representing the degree of similarity of each of the selected 2D models to the line drawing 140. In other words, the 2D model selection unit 28 narrows down the multiple 2D models based on the objects surrounding the line drawing 140. If the scores are the same, a selection window 500 that accepts a selection operation from the user is displayed on the simulation screen 100, as shown in FIG. 9B . Then, as shown in FIG. 9C , the 2D model selection unit 28 selects a 2D model selected by a user operation in the selection window 500 as the 2D model 220 to be placed in the simulation space image 110. The selected 2D model 220 is then placed in the simulation space image 110 based on the position of the line drawing 140. By considering the placement conditions when selecting a 2D model in this way, 2D models that do not satisfy the placement conditions can be excluded from the score calculation, thereby shortening the total time for score calculation. Furthermore, the accuracy of object selection can be improved simply by reducing the population. The 2D model selection unit 28 can determine whether the line drawing 140 is drawn on the conveyor model 400 based on whether at least a portion of the line drawing 140 overlaps with the conveyor model 400. The 2D model selection unit 28 may also determine a 2D model (including a floor) specified by a click operation before drawing the line drawing as the object on which the object corresponding to the line drawing is to be placed. For example, when the conveyor model 400 is specified by a click operation or the like and then the line drawing 140 is drawn, the two-dimensional model selection unit 28 can determine that the object corresponding to the line drawing 140 is an object to be placed on the conveyor.

[0037] In this embodiment, the three-dimensional model placement unit 30 places the three-dimensional model selected by the three-dimensional model selection unit 29 in the simulation space based on the position of the line drawing drawn by the user on the simulation space image. In this placement process, rules for limiting the placement position of the three-dimensional model of the object corresponding to the line drawing may be set in advance. For example, the three-dimensional model placement unit 30 determines an object selected by a click operation or the like by the user before the line drawing is drawn as the target for placing the three-dimensional model of the object corresponding to the line drawing. Specifically, based on the two-dimensional model of the conveyor being designated by a click operation or the like before the line drawing is drawn, the three-dimensional model placement unit 30 places the three-dimensional model of the object corresponding to the line drawing in a position that contacts the conveyance surface of the three-dimensional model of the conveyor in the simulation space. Similarly, based on the floor being designated by a click operation or the like before the line drawing is drawn, the three-dimensional model placement unit 30 places the three-dimensional model of the object corresponding to the line drawing in a position that contacts the floor surface in the simulation space. In this way, by setting the above-mentioned rules in the three-dimensional model placement process by the three-dimensional model placement unit 30, even if a two-dimensional model overlaps in the simulation space image in which the line drawing is drawn, the three-dimensional model of the object corresponding to the line drawing can be placed in the position desired by the user in the three-dimensional simulation space.

[0038] In this embodiment, a three-dimensional model is ultimately placed within the simulation space. However, when simply considering the placement of each device constituting a robot system, a two-dimensional simulation space image corresponding to a specific line of sight direction may be sufficient. That is, a simulation device 2′ according to a modified example of this embodiment may include a storage unit 24 that stores data on multiple two-dimensional models corresponding to multiple objects as candidates for placement in the simulation space, a display unit 22 that displays the simulation space, a reception unit 21 that receives user input for drawing a line drawing freehand within the simulation space, a two-dimensional model selection unit 28 that selects a two-dimensional model that approximates the line drawing from multiple two-dimensional models, and a two-dimensional model placement unit 31 that places the 2D model selected by the 2D model selection unit 28 within the simulation space displayed on the display unit 22 based on the position of the line drawing. The two-dimensional model placement unit 31 corresponds to the three-dimensional model placement unit 30 of the simulation device 2 according to this embodiment. That is, the two-dimensional model placement unit 31 places the two-dimensional models on the simulation space image so as to achieve the positional relationship that maximizes the score.

[0039] In the simulation device 2 according to this embodiment, 3D model data is stored in advance in the storage unit 24. However, the simulation device may also function as a 3D model creation unit that creates 3D model data. For example, the storage unit stores information on a reference object shape (reference object shape) and dimensions in association with object information. When an object that approximates a line drawing is determined by the processing of the 2D model selection unit 28, the 3D model creation unit creates a simple 3D model of the object by combining basic 3D models such as a cube, rectangular parallelepiped, sphere, and cylinder based on the shape and size of the drawn object and the reference object shape, in accordance with the object's dimensional information. This eliminates the need to store 3D models of the object in advance, thereby shortening the preparation time required to use the simulation device. Furthermore, since there is no need to maintain a 3D model, the barrier to using the simulation device is lowered. Furthermore, the layout of each device constituting the robot system can be easily considered ahead of schedule, even when a 3D model has not yet been created.

[0040] The following supplementary notes are further disclosed regarding this embodiment and its modified examples. (Supplementary Note 1) The simulation device 2 includes a storage unit 24 that stores data on multiple 3D models corresponding to multiple objects as candidates to be placed in the simulation space and data on multiple 2D models corresponding to the multiple objects, a display unit 22 that displays the simulation space, a reception unit 21 that receives a user operation to draw a line drawing freehand in the simulation space, a 2D model selection unit 28 that selects a 2D model that approximates the line drawing from the multiple 2D models, a 3D model selection unit 29 that selects a 3D model of an object corresponding to the selected 2D model from the multiple 3D models, and an arrangement unit 30 that arranges the selected 3D model in the simulation space based on the position of the line drawing. (Supplementary Note 2) In the simulation device 2 described in Supplementary Note 1, the 2D model selection unit 28 selects two or more 2D models that approximate the line drawing from the multiple 2D models, and selects one 2D model from the two or more 2D models based on the user operation. (Supplementary Note 3) In the simulation device 2 described in Supplementary Note 1 or Supplementary Note 2, the 2D model selection unit 28 narrows down the multiple 2D models based on information about objects arranged around the line drawing, and selects a 2D model that approximates the line drawing from the narrowed down 2D models. (Supplementary Note 4) In the simulation device 2 described in any of Supplementary Notes 1 to 3, the multiple 2D models include multiple types of 2D models of the same object but with different poses. (Supplementary Note 5) In the simulation device 2 described in any of Supplementary Notes 1 to 4, the display unit 22 projects and displays the simulation space from a predetermined line of sight direction, and the 2D model is a model obtained by projecting a 3D model having a common object from the line of sight direction.(Supplementary Note 6) The simulation device 2 includes a memory unit 24 that stores data of a plurality of two-dimensional models corresponding to a plurality of objects that are candidates for placement in the simulation space, a display unit 22 that displays the simulation space, a reception unit 21 that receives user operations for drawing a line drawing freehand in the simulation space, a two-dimensional model selection unit 28 that selects a two-dimensional model that approximates the line drawing from the plurality of two-dimensional models, and a two-dimensional model placement unit that places the selected two-dimensional model in the simulation space displayed on the display unit 22 based on the position of the line drawing. (Supplementary Note 7) The simulation program causes a computer that stores data of a plurality of three-dimensional models corresponding to a plurality of objects as candidates to be placed in the simulation space and data of a plurality of two-dimensional models corresponding to a plurality of objects, to realize the following: means for displaying the simulation space; means for accepting user operations to draw a line drawing freehand in the simulation space; means for selecting a two-dimensional model that approximates the line drawing from the plurality of two-dimensional models; means for selecting a three-dimensional model of an object corresponding to the two-dimensional model from the plurality of three-dimensional models; and means for placing the selected three-dimensional model in the simulation space based on the position of the line drawing.

[0041] Although the embodiments of the present disclosure have been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the invention or the concept and spirit of the present invention derived from the content of the claims and their equivalents. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0042] 1...simulation system, 2...simulation device, 6...robot, 7...control device, 10...data / control bus, 11...processor, 12...RAM, 13...ROM, 14...storage device, 15...input controller, 16...display controller, 17...communication device, 18...input device, 19...display device, 21...reception unit, 22...display unit, 23...transmission / reception unit, 24...storage unit, 25...screen creation unit, 26...simulation space creation unit, 27...projection processing unit, 28...2D model selection unit, 29...3D model selection unit, 30...3D model placement unit.

Claims

1. A storage unit that stores data for multiple 3D models corresponding to multiple objects as candidates to be placed in the simulation space, and data for multiple 2D models corresponding to each of the multiple objects, A display unit for displaying the aforementioned simulation space, A reception unit that accepts user input for drawing line drawings freehand within the aforementioned simulation space, A two-dimensional model selection unit selects a two-dimensional model from the plurality of two-dimensional models that approximates the line drawing, A 3D model selection unit selects a 3D model of an object corresponding to the selected 2D model from the plurality of 3D models, A placement unit that places the selected three-dimensional model in the simulation space based on the position of the line drawing, A simulation device equipped with the following features.

2. The simulation apparatus according to claim 1, wherein the two-dimensional model selection unit selects two or more two-dimensional models that approximate the line drawing from the plurality of two-dimensional models, and selects one two-dimensional model from the two or more two-dimensional models based on user operation.

3. The two-dimensional model selection unit narrows down the plurality of two-dimensional models based on object information arranged around the line drawing, and selects a two-dimensional model that approximates the line drawing from the narrowed-down two-dimensional models. The simulation apparatus according to claim 1 or claim 2.

4. The aforementioned plurality of two-dimensional models include multiple types of two-dimensional models of the same object but with different orientations. The simulation apparatus according to claim 1 or claim 2.

5. The display unit projects and displays the simulation space from a predetermined line of sight direction. The aforementioned two-dimensional model is a model obtained by projecting the aforementioned three-dimensional model, which shares common objects, from the direction of the line of sight. The simulation apparatus according to claim 1 or claim 2.

6. A memory unit that stores data for multiple 2D models corresponding to multiple objects that are candidates to be placed in the simulation space, A display unit for displaying the aforementioned simulation space, A reception unit that accepts user input for drawing line drawings freehand within the aforementioned simulation space, A two-dimensional model selection unit selects a two-dimensional model from the plurality of two-dimensional models that approximates the line drawing, A two-dimensional model placement unit that places the selected two-dimensional model in the simulation space displayed on the display unit based on the position of the line drawing, A simulation device equipped with the following features.

7. A computer that stores data for multiple 3D models corresponding to multiple objects as candidates to be placed in the simulation space, and data for multiple 2D models corresponding to each of the multiple objects, Means for displaying the aforementioned simulation space, A means for receiving user input to draw line drawings freehand within the aforementioned simulation space, A means for selecting a two-dimensional model that approximates the line drawing from the aforementioned plurality of two-dimensional models, A means for selecting a 3D model of an object corresponding to the selected 2D model from the plurality of 3D models, Means for arranging the selected three-dimensional model in the simulation space based on the position of the line drawing, A simulation program to achieve this.