Design system for component layout and design method for component layout

Abstract

The present invention comprises: a virtual space generation program (31); an image generation program (32); a virtual space display means (6); a hand tracking means (7) for detecting the movement of an operator's hands and fingers; a layout modification means (42) for modifying the layout of components on the basis of detection information from the hand tracking means (7), and generating an image of the components that reflects the changes in the layout of the components according to the image generation program (32); and a hand image generation means (43) for generating an image of the operator's hands and fingers that changes from moment to moment, on the basis of the detection information from the hand tracking means (7), according to the image generation program (32). The layout modification means (42) allows the layout of the components to be modified by using the intersection of the extension line of the operator's finger and a component as the starting point for layout modification.

Description

Design System and Design Method for Component Layout

[0001] The present invention relates to a design system for component layout and a design method for component layout for performing design related to the layout of components in a virtual space.

[0002] Conventionally, design support for determining the routing of wiring and piping and performing the design of the final shape using virtual space technology is known. For example, the wiring / piping design device described in Patent Document 1 includes a head-mounted display worn on the head of an evaluation designer, a head (line-of-sight) detection transmitter for detecting the orientation or line-of-sight direction of the evaluation designer's head, a finger detection transmitter for detecting the position of the fingers of the evaluation designer's hand, a position detection sensor which is a receiver for detecting the positions of the head (line-of-sight) detection transmitter and the finger detection transmitter, a controller operated by the design evaluator, and a processing device.

[0003] In this configuration, the processing device moves and corrects the position of the grasped wire / piping tube (hereinafter referred to as "wire etc.") in the 3D data in accordance with the movement of the finger. For example, when in the virtual space, the evaluation designer grasps an arbitrary position of the wire etc. with two fingers of the left hand and grasps another position of the same wire etc. with two fingers of the other right hand, and moves the right hand without moving the left hand, the processing device performs position correction on the 3D data of the wire etc. so that the portion grasped by the right hand moves while the position grasped by the left hand is fixed.

[0004] That is, when the evaluation designer performs an operation of stretching and contracting the wire etc. while grasping the wire etc. in the virtual space according to the detection result of the position detection sensor, the wiring / piping design device described in Patent Document 1 can change the length of the component in the virtual space according to the operation and reflect it in the output image.

[0005] Japanese Unexamined Patent Application Publication No. 2016-189122

[0006] However, in the wiring and piping design device described in Patent Document 1, in order to correct the position of wires and the like in 3D data, the evaluation designer had to grasp the wires and the like with their fingers and then stretch or contract them. In this configuration, the evaluation designer had to move close to the wires and the like in order to grasp them, and in order to move the wires and the like to the target position, the evaluation designer had to move close to the target position.

[0007] For example, if the target location of a wire or other element is deep within the virtual space, there will be many interfering structures between the target and the target, making the layout work difficult. Also, if the target location of a wire or other element is high up, such as on the ceiling of the virtual space, the evaluation designer will need to use a ladder or similar equipment, which also makes the layout work difficult.

[0008] The present invention aims to solve the aforementioned conventional problems and to provide a design system and method for designing component layouts that enable the operator to perform work while remaining in the same position, regardless of the position of the components.

[0009] To achieve the above objective, the present invention provides a component layout design system for designing the layout of components in a virtual space, comprising: a storage means storing a virtual space generation program for generating the virtual space; an image generation program for generating images of the components and images of the operator's fingers in the virtual space; a virtual space display means for displaying the virtual space; a hand tracking means for detecting the movement of the operator's fingers; and a control means for performing control according to the virtual space generation program and the image generation program, wherein the control means modifies the layout of the components based on the detection information from the hand tracking means, and The system includes a layout modification means for generating an image of the member that reflects changes in the member's layout according to the image generation program, and a hand image generation means for generating an image of the operator's fingers that changes moment by moment according to the image generation program, based on detection information from the hand tracking means, wherein the layout modification means uses the intersection of the extension line of the operator's finger image in the finger image and the member, which is spaced apart from the finger image, as the starting point for layout modification, and moves the starting point to a new destination detected by the hand tracking means, thereby stretching and contracting the member and modifying the member's layout.

[0010] The present invention relates to a design method for component layout, which is a design method for performing a design related to the layout of components in a virtual space, comprising a virtual space generation program for generating the virtual space, The system uses an image generation program for generating images of the member and the operator's fingers in the virtual space, a virtual space display means for displaying the virtual space, a hand tracking means for detecting the movement of the operator's fingers, and a control means for performing control according to the virtual space generation program and the image generation program. The control means includes a layout modification means for modifying the layout of the member based on detection information from the hand tracking means and generating an image of the member that reflects the change in the layout of the member according to the image generation program, and a hand image generation means for generating an image of the operator's fingers that changes moment by moment according to the image generation program based on detection information from the hand tracking means. The layout modification means uses the intersection of the extension line of the operator's finger image in the finger image and the member, which is spaced apart from the finger image, as the starting point for layout modification, and moves the starting point to a new destination detected by the hand tracking means, thereby stretching and contracting the member and modifying its layout.

[0011] In the component layout design system and component layout design method of the present invention, it is preferable that marks are placed at the intersections.

[0012] The method for modifying the layout of a component according to the present invention involves using the intersection of the extension of the operator's fingers and the component as the starting point for the layout modification, and then repositioning the component. Therefore, when modifying the layout of a component, it is not necessary to directly grasp the component with the fingers, and the operator does not need to move close to the component. Consequently, regardless of the position of the component, the operator can modify the layout of the component while remaining in the same position.

[0013] A preferred configuration in which a mark is placed at the intersection of the extension of the operator's finger and the aforementioned member makes it easier to perform layout correction work using the mark as a guide.

[0014] A block diagram showing the configuration of a design system according to one embodiment of the present invention. A diagram showing an example of a virtual space viewed by an operator in one embodiment of the present invention. A diagram showing the basic operation of modifying the layout of a component according to one embodiment of the present invention. A flowchart showing the design procedure by the design system according to one embodiment of the present invention in order of steps. A diagram showing an image in the virtual space at the start of design in one embodiment of the present invention. A diagram showing a finger pointing to part B of a concave connector in the virtual space according to one embodiment of the present invention. A diagram showing part A and part B of an electrical component connected by a wire in the virtual space according to one embodiment of the present invention. A diagram showing a finger pointing to a modification point on the wire in the virtual space according to one embodiment of the present invention. A diagram showing a finger pointing to a new destination for a mark in the virtual space according to one embodiment of the invention. A diagram showing a mark moving to a new destination in the virtual space according to one embodiment of the invention. A diagram showing a finger pointing to the second modification point on the wire in the virtual space according to one embodiment of the invention. A diagram showing a mark moving to a new destination in the virtual space according to one embodiment of the invention. A diagram showing a state where the wire layout modification has been completed in the virtual space according to one embodiment of the invention.

[0015] This invention relates to a design system that utilizes virtual space technology. The design content concerns the layout of components, and it is possible to design the paths and lengths of components by modifying the layout (rerouting). There are no particular limitations on the type of component; typical examples include wires and cables for wiring, pipes, tubes, and hoses for piping, but it may also be metal rods, food products (e.g., noodles), textile products (e.g., belts, strings, robes), etc. Furthermore, there are no limitations on the size or length of the component; for example, it may be a long wire or pipe in a large piece of equipment, or a tiny wire in a semiconductor chip.

[0016] First, with reference to Figure 1, an overview of the design system for component layout according to the present invention (hereinafter simply referred to as the "design system") will be described. The design system 1 comprises an input means 2, a storage means 3, a control means 4, an output means 5, a virtual space display means 6, and a hand tracking means 7.

[0017] A typical example of design system 1 is one in which a virtual space generation program 31 and an image generation program 32 are installed on the storage means 3 of a terminal such as a personal computer, but these programs may also be provided on the cloud. When design system 1 is configured on a personal computer, the input means 2 is an import means or a keyboard, and the output means 5 is the display provided by the personal computer.

[0018] The input means 2 stores 3D (three-dimensional) data 33 in the storage means 3. The 3D data 33 is 3D data of structures, equipment, etc., that are the subject of the component layout. In the case of wiring design for a power distribution board, it is 3D data showing the internal structure of the power distribution board. The object data 34 is data related to the component being designed, and includes data on the type of component (wire, pipe, etc.), diameter, length, model, etc.

[0019] The virtual space generation program 31 can utilize commercially available programs. This program is, for example, what is called an MR (Mixed Reality) application, which generates a virtual space and links and displays the generated virtual space in the real world. The virtual space is generated by the virtual space generation means 41 according to the virtual space generation program 31, based on 3D data 33. The generated virtual space is displayed on the head-mounted display 61.

[0020] The image generation program 32 is a development tool such as Unity, and generates images of the component being designed and images of the operator's fingers. The layout of the component in the virtual space changes constantly due to layout modifications. As will be described in detail later, the layout modification means 42 modifies the layout of the component based on detection information from the hand tracking means 7, and generates an image of the component that reflects the constantly changing layout according to the image generation program 32, and displays it in the virtual space.

[0021] The virtual space display means 6 can be anything that can display a virtual space, such as goggles or glasses that can display a virtual space inside. Also, the virtual space display means 6 does not have to be worn by the operator, and may be an external display or a spatial (3D) hologram. In this embodiment, it is a head-mounted display 61. The head-mounted display 61 is worn on the operator's head, and the operator wearing it can see the virtual space displayed inside the head-mounted display 61. Figure 2 shows an example of a virtual space seen by the operator 10. In Figure 2, the virtual space 30 is displayed inside the head-mounted display 61, but for convenience, it is shown outside the operator 10. The virtual space 30 is an example of wiring inside a distribution panel, and the main part of the displayed image is shown, while the individual structures inside the distribution panel are not shown.

[0022] In the virtual space 30, a wire 20 is displayed, with convex connectors 21a and 21b connected to both ends of the wire 20. In addition, the operator's left and right hands 11 and fingers 12 are displayed in the virtual space 30. The left and right hands 11 are not the operator's actual hands, but rather, as will be described in detail later, in Figure 1, images generated by the hand image generation means 43 according to the image generation program 32 based on detection information from the hand tracking means 7.

[0023] In Figure 1, the hand tracking means 7 is equipped with a sensor that detects the movement of the operator's fingers 10, and the design system 1 can track the movement of the operator's fingers 10. Specifically, in Figure 1, the hand image generation means 43 generates images of the operator's hand 11 and fingers 12, as shown in Figure 2, according to the image generation program 32 based on the detection information from the hand tracking means 7. In other words, the images of the hand 11 and fingers 12 move in accordance with the movement of the operator's fingers 10 within the virtual space 30 in Figure 2.

[0024] The hand tracking means 7 may use a sensor as described above, but instead, multiple cameras may be installed to track the movement of the hands and fingers. In this case, the images of the hand 11 and fingers 12 will be synchronized with the movement of the operator's hands and fingers based on the image data from the cameras.

[0025] The design system 1 according to the present invention enables the operator to modify the layout of components while remaining in the same position, regardless of the position of the components. The basic operation of layout modification will be explained with reference to Figure 3. Figure 3(a) shows the state in which the operator 12 is pointing at the modification point of the wire 20. In this state, as in Figure 2, the operator 10 is looking at the virtual space 30 in the head-mounted display 61 and has the operator 10's actual finger pointed at the wire 20 in the virtual space 30. As a result, the images of the hand 11 and finger 12 in the virtual space 30 move to point towards the wire 20.

[0026] More specifically, in Figure 3(a), a beam 22 is emitted from the tip of the finger 12 facing the wire 20, and the beam 22 reaches the wire 20. The intersection of the wire 20 and the beam 22 is the correction point, and a mark 23 is placed at the intersection by a selection operation. As a result, as shown in Figure 3(a), the tip of the beam 22 points to the mark 23. In other words, a mark 23, which serves as the starting point for layout correction, is placed at the intersection of the longitudinal extension of the operator's finger and the wire 20.

[0027] The hand tracking means 7 (Figure 1) detects that a specific point has been selected through the selection operation described above. The generation, display, and application of the beam 22 and the mark 23 are performed by the layout correction means 42 in accordance with the image generation program 32 based on the detection information of the hand tracking means 7 in Figure 1 (the same applies hereafter).

[0028] A selection operation is, for example, pinching with the index finger and thumb. In this case, the operation is completed simply by moving the fingers, so a controller is not necessary. If a controller is used, an image of the controller is displayed in the virtual space 30 in Figure 2, and the operation is performed using this controller.

[0029] As shown in Figure 3(a), when the operator moves their finger from a state where the wire 20 has a mark 23 attached to it, the trajectory of the beam 22 also changes accordingly, and the length of the beam 22 usually extends or retracts. Figure 3(b) shows the state where the finger 12 is pointing to the new destination of the mark 23. In this state, the tip 22a of the beam 22 becomes the new destination of the mark 23. If selected from this state using the selection operation described above, the mark 23 will move to the new destination.

[0030] Figure 3(c) shows the state after the mark 23 has moved to a new location. As the mark 23 moves, the layout of the wire 20 also changes, as shown in Figure 3(c). In the example in Figure 3, the convex connectors 21a and 21b at both ends of the wire 20 are inserted into and fixed to the object, so as the mark 23 moves, the wire 20 changes into a mountain shape with the mark 23 as the apex.

[0031] As shown in Figure 3(c), the wire 20 has become V-shaped, and its length is longer than that of the wire 20 in Figure 3(b) (in a nearly straight state). Conversely, by performing the reverse operation, the V-shaped wire 20 in Figure 3(c) is returned to a nearly straight state as in Figure 3(b), and the length of the wire 20 becomes shorter. In other words, the wire 20 expands and contracts in response to the operation of modifying the layout of the wire 20. The length of the wire 20 due to expansion and contraction is calculated by the calculation means 44 (Figure 1).

[0032] In the state shown in Figure 3(c), once the layout correction of wire 20 is completed, in Figure 1, the calculation means 44 calculates the length of wire 20 based on the image data of wire 20. The operation to complete the layout correction may be performed by finger movements, similar to the selection operation described above, or by using a controller image that appears in the virtual space. The length of wire 20 after the layout correction can be displayed on the output stage 44 (Figure 1).

[0033] The outline of the design system 1 according to the present invention has been described above. Now, the design procedure using the design system 1 will be described in more detail with reference to the flowchart in Figure 4. Figure 4 is a flowchart showing the design procedure using the design system 1 in order of steps. Figures 5 to 13 show images in the virtual space in order of steps from the start of the design to the end of the design. In each of these figures, one operator is performing the work, but multiple operators may each wear a head-mounted display 61 and perform collaborative work.

[0034] Figure 5 shows an image of the virtual space at the start of the design process. Similar to Figure 2, the virtual space is the inside of a power distribution panel. The displayed image shows the main parts, and the individual structures inside the power distribution panel are not shown (the same applies to Figures 6 to 13). Although not shown in Figure 5, the object of the layout design is the wire 20 shown in Figures 2 and 3. The operator selects connection points A and B for the wire 20 while viewing the image on the head-mounted display 61 (Figure 2) (step 100 in Figure 4).

[0035] In Figure 5, electrical components 30 and 32 are installed in the virtual space. Electrical component 30 is provided with a recessed connector 31, and electrical component 32 is provided with a recessed connector 33. Finger 12 is pointing to part A of the recessed connector 31, and a beam 22 is emitted toward part A. If the operator performs a selection operation in this state, the recessed connector 31 of part A will be selected as the connection destination for the convex connector 21a (Figure 2) of the wire 20.

[0036] Figure 6 shows the state where finger 12 is pointing to part B of the concave connector 33. In this state, a beam 22 is emitted toward part B. If the operator performs a selection operation in this state, the concave connector 33 of part B will be selected as the connection destination for the convex connector 21b (Figure 2) of the wire 20.

[0037] Figure 7 shows the state in which part A of electrical component 30 and part B of electrical component 32 are connected by wire 20 (step 101 in Figure 4). In the state shown in Figure 7, wire 20 is connected between A and B with approximately the shortest possible length. Subsequently, the operator modifies the layout of wire 20 (step 102 in Figure 4). In Figure 1, the image of wire 20 is generated by the layout modification means 42 according to the image generation program 32 based on the object data 34.

[0038] Figure 8 shows the state where the correction point on the wire 20 is pointed to by finger 12. A beam 22 is emitted from the tip of finger 12 facing the wire 20, and the beam 22 reaches the wire 20. A mark 24 is placed at the intersection of the wire 20 and the beam 22 by the operator's selection.

[0039] Figure 9 shows the state where finger 12 is pointing to the new destination of mark 24. The tip 22a of beam 22 becomes the new destination of mark 23. From this state, a selection operation moves mark 24 to the new destination. Figure 10 shows the state after mark 24 has moved to the new destination. As mark 24 moves, wire 20 is curved.

[0040] Figure 11 shows the second modification point on wire 20 being pointed to by finger 12. The subsequent operations are the same as for the first modification, so only the key points will be explained. A new mark 25 has been added to the intersection of wire 20 and beam 22. Figure 12 shows the state after mark 25 has moved to its new location. As mark 25 moves, a second curve is formed in wire 20.

[0041] Figure 13 shows the state where the layout correction of the wire 20 has been completed once. If the operator determines that the layout of the wire 20 in Figure 13 is still inappropriate (step 103 in Figure 4), the above operation is repeated to re-correct the layout of the wire 20. On the contrary, if the operator determines that the layout is appropriate in the state of the wire 20 in Figure 13 (step 103 in Figure 4), the layout correction for the wire 20 is completed by a confirmation operation.

[0042] Figures 5 to 13 only show the electrical components 30 and 32 for the sake of illustration. However, there are many connection points by wires in the distribution board. Therefore, usually, once the layout design of the first wire 20 is completed, the layout design for the next wire is carried out. That is, if there are other wires to be the object of the layout design (step 104 in Figure 4), the layout design for the new other wire is repeated (steps 100 to 102 in Figure 4). When the layout design for all wires is completed, the design data such as the length of each wire is output to the output means 5 (Figure 1).

[0043] The above embodiment is an example, and the following configuration may also be adopted. In the above embodiment, marks 23 to 25 are attached to the intersection of the wire 20 and the beam 22, so that the layout correction work is facilitated by using the marks as a guide. However, a configuration in which the attachment of the marks is omitted may also be adopted. Similarly, since the layout correction work is facilitated by the beam 22 being emitted from the finger 12, a configuration in which the generation of the beam 22 is omitted may also be adopted.

[0044] In the above embodiment, as shown in Figure 3(b), after the finger 12 once selects a new destination for the mark 23, as shown in Figure 3(c), the layout of the wire 20 is corrected. However, a configuration may also be adopted in which the mark 23 is made to follow the change in the position pointed by the finger 12 as it is, and the layout of the wire 20 is corrected at once.

[0045] In the examples of FIGS. 8 to 13, after attaching the mark 24, the layout of the wire 20 was corrected, and after attaching a new mark 25, the layout of the wire 20 was corrected again. However, after attaching the marks 24 and 25, the destinations of movement of the marks 24 and 25 may be selected respectively, and then the layout of the wire 20 may be corrected at once.

[0046] Also, in the examples of FIGS. 8 to 13, both ends of the wire 20 are fixed, but one end may be a free end, and the wire 20 may be configured to expand and contract starting from the intersection of the wire 20 and the beam 22. Further, both ends of the wire 20 may be free ends, and the wire 20 may be configured to move starting from the intersection of the wire 20 and the beam 22. In this case, an image of the wire 20 before movement may be left, and a new wire 20 may be copied.

[0047] As described above, one embodiment of the present invention has been described. The layout correction of the member according to the present invention is to pull around the member with the intersection of the extension line of the operator's finger and the member as the starting point of the layout correction. Therefore, when correcting the layout of the member, it is not necessary to directly grasp the member with a finger, and the operator does not need to move close to the member. Therefore, regardless of the position of the member, the operator can correct the layout of the member while staying at the same position.

[0048] For example, when the target movement destination of the member is in the depth of the virtual space, in the conventional design support device, the operator needs to approach the depth in order to grasp the member, and moreover, there are many structures that interfere with the fingers between the operator and the depth. Also, when the target movement destination of the member is at a high place such as the ceiling of the virtual space, in the conventional design support device, the operator needs to stand on a stepladder or the like. On the other hand, in the present invention, since it is not necessary to directly grasp the member with a finger, the layout correction becomes easy even if the target movement destination of the member is in the depth of the virtual space or at a high place.

[0049] Furthermore, as mentioned above, according to the present invention, the operator does not need to approach the component, so the present invention is also useful for operators with physical disabilities. Moreover, since the present invention modifies the layout of the component by using the intersection of the extension of the operator's finger and the component as the starting point for layout modification, it becomes possible to operate with one hand, as shown in the work examples in Figures 5 to 13.

[0050] 1 Design system 2 Input means 3 Memory means 4 Control means 5 Output means 6 Virtual space display means 7 Hand tracking means 10 Operator 11 Hand (image) 12 Finger (image) 20 Wire (component) 23, 24, 25 Mark 30 Virtual space 31 Virtual space generation program 32 Image generation program 33 3D data 41 Virtual space generation means 42 Layout modification means 43 Hand image generation means 44 Calculation means 61 Head-mounted display

Claims

1. A component layout design system for designing the layout of components in a virtual space, comprising: a storage means storing a virtual space generation program for generating the virtual space; an image generation program for generating images of the components and images of the operator's fingers in the virtual space; a virtual space display means for displaying the virtual space; a hand tracking means for detecting the movements of the operator's fingers; and a control means for performing control according to the virtual space generation program and the image generation program, wherein the control means comprises: a layout modification means for modifying the layout of the components based on detection information from the hand tracking means and generating images of the components that reflect the changes in the layout of the components according to the image generation program; and a hand image generation means for generating images of the operator's fingers that change moment by moment according to the image generation program based on detection information from the hand tracking means. The layout modification means is characterized in that it uses the intersection of the extension line of the operator's finger image in the finger image and the member separated from the finger image as the starting point for layout modification, and moves the starting point to a new destination detected by the hand tracking means, thereby stretching and contracting the member, and thus being able to reposition the member and modify the layout of the member.

2. The member layout design system according to claim 1, wherein a mark is placed at the intersection.

3. A component layout design method for designing the layout of components in a virtual space, comprising: a virtual space generation program for generating the virtual space; an image generation program for generating images of the components and images of the operator's fingers in the virtual space; a virtual space display means for displaying the virtual space; a hand tracking means for detecting the movements of the operator's fingers; and a control means for performing control according to the virtual space generation program and the image generation program, wherein the control means comprises: a layout modification means for modifying the layout of the components based on detection information from the hand tracking means and generating images of the components that reflect the changes in the layout of the components according to the image generation program; and a hand image generation means for generating images of the operator's fingers that change moment by moment according to the image generation program based on detection information from the hand tracking means. A design method for designing member layouts, characterized in that the layout modification means uses the intersection of the extension line of the operator's finger image in the finger image and the member spaced apart from the finger image as the starting point for layout modification, moves the starting point to a new destination detected by the hand tracking means, and stretches or contracts the member, thereby repositioning the member and modifying the layout of the member.

4. The design method for member layout according to claim 3, wherein a mark is placed at the intersection.

Images