Design system and design method for component layouts

Abstract

This invention provides a design system and method for designing component layouts that allow an operator to perform tasks while remaining in the same position, regardless of the position of the components. [Solution] The system comprises a virtual space generation program 31, an image generation program 32, a virtual space display means 6, a hand tracking means 7 for detecting the movement of the operator's fingers, a layout modification means 42 for modifying the layout of the components based on the detection information from the hand tracking means 7 and generating an image of the components that reflects the changes in the layout of the components according to the image generation program 32, and a hand image generation means 43 for generating an image of the operator's fingers that changes moment by moment according to the image generation program 32 based on the detection information from the hand tracking means 7. The layout modification means 42 is configured to modify the layout of the components by using the intersection of the extension of the operator's fingers and the components as the starting point for layout modification.

Description

Technical Field

[0001] The present invention relates to a member layout design system and a member layout design method for performing design related to the layout of members in a virtual space.

Background Art

[0002] Conventionally, design support for determining the routing of wiring and piping and performing the design of the final shape using virtual space technology has been known. For example, the wiring / piping design device described in Patent Document 1 includes a head-mounted display worn on the head of an evaluation designer, a head (line-of-sight) detection transmitter for detecting the orientation or line-of-sight direction of the evaluation designer's head, a finger detection transmitter for detecting the position of the fingers of the evaluation designer's hand, a position detection sensor that is a receiver for detecting the positions of the head (line-of-sight) detection transmitter and the finger detection transmitter, a controller operated by the design evaluator, and a processing device.

[0003] In this configuration, the processing device moves and corrects the position of the grasped wire / piping tube (hereinafter referred to as "wire etc.") in the 3D data according to the movement of the finger. For example, in the virtual space, when the evaluation designer grasps an arbitrary position of the wire etc. with two fingers of the left hand and grasps another position of the same wire etc. with two fingers of the other right hand and moves the right hand without moving the left hand, the processing device performs position correction on the 3D data of the wire etc. so that the part grasped by the right hand moves while the position grasped by the left hand is fixed.

[0004] That is, the wiring / piping design device described in Patent Document 1 can change the length of the member in the virtual space according to the operation and reflect it in the output image when the evaluation designer performs an operation of stretching and contracting the wire etc. while grasping the wire etc. in the virtual space based on the detection result of the position detection sensor.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2016-189122 [Overview of the project] [Problems that the invention aims to solve]

[0006] However, in the wiring and piping design device described in Patent Document 1, in order to correct the position of wires and the like in 3D data, the evaluation designer had to grasp the wires and the like with their fingers and then stretch or contract them. In this configuration, the evaluation designer had to move close to the wires and the like in order to grasp them, and in order to move the wires and the like to the target position, the evaluation designer had to move close to the target position.

[0007] For example, if the target location of a wire or other element is deep within the virtual space, there will be many interfering structures between the target and the target, making the layout work difficult. Also, if the target location of a wire or other element is high up, such as on the ceiling of the virtual space, the evaluation designer will need to use a ladder or similar equipment, which also makes the layout work difficult.

[0008] The present invention aims to solve the aforementioned conventional problems and to provide a design system and method for designing component layouts that enable the operator to perform work while remaining in the same position, regardless of the position of the components. [Means for solving the problem]

[0009] To achieve the above objective, the present invention provides a component layout design system for designing the layout of components in a virtual space, comprising: a storage means storing a virtual space generation program for generating the virtual space; an image generation program for generating images of the components and images of the operator's fingers in the virtual space; a virtual space display means for displaying the virtual space; a hand tracking means for detecting the movement of the operator's fingers; and a control means for performing control according to the virtual space generation program and the image generation program, wherein the control means comprises: a layout modification means for modifying the layout of the components based on detection information from the hand tracking means and generating images of the components that reflect the changes in the layout of the components according to the image generation program; and a hand image generation means for generating images of the operator's fingers that change moment by moment according to the image generation program based on detection information from the hand tracking means, wherein the layout modification means is configured to modify the layout of the components using the intersection of the extension of the operator's fingers and the components as the starting point for layout modification.

[0010] The present invention relates to a component layout design method for designing the layout of components in a virtual space, comprising: a virtual space generation program for generating the virtual space; an image generation program for generating images of the components and images of the operator's fingers in the virtual space; a virtual space display means for displaying the virtual space; a hand tracking means for detecting the movement of the operator's fingers; and a control means for performing control according to the virtual space generation program and the image generation program, wherein the control means comprises: a layout modification means for modifying the layout of the components based on detection information from the hand tracking means and generating images of the components that reflect the changes in the layout of the components according to the image generation program; and a hand image generation means for generating images of the operator's fingers that change moment by moment according to the image generation program based on detection information from the hand tracking means, characterized in that the layout modification means modifies the layout of the components using the intersection of the extension of the operator's fingers and the components as the starting point for layout modification.

[0011] In the component layout design system and component layout design method of the present invention, it is preferable that marks are placed at the intersections. [Effects of the Invention]

[0012] The method for modifying the layout of a component according to the present invention involves using the intersection of the extension of the operator's fingers and the component as the starting point for the layout modification, and then repositioning the component. Therefore, when modifying the layout of a component, it is not necessary to directly grasp the component with the fingers, and the operator does not need to move close to the component. Consequently, regardless of the position of the component, the operator can modify the layout of the component while remaining in the same position.

[0013] A preferred configuration in which a mark is placed at the intersection of the extension of the operator's finger and the aforementioned member makes it easier to perform layout correction work using the mark as a guide. [Brief explanation of the drawing]

[0014] [Figure 1] Block diagram showing the configuration of a design system according to an embodiment of the present invention. [Figure 2] Diagram showing an example of a virtual space viewed by an operator in an embodiment of the present invention. [Figure 3] Diagram showing the basic operation of layout modification of members according to an embodiment of the present invention. [Figure 4] Flowchart showing the design procedure by a design system according to an embodiment of the present invention in the order of steps. [Figure 5] Diagram showing an image in a virtual space at the start of design in an embodiment of the present invention. [Figure 6] Diagram showing a state in which a finger points to part B of a concave connector in a virtual space according to an embodiment of the present invention. [Figure 7] Diagram showing a state in which part A and part B of an electrical component are connected by a wire in a virtual space according to an embodiment of the present invention. [Figure 8] Diagram showing a state in which a finger points to a correction point of a wire in a virtual space according to an embodiment of the present invention. [Figure 9] Diagram showing a state in which a finger points to a new movement destination of a mark in a virtual space according to an embodiment of the present invention. [Figure 10] Diagram showing a state in which a mark has moved to a new movement destination in a virtual space according to an embodiment of the invention. [Figure 11] Diagram showing a state in which a finger points to the second correction point of a wire in a virtual space according to an embodiment of the invention. [Figure 12] Diagram showing a state in which a mark has moved to a new movement destination in a virtual space according to an embodiment of the invention. [Figure 13] Diagram showing a state in which the layout correction of a wire has been once completed in a virtual space according to an embodiment of the invention.

Mode for Carrying Out the Invention

[0015] The present invention relates to a design system utilizing virtual space technology. The design content pertains to the layout of members, and it is possible to design the path and length of members through layout modification (rearrangement). The types of members are not particularly limited. Typical examples include wires, cables for wiring, pipes, tubes, and hoses for piping. However, it may also be metal bars, food (e.g., noodles), textile products (e.g., belts, strings, robes), etc. Also, there are no limitations regarding the size and length of the members. For example, it may be long wires, pipes, etc. within large equipment, or even tiny wires within semiconductor chips.

[0016] First, while referring to FIG. 1, the outline of a design system for member layout according to the present invention (hereinafter simply referred to as the "design system") will be described. The design system 1 includes an input means 2, a storage means 3, a control means 4, an output means 5, a virtual space display means 6, and a hand tracking means 7.

[0017] A typical example of the design system 1 is one in which a virtual space generation program 31 and an image generation program 32 are installed in the storage means 3 of a terminal such as a personal computer. However, these programs may also be prepared on the cloud. When the design system 1 is configured by a personal computer, the input means 2 is an import means, a keyboard, etc., and the output means 5 is a display provided in the personal computer.

[0018] 3D (three-dimensional) data 33 is stored in the storage means 3 by the input means 2. The 3D data 33 is 3D data of a structure, equipment, etc. that is the target of the member layout. In the case of wiring design of a switchboard, it is 3D data showing the internal structure of the switchboard. The object data 34 is data regarding the member that is the design target, and is data regarding the type of member (wire, pipe, etc.), diameter, length, type, etc.

[0019] The virtual space generation program 31 can utilize commercially available programs. This program is, for example, what is called an MR (Mixed Reality) application, which generates a virtual space and links the generated virtual space to the real world for display. The virtual space is generated by the virtual space generation means 41 according to the virtual space generation program 31, based on 3D data 33. The generated virtual space is displayed on the head-mounted display 61.

[0020] The image generation program 32 is a development tool such as Unity, and generates images of the components being designed and images of the operator's fingers. The layout of the components in the virtual space changes constantly due to layout modifications. As will be described in detail later, the layout modification means 42 modifies the layout of the components based on detection information from the hand tracking means 7, and generates an image of the components that reflects the constantly changing layout according to the image generation program 32, and displays it in the virtual space.

[0021] The virtual space display means 6 can be anything that can display a virtual space, such as goggles or glasses that can display a virtual space inside. Also, the virtual space display means 6 does not have to be worn by the operator, and may be an external display or a spatial (3D) hologram. In this embodiment, it is a head-mounted display 61. The head-mounted display 61 is worn on the operator's head, and the operator wearing it can see the virtual space displayed inside the head-mounted display 61. Figure 2 shows an example of the virtual space seen by the operator 10. In Figure 2, the virtual space 30 is displayed inside the head-mounted display 61, but for convenience it is shown outside the operator 10. The virtual space 30 is an example of wiring inside a distribution panel, and the main part of the displayed image is shown, while the individual structures inside the distribution panel are not shown.

[0022] In the virtual space 30, a wire 20 is displayed, with convex connectors 21a and 21b connected to both ends of the wire 20. In addition, the operator's left and right hands 11 and fingers 12 are displayed in the virtual space 30. The left and right hands 11 are not the operator's actual hands, but rather, as will be described in detail later, in Figure 1, images generated by the hand image generation means 43 according to the image generation program 32 based on detection information from the hand tracking means 7.

[0023] In Figure 1, the hand tracking means 7 is equipped with a sensor that detects the movement of the operator's fingers 10, and the design system 1 can track the movement of the operator's fingers 10. Specifically, in Figure 1, the hand image generation means 43 generates images of the operator's hand 11 and fingers 12 according to the image generation program 32 based on the detection information from the hand tracking means 7, as shown in Figure 2. That is, in accordance with the movement of the operator's fingers 10, the images of the hand 11 and fingers 12 move within the virtual space 30 in Figure 2.

[0024] The hand tracking means 7 may use a sensor as described above, but instead, multiple cameras may be installed to track the movement of the hands and fingers. In this case, the images of the hand 11 and fingers 12 will be synchronized with the movement of the operator's hands and fingers based on the image data from the cameras.

[0025] The design system 1 according to the present invention enables the operator to modify the layout of components while remaining in the same position, regardless of the position of the components. The basic operation of layout modification will be explained with reference to Figure 3. Figure 3(a) shows the state in which the operator 11 is pointing to the modification point of the wire 20. In Figure 2, the operator 10 is looking at the virtual space 30 in the head-mounted display 61 and pointing the operator 10's actual finger towards the wire 20 in the virtual space 30. As a result, the images of the hand 11 and finger 12 in the virtual space 30 move to face the wire 20.

[0026] More specifically, in Figure 3(a), a beam 22 is emitted from the tip of the finger 12 facing the wire 20, and the beam 22 reaches the wire 20. The intersection of the wire 20 and the beam 22 is the correction point, and a mark 23 is placed at the intersection by a selection operation. As a result, as shown in Figure 3(a), the tip of the beam 22 points to the mark 23. In other words, a mark 23, which serves as the starting point for layout correction, is placed at the intersection of the longitudinal extension of the operator's finger and the wire 20.

[0027] The hand tracking means 7 (Figure 1) detects that a specific point has been selected through the selection operation described above. The generation, display, and application of the beam 22 and the mark 23 are performed by the layout correction means 42 in accordance with the image generation program 32 based on the detection information of the hand tracking means 7 in Figure 1 (the same applies hereafter).

[0028] A selection operation is, for example, pinching with the index finger and thumb. In this case, the operation is completed simply by moving the fingers, so a controller is not necessary. If a controller is used, an image of the controller will appear in the virtual space 30 in Figure 2, and the operation will be performed using this controller.

[0029] As shown in Figure 3(a), when the operator moves their finger from a state where the wire 20 has a mark 23 attached to it, the trajectory of the beam 22 also changes accordingly, and the length of the beam 22 usually extends or retracts. Figure 3(b) shows the state where the finger 12 is pointing to the new destination of the mark 23. In this state, the tip 22a of the beam 22 becomes the new destination of the mark 23. If selected from this state using the selection operation described above, the mark 23 will move to the new destination.

[0030] Figure 3(c) shows the state after mark 23 has moved to a new location. As mark 23 moves, the layout of wire 20 also changes, as shown in Figure 3(c). In the example in Figure 3, the convex connectors 21a and 21b at both ends of wire 20 are inserted into and fixed to the object, so as mark 23 moves, wire 20 changes into a mountain shape with mark 23 as the apex.

[0031] As shown in Figure 3(c), the wire 20 has become V-shaped, and its length is longer than that of the wire 20 in Figure 3(b) (in a nearly straight state). Conversely, by performing the reverse operation, the V-shaped wire 20 in Figure 3(c) is returned to a nearly straight state as in Figure 3(b), and the length of the wire 20 becomes shorter. In other words, the wire 20 expands and contracts in response to the operation of modifying the layout of the wire 20. The length of the wire 20 due to expansion and contraction is calculated by the calculation means 44 (Figure 1).

[0032] In the state shown in Figure 3(c), once the layout correction of wire 20 is completed, in Figure 1, the calculation means 44 calculates the length of wire 20 based on the image data of wire 20. The operation to complete the layout correction may be performed by finger movements, similar to the selection operation described above, or by using a controller image that appears in the virtual space. The length of wire 20 after the layout correction can be displayed on the output stage 44 (Figure 1).

[0033] The outline of the design system 1 according to the present invention has been described above. Below, the design procedure using the design system 1 will be described in more detail with reference to the flowchart in Figure 4. Figure 4 is a flowchart showing the design procedure using the design system 1 in order of steps. Figures 5 to 13 are diagrams showing images in the virtual space in order of steps from the start of the design to the end of the design. In each of these diagrams, one operator is performing the work, but multiple operators may each wear a head-mounted display 61 and perform collaborative work.

[0034] Figure 5 shows an image of the virtual space at the start of the design process. Similar to Figure 2, the virtual space is the inside of a power distribution panel. The displayed image shows the main parts, and the individual structures inside the power distribution panel are not shown (the same applies to Figures 6 to 13). Although not shown in Figure 5, the object of the layout design is the wire 20 shown in Figures 2 and 3. The operator selects connection points A and B for the wire 20 while viewing the image on the head-mounted display 61 (Figure 2) (step 100 in Figure 4).

[0035] In Figure 5, electrical components 30 and 32 are installed in the virtual space. Electrical component 30 is provided with a recessed connector 31, and electrical component 32 is provided with a recessed connector 33. Finger 12 is pointing to part A of the recessed connector 31, and a beam 22 is emitted toward part A. If the operator performs a selection operation in this state, the recessed connector 31 of part A will be selected as the connection destination for the convex connector 21a (Figure 2) of the wire 20.

[0036] Figure 6 shows finger 12 pointing to part B of the recessed connector 33. In this state, beam 22 is emitted toward part B. If the operator performs a selection operation in this state, recessed connector 33 of part B will be selected as the connection destination for the convex connector 21b (Figure 2) of the wire 20.

[0037] Figure 7 shows the state in which part A of electrical component 30 and part B of electrical component 32 are connected by wire 20 (step 101 in Figure 4). In the state shown in Figure 7, wire 20 is connected between A and B with approximately the shortest possible length. Subsequently, the operator modifies the layout of wire 20 (step 102 in Figure 4). In Figure 1, the image of wire 20 is generated by the layout modification means 42 according to the image generation program 32 based on the object data 34.

[0038] Figure 8 shows the state where the correction point on wire 20 is pointed to by finger 12. A beam 22 is emitted from the tip of finger 12 facing wire 20, and the beam 22 reaches wire 20. A mark 24 is placed at the intersection of wire 20 and beam 22 by the operator's selection.

[0039] Figure 9 shows the finger 12 pointing to the new destination of mark 24. The tip 22a of beam 22 becomes the new destination of mark 23. From this state, a selection operation moves mark 24 to the new destination. Figure 10 shows the state after mark 24 has moved to the new destination. As mark 24 moves, wire 20 is curved.

[0040] Figure 11 shows the second modification point on wire 20 pointed to by finger 12. The subsequent operations are the same as for the first modification, so only the key points will be explained. A new mark 25 has been added to the intersection of wire 20 and beam 22. Figure 12 shows the state after mark 25 has moved to its new location. As mark 25 moves, a second curve is formed in wire 20.

[0041] Figure 13 shows the state after the initial layout correction of wire 20 has been completed. If the operator determines that the layout of wire 20 in Figure 13 is still not appropriate (step 103 in Figure 4), they repeat the above operation to correct the layout of wire 20 again. On the other hand, if the operator determines that the layout of wire 20 in the state of wire 20 in Figure 13 is appropriate (step 103 in Figure 4), the layout correction for wire 20 is completed by the confirmation operation.

[0042] Figures 5 to 13 only show electrical components 30 and 32 for illustrative purposes, but the distribution board has numerous wire connection points. Therefore, normally, once the layout design for the first wire 20 is completed, the layout design for the next wire is moved on. That is, if there are other wires to be designed (step 104 in Figure 4), the layout design is repeated for the new wires (steps 100 to 102 in Figure 4). Once the layout design for all wires is complete, the design data such as the length of each wire is output to the output means 5 (Figure 1).

[0043] The above embodiment is merely an example, and the following configuration may also be used. In the above embodiment, marks 23 to 25 are attached to the intersection of the wire 20 and the beam 22, making layout correction work easier using the marks as guides, but the configuration may be made without the attachment of marks. Similarly, the beam 22 is emitted from the finger 12, making layout correction work easier, but the configuration may be made without the generation of the beam 22.

[0044] In the above embodiment, as shown in Figure 3(b), the finger 12 first selects a new destination for the mark 23, and then, as shown in Figure 3(c), the layout of the wire 20 is modified. However, it is also possible to have the mark 23 follow the change in the position pointed to by the finger 12, thereby modifying the layout of the wire 20 all at once.

[0045] In the examples shown in Figures 8 to 13, the configuration involved attaching mark 24, then repositioning the wire 20, attaching a new mark 25, and then repositioning the wire 20 again. However, it is also possible to attach marks 24 and 25, select the destinations for marks 24 and 25 respectively, and then reposition the wire 20 all at once.

[0046] Furthermore, in the examples shown in Figures 8 to 13, both ends of the wire 20 are fixed, but one end may be a free end, and the wire 20 may be extended or retracted starting from the intersection of the wire 20 and the beam 22. In addition, both ends of the wire 20 may be free ends, and the wire 20 may be moved starting from the intersection of the wire 20 and the beam 22. In this case, the image of the wire 20 before movement may be retained, and the new wire 20 may be copied.

[0047] One embodiment of the present invention has been described above. The method for correcting the layout of a component according to the present invention involves using the intersection of the extension of the operator's fingers and the component as the starting point for the layout correction, and then repositioning the component. Therefore, when correcting the layout of a component, it is not necessary to directly grasp the component with the fingers, and the operator does not need to move close to the component. Consequently, regardless of the position of the component, the operator can perform the component layout correction work while remaining in the same position.

[0048] For example, if the target destination of a component is at the back of the virtual space, conventional design support devices require the operator to approach the back to grasp the component, and there are many structures between the operator and the back that interfere with the operator's fingers. Also, if the target destination of a component is at a high place such as the ceiling of the virtual space, conventional design support devices require the operator to stand on a ladder or similar. In contrast, with the present invention, there is no need to directly grasp the component with the fingers, so layout modifications become easy even if the target destination of a component is at the back of the virtual space or at a high place.

[0049] Furthermore, as mentioned above, according to the present invention, the operator does not need to approach the component, so the present invention is also useful for operators with physical disabilities. Moreover, since the present invention modifies the layout of the component by using the intersection of the extension of the operator's finger and the component as the starting point for layout modification, it becomes possible to operate with one hand, as shown in the work examples in Figures 5 to 13. [Explanation of symbols]

[0050] 1. Design System 2. Input means 3 Memory means 4. Control means 5 Output means 6. Virtual Space Display Means 7. Hand tracking means 10 Operator 11 Hands (Image) 12 fingers (image) 20 Wire (component) Marks 23, 24, and 25 30 Virtual Space 31 Virtual Space Generation Program 32 Image Generation Programs 33 3D data 41 Virtual space generation means 42 Layout modification means 43 Hand Image Generation Means 44 Calculation means 61 Head-mounted displays

Claims

1. A design system for designing the layout of components in a virtual space, A storage means storing a virtual space generation program for generating the virtual space, and an image generation program for generating images of the member and the operator's fingers within the virtual space, A virtual space display means for displaying the aforementioned virtual space, A hand tracking means that detects the movement of the operator's fingers, The system includes a control means that performs control according to the virtual space generation program and the image generation program, The control means is A layout modification means for modifying the layout of the member based on detection information from the hand tracking means and generating an image of the member that reflects the change in the layout of the member according to the image generation program, The system includes a hand image generation means for generating images of the operator's fingers that change moment by moment, based on detection information from the hand tracking means, according to the image generation program. The layout modification means is characterized in that it allows the layout of the member to be modified by using the intersection of the extension line of the operator's finger and the member as the starting point for the layout modification.

2. The member layout design system according to claim 1, wherein a mark is placed at the intersection.

3. A design method for designing the layout of components in a virtual space, A virtual space generation program for generating the aforementioned virtual space, An image generation program for generating images of the aforementioned components and images of the operator's fingers within the virtual space, A virtual space display means for displaying the aforementioned virtual space, A hand tracking means that detects the movement of the operator's fingers, Using the virtual space generation program and the control means that perform control according to the image generation program, The control means is A layout modification means for modifying the layout of the member based on detection information from the hand tracking means and generating an image of the member that reflects the change in the layout of the member according to the image generation program, The system includes a hand image generation means for generating images of the operator's fingers that change moment by moment, based on detection information from the hand tracking means, according to the image generation program. A design method for designing member layouts, characterized in that the layout of the member is modified by using the layout modification means to modify the layout of the member, with the intersection of the extension line of the operator's finger and the member as the starting point for the layout modification.

4. The design method for member layout according to claim 3, wherein a mark is placed at the intersection.

Images