Lifting simulation device, lifting simulation system, and lifting simulation method

Abstract

To provide a suspension simulation device that can display a three-dimensional model of a suspended load including actual physical behavior associated with movement.SOLUTION: A suspension simulation device 100 includes a camera 151. The suspension simulation device 100 further includes a display unit 118 that superimposes a captured image taken by the camera 151 and an image of a three-dimensional model of a suspended load generated from animation data 134 describing movement of the suspended load including vibration, and displays it on a display device (display 152).SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to a lifting simulation device, a lifting simulation system, and a lifting simulation method for displaying a three-dimensional model of a suspended load to be transported. [Background technology]

[0002] At construction sites, the task of lifting and transporting structures (architectural elements, construction elements) with cranes and chain blocks is repeated. In particular, in the case of plant construction, large equipment and piping with complex shapes often need to be transported through narrow spaces and installed in narrow spaces, so meticulous planning is required. A system for calculating a transport route for transporting a suspended load (suspension transport, hoisting) is the transport route calculation system described in Patent Document 1. This system calculates a transport route that does not interfere with the building and includes an appropriate hanging posture and arc trajectory based on building data, transported object data, mechanical parameters of the transport equipment, waypoint information, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-68019 A Summary of the Invention [Problem to be solved by the invention]

[0004] The system of Patent Document 1 generates the hanging posture and transportation route of the transported object (suspended load). However, the actual building data does not necessarily include all objects that exist on the construction site and may interfere with the transported load, such as temporary scaffolding and removable handrails. In addition, the suspended load vibrates back and forth and left and right and rotates as it is transported (moved), but Patent Document 1 does not take such physical behavior of the suspended load into consideration. For this reason, there is a possibility that uncalculated interference may occur.

[0005] The present invention has been made in consideration of the above background, and has an objective of providing a lifting simulation device, a lifting simulation system, and a lifting simulation method that enable the display of a three-dimensional model of a suspended load, including the actual physical behavior associated with the load as it moves. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention provides a suspension simulation device comprising: a camera; an animation generation unit that receives as input model setting information including the size, weight, and center of gravity position of a suspended load, model movement information including the movement direction, movement amount, and movement time of the center of gravity, and model rotation information including the rotation axis, rotation direction, rotation amount, and rotation time of the suspended load, and calculates physical behavior of the suspended load, including vibration, to generate animation data describing the movement of the suspended load; An image captured by the camera; and The above The apparatus is provided with a display unit that displays an image of a three-dimensional model of the suspended load generated from the animation data on a display device in a superimposed manner. Effect of the Invention

[0007] According to the present invention, it is possible to provide a lifting simulation device, a lifting simulation system, and a lifting simulation method that enable display of a three-dimensional model of a suspended load including the actual physical behavior associated with the movement. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0008] [Figure 1] FIG. 13 is a diagram showing an overview of the building into which beam materials are delivered. [Diagram 2] 13 is a diagram showing the initial state immediately before the lifting device is hung on the beam and lifted. FIG. [Diagram 3] This figure shows the state in which the beam is lifted from the initial state shown in Figure 2 and tilted 30 degrees by operating the chain block. [Figure 4] 4 is a diagram showing a state in which the beam material is lifted by a crane from the state shown in FIG. 3. FIG. [Diagram 5] 5 is a diagram showing a state in which the beam material is moved horizontally in the X direction by a crane from the state shown in FIG. 4. FIG. [Figure 6] 6 is a diagram showing a state in which the beam material has been lifted by a crane to its lifting limit from the state shown in FIG. 5. FIG. [Figure 7] 7 is a diagram showing a state in which the beam material is moved horizontally in the X direction by a crane from the state shown in FIG. 6. FIG. [Figure 8] 8 is a diagram showing a state in which the inclination of the beam material is returned to a horizontal state by operating the chain block from the state shown in FIG. 7. FIG. [Figure 9] 9 is a diagram showing a state in which the beam material is moved horizontally in the X direction by a crane from the state shown in FIG. 8. FIG. [Figure 10] 10 is a flowchart showing a procedure for using the lifting simulation device according to the present embodiment. [Figure 11] FIG. 2 is a functional configuration diagram of the lifting simulation device according to the present embodiment. [Figure 12] FIG. 4 is a configuration diagram of a scene setting screen according to the embodiment. [Figure 13] FIG. 4 is a diagram illustrating the configuration of a model position / posture setting screen according to the present embodiment. [Figure 14] 11 is a diagram for explaining the processing content of a physical calculation unit according to the embodiment. FIG. [Figure 15] 13 is a diagram for explaining an image in which a three-dimensional model of a beam material according to the present embodiment is displayed superimposed on an image captured by a camera. FIG. [Figure 16] 11 is a diagram for explaining calculation of the distance between a suspended load and an object at the site according to the embodiment. FIG. [Figure 17] 13 is a flowchart of a lifting simulation process according to the present embodiment. [Figure 18] 10 is a flowchart of an animation display process according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] <Summary of lifting work> Before describing a lifting simulation device in a mode (embodiment) for carrying out the present invention, an example of a lifting operation for carrying in a suspended load will be described. Fig. 1 is a diagram showing an outline of a building 890 into which a beam material 800 is carried in. With reference to Figs. 2 to 9 described later, the operation of carrying the beam material 800, which has been carried in from a carrying-in entrance 810 of the building 890 to a first floor 830, to a second floor 840 through an opening 820 in the second floor 840 will be described.

[0010] 2 is a diagram showing the initial state immediately before hoisting by hanging a hoist 850 on a beam 800. The hoist 850 is a general term for a machine or tool used to hoist and move a heavy object to be hoisted, such as a crane, a hoisting fitting, a chain block, a wire, or a sling. FIG. 3 is a diagram showing a state in which the beam 800 is lifted from the initial state shown in FIG. 2 and tilted by 30 degrees by operating the chain block. Fig. 4 is a diagram showing a state in which beam 800 has been lifted up by a crane from the state shown in Fig. 3. The end point of beam 800 in the X direction passes through opening 820 and is positioned above second floor 840.

[0011] FIG. 5 is a diagram showing a state in which the beam 800 has been moved horizontally in the X direction by a crane from the state shown in FIG. Fig. 6 is a diagram showing a state in which the beam 800 has been lifted up to its lifting limit by a crane from the state shown in Fig. 5. The end point of the beam 800 in the -X direction is below the opening 820. FIG. 7 is a diagram showing a state in which the beam 800 has been moved horizontally in the X direction by a crane from the state shown in FIG.

[0012] FIG. 8 is a diagram showing a state in which the inclination of the beam 800 has been returned to a horizontal state by operating the chain block from the state shown in FIG. Fig. 9 is a diagram showing a state in which the beam material 800 has been moved horizontally in the X direction by a crane from the state shown in Fig. 8. From this state, the crane is operated to lower the beam material 800 onto the second floor 840, completing the delivery work.

[0013] <Problems with existing lifting simulations> 2 to 9, it appears possible for beam 800 to be brought into building 890 without interfering with second floor 840 (the edge of opening 820). However, when beam 800 is suspended and moved during actual delivery work, beam 800 shakes (vibrates) back and forth and side to side, and for example, a portion 811 (see FIG. 5) of beam 800 close to the edge of opening 820 may interfere with the edge of opening 820. For this reason, the presence or absence of interference cannot be confirmed in a suspension simulation in which a three-dimensional model of beam 800, which is the suspended load, is moved up and down and side to side within a model of building 890, or the angle of the attitude of beam 800 is changed.

[0014] Furthermore, the magnitude and direction of the vibrations are determined by the size and weight of the beam 800 that is the suspended load, the length of the wires included in the hoisting device 850, as well as the speed of movement (changes in speed). For this reason, it cannot be said that interference will not occur just because there is a gap of a certain value or more between the beam 800 and the building 890. When simulating suspension, it is necessary to reproduce the movement (physical behavior) of the suspended load, including vibration, taking into account the speed of movement (travel time), so that it can be confirmed that there will be no interference.

[0015] When simulating the lifting, it is necessary to model the site (building 890) in addition to the lifted load (beam 800). However, at the actual site, there are objects such as temporary structures that are not modeled but interfere with the lifted load, such as scaffolding that is temporarily installed. Although not shown in Figs. 1 to 9, removable handrails are installed around the opening 820, and are removed if they get in the way of the delivery. If the temporary structure is easily movable, it can be moved during the delivery, but if it is difficult to move, it becomes necessary to add a model of the temporary structure to the site model and simulate the delivery. However, the existence of the temporary structure, its location, size, etc. cannot always be grasped in advance (when the delivery plan is drawn up), and in many cases, it is only discovered on the site on the day of delivery. It is not realistic to measure the location and size of the temporary structure at the site and model it to determine the delivery route at the site where time is limited.

[0016] <Overview of the lifting simulation device> As described above, the lifting simulation device is required to be able to simulate the movement of the lifted load, including vibration, and check for interference. It is also required to be able to check for interference with objects that are on-site but not modeled, such as temporary structures.

[0017] The lifting simulation device according to this embodiment (see FIG. 11 described later) meets these requirements by displaying a three-dimensional model of the load superimposed on the image of the site. In detail, a preset three-dimensional model of the load is displayed superimposed on the image of the site captured by a camera. The three-dimensional model of the load moves according to a set delivery process (for example, the process described in FIG. 2 to FIG. 9). At this time, the three-dimensional model is displayed including not only the vertical and horizontal movement due to the crane operation and the change in posture due to the chain block, but also the physical behavior including the swing and rotation of the load. The swing refers to the swing of the load back and forth and side to side like a pendulum. The rotation refers to the rotation of the load while changing one direction or direction. The physical behavior of the load including the swing and rotation, which is different from the vertical and horizontal movement due to the crane operation and the change in posture due to the chain block, is also referred to as vibration.

[0018] By overlaying the movement of a 3D model of the load, including movement due to crane operation, changes in posture due to the chain block, and vibration, on the video of the site, it becomes possible to check whether the load will cause interference during actual transport work.The lifting simulation device is capable of recognizing its own position on site, and can capture images of the site from the position and direction desired by the user, and then overlay a 3D model of the load on the captured image. In addition, the lifting simulation device can recognize the spatial shape of the site (walls, ceilings, floors, equipment, installed objects, etc.) and issues a warning if the distance to the suspended load is below a specified value.

[0019] By using this type of lifting simulation device, the user (the worker who delivers the load) can move the 3D model of the load while watching the video of the site, and check where the load will pass, how much it will sway, and the distance between the load and walls or other installed objects. Even if the delivery route must be changed due to an unexpected installed object causing interference, the delivery route can be changed by replacing the delivery process or changing parameters (travel distance, direction, etc.). By overlaying the 3D model of the load passing through the new delivery route on the video of the site, it becomes possible to confirm that there will be no interference before carrying out the actual delivery work.

[0020] Before describing the above-mentioned lifting simulation device 100 (see FIG. 11 described later), the procedure for using the lifting simulation device 100 will be described. In the following, when confusion is not particularly problematic, the suspended load and the 3D model of the load may be described as being the same. For example, checking that the 3D model of the load does not interfere with the wall on the screen is described as checking that the load does not interfere with the wall on the screen.

[0021] FIG. 10 is a flowchart showing a procedure for using the lifting simulation device 100 according to this embodiment. In step S11, the user creates a 3D model of the load and the hoisting equipment (e.g., a hook). In detail, the user sets the size, shape, weight, center of gravity, and hoisting point (point for hanging a wire sling, etc.) of the load, as well as the positional relationship (constraint condition) between the load and the hook.

[0022] In step S12, the user creates (sets) a delivery scene. A scene is one delivery process, such as raising a suspended load, lowering it, moving it in one direction, changing its position, or changing its direction. The delivery of the beam 800 (see FIG. 1) to the second floor 840 described above is made up of a total of eight scenes (delivery processes, steps), starting with the scene of lifting the beam 800 from the initial state shown in FIG. 2 as shown in FIG. 3, and ending with the scene of lowering the beam 800 to the second floor 840 from the state shown in FIG. 9. Note that FIG. 12 and FIG. 13, which will be described later, show the screen of the lifting simulation device 100 when a scene is set.

[0023] In step S13, the user instructs the creation of an animation. Upon receiving the instruction, the lifting simulation device 100 generates animation data showing the movement of the suspended load, including vibration. The above steps S11 to S13 are the steps for advance preparation. The following are the steps at the delivery site. In step S14, the user places the marker at the site (delivery site).

[0024] In step S15, the user checks the spatial recognition by the suspension simulation device 100. More specifically, the user photographs the markers and the space at the site with the camera of the suspension simulation device 100 while moving around. During this time, the suspension simulation device 100 recognizes the spatial shape of the site. The suspension simulation device 100 displays the spatial shape (surfaces of objects at the site, such as walls and installations) that is the recognition result as a mesh superimposed on the image. The user can confirm that the suspension simulation device 100 has correctly recognized the spatial shape by seeing that the mesh is correctly displayed.

[0025] In step S16, the user plays back the animation created in step S13 (by moving the 3D model of the suspended load) to check for interference. More specifically, the lifting simulation device 100 recognizes its own position and orientation at the site, using the marker as a base point. The lifting simulation device 100 displays a 3D model of the suspended load visible from the recognized position and orientation, superimposed on the video of the site. The user plays and stops the animation, and changes the position and orientation of the lifting simulation device 100, to check for interference between the displayed 3D model and walls or installed objects shown in the video of the site.

[0026] In step S17, if there is interference, the user proceeds to step S18, and if there is no interference, the user proceeds to step S19. Regardless of whether there is interference or not, the user may determine that there is a high possibility of interference (the distance between the suspended load and the object is small) and proceed to step S18. In step S18, the user modifies the scene, which is the delivery work process, creates an animation, and then returns to step S16. In step S19, the user actually carries out the carrying-in work.

[0027] <Configuration of the lifting simulation device> 11 is a functional configuration diagram of the lifting simulation device 100 according to this embodiment. The lifting simulation device 100 includes a control unit 110, a storage unit 130, a camera 151, a display 152, a depth sensor 153, and an inertial sensor 154. The camera 151 is a camera that captures an image of the site including the marker. The display 152 is a display with a touch panel that accepts user operations. The depth sensor 153 is, for example, a ToF (Time-of-Flight) depth sensor, and measures the distance to an object (object surface) at the site such as a wall, floor, equipment, installation, or temporary structure. The inertial sensor 154 includes a three-directional acceleration sensor, a gyroscope (angular velocity sensor), and the like.

[0028] <Configuration of the lifting simulation device: Memory unit> The storage unit 130 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, etc. The storage unit 130 stores model data 131, model position and orientation data 132, motion data 133, animation data 134, marker data 135, spatial shape data 136, etc.

[0029] The model data 131 is data relating to a three-dimensional model of the load and the hoisting tool. The model data 131 includes the size, shape, weight, center of gravity position, hoisting point position, etc. of the load, as well as a three-dimensional model of the hoisting tool hook and data relating to the positional relationship (constraint conditions) between the load and the hoisting tool (crane, hook, etc.). The model data 131 is set during the creation of the three-dimensional model in step S11 (see FIG. 10). The model position / posture data 132 is data related to the position and posture of a three-dimensional model including a suspended load. The model position / posture data 132 includes data such as the movement direction and movement amount, or the rotation axis, rotation direction, and rotation amount (rotation angle) for each scene (carry-in process), data on the sequence of scenes, and the length of each scene (process time). The model position / posture data 132 is set during scene creation in step S12 (see FIG. 10).

[0030] The motion data 133 is data indicating the physical behavior of the suspended load. For example, the motion data 133 is time history coordinate value data of the center of gravity or the suspension point of the suspended load (three-dimensional model of the suspended load), and is data indicating the physical behavior including the vibration of the suspended load. The animation data 134 is data that indicates the movement of the three-dimensional model, including vibrations. The animation data 134 is, for example, time history data of the positions and orientations of polygons that make up the three-dimensional model.

[0031] The marker data 135 is data indicating a marker, for example, image data of the marker. The marker data 135 also includes data on the position and orientation of the marker placed in step S14 (see FIG. 10). The spatial shape data 136 is shape data of the space that is the delivery site, and is shape data including the positions of objects at the site. The spatial shape data 136 is generated by the space recognition unit 117, which will be described later.

[0032] <Configuration of the lifting simulation device: Control unit> The control unit 110 is configured to include a CPU (Central Processing Unit), and includes a model setting unit 111, a scene setting unit 112, a physics calculation unit 113, an animation creation unit 114, a marker setting unit 115, a marker recognition unit 116, a space recognition unit 117, a display unit 118, and an interference determination unit 119. The model setting unit 111 accepts settings for a three-dimensional model from the user (see step S11 in FIG. 10) and stores the settings in the model data 131. The model setting unit 111 also displays the set three-dimensional model on the display 152 while changing the viewpoint and direction, and accepts corrections to the settings from the user.

[0033] The scene setting unit 112 accepts a scene (delivery process) setting from the user (see step S12 in FIG. 10) and stores it in the model position / posture data 132. The user sets the scene using a scene setting screen 210 (see FIG. 12) described later. The setting result is stored in the model position / posture data 132.

[0034] FIG. 12 is a configuration diagram of a scene setting screen 210 according to this embodiment. A user uses the scene setting screen 210 to add, delete, and change the order of scenes. In a scene string display area 221 on the left side of the screen, scenes that have been set at the current time are displayed in the order of the delivery process along with their "length." "Length" refers to the time required for a process, for example, the time it takes to raise a suspended load by 1 m. Even for the same scene, if the "length" value is large, the movement will be slower. The top scene shows the "initial state", which is the first state of the loading operation, and shows the position and orientation of the suspended load (see FIG. 2). The scenes after the "initial state" show the movement and change in posture (rotation) of the suspended load. In this embodiment, one scene is a movement with the same movement direction, or a rotation (change in posture) with the same rotation axis and rotation direction.

[0035] When the save button 235 is tapped, the set scenes, their lengths, the order of the scenes, etc. are stored as the model position / posture data 132. When the open button 234 is tapped, the stored model position / posture data 132 is read out, and the scene names and lengths are displayed in the scene string display area 221. When a scene is tapped to select it in the scene string display area 221 and the scene delete button 233 is tapped, the selected scene is deleted. When a scene is tapped to select it in the scene string display area 221 and the scene insert button 232 is tapped, a new scene is added next to the selected scene. When a scene is tapped to select it in the scene string display area 221 and the scene change button 231 is tapped, a model position / posture setting screen 240 (see FIG. 13) described later is displayed, and the selected scene can be changed.

[0036] Fig. 13 is a configuration diagram of the model position / posture setting screen 240 according to this embodiment. When the scene change button 231 (see Fig. 12) is tapped, the model position / posture setting screen 240 is displayed. "Model#302" which is the "object name" is identification information of the 3D model of the load included in the scene, and indicates the load to be set. The "X-axis movement" setting area 251 is an area for setting the amount of movement of the suspended load in the X-axis direction, and the amount of movement is set by tapping the "-" button or the "+" button in the setting area 251, or by selecting the amount of movement from the "movement amount" combo box 261. The same applies to "Y-axis movement" and "Z-axis movement."

[0037] The "X-axis rotation" setting area 252 is an area for setting the amount of rotation about the X-axis of the suspended load, and the amount of rotation is set by moving the knob 253 in the setting area 252 left and right, tapping the "<" button or ">" button, or selecting the amount of rotation from the "rotation amount" combo box 262. The same applies to "Y-axis rotation" and "Z-axis rotation." When the OK button 263 is tapped, the settings are reflected, the model position / posture setting screen 240 is closed, and the screen returns to the scene setting screen 210 (see FIG. 12). When the cancel button 264 is tapped, the settings are not reflected, the model position / posture setting screen 240 is closed, and the screen returns to the scene setting screen 210.

[0038] Returning to FIG. 11 , the physical calculation unit 113 calculates the physical behavior (movement) of the load, including vibration, from information such as the size, shape, and weight of the load stored in model data 131, its positional relationship (constraint condition) with the hoisting device (crane, hook, etc.), and information on the movement and rotation of the load stored in model position and orientation data 132, and stores the calculation results in motion data 133.

[0039] 14 is a diagram for explaining the processing contents of the physical calculation unit 113 according to this embodiment. When a trolley 856 provided on a crane moves on a girder 857, a beam 800 suspended by a hoisting tool 850 moves in the ±X direction. If the displacement of the trolley 856 in the -X direction is u, the angle that the hoisting tool 850 makes with the vertical direction is θ, the length of the hoisting tool 850 (wire or chain block) is l, the hoisting speed of the hoisting tool 850 is lv, and the gravitational acceleration is g, then (Equation 1) holds. Here, the differentiation is a differentiation with respect to time.

[0040]

number

[0041] The physical calculation unit 113 finds u, which is the displacement of the trolley 856, from the information on the movement of the suspended load stored in the model position / posture data 132, and calculates θ from (Equation 1) to calculate the physical behavior (movement) of the suspended load, including vibration, and stores the calculation result in the motion data 133. Similarly, the physical behavior of the rotation of the suspended load is calculated from a differential equation of the rotation angle.

[0042] The animation creation unit 114 calculates the movement of the three-dimensional model, including vibration, from the motion data 133 and the setting information of the three-dimensional model contained in the model data 131, and stores the calculation results in the animation data . The marker setting unit 115 acquires setting information related to the shape data of the marker (for example, image data of the marker) and the installation position and orientation, and stores the information in the marker data 135.

[0043] The marker recognition unit 116 recognizes markers installed at the site from the images captured by the camera 151 and the distance acquired by the depth sensor 153, and calculates the position and orientation of the suspension simulation device 100 itself from information such as the direction and distance of the marker relative to the suspension simulation device 100 and the installation position and orientation of the marker stored in the marker data 135. The position and orientation of the suspension simulation device 100 after marker recognition can be calculated from the amount of change in position and orientation of the suspension simulation device 100 calculated from the measurement results of the inertial sensor 154 and the position and orientation at the time of marker recognition calculated by the marker recognition unit 116.

[0044] The space recognition unit 117 calculates a spatial shape from the image captured by the camera 151 , the position and orientation of the lifting simulation device 100 , and distance measurement information from the depth sensor 153 , and stores the calculated shape in the spatial shape data 136 . The display unit 118 displays the three-dimensional model described in the animation data 134 by superimposing it on the image captured by the camera 151. The display unit 118 generates and displays an image of the three-dimensional model with the suspension simulation device 100 as the viewpoint, based on the position and orientation of the suspension simulation device 100 and the position and orientation of the three-dimensional model described in the animation data 134. The animation data 134 is time history data, and the display unit 118 successively displays the three-dimensional models, thereby making it possible to display the movement of the three-dimensional model, including vibration.

[0045] FIG. 15 is a diagram for explaining an image in which a three-dimensional model of a beam 800 according to this embodiment is displayed superimposed on an image captured by the camera 151. The captured image includes a handrail 860 and an elevator door 870 installed on the edge of an opening 820 of a second floor 840. The display unit 118 displays a three-dimensional model of a beam 800 (see FIG. 7) whose tip in the X direction has passed through the opening 820 to the second floor 840 superimposed on an image captured by the camera 151. The captured image of a part of the handrail 860 and wall that is hidden by the three-dimensional model and can be calculated from the position and shape data of the wall and handrail 860 of the second floor 840 included in the spatial shape data 136, and the position, size, and shape data of the three-dimensional model of the beam 800, is not displayed. Therefore, the three-dimensional model of the beam 800 is displayed as if it is in front of the handrail 860 and the wall.

[0046] Returning to Fig. 11, the interference determination unit 119 calculates the distance between the suspended load and an object at the site, such as a wall. In detail, it calculates the distance from a preset interference measurement point 801 (see Fig. 16 described later) on the three-dimensional model of the suspended load to the object. If the distance is closer than a predetermined value, the interference determination unit 119 issues an alarm.

[0047] 16 is a diagram for explaining calculation of the distance between a suspended load and an object at the site according to this embodiment. The interference determination unit 119 calculates the distance from a preset interference measurement point 801 as the starting point to an object (e.g., a wall of a building 890) in a preset interference measurement direction 802. Position information of the object is stored in the spatial shape data 136.

[0048] <Lifting simulation processing> 17 is a flowchart of the lifting simulation process according to this embodiment. This process is the process of the lifting simulation device 100, which corresponds to steps S11 to S18 shown in FIG. In step S31, the model setting unit 111 accepts settings for a three-dimensional model from a user, and stores the settings in the model data 131. Step S31 is a process of the model setting unit 111 that corresponds to step S11 in FIG.

[0049] In step S32, the scene setting unit 112 accepts a scene (delivery process) setting from the user, and stores the setting contents in the model position / posture data 132. Step S32 is a process of the scene setting unit 112 corresponding to step S12 described in FIG. In step S 33 , the physical calculation unit 113 calculates the physical behavior (movement) of the suspended load, including vibration, from the model data 131 and the model position and orientation data 132 , and stores the calculation result in the motion data 133 . In step S34, animation creation unit 114 calculates the movement of the three-dimensional model, including vibration, from motion data 133 and model data 131, and stores the calculation results in animation data 134. Steps S33 and S34 are processes by physics calculation unit 113 and animation creation unit 114, which correspond to step S13 in FIG.

[0050] In step S35, the space recognition unit 117 calculates a spatial shape from the position and orientation of the lifting simulation device 100 at the time of marker recognition calculated by the marker recognition unit 116, the image captured by the camera 151, the amount of change in movement and orientation measured by the inertial sensor 154, and the distance measurement information of the depth sensor 153, and stores the calculated spatial shape in the spatial shape data 136. The space recognition unit 117 displays the recognized spatial shape as a mesh by superimposing it on the image captured by the camera 151. In step S36, the space recognition unit 117 confirms with the user whether the recognized and displayed spatial shape is correct. If the confirmation is OK (the recognition result is correct) (step S36→YES), the process proceeds to step S37, and if it is not OK (step S36→NO), the process returns to step S35 and the spatial shape is generated again. Steps S35 and S36 are processes of the space recognition unit 117 corresponding to step S15 in FIG. 10.

[0051] In step S37, the display unit 118 executes an animation display process. Details of the animation display process will be described later with reference to Fig. 18. Step S37 is a process of the display unit 118 that corresponds to step S16 in Fig. 10. In step S38, if there is an instruction to modify the scene (delivery process) during the animation display processing in step S37 (step S38→YES), the display unit 118 proceeds to step S39, and if there is no instruction to modify (step S38→NO), the display unit 118 proceeds to step S42. The instruction to modify is an instruction to modify given by an operation in step S59 in Fig. 18, which will be described later. Step S38 is processing by the display unit 118 corresponding to step S17 in Fig. 10. In step S 39 , the scene setting unit 112 accepts any corrections to the scene settings from the user, and stores the corrected settings in the model position and orientation data 132 .

[0052] Steps S40 and S41 are similar to steps S33 and S34, respectively. After step S41 is executed, the process returns to step S37. Steps S39 to S41 are processes performed by the scene setting unit 112, the physics calculation unit 113, and the animation creation unit 114, which correspond to step S18 in FIG. In step S42, the display unit 118 ends the lifting simulation process if there is an instruction to end the animation display process (step S42→YES), and returns to step S37 if there is no instruction to end (step S42→NO). The instruction to end is an instruction to end given by an operation in step S59 of FIG. 18, which will be described later.

[0053] <Animation display processing> 18 is a flowchart of the animation display process according to this embodiment. The animation display process in step S37 will be described in detail with reference to FIG. In step S51, the display unit 118 acquires the position and orientation of the suspension simulation device 100. In detail, the display unit 118 calculates the current position and orientation of the suspension simulation device 100 from the position and orientation of the suspension simulation device 100 at the time of marker recognition calculated by the marker recognition unit 116 and the amount of change in movement and orientation measured by the inertial sensor 154.

[0054] In step S52, the display unit 118 calculates the display position and orientation of the three-dimensional model image on the image captured by the camera 151 from the position and orientation of the lifting simulation device 100 calculated in step S51 and the position and orientation of the three-dimensional model described in the animation data 134. In step S53, display unit 118 generates a three-dimensional model image that matches the display position and orientation of the three-dimensional model image calculated in step S52, and displays it superimposed on the image captured by camera 151.

[0055] In step S54, the interference determination unit 119 calculates the distance from the suspended load to an object such as a wall or an installation at the site. When multiple interference measurement points or multiple interference measurement directions (see FIG. 16) are set, the interference determination unit 119 calculates multiple distances and calculates the smallest distance. In step S55, if the distance calculated in step S54 is equal to or less than the first threshold (step S55→YES), the interference determination unit 119 proceeds to step S56, and if the distance exceeds the first threshold (step S55→NO), the interference determination unit 119 proceeds to step S57.

[0056] In step S56, the interference determination unit 119 displays the distance in red. The interference determination unit 119 may also display the interference measurement point and interference measurement direction corresponding to the distance by superimposing them on the three-dimensional model image. In step S57, if the distance calculated in step S54 is equal to or less than the second threshold (step S57→YES), the interference determination unit 119 proceeds to step S58, and if the distance exceeds the second threshold (step S57→NO), the interference determination unit 119 proceeds to step S59.

[0057] In step S58, the interference determination unit 119 displays the distance in yellow. The interference determination unit 119 may display the interference measurement point and interference measurement direction corresponding to the distance by superimposing them on the three-dimensional model image. In step S59, if there is an operation from the user (step S59→YES), display unit 118 ends the animation processing and proceeds to step S38 (see FIG. 17), and if there is no operation (step S59→NO), returns to step S51. The operation is an operation to correct the scene or to instruct the end of the animation display (see steps S38 and S42 in FIG. 17).

[0058] <Features of the lifting simulation device> The lifting simulation device 100 calculates the physical behavior (movement) of the load from the information of the 3D model of the load (size, shape, weight, etc.) and the delivery process information (movement and rotation), and displays it superimposed on the video information of the site. The physical behavior includes not only the movement and rotation caused by the delivery work itself, but also the vibration caused by the start and end of the movement and rotation. By viewing the 3D model, which includes physical behavior including vibration and is superimposed on the video of the site, the user can confirm how the delivery work will proceed. In addition, points where interference is likely to occur (interfering parts of the load and objects) can be confirmed.

[0059] By setting the interference measurement point and interference measurement direction (see FIG. 16), the lifting simulation device 100 measures the distance between the suspended load and the object (a wall or an installed object at the site) and issues an alarm if the distance is equal to or less than a predetermined threshold (see steps S55 to S58 in FIG. 18). The user can confirm the distance between the suspended load and the object not only by image but also by numerical value. If it is determined that the distance is small (the margin is small), the scene (delivery process) can be changed and the delivery work can be confirmed again (see step S17→YES, steps S18, S16 in FIG. 10, step S38→YES, steps S39 to S41, S37 in FIG. 17).

[0060] Delivery sites for plants under construction change daily, and it may become clear on the day of delivery that temporary structures are located near the delivery route. Even in such cases, the delivery process can be confirmed by checking the 3D model images overlaid on the on-site video and the distance to the temporary structures. Also, if there is a high possibility of interference, the delivery process can be reviewed and checked again on the spot.

[0061] <<Variation: Scene Setting>> In the above embodiment, the movement direction, movement amount, rotation axis, and rotation amount are set for each scene, and the length (time) is set (see Figs. 12 and 13). Instead of setting changes such as movement and rotation, the position and posture of the load at the end of each scene may be set. In this case, the physical calculation unit 113 calculates the physical behavior of the load from the set changes in position and posture. In the above embodiment, the movement of the suspended load is limited to linear movement, but it is also possible to define movement including a curve in response to a crane equipped with a curved rail.

[0062] <Modification: Lifting simulation system> In the above embodiment, the suspension simulation device 100 has all the functions, but some of the functions may be realized by other devices. For example, the suspension simulation system may have different functions, with the physics calculation unit 113 and the animation creation unit 114 serving as animation generation devices and the remaining units serving as portable display devices.

[0063] Other variations Although several embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. For example, in the animation display process (see FIG. 18), the display process (steps S51 to S53) and the distance calculation process (steps S54 to S57) may be processed in parallel. Alternatively, each process may be repeated in parallel.

[0064] Also, the physical calculation unit 113 that calculates the physical behavior of the suspended load and the animation creation unit 114 that calculates the movement of the three-dimensional model may be integrated into an animation generation unit. In the above embodiment, the space recognition unit 117 calculates the spatial shape in step S35 shown in Fig. 17, but the calculation may also be performed during the animation display processing in step S37 to correct the spatial shape data 136.

[0065] The present invention can take various other embodiments, and furthermore, various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents. [Explanation of symbols]

[0066] 100 Lifting simulation device 111 Model Setting Section 112 Scene setting section 113 Physics calculation unit (animation generation unit) 114 Animation Creation Department (Animation Generation Department) 115 Marker setting section 116 Marker Recognition Unit 117 Spatial recognition section 118 Display section 119 Interference detection section 131 Model data (model setting information) 132 Model position and orientation data (model movement information, model rotation information) 133 Motion Data 134 Animation Data 136 Spatial Shape Data 151 Camera 152 Display (display device) 153 Depth sensor (spatial sensor) 800 Beam (hanging load, 3D model of hanging load)

Claims

1. Camera and Model setting information including the size, weight, and center of gravity of the suspended load, and Model movement information including the movement direction, movement amount, and movement time of the center of gravity, and model rotation information including the rotation axis, rotation direction, rotation amount, and rotation time of the suspended load an animation generation unit that calculates the physical behavior of the suspended load, including vibration, as an input, and generates animation data describing the movement of the suspended load; a display unit that displays on a display device a captured image captured by the camera and an image of a three-dimensional model of the suspended load generated from the animation data in a superimposed manner; A lifting simulation device comprising:

2. The model movement information includes a plurality of pieces of model movement information each having a different movement direction.

2. The lifting simulation device according to claim 1 .

3. The model rotation information includes a plurality of pieces of model rotation information in which the rotation axis or the rotation direction is different.

2. The lifting simulation device according to claim 1 .

4. Camera and a display unit that displays on a display device a captured image captured by the camera and an image of a three-dimensional model of the suspended load generated from animation data describing the movement of the suspended load including vibration in a superimposed manner; A spatial sensor for measuring a distance to an object; a space recognition unit that generates space shape data from the measurement results of the space sensor; an interference determination unit that displays a warning when a distance from a predetermined point on the three-dimensional model to an object included in the spatial shape data and located in a predetermined direction with respect to the three-dimensional model is within a predetermined value; A lifting simulation device comprising:

5. A load lifting simulation system comprising an animation generating device and a portable display device having a camera, The animation generating device comprises: Model setting information including the size, weight, and center of gravity position of the load; and an animation generation unit that receives as input model movement information including the movement direction, amount, and time of the center of gravity, and model rotation information including the rotation axis, direction, amount, and time of rotation of the load, calculates physical behavior of the load including vibration, and generates animation data describing the movement of the load, The portable display device includes: and a display unit that displays on a display device a captured image captured by the camera and an image of a three-dimensional model of the suspended load generated from the animation data in a superimposed manner. A lifting simulation system comprising:

6. A suspension simulation method for a suspension simulation device equipped with a camera, comprising: The lifting simulation device, Model setting information including the size, weight, and center of gravity of the suspended load, and Model movement information including the movement direction, movement amount, and movement time of the center of gravity, and model rotation information including the rotation axis, rotation direction, rotation amount, and rotation time of the suspended load As an input, calculating the physical behavior of the load, including vibrations, to generate animation data describing the movement of the load; and displaying, on a display device, a captured image captured by the camera and an image of a three-dimensional model of the suspended load generated from the animation data in a superimposed manner. A lifting simulation method comprising:

Images