Suspended load movement path generation device, truck-mounted crane and program

Abstract

To reduce a cognitive load and fatigue of a crane operator related to complex repeated operations for loading a cargo onto a loading platform of a truck.SOLUTION: A lifted load route generation device includes: an image acquisition unit 10 that acquires an overhead image taken from a boom tip; an input unit 13 that receives an input specifying a lifted load to be loaded in an overhead image; a depth acquisition unit 11 that acquires a distance from the boom tip to the lifted load; an object recognition unit 14 that recognizes a position and a shape of the lifted load from the overhead image and the distance; a loading position acquisition unit 15 that acquires a loading position for loading the lifted load on a loading platform of a loading type truck crane; and a route generation unit 16 that generates a hook route as a recommended travel route that passes through a rise position where the hook on which the lifted load is slung is located from a starting position of the slung load to a position where the lifted load is separated from the loading platform by a side margin and is separated from the ground surface by a safety margin, and through an outer circumference position where the lifted load passes over the outer circumference at a safe height from the outer circumference of the loading platform and reaches a target position at a predetermined height from the loading platform directly above the loading position.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for generating a path of a suspended load when loading a load onto a platform of a truck-mounted crane.

Background Art

[0002] Crane operators are required to grasp the situation at work sites with various different conditions and move the suspended load safely and surely. In order to reduce the load of crane work, techniques for assisting crane operators have been proposed.

[0003] For example, in the crane of Patent Document 1, three-dimensional information of a suspended load is extracted based on three-dimensional information obtained by a stereo camera or the like, and a plan view of the work site is created. Based on the three-dimensional information, the crane creates a developed cross-sectional view along the movement path input on the plan view. An operator can input a movement path in the height direction of the suspended load on the developed cross-sectional view. Patent Document 1 describes that the crane can automatically move the suspended load according to the input movement path.

[0004] Patent Document 2 describes a technique for providing guide information that accurately represents the flow line of a suspended load. The guide information display device of Patent Document 2 includes a camera that takes an image of a region including the suspended load and the ground surface from above the suspended load that is suspended by a wire hanging from a boom and moved according to the operation of the boom, and a laser scanner that acquires point cloud data from above the suspended load, a data acquisition unit, a data display unit that displays the image of the camera, and a data processing unit that generates guide information to be superimposed and displayed on the image of the camera based on the point cloud data. The data processing unit estimates a reference height of the ground surface based on the point cloud data acquired by the laser scanner, calculates the shooting range of the camera based on the reference height of the ground surface, and generates guide information representing the flow line of the suspended load based on the shooting range of the camera.

[0005] Patent Document 3 describes a method for controlling a climbing crane. In the control method described in Patent Document 3, the position and size of the suspended load are automatically determined from three-dimensional measurement data, and the position and size of collision avoidance objects that the suspended load must not collide with are also determined. When the distance between the suspended load and the collision avoidance object falls below a threshold, a drive stop signal is output to the climbing crane.

[0006] In the cable crane of Patent Document 4, an automatic driving device, a sway-preventing device, and a trolley positioning device are provided. The automatic driving device, sway-preventing device, and positioning device are activated by command signals from the operator console to hoist the load, while simultaneously preventing the load from swaying by moving the trolley horizontally. When the load reaches directly above the bottom position, the automatic landing device is activated automatically or by command signals from the operator console to automatically lower the load, and the load is lowered to the target position using height and position signals from a visual sensor and an image processing device. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2018-095370 [Patent Document 2] Japanese Patent Publication No. 2019-023116 [Patent Document 3] Japanese Patent Publication No. 2019-167221 [Patent Document 4] Japanese Patent Application Publication No. 08-175788 [Overview of the project] [Problems that the invention aims to solve]

[0008] When a loading operation is presented, the crane operator must repeatedly perform similar, complex motion operations for each load that needs to be loaded. In most cases, these loads are placed close to the ground and require almost the same movements to be positioned on the truck bed. However, the shape and weight of the loads, as well as the conditions of the work site where the loads are moved, vary greatly, and the crane operator must visualize the appropriate movement path for each load.

[0009] While the technology described in the patent document assists crane operators, there was room to reduce the cognitive load and fatigue of the crane operators.

[0010] The present invention aims to reduce the involvement of crane operators in the process of loading cargo onto truck beds, thereby reducing the cognitive load and fatigue of crane operators in complex, repetitive tasks. [Means for solving the problem]

[0011] A load movement path generation device for a truck-mounted crane according to a first aspect of the present invention is: An image acquisition unit that acquires an overhead image taken from the tip of the boom of a truck-mounted crane, An input unit that receives input specifying the suspended load to be loaded in the aforementioned overhead view image, A depth acquisition unit that acquires the distance from the boom tip to the suspended load, A target recognition unit that recognizes the position and shape of the suspended load from the overhead image and the distance, A loading position acquisition unit that acquires the loading position for loading the suspended load onto the loading platform of the aforementioned truck crane, A path generation unit generates a recommended movement path for the hook to which the load is attached, passing from the starting position where the load is attached, through a raised position where the load is separated from the loading platform by a lateral margin and safely separated from the ground surface, and through an outer perimeter position where the load is separated from the outer perimeter of the loading platform at a safe height and passes over the outer perimeter, until the load reaches a target position which is a predetermined height from the loading platform directly above the loading position. It is equipped with.

[0012] Preferably, The aforementioned path generation unit, In a plan view projected onto a horizontal plane, if the shortest line segment connecting the starting position and the loading position intersects with the rear edge of the loading platform, or if, in the plan view, the outer circle with the center of rotation of the boom as its center and a radius equal to the distance between the center of rotation and the loading position plus the diameter of the circumscribed circle of the suspended load's footprint does not intersect with the side edge of the loading platform, then the intersection point of the shortest line segment and the outer circumference of the loading platform in the plan view is defined as the outer circumference position, and the position on the shortest line segment in the plan view, separated from the outer circumference of the loading platform by the lateral margin, is calculated as the rising position. In the plan view, if the shortest line segment intersects the lateral side of the cargo bed, and in the plan view, if the outer circle intersects the lateral side of the cargo bed in the case of a back bias, the outer circle intersects the lateral side of the cargo bed near the starting position in the plan view, and the position on the line bisector of the angle formed by the line segment connecting the loading position and the outer circle position and the shortest line segment, which is separated from the outer circumference of the cargo bed by a lateral margin, is calculated as the rising position. A Bézier curve passing through the starting position, the rising position, the outer perimeter position, and the target position is generated as the recommended movement path.

[0013] Preferably, The aforementioned path generation unit, The start control point is located directly above the aforementioned start position, at a safe height higher than the aforementioned start position. The point directly below the aforementioned rising position, where the suspended load maintains a safety margin from the ground surface, is called the rising tangential control point. The point directly above the aforementioned raised position, at the same height as the aforementioned outer perimeter position, is defined as the raised control point. On the extension of the straight line from the point obtained by projecting the target position onto the horizontal plane at the height of the outer perimeter position to the outer perimeter position, a point at a predetermined distance from the outer perimeter position is designated as the outer perimeter tangent control point. On the horizontal plane at the height of the aforementioned outer perimeter position, a point at a predetermined distance from the point obtained by projecting the aforementioned target position from the aforementioned outer perimeter position is defined as the outer perimeter control point, and A point directly above the aforementioned target position, at a predetermined height, is designated as the target tangent control point. as, A cubic Bézier curve with the starting position, the starting control point, the ascending tangent control point, and the ascending position as control points, A cubic Bézier curve with the ascending position, the ascending control point, the outer peripheral tangent control point, and the outer peripheral position as control points, and A cubic Bézier curve with the outer peripheral position, the outer peripheral control point, the target tangent control point, and the target position as control points, are connected to generate a curve as the recommended movement path.

[0014] Preferably, the suspended load movement path generation device includes a selection unit that receives a selection of side bias from the crane operator in the case of the back bias, when the side bias is selected, the path generation unit in the plan view, the foot of the perpendicular dropped to the lateral side close to the starting position from the loading position is taken as the outer peripheral position, and on the bisector of the angle formed by the line segment connecting the loading position and the outer peripheral position and the shortest line segment, a position separated from the outer periphery of the load platform by a lateral margin is calculated as the ascending position, and a Bézier curve passing through the starting position, the ascending position, the outer peripheral position, and the target position is generated as the recommended movement path.

[0015] Preferably, the suspended load movement path generation device includes a ground object recognition unit that recognizes ground objects from the bird's-eye view image and the distance, when there is an obstacle interfering with the generated recommended movement path, the path generation unit Projected onto a horizontal plane in the plan view, if a square having a vertex on the straight line passing through the center of the circumcircle of the obstacle and the starting position does not overlap with the circumcircle of the footprint of the suspended load when it is circumscribed by an avoidance circle obtained by expanding the radius of the circumcircle of the obstacle by a safety margin, an avoidance path for moving the suspended load parallel to the side of the square is generated, if the square overlaps with the circumcircle of the footprint of the suspended load, an avoidance path for moving the suspended load by an avoidance height obtained by adding a safety height to the height of the obstacle is generated, The recommended movement path is generated using the position reached via the aforementioned avoidance path as the new starting position.

[0016] A truck-mounted crane according to a second aspect of the present invention is: Vehicles with a cargo bed, The crane mounted on the aforementioned vehicle, A device for generating a suspended load movement path relating to the first aspect, It is equipped with.

[0017] A program relating to the third aspect of the present invention is: Computers, An image acquisition unit that acquires an overhead image taken from the tip of the boom of a truck-mounted crane. An input unit that receives input specifying the suspended load to be loaded in the aforementioned overhead view image, A depth acquisition unit that acquires the distance from the boom tip to the suspended load, A target recognition unit that recognizes the position and shape of the suspended load from the overhead image and the distance. A loading position acquisition unit for acquiring the loading position on the loading platform of the aforementioned loading-type truck crane for loading the suspended load, and A path generation unit generates a recommended movement path for the hook to which the aforementioned load is attached, passing from the starting position where the load is attached, through an elevated position where the load is separated from the loading platform by a lateral margin and safely separated from the ground surface, and an outer perimeter position where the load is separated from the outer perimeter of the loading platform by a safe height and passes over the outer perimeter, until the load reaches a target position which is a predetermined height from the loading platform directly above the loading position. To make it function as such. [Effects of the Invention]

[0018] According to the present invention, the load to be loaded onto the truck crane is specified, and the loading position on the truck bed is obtained, and the load movement path is generated, thereby reducing the cognitive load and fatigue of the crane operator. [Brief explanation of the drawing]

[0019] [Figure 1]Block diagram showing the functional configuration of a suspended load transfer path generation device according to an embodiment of the present invention. [Figure 2] External perspective view of a truck-mounted crane according to an embodiment. [Figure 3] Plan view showing an example of a recommended movement path generated by the suspended load movement path generation device according to the embodiment. [Figure 4] Plan view showing an example of a recommended travel path in the case of back bias according to the embodiment. [Figure 5] Elevation view showing an example of a recommended travel route according to the embodiment. [Figure 6] Plan view showing an example of a recommended travel path in the case of side bias according to the embodiment. [Figure 7] A flowchart illustrating an example of the operation of generating a suspended load movement path according to the embodiment. [Figure 8] Plan view showing an example of a recommended movement path to avoid obstacles. [Figure 9] Block diagram showing an example of the hardware configuration of a suspended load movement path generation device according to an embodiment. [Modes for carrying out the invention]

[0020] This disclosure relates to a load movement path generation device that generates a load movement path for moving a load from an initial position to a desired position on a truck bed. Embodiments of the present invention will be described in detail below with reference to the drawings. In the drawings, the same or equivalent parts are denoted by the same reference numerals.

[0021] Embodiment. Figure 1 is a block diagram showing the functional configuration of a suspended load movement path generation device according to an embodiment of the present invention. The suspended load movement path generation device 1 acquires image data from an imaging device 2 installed at the tip of the crane boom, recognizes local features and the load to be loaded from the image data, and generates a recommended movement path for moving the load to the loading position on the loading platform. Hereinafter, the suspended load movement path generation device 1 may be abbreviated as the path generation device 1. The crane operator or the crane control device can move the suspended load along the recommended movement path generated by the path generation device 1.

[0022] Figure 2 is an external perspective view of a truck-mounted crane according to an embodiment. The route generation device 1 is provided on the truck-mounted crane. The truck-mounted crane has a crane 40 mounted near the cab 31 of a vehicle 30 equipped with a cargo bed 32. The crane 40 includes outriggers 41, an extendable boom 42, a luffing cylinder 43 for luffing the boom 42, and a hook 44 suspended by a wire rope attached to a pulley provided at the tip 45.

[0023] The outriggers 41 extend to the left and right when the crane 40 is lifting a load, and are extended downward and grounded to support the crane. When the vehicle 30 is moving, the outriggers 41 are retracted and stored near the vehicle body. The boom 42 and luffing cylinder 43 are supported so as to be able to swivel around a vertical axis while maintaining their relative positions. The hook 44 can be raised and lowered by winding in and unwinding the wire rope that suspends the hook 44 using a hoisting device (not shown).

[0024] An imaging device 2 is installed at the tip 45 of the boom 42 to capture images of the area vertically downward. The imaging device 2 is supported, for example, by a gimbal and is always positioned to face vertically downward. By raising and extending the boom 42 of the crane 40, an overhead image is obtained by capturing images of the area downward with the imaging device 2 at the tip 45. The path generation device 1 recognizes the terrain and the load to be loaded from the overhead image captured by the imaging device 2. There may be more than one overhead image for recognizing the terrain and the load to be loaded. In order to cover the entire area where the load loading work is performed, the position of the imaging device 2 may be changed to capture multiple overhead images.

[0025] As shown in Figure 1, the route generation device 1 comprises an image acquisition unit 10, a depth acquisition unit 11, a feature recognition unit 12, an input unit 13, a target recognition unit 14, a loading position acquisition unit 15, a route generation unit 16, and a route display unit 17. The route generation device 1 is connected to an operation unit 3 for the worker to input the specifications of the suspended load, the loading position of the suspended load, etc., and a display device 4 that displays an overhead image captured by the imaging device 2 and a recommended movement route.

[0026] The image acquisition unit 10 acquires images captured by the imaging device 2. The imaging device 2 is, for example, a stereo camera. Since the imaging device 2 is installed at the tip 45 of the boom 42 on the crane 40, the image acquisition unit 10 can calculate the imaging position based on the ground contact position of the outrigger 41 from the slewing angle, inclination angle and length of the boom 42, as well as the extension length of the outrigger 41. The image acquisition unit 10 sends the acquired image data and imaging position to the depth acquisition unit 11.

[0027] The depth acquisition unit 11 measures the depth of each point of a feature from the imaging position by using the parallax of each point in the image data, for example, a stereo image. The imaging device 2 is not limited to a stereo camera. For example, it may be a LiDAR, a laser scanner, or a monocular camera. In the case of a monocular camera, the actual distance to each point in the image can be measured by the position of the lens when the point is in focus. With a monocular camera, the distance to the subject can be measured with the accuracy of the camera's depth of field and the focusing error. Alternatively, the boom 42 can be rotated to capture images from two viewpoints at different positions, and the depth can be measured from the parallax of corresponding points.

[0028] The object recognition unit 12 recognizes objects from the overhead image and the depth of each point in the image. The object recognition unit 12 calculates the coordinates of each point relative to the truck-mounted crane from the image capture location and the direction and depth of each point in the image, and generates a 3D map of the work site. The object recognition unit 12 converts the three-dimensional information represented in the camera coordinate system into three-dimensional information represented in the reference coordinate system of the truck-mounted crane using the image capture location.

[0029] The feature recognition unit 12 recognizes features in the following manner, for example. First, the feature recognition unit 12 acquires point cloud data for one frame. The point cloud data consists of the coordinates of each point in the region including the suspended load L, the feature C, and the ground surface F, starting from above the suspended load L and the feature C, which are the objects to be measured.

[0030] The feature recognition unit 12 divides the area of ​​the overhead image into multiple small regions S in a grid pattern. In each small region S, the feature recognition unit 12 extracts the point data p with the largest depth (distance from the imaging position) h. The point data p with the maximum depth hmax is presumed to be at the lowest position in that small region S. The feature recognition unit 12 then calculates the amount of deviation D of the depth h of the other point data relative to the point data p with the maximum depth hmax. Using the maximum depth hmax as a reference, the feature recognition unit 12 extracts point data p whose depth deviation D is within a predetermined threshold r1, for example, a deviation D of 7 cm or less, as point data that constitutes the ground surface F.

[0031] Next, the feature recognition unit 12 estimates the reference elevation HS of the ground surface F in each sub-region S based on the depth h of the extracted point data p. In this embodiment, the feature recognition unit 12 uses the average value of the depth h of the extracted point data p as the reference elevation HS of the ground surface F in the sub-region S. With this configuration, the feature recognition unit 12 can estimate the reference elevation HS of the ground surface F in any sub-region S.

[0032] The feature recognition unit 12 estimates the reference height H0 of the ground surface F for the entire region based on the reference height HS of the ground surface F in each sub-region S. In this embodiment, the feature recognition unit 12 uses the average value of the reference height HS (average value of depth h) of the ground surface F in each sub-region S as the reference height H0 of the ground surface F for the entire region.

[0033] With this configuration, the feature recognition unit 12 can estimate the reference height H0 of the ground surface F for the entire area of ​​the overhead image. Then, the feature recognition unit 12 calculates the altitude H of the point data p from the depth h and the reference height H0. The altitude H is the height of the point data p from the reference height H0.

[0034] The feature recognition unit 12 may determine that point data p that does not constitute the ground surface F has been extracted when the difference between the reference height H0 of the ground surface F in the entire region and the reference height HS of the ground surface F in one sub-region S is greater than a predetermined threshold, and may correct the reference height H0 of the ground surface F in the entire region by using the reference height HS of the ground surface F in one sub-region S, which is an adjacent sub-region S, and the difference is less than a predetermined threshold, instead of the reference height HS of the ground surface F in one sub-region S.

[0035] In the feature recognition unit 12, if it is estimated that point data p that do not constitute the ground surface F have been extracted with this configuration, the reference height H0 of the ground surface F of the entire region can be estimated more accurately by using the reference height H0 of the ground surface F of a sub-region S adjacent to the first sub-region S whose difference is less than a predetermined threshold, instead of the reference height H0 of the first sub-region S.

[0036] When estimating the reference height H0 of the ground surface F across the entire region, it is possible to exclude sub-regions S from all sub-regions S that are estimated to contain point data p that do not constitute the ground surface F. For example, the average value may be calculated using only the reference height H0 of the sub-regions S that are within a predetermined threshold, with the lowest reference height H0 of the ground surface F calculated in each sub-region S being used as the reference height H0. As described above, when estimating the reference height H0 of the ground surface F across the entire region, it is not necessary to use the reference height H0 of all sub-regions S, and it is possible to use only the reference height H0 of a specific sub-region S.

[0037] With this configuration, the feature recognition unit 12 can exclude small regions S from which it is estimated that point data constituting the ground surface F has not been extracted. Therefore, it is possible to accurately estimate the reference elevation H0 for the entire region.

[0038] The feature recognition unit 12 may estimate the ground surface F based on a specific location in the overhead view image. In the path generation device 1, the reference ground surface F can be determined by specifying the location of the ground surface in the overhead view image displayed on the display device 4 using the operation unit 3. The feature recognition unit 12 may also be configured to automatically determine and specify a specific location in the overhead view image, for example, a location around the outrigger 41 or around the loading platform 32.

[0039] In manual operation, the operator uses the control unit 3 to specify a location that is clearly the ground surface within the overhead image displayed on the display device 4. The feature recognition unit 12 then generates a reference circle with a predetermined radius centered on the specified location (point). The feature recognition unit 12 then selects multiple point data p included in the reference circle.

[0040] The feature recognition unit 12 first extracts the point data p with the largest depth h (the maximum depth hmax) from a selection of multiple point data p. Then, the feature recognition unit 12 calculates the difference D in depth h of the other point data relative to the point data p with the maximum depth hmax. Using the maximum depth hmax as a reference, the feature recognition unit 12 extracts point data p whose difference D in depth h is within a predetermined threshold r1, for example, a difference D of 7 cm or less, as point data constituting the ground surface F. The feature recognition unit 12 estimates the reference height H0 of the ground surface F based on the depth h of the extracted point data p. The feature recognition unit 12 adopts the average value of the depth h of the extracted point data p as the reference height H0 of the ground surface F.

[0041] In the data processing by the feature recognition unit 12, the next step is to perform plane estimation. The feature recognition unit 12 estimates the upper surfaces of the suspended load L and feature C, which are the objects to be measured and are present throughout the entire area, using the upper surface estimation method shown below.

[0042] The feature recognition unit 12 divides the point cloud data P acquired from the overhead image into layers with a predetermined thickness d in the height direction, and distributes the point cloud data P into multiple layers. At this time, the feature recognition unit 12 assigns a unique layer ID to each divided layer and associates each point data p with the layer ID.

[0043] The feature recognition unit 12 estimates a plane in each layer using multiple point data p contained in that layer. Here, "plane" refers to the plane that exists upward in the suspended load L and feature C, that is, the upper surface of the suspended load L and feature C.

[0044] The feature recognition unit 12 first selects two point data, pi and pj, from multiple point data p1, p2, ... included in the same layer. The feature recognition unit 12 then calculates the distance L1 between the two selected point data, pi and pj.

[0045] Next, the feature recognition unit 12 considers two points, pi and pj, to be on the same plane if the distance L1 is less than or equal to a predetermined threshold r2. The threshold r2 is less than or equal to twice the resolution of the overhead image of the point cloud data P. In other words, it determines that two points, pi and pj, are on the same plane if they are at a distance that allows them to be considered adjacent in terms of resolution. If the distance L1 is greater than the threshold r2, it selects two new points and calculates the distance L1.

[0046] The feature recognition unit 12 searches for points within the same layer whose distance from either point pi or pj, which it has determined to be on the same plane, is less than or equal to a threshold r2. If the feature recognition unit 12 finds a neighboring point data pk, it considers that neighboring point data pk to be on the same plane as the two previously selected point data pi and pj. The feature recognition unit 12 searches for points within the same layer whose distance from any point in the set of points {pi} that it has determined to be on the same plane is less than or equal to a threshold r2, and successively adds neighboring points as points on the same plane.

[0047] If, among the points in the same layer, there are no points whose distance from any point in the set of points determined to be on the same plane is less than or equal to the threshold r2, the feature recognition unit 12 searches for two points that were not included in the set of points determined to be on the same plane and whose distance is less than or equal to the threshold r2, and extracts a set of points that are on different planes based on these two points. By repeating this operation, a group of clusters, which are sets of points that can be considered to be on different planes, are extracted from the points in the same layer. The feature recognition unit 12 considers one cluster to be one plane.

[0048] The feature recognition unit 12 divides the point cloud data P into point data p that are considered to be on the same plane and sets up planar clusters. The upper surfaces of the suspended load L and feature C can be defined by each point data p belonging to the planar cluster. Multiple planar clusters may exist in a layer that has been assigned the same layer ID. For each cluster, the feature recognition unit 12 calculates the centroid G of the set of points included in the cluster.

[0049] While one cluster can be estimated as the top surface of a feature, the top surface of a feature is not necessarily a single cluster. For example, an object with a spherical top surface is composed of multiple annular clusters, each belonging to a different layer. A group of clusters whose projected centroids on a plane approximately coincide may be considered the top surface of a single feature. Similarly, a cylindrical object placed with its central axis horizontal is composed of parallel band-shaped clusters in each layer. In this case, a group of clusters belonging to different layers whose projected centers on a plane approximately coincide may be considered the top surface of a single feature.

[0050] Next, the feature recognition unit 12 combines the estimated planar clusters (upper surface). The feature recognition unit 12 selects two planar clusters with different layer IDs from the estimated planar clusters and calculates the difference in elevation H between the two planar clusters, dH. The elevation H of a planar cluster is the average height of each point data p belonging to the planar cluster from the reference elevation H0.

[0051] The feature recognition unit 12 searches for combinations of planar clusters in which the difference dH is within the threshold r3. When the feature recognition unit 12 detects a combination of planar clusters in which the difference dH of height H is within the threshold r3, it detects the horizontal overlap dW for those planar clusters. Here, "overlap" refers to the degree of overlap and separation in the horizontal direction of the plane defined by the planar clusters, and "overlap" is detected when an overlap amount dW1 is detected in either of the two orthogonal directions of the horizontal plane (dW1>0), or when the separation amount dW2 is less than or equal to a predetermined threshold r4 (0≦dW2≦r4).

[0052] When the feature recognition unit 12 detects "overlap," it considers the point data p belonging to those planar clusters to be on the same plane, merges the two planar clusters, and updates them as a new planar cluster. At this time, it also calculates a new altitude H from each point data p belonging to the new planar cluster.

[0053] The feature recognition unit 12 repeats the above process until there are no more combinations of planar clusters that satisfy the conditions, and estimates the planes that exist across multiple layers. The feature recognition unit 12 outputs the planes (i.e., planar clusters) that have been combined by the above combination process. The planes defined by the planar clusters are the planes that exist facing upwards on the suspended load L and the feature C, that is, the upper surfaces of the suspended load L and the feature C.

[0054] The plane estimation method described above allows for the estimation of a plane without using the normal vector of the point cloud data P. Therefore, it has the advantage of requiring less computation compared to the method of estimating a plane using the normal vector of the point cloud data P. With this plane estimation method, by estimating the top surface of the suspended load L or feature C, the three-dimensional shape of the suspended load L or feature C can be understood without obtaining point data p for the sides of the suspended load L or feature C.

[0055] The feature recognition unit 12 then classifies the point cloud data P into clusters of the same region. Clustering of the same region is a process that clusters the generated planar clusters from a different perspective: whether or not they exist in the same region.

[0056] The feature recognition unit 12 extracts planar clusters containing point data p where the altitude H is at its maximum value Hh, and planar clusters that are not connected to this planar cluster. If the difference ΔH of the altitude H between the two extracted planar clusters is less than or equal to a predetermined threshold, the feature recognition unit 12 checks for overlap in the height direction between the two planar clusters.

[0057] If two planar clusters overlap in the height direction, the feature recognition unit 12 considers these planar clusters to be in the "same region" and forms a same region cluster with these planar clusters.

[0058] The feature recognition unit 12 further searches for planar clusters containing point data p having a maximum value Hh of altitude H, and planar clusters that are not connected to this planar cluster. If an unconnected planar cluster is extracted, it performs a determination using the difference ΔH and checks for overlap in the height direction. If there is a planar cluster that matches the above conditions, it is added to the above same-region cluster.

[0059] The feature recognition unit 12 repeats this process for planar clusters containing point data p with maximum height Hh until no unconnected planar clusters are found. Through this process, the feature recognition unit 12 forms identical area clusters.

[0060] The feature recognition unit 12 treats the point data p belonging to the same region cluster formed in this way as a single unit in terms of shape, and sets a guide frame to surround the same region cluster.

[0061] Such clustering of identical regions is preferably performed using hierarchical clustering with a height-based tree structure. The feature recognition unit 12 creates a tree structure for each feature C using the height H through clustering of identical regions.

[0062] In hierarchical clustering using a height-based tree structure, the feature recognition unit 12 sets the planar cluster with the smallest average height H as the root. If there are planar clusters that overlap with the planar clusters constituting the root when viewed in the height direction, the feature recognition unit 12 extends a branch from the root and adds the overlapping planar cluster to the end of the branch. The feature recognition unit 12 sets the planar cluster with the largest average height H as a child.

[0063] The feature recognition unit 12 also recognizes the cargo loaded in the area of ​​the loading platform 32. The area of ​​the loading platform 32 in the reference coordinate system is stored in advance. Since the extent of the loading platform 32 is known, the area of ​​the loading platform 32 can be defined in the overhead view image. Since the position of the floor of the loading platform 32 is known, the floor of the loading platform can be extracted from the area of ​​the loading platform 32 in the overhead view image. The feature recognition unit 12 can recognize the tree structure located in the loading platform area from the generated tree structure as the loaded cargo. The area excluding the portion containing the loaded cargo is the empty area.

[0064] The feature recognition unit 12 generates an outline as a guide frame that encloses all planar clusters included in a hierarchical cluster of a single tree structure. Once the feature recognition unit 12 has classified all planar clusters in the entire region into one of the tree structures, it sends those tree structures to the object recognition unit 14 and the path generation unit 16.

[0065] The input unit 13 receives input specifying the suspended load to be loaded in the overhead view image. The input unit 13 displays the overhead view image on the display device 4 and sends the position of the image input from the operation unit 3 to the object recognition unit 14. The input unit 13 may also recognize a gesture in which the operator indicates the suspended load in the overhead view image and send the position in the overhead view image pointed to by the operator to the object recognition unit 14. Alternatively, when the operator operates the crane 40 and stops the hook 44 directly above a certain object C, and inputs the suspension load specification to the operation unit 3, the object recognition unit 14 may recognize the object C directly below the hook 44 as the suspended load L to be loaded.

[0066] The object recognition unit 14 identifies the suspended load L to be loaded from the tree structure corresponding to the specified location. That is, the feature C represented by a planar cluster corresponding to the location specified in the overhead image is identified as the suspended load L to be loaded. The object recognition unit 14 may also identify the three-dimensional information (point cloud data) of the suspended load from the three-dimensional information (point cloud data) by object recognition. The object recognition method for the suspended load may be appearance-based object recognition or model-based object recognition.

[0067] The loading position acquisition unit 15 receives input specifying the loading position of the suspended load L within the empty area of ​​the cargo bed 32 in the overhead view image. The loading position acquisition unit 15 adjusts the loading position from the footprint of the suspended load L, that is, the shape obtained by projecting the outermost shape of the planar cluster constituting the suspended load L onto the horizontal plane. The loading position acquisition unit 15 overlays the footprint of the suspended load L at the specified position in the empty area of ​​the cargo bed 32 in the overhead view image, rotates and translates the footprint, and sets the loading position PL of the suspended load L at the position where the footprint fits within the empty area. The loading position acquisition unit 15 represents the loading position PL of the suspended load L at the position of the center of gravity of the footprint of the suspended load L. The loading position acquisition unit 15 sends the acquired loading position PL to the path generation unit 16.

[0068] The input unit 13 receives input that the rigging of the suspended load L is complete. When notified that the rigging of the suspended load L is complete, the object recognition unit 14 obtains the position of the hook 44 attached to the suspended load L from the overhead image, or from the slewing angle, inclination angle and length of the boom 42, and the amount of wire rope extended, and sends it to the path generation unit 16.

[0069] The route generation unit 16 sets the starting position P0 as the position of the hook 44 with the load L attached, and the target position P3 as the position of the hook 44 directly above the loading position PL, where the load L has not yet touched the loading platform 32 and is at a predetermined height. The route generation unit 16 generates a recommended movement path for the hook 44, starting from the starting position P0, passing through an upward position P1 where the load L is separated from the loading platform 32 by a lateral margin and safely separated from the ground surface, and then through an outer perimeter position P2 where the load L passes over the outer perimeter of the loading platform 32 at a safe height away from the outer perimeter, to reach the target position P3. The predetermined height from the loading position PL to the target position P3, the lateral margin, the safety margin, and the safety height are set in advance in the route generation device 1.

[0070] The path generation unit 16 generates recommended movement paths by considering different cases based on the positions of the starting position P0 and the target position P3 relative to the cargo bed 32 in a plan view projected onto a horizontal plane. The recommended movement path is represented by a line connecting the starting position P0, the rising position P1, the outer perimeter position P2, and the target position P3 in that order. The recommended movement path is a Bézier curve passing through each point. That is, a first Bézier curve from the starting position P0 to the rising position P1 in a vertical plane passing through the starting position P0 and the rising position P1, a second Bézier curve from the rising position P1 to the outer perimeter position P2, and a third Bézier curve from the outer perimeter position P2 to the target position P3 in a vertical plane passing through the outer perimeter position P2 and the target position P3. The first and second Bézier curves are smoothly connected at the rising position P1, and the second and third Bézier curves are smoothly connected at the outer perimeter position P2.

[0071] Figure 3 is a plan view showing an example of a recommended movement path generated by the load movement path generation device according to the embodiment. Figure 3 shows the case where the positions of the starting position P0 and the target position P3 relative to the loading platform 32 are rear-biased. Rear-biased refers to the case where, in the plan view, the shortest line segment connecting the starting position P0 and the loading position PL (target position P3) intersects the rear edge 33 of the loading platform 32. In the case of rear-biased, the control point of the Bézier curve lies in the vertical plane containing the shortest line segment, so the recommended movement path is represented by the shortest line segment connecting the starting position P0 and the target position P3 in the plan view.

[0072] In the case of rear bias, the lifting position P1 is the position in the plan view where the circumscribed circle CL of the suspended load, which is the circumscribed circle of the footprint of the suspended load L, is located at a lateral clearance Lm from the rear edge 33 of the loading platform 32. In the vertical direction, the lifting position P1 is the position where the suspended load L is safely distanced from the ground surface. The outer perimeter position P2 is the intersection point of the shortest line segment and the rear edge 33 in the plan view. In the vertical direction, the outer perimeter position P2 is the position where the suspended load L is safely distanced from the outer perimeter of the loading platform 32. In the case of rear bias, the first Bézier curve B1, the second Bézier curve B2, and the third Bézier curve B3 coincide with the shortest line segment in the plan view.

[0073] Figure 4 is a plan view showing an example of a recommended movement path in the case of back bias according to the embodiment. Back bias is a case in the plan view where the shortest line segment Ss intersects the lateral side 34 of the loading platform 32, and in the plan view where the outer circle Cs, with the pivot center 46 of the boom 42 as its center and a radius equal to the distance between the pivot center 46 and the loading position PL plus the diameter of the circumscribed circle CL of the suspended load, intersects the lateral side 34 of the loading platform 32. In Figure 4, a circle with the pivot center 46 as its center and a radius equal to the distance between the pivot center 46 and the loading position PL is drawn as the loading circle Cp.

[0074] In the case of back bias, the path generation unit 16 sets the outer perimeter position P2 at the intersection of the outer perimeter circle Cs and the lateral side 34 in the plan view. The outer perimeter position P2 is a position where the suspended load L is safely away from the outer perimeter of the loading platform 32 in the vertical direction. The path generation unit 16 calculates the rising position P1 as the position where the circumscribed circle CL is a lateral margin Lm away from the outer perimeter of the loading platform 32 on the angle bisector Bi of the angle formed by the line segment connecting the loading position PL and the outer perimeter position P2 and the shortest line segment Ss in the plan view. The rising position P1 is a position where the suspended load L is safely away from the ground surface in the vertical direction.

[0075] Even in the case of back bias, the recommended movement path is a Bézier curve passing through each point. Specifically, it is a first Bézier curve B1 from the starting position P0 to the rising position P1 in a vertical plane passing through the starting position P0 and the rising position P1, a second Bézier curve B2 from the rising position P1 to the outer perimeter position P2, and a third Bézier curve B3 from the outer perimeter position P2 to the target position P3 in a vertical plane passing through the outer perimeter position P2 and the target position P3. The first Bézier curve B1 and the second Bézier curve B2 are smoothly connected at the rising position P1, and the second Bézier curve B2 and the third Bézier curve B3 are smoothly connected at the outer perimeter position P2.

[0076] In the case of back bias, in order to smoothly connect the second Bézier curve B2 and the third Bézier curve B3, the control point on the outer perimeter position P2 side of the second Bézier curve B2 (outer perimeter tangent control point tP2) is on the extension of the line segment from the target position P3 to the outer perimeter position P2 in the plan view. Therefore, the second Bézier curve B2 is generally a curve in the plan view.

[0077] Even if the shortest line segment Ss intersects the lateral side 34 in the plan view, if the outer circle Cs does not intersect the lateral side 34, the path generation unit 16 generates a recommended movement path represented by a line segment that overlaps the shortest line segment Ss in the plan view, just as in the case of rear bias. In that case, the rising position P1 is the position where the circumscribing circle CL of the suspended load is at a lateral clearance Lm away from the lateral side 34 of the loading platform 32 in the plan view, and the outer position P2 is the intersection point of the shortest line segment Ss and the lateral side 34 in the plan view.

[0078] Figure 5 is an elevation view showing an example of a recommended movement path according to the embodiment. Figure 5 corresponds to a view of the recommended movement path in Figure 4 from the rear of the vehicle 30. In Figure 5, the positions of the suspended load L and the hook 44 are shown, but the lifting equipment is omitted. The recommended movement path of the hook 44 is represented by a curve connecting the first Bézier curve B1, the second Bézier curve B2, and the third Bézier curve B3.

[0079] The first Bézier curve is a cubic Bézier curve with control points at the starting position P0, the starting control point cP0, the ascending tangent control point tP1, and the ascending position P1. The starting control point cP0 is directly above the starting position P0 and at a safe height higher than the starting position P0. The ascending tangent control point tP1 is directly below the ascending position P1 and at a height where the suspended load L maintains a safe margin from the ground surface.

[0080] The second Bézier curve B2 is a cubic Bézier curve with control points at the ascending position P1, the ascending control point cP1, the outer perimeter tangent control point tP2, and the outer perimeter position P2. The ascending control point cP1 is a point directly above the ascending position P1 and at the same height as the outer perimeter position P2. The outer perimeter tangent control point tP2 is a point at a predetermined distance from the outer perimeter position P2 on the extension of the straight line from the point obtained by projecting the target position P3 onto a horizontal plane at the height of the outer perimeter position P2 to the outer perimeter position P2. Since the ascending tangent control point tP1 of the first Bézier curve B1 is directly below the ascending position P1, and the ascending control point cP1 of the second Bézier curve B2 is directly above the ascending position P1, the ascending tangent control point tP1, the ascending position P1, and the ascending control point cP1 lie on a single vertical line, and therefore the first Bézier curve B1 and the second Bézier curve B2 are smoothly connected at the ascending position P1.

[0081] The third Bézier curve B3 is a cubic Bézier curve with control points at the outer perimeter position P2, the outer perimeter control point cP2, the target tangent control point tP3, and the target position P3. The outer perimeter control point cP2 is a point at a predetermined distance on the line segment extending from the outer perimeter position P2 to the point obtained by projecting the target position P3 onto the horizontal plane at the height of the outer perimeter position P2. The target tangent control point tP3 is a point directly above the target position P3, at a predetermined height. The outer perimeter position P2, the outer perimeter control point cP2, the target tangent control point tP3, and the target position P3 are all located on a single vertical plane. The outer perimeter tangent control point tP2 of the second Bézier curve B2 and the outer perimeter control point cP2 of the third Bézier curve B3 are located on a straight line passing through the point obtained by projecting the outer perimeter position P2 and the target position P3 onto a horizontal plane passing through the outer perimeter position P2. Therefore, the outer perimeter tangent control point tP2, the outer perimeter position P2, and the outer perimeter control point cP3 lie on a single straight line, and thus the second Bézier curve B2 and the third Bézier curve B3 are smoothly connected at the outer perimeter position P2.

[0082] Even in the case of rear bias as shown in Figure 3, the control points for the first Bézier curve B1, the second Bézier curve B2, and the third Bézier curve B3 are set in the same way as in back bias. The recommended movement path for rear bias viewed horizontally is represented by a curve similar to that in Figure 5. In the case of rear bias, since the control points for the first Bézier curve B1, the second Bézier curve B2, and the third Bézier curve B3 are located on a single vertical plane, the recommended movement path in the plan view becomes a single line segment.

[0083] Figure 6 is a plan view showing an example of a recommended movement path in the case of side bias according to the embodiment. Side bias is used when there is not enough space to the side of the loading platform 32. Side bias is selected by the crane operator when the starting position P0 and target position P3 relative to the loading platform 32 are under the same conditions as in back bias.

[0084] When side bias is selected, the path generation unit 16 sets the foot of the perpendicular line drawn from the loading position PL to the lateral side 34 closest to the starting position P0 in the plan view as the outer perimeter position P2. Then, in the plan view, it calculates the rising position P1 as the position where the circumscribed circle CL of the suspended load is separated from the outer perimeter of the loading platform 32 by a lateral margin Lm on the angle bisector Bi of the angle formed by the line segment connecting the loading position PL and the outer perimeter position P2 and the shortest line segment Ss. In the case of side bias, the heights of the starting position P0, rising position P1, outer perimeter position P2, and target position P3 are the same as in rear bias and back bias. The path generation unit 16 generates a Bézier curve passing through the starting position P0, rising position P1, outer perimeter position P2, and target position P3 as the recommended movement path.

[0085] Even in the side bias configuration, the control points for the first Bézier curve B1, the second Bézier curve B2, and the third Bézier curve B3 are set in the same way as in the back bias configuration. Therefore, if the starting position P0, the loading position PL, and the suspended load L are the same as those for the cargo bed 32, the recommended movement path for the side bias configuration is the same as the recommended movement path for the back bias configuration in the elevation view of the vehicle 30 from the rear.

[0086] The recommended side bias route is designed for use in space-constrained environments, such as roadside or indoor operations. A side bias is suitable, for example, when vehicle 30 is parked on the side of a road with another vehicle parked behind it. Alternatively, it is suitable, for example, when vehicle 30 is indoors and an obstacle is preventing access to the rear of vehicle 30. In general, a side bias can be chosen to avoid an object present behind vehicle 30.

[0087] Figure 7 is a flowchart showing an example of the operation of generating a suspended load movement path according to the embodiment. The process of generating a suspended load movement path is started by the crane operator. When the suspended load movement path generation device 1 is started, the image acquisition unit 10 acquires an overhead image taken from the tip 45 of the boom 42 (step S10). The depth acquisition unit 11 acquires the depth of each point of the feature from the imaging position (step S11). The feature recognition unit 12 recognizes the feature and the load loaded in the area of ​​the loading platform 32 from the overhead image and the depth of each point in the image (step S12).

[0088] The input unit 13 receives input specifying the suspended load L to be loaded in the overhead view image (step S13). Once the suspended load L is specified, the object recognition unit 14 recognizes the position and shape of the suspended load L (step S14). The loading position acquisition unit 15 receives input specifying the position in which the suspended load L will be loaded within the range of the empty area of ​​the loading platform 32 in the overhead view image (step S15).

[0089] The route generation device 1 waits for the rigging of the suspended load L to be completed (step S16; N), and when the completion of rigging is input (step S16; Y), it acquires the position of the hook 44 in the completed rigging state as the starting position P0 (step S17). The route generation device 1 determines whether or not it is a back bias based on the relationship between the loading position PL on the loading platform 32 and the starting position P0 (step S18).

[0090] If the loading position PL and starting position P0 are in the back bias position (step S18; Y), the path generation device 1 accepts a side bias selection input (step S19). If the side bias is not selected (step S20; N), the path generation unit 16 generates a back bias movement path as shown in Figure 4 (step S21). If the side bias is selected (step S20; Y), the path generation unit 16 generates a side bias movement path as shown in Figure 6 (step S22).

[0091] On the other hand, if the loading position PL and starting position P0 are not back-biased (step S18;N), the route generation unit 16 generates a movement path as rear-biased as shown in Figure 3 (step S23). In all cases of back-biased (step S21), side-biased (step S22), and rear-biased (step S23), the route display unit 17 displays the generated movement path as the recommended movement path on the display device 4 (step S24), and the route generation device 1 terminates the process of generating the suspended load movement path.

[0092] Up until now, it was assumed that there would be no obstacles in the movement path, but in actual work sites, obstacles are often located on the ground. If there are obstacles in the generated recommended movement path, the path generation unit 16 generates a recommended movement path as described below.

[0093] Figure 8 is a plan view showing an example of a recommended movement path that avoids an obstacle. Figure 8 shows the case where the recommended movement path generated by the back vise has an obstacle Ob. If the recommended movement path interferes with the obstacle Ob, the path generation unit 16 virtually draws a circumscribed square Sc in the plan view, which is circumscribed around a avoidance circle Ca obtained by expanding the radius of the circumscribed circle Co of the obstacle Ob by a safety margin, and whose vertex lies on the line Sb passing through the center of the circumscribed circle Co of the obstacle Ob and the starting position P0.

[0094] If the circumscribing square Sc does not overlap with the circumscribing circle CL of the suspended load, the path generation unit 16 generates an avoidance path that moves the suspended load L parallel to the sides of the circumscribing square Sc. Figure 8 shows the case where the circumscribing square Sc does not overlap with the circumscribing circle CL of the suspended load. The path generation unit 16 generates an avoidance path that moves the suspended load L to a new starting position nP0 from the loading position PL, avoiding the circumscribing square Sc and allowing a clear view of the suspended load L. There are two mutually orthogonal directions parallel to the sides of the circumscribing square Sc, but the path generation unit 16 selects the one in which the position after moving the suspended load L is closer to the pivot center 46 of the boom 42. The height of the new starting position nP0 is the same as the height of the rising position P1 when there are no obstacles.

[0095] If the circumscribing square Sc overlaps with the circumscribing circle CL of the suspended load, the path generation unit 16 generates an avoidance path that moves the suspended load L by an avoidance height equal to the height of the obstacle Ob plus a safety height. In this case, the avoidance path is one that raises the suspended load L straight up by the avoidance height, and the new starting position nP0 is a position that is the avoidance height higher than the original starting position P0.

[0096] The path generation unit 16 generates recommended movement paths for rear bias, back bias, or side bias, setting the new starting position nP0 as the starting position P0, regardless of whether the circumscribing square Sc overlaps with the circumscribing circle CL of the suspended load or not.

[0097] After this, the crane operator can move the hook 44, which is rigged with the suspended load L, from the starting position P0 to the target position P3 by operating the crane 40 and moving the hook 44 along the recommended movement path displayed on the display device 4. Alternatively, if the control device of the crane 40 can control the crane 40 to move the hook 44 along a given movement path, the recommended movement path information can be sent to the control device to move the hook 44, which is rigged with the suspended load L, from the starting position P0 to the target position P3.

[0098] As described above, with the load path generation device 1 according to the embodiment, if the crane operator specifies the loading position PL and the load L, a movement path is generated that moves the hook 44 to which the load L is attached from the starting position P0 to the target position P3 directly above the loading position PL, thereby reducing the cognitive load and fatigue of the operator.

[0099] Figure 9 is a block diagram showing an example of the hardware configuration of a suspended load movement path generation device according to an embodiment. As shown in Figure 9, the path generation device 1 includes a control unit 51, a main memory unit 52, an external memory unit 53, an operation unit 54, a display unit 55, an input / output unit 56, and a transmitting / receiving unit 57. The main memory unit 52, external memory unit 53, operation unit 54, display unit 55, input / output unit 56, and transmitting / receiving unit 57 are all connected to the control unit 51 via an internal bus 50.

[0100] The control unit 51 consists of a CPU (Central Processing Unit) and the like, and executes the processes of the image acquisition unit 10, depth acquisition unit 11, feature recognition unit 12, input unit 13, target recognition unit 14, loading position acquisition unit 15, route generation unit 16, and route display unit 17 of the route generation device 1 according to the control program 58 stored in the external storage unit 53.

[0101] The main memory unit 52 consists of RAM (Random-Access Memory) and the like, loads the control program 58 stored in the external memory unit 53, and is used as a work area for the control unit 51.

[0102] The external storage unit 53 consists of non-volatile memory such as flash memory, hard disk, DVD-RAM (Digital Versatile Disc Random-Access Memory), and DVD-RW (Digital Versatile Disc ReWritable). It stores a program for causing the control unit 51 to process the route generation device 1, as well as data such as the position of the loading platform 32 relative to the pivot center of the boom 42. It also supplies the data stored in this program to the control unit 51 according to the instructions of the control unit 51, and stores data such as the position of the suspended load L, the loading position PL, and the recommended movement route supplied by the control unit 51.

[0103] The operation unit 54 consists of a keyboard and a pointing device such as a mouse, and an interface device that connects the keyboard and pointing device to the internal bus 50. Inputs such as the specification of the suspended load L, completion of rigging, and selection of side bias are received via the operation unit 54 and supplied to the control unit 51. The operation unit 54 includes the operation unit 3 in Figure 1.

[0104] The display unit 55 consists of a display device such as an LCD (Liquid Crystal Display) or an organic EL display and an interface device that connects the display device to the internal bus 50, and displays an overhead view image, a specified suspended load L, a loading position PL, and a recommended movement route. The display unit 55 includes the display device 4 shown in Figure 1.

[0105] The input / output unit 56 consists of a serial interface or a parallel interface. The imaging device 2 and the crane control device are connected to the input / output unit 56, and the control unit 51 acquires an overhead image via the input / output unit 56.

[0106] The transmitting / receiving unit 57 consists of a wireless communication device and a serial interface or LAN (Local Area Network) interface. The transmitting / receiving unit 57 communicates with a mobile terminal that inputs, for example, the selection of the suspended load L, via wireless communication. It also updates various data or downloads control programs.

[0107] The processing of the image acquisition unit 10, depth acquisition unit 11, feature recognition unit 12, input unit 13, target recognition unit 14, loading position acquisition unit 15, route generation unit 16, and route display unit 17 of the route generation device 1 shown in Figure 1 is performed by the control program 58, which uses the control unit 51, main memory unit 52, external memory unit 53, operation unit 54, display unit 55, input / output unit 56, and transmission / reception unit 57 as resources.

[0108] The configuration of the route generation device 1 described in each embodiment is merely an example and can be arbitrarily changed and modified. The configuration of the route generation device 1 is not limited to those shown in the embodiments. For example, a tablet terminal may be used as the route generation device 1. Alternatively, the route generation device 1 may be installed on a network, and its functions may be provided via the network.

[0109] Furthermore, the aforementioned hardware configuration and flowchart are examples only and can be changed and modified as needed.

[0110] The core part of the route generation device 1, which consists of an image acquisition unit 10, a depth acquisition unit 11, a feature recognition unit 12, an input unit 13, a target recognition unit 14, a loading position acquisition unit 15, a route generation unit 16, and a route display unit 17, can be implemented using a standard computer system, not a dedicated system. For example, the route generation device 1 that performs the above processing may be configured by distributing a computer program for executing the above operations on a computer-readable recording medium (USB memory, CD-ROM, DVD-ROM, etc.) and installing the computer program on a computer. Alternatively, the computer program may be stored on a storage device of a server on a communication network such as the Internet, and the route generation device 1 may be configured by downloading it from a standard computer system.

[0111] Furthermore, if the route generation device 1 is implemented through a division of labor between the OS (operating system) and the application program, or through collaboration between the OS and the application program, only the application program portion may be stored on the recording medium or storage device.

[0112] Furthermore, it is possible to superimpose a computer program onto a carrier wave and distribute it via a communication network. For example, the computer program may be posted on a bulletin board system (BBS) on the communication network and distributed via the network. The computer program may then be launched and executed under the control of the OS, similar to other application programs, to perform the aforementioned processing. [Explanation of symbols]

[0113] 1. Suspended load movement path generation device 2. Imaging device 3 Control section 4 Display device 10 Image acquisition unit 11 Depth acquisition section 12 Feature recognition section 13 Input section 14. Object Recognition Unit 15 Loading position acquisition unit 16 Route generation unit 17 Route display section 30 vehicles 31 Cab 32 cargo bed 33 Trailing edge 34 Side edges 40 Cranes 41 Outrigger 42 Boom 43. Relief cylinder 44 hooks 45 Tip 46. ​​Center of rotation

Claims

1. An image acquisition unit that acquires an overhead image taken from the tip of the boom of a truck-mounted crane, An input unit that receives input specifying the suspended load to be loaded in the aforementioned overhead view image, A depth acquisition unit that acquires the distance from the boom tip to the suspended load, A target recognition unit that recognizes the position and shape of the suspended load from the overhead image and the distance, A loading position acquisition unit that acquires the loading position for loading the suspended load onto the loading platform of the aforementioned truck crane, A path generation unit generates a recommended movement path for the hook to which the load is attached, passing from the starting position where the load is attached, through a raised position where the load is separated from the loading platform by a lateral margin and safely separated from the ground surface, and through an outer perimeter position where the load is separated from the outer perimeter of the loading platform at a safe height and passes over the outer perimeter, until the load reaches a target position which is a predetermined height from the loading platform directly above the loading position. A device for generating a suspended load movement path, equipped with the following features.

2. The aforementioned path generation unit, In a plan view projected onto a horizontal plane, if the shortest line segment connecting the starting position and the loading position intersects with the rear edge of the loading platform, or if, in the plan view, the outer circle with the center of rotation of the boom as its center and a radius equal to the distance between the center of rotation and the loading position plus the diameter of the circumscribed circle of the suspended load's footprint does not intersect with the side edge of the loading platform, then the intersection point of the shortest line segment and the outer circumference of the loading platform in the plan view is defined as the outer circumference position, and the position on the shortest line segment in the plan view, separated from the outer circumference of the loading platform by the lateral margin, is calculated as the rising position. In the plan view, if the shortest line segment intersects the lateral side of the cargo bed, and in the plan view, if the outer circle intersects the lateral side of the cargo bed in the case of a back bias, the outer circle intersects the lateral side of the cargo bed near the starting position in the plan view, and the position on the line bisector of the angle formed by the line segment connecting the loading position and the outer circle position and the shortest line segment, which is separated from the outer circumference of the cargo bed by a lateral margin, is calculated as the rising position. A Bézier curve passing through the starting position, the rising position, the outer perimeter position, and the target position is generated as the recommended movement path. A device for generating a suspended load movement path according to claim 1.

3. The aforementioned path generation unit, The start control point is located directly above the aforementioned start position, at a safe height higher than the aforementioned start position. The point directly below the aforementioned rising position, where the suspended load maintains a safety margin from the ground surface, is called the rising tangential control point. The point directly above the aforementioned raised position, at the same height as the aforementioned outer perimeter position, is defined as the raised control point. On the extension of the straight line from the point obtained by projecting the target position onto the horizontal plane at the height of the outer perimeter position to the outer perimeter position, a point at a predetermined distance from the outer perimeter position is designated as the outer perimeter tangent control point. On the horizontal plane at the height of the aforementioned outer perimeter position, a point at a predetermined distance from the point obtained by projecting the aforementioned target position from the aforementioned outer perimeter position is defined as the outer perimeter control point, and A point directly above the aforementioned target position, at a predetermined height, is designated as the target tangent control point. as, A cubic Bézier curve with the aforementioned starting position, the aforementioned starting control point, the aforementioned rising tangent control point, and the aforementioned rising position as control points, A cubic Bézier curve with the aforementioned rising position, the aforementioned rising control point, the aforementioned outer perimeter tangent control point, and the aforementioned outer perimeter position as control points, and A cubic Bézier curve with the outer perimeter position, the outer perimeter control point, the target tangent control point, and the target position as control points, A curve is generated by connecting the points as the recommended travel path. The device for generating a suspended load movement path according to claim 2.

4. In the case of the aforementioned back bias, the crane operator selects a side bias, and the system is equipped with a selection unit that accepts this selection from the crane operator. The path generation unit, when the side bias is selected, In the aforementioned plan view, the foot of the perpendicular line drawn from the loading position to the side edge closest to the starting position is defined as the outer perimeter position, and the rising position is calculated as the position on the line bisector of the angle formed by the line segment connecting the loading position and the outer perimeter position and the shortest line segment, which is separated from the outer perimeter of the cargo bed by a lateral margin. A Bézier curve passing through the starting position, the rising position, the outer perimeter position, and the target position is generated as a recommended movement path. A device for generating a suspended load movement path according to claim 2 or 3.

5. The system includes a feature recognition unit that recognizes features from the aforementioned overhead image and the aforementioned distance, If there are obstacles that interfere with the generated recommended travel path, the path generation unit will, in a plan view projected onto a horizontal plane, If a square circumscribing a avoidance circle with a radius increased by a safety margin from the circumscribed circle of the obstacle, and whose vertices lie on a line passing through the center of the circumscribed circle of the obstacle and the starting position, does not overlap with the circumscribed circle of the footprint of the suspended load, then an avoidance path is generated that moves the suspended load parallel to the sides of the square. If the square overlaps with the circumscribed circle of the footprint of the suspended load, a avoidance path is generated that moves the suspended load by an avoidance height equal to the height of the obstacle plus a safety height. The recommended movement path is generated using the position reached by the aforementioned avoidance path as the new starting position. A device for generating a suspended load movement path according to any one of claims 1 to 4.

6. Vehicles with a cargo bed, The crane mounted on the aforementioned vehicle, A suspended load movement path generation device according to any one of claims 1 to 5, A truck-mounted crane equipped with a loading-type crane.

7. Computers, An image acquisition unit that acquires an overhead image taken from the tip of the boom of a truck-mounted crane. An input unit that receives input specifying the suspended load to be loaded in the aforementioned overhead view image, A depth acquisition unit that acquires the distance from the boom tip to the suspended load, A target recognition unit that recognizes the position and shape of the suspended load from the overhead image and the distance. A loading position acquisition unit for acquiring the loading position on the loading platform of the aforementioned loading-type truck crane for loading the suspended load, and A path generation unit generates a recommended movement path for the hook to which the aforementioned load is attached, passing from the starting position where the load is attached, through an elevated position where the load is separated from the loading platform by a lateral margin and safely separated from the ground surface, and an outer perimeter position where the load is separated from the outer perimeter of the loading platform by a safe height and passes over the outer perimeter, until the load reaches a target position which is a predetermined height from the loading platform directly above the loading position. A program that makes it function as such.

Images