Identification system, identification method, and work vehicle

Abstract

An identification system 1 comprises: a three-dimensional sensor 11 which is attached to a work vehicle and detects objects existing around the work vehicle; and a controller 20. The controller 20 acquires point cloud data that is object detection data from the three-dimensional sensor 11, estimates a plane constituting at least a portion of the outer shape of a target object from the point cloud data, extracts a point cloud located on the estimated plane to generate a binary image, and identifies the target object from the binary image.

Description

Identification system, identification method, and work vehicle

[0001] The present disclosure relates to an identification system, an identification method, and a work vehicle.

[0002] For example, in order to perform accurate loading operations using work vehicles such as dump trucks, wheel loaders, shovels, and forklifts, it is necessary to improve the work vehicle's ability to identify work objects. Patent Document 1 discloses a known technology relating to an automatic excavator that automatically performs a series of processes, including excavating an excavation object and loading the excavated material into a loading object. The technology described in Patent Document 1 calculates the relative position of the loading object relative to the excavator and the relative attitude of the excavator relative to the loading object using images captured by a stereo camera. Patent Document 2 discloses a known technology relating to a loading vehicle that extracts an observation point cloud from three-dimensional distance data, and, if the extracted observation point cloud includes a feature representing the front of the loading object, extracts the feature and identifies the position, direction, and width of the loading object.

[0003] JP 10-088625 A JP 2016-210586 A

[0004] When performing object identification processing using an image, it may be difficult to identify the object depending on the shooting conditions such as brightness when the image was taken. When performing object identification processing using three-dimensional data of an object detected by a three-dimensional sensor, the processing load increases, and it may become difficult to perform calculations in real time.

[0005] An aspect of the present disclosure aims to provide an identification system, an identification method, and a work vehicle that can appropriately identify objects from point cloud data, which is three-dimensional data.

[0006] According to an aspect of the present disclosure, there is provided an identification system for identifying an object from objects present in the vicinity of a work vehicle, the system including an object detection sensor attached to a work vehicle and detecting objects present in the vicinity of the work vehicle, and a controller. The controller acquires point cloud data, which is object detection data, from the object detection sensor, estimates a plane that constitutes at least a part of the outline of the object from the point cloud data, extracts the point cloud located on the estimated plane to generate a binary image, and identifies the object from the binary image.

[0007] According to an aspect of the present disclosure, there is provided an identification method for identifying an object from objects present in the vicinity of a work vehicle, comprising the steps of: acquiring point cloud data, which is object detection data, from an object detection sensor attached to the work vehicle and detecting objects present in the vicinity of the work vehicle; estimating a plane that constitutes at least a part of the outline of the object from the point cloud data; extracting the point cloud located on the estimated plane to generate a binary image; and identifying the object from the binary image.

[0008] According to an aspect of the present disclosure, there is provided a work vehicle including a vehicle body, an object detection sensor attached to the vehicle body for detecting objects present around the work vehicle, and a controller. The controller acquires point cloud data, which is object detection data, from the object detection sensor, estimates a plane that constitutes at least a part of the outline of the object from the point cloud data, extracts the point cloud located on the estimated plane to generate a binary image, and identifies the object from the binary image.

[0009] According to aspects of the present disclosure, an identification system and a work vehicle are provided that can appropriately identify objects from point cloud data, which is three-dimensional data.

[0010] FIG. 1 is a side view showing a work vehicle according to the first embodiment. FIG. 2 is a functional block diagram showing an identification system according to the first embodiment. FIG. 3 is a block diagram showing a computer system according to the first embodiment. FIG. 4 is a schematic diagram showing the operation of the work vehicle according to the first embodiment. FIG. 5 is a diagram showing an example of a point cloud indicating an object. FIG. 6 is a schematic diagram explaining an example of an object identification process. FIG. 7 is a diagram showing an example of a binary image of an object. FIG. 8 is a diagram showing an example of a feature pattern of an object. FIG. 9 is a flowchart showing an example of processing in the object identification method according to the first embodiment. FIG. 10 is a perspective view showing a work vehicle according to a second embodiment. FIG. 11 is a schematic diagram showing the operation of the work vehicle according to the second embodiment. FIG. 12 is a schematic perspective view explaining another example of the object identification process. FIG. 13 is a schematic plan view explaining another example of the object identification process. FIG. 14 is a diagram explaining another example of the object identification process. FIG. 15 is a diagram showing another example of a binary image of an object. FIG. 16 is a diagram showing an example of a feature pattern of an object. FIG. 17 is a flowchart showing an example of processing in the object identification method according to the second embodiment.

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings, but the present disclosure is not limited thereto. The components of each embodiment described below can be combined as appropriate. In addition, some components may not be used.

[0012] First Embodiment <Work Vehicle> A work vehicle identifies an object from objects present around the work vehicle at a work site. Examples of the work vehicle include a wheel loader and a forklift.

[0013] FIG. 1 is a side view showing a work vehicle according to a first embodiment. In this embodiment, the work vehicle 100 is a wheel loader. The work vehicle 100 may be a manned vehicle operated by a driver, or an unmanned vehicle operated without a driver. The wheel loader 100 performs loading work, loading transported objects picked up by a work implement 110.

[0014] In this embodiment, the object to be identified by the wheel loader 100 is the vessel of a dump truck.

[0015] As shown in FIG. 1 , the wheel loader 100 includes a vehicle body 102 , a traveling device 104 , and a work implement 110 .

[0016] The vehicle body 102 includes a front vehicle body portion 102F and a rear vehicle body portion 102R disposed rearward of the front vehicle body portion 102F. The front vehicle body portion 102F and the rear vehicle body portion 102R are flexibly connected via a joint mechanism 109.

[0017] The traveling device 104 supports the vehicle body 102 and travels on the ground surface RS. The traveling device 104 has a power source 104A, a steering device 104B, wheels 106, and a power transmission device 108.

[0018] The power source 104A generates power for driving the wheel loader 100. The power source 104A is, for example, an electric motor.

[0019] The steering device 104B is provided to the joint mechanism 109 in order to steer the wheel loader 100. When the steering device 104B is actuated, the front vehicle body 102F bends relative to the rear vehicle body 102R. When the front vehicle body 102F bends relative to the rear vehicle body 102R, the traveling direction of the wheel loader 100 is adjusted. The steering device 104B is, for example, a hydraulic cylinder.

[0020] The wheels 106 include two front wheels 106F rotatably supported on the vehicle body front portion 102F, and two rear wheels 106R rotatably supported on the vehicle body rear portion 102R.

[0021] In the embodiment, the left-right direction is a direction parallel to the rotation axes of the front wheels 106F and rear wheels 106R when the wheel loader 100 travels in a straight line. The up-down direction is a direction perpendicular to the contact surfaces of the front wheels 106F and rear wheels 106R. The front-rear direction is a direction perpendicular to both the left-right direction and the up-down direction.

[0022] The power transmission device 108 transmits the driving force of the power source 104A to the wheels 106. The power transmission device 108 changes the output from the power source 104A, thereby accelerating or decelerating the wheel loader 100. The power transmission device 108 switches the rotation direction of the wheels 106, thereby causing the wheel loader 100 to move forward or backward.

[0023] The work implement 110 is supported by the vehicle body front section 102F. At least a portion of the work implement 110 is disposed forward of the vehicle body front section 102F. The work implement 110 has a boom 111, a bucket 112, a bell crank 113, a link 114, a boom cylinder 115, and a bucket cylinder 116.

[0024] The boom 111 is rotatably connected to the front vehicle body 102F. The bell crank 113 is connected to the bucket 112 via a link 114. The bucket 112 is rotatably connected to the tip of the boom 111. The bucket 112 is a working member having a tip 112B including a cutting edge. The bucket 112 is positioned forward of the front tires 106F.

[0025] The boom cylinder 115 is a hydraulic cylinder that generates power to operate the boom 111. The boom cylinder 115 extends and retracts, causing the boom 111 to move up and down.

[0026] The bucket cylinder 116 is a hydraulic cylinder that generates power to operate the bucket 112. The bucket 112 performs a dumping operation or a tilting operation when the bucket cylinder 116 extends and retracts. When the bucket 112 performs a tilting operation, the bucket 112 scoops up the excavated material. When the bucket 112 performs a dumping operation, the excavated material held in the bucket 112 is ejected from the bucket 112.

[0027] <Work vehicle identification system> Figure 2 is a functional block diagram showing the identification system according to the first embodiment. The identification system 1 for the wheel loader 100 comprises a three-dimensional sensor (object detection sensor) 11, an output unit 19, and a controller 20. The three-dimensional sensor 11, the output unit 19, and the controller 20 are connected to each other so as to be able to communicate data wirelessly or via a wire.

[0028] The three-dimensional sensor 11 is attached to the wheel loader 100. In the embodiment, the three-dimensional sensor 11 is attached to the front of the wheel loader 100. The number of three-dimensional sensors 11 is not particularly limited, and, for example, a plurality of three-dimensional sensors 11 may be arranged on the wheel loader 100. The three-dimensional sensor 11 is an object detection sensor that detects objects present in the vicinity of the wheel loader 100. In the embodiment, the three-dimensional sensor 11 is an object detection sensor that detects objects present in front of the wheel loader 100. The three-dimensional sensor 11 is capable of detecting a range of, for example, several tens of meters. The three-dimensional sensor 11 transmits detected object detection data to the controller 20.

[0029] The three-dimensional sensor 11 acquires three-dimensional data of an object. The three-dimensional data of the object includes a point cloud consisting of multiple detection points defined on the surface of the object. The point cloud indicates the relative distance and relative position between the three-dimensional sensor 11 and each of the multiple detection points defined on the surface of the object. An example of the three-dimensional sensor 11 is a laser sensor (LiDAR: Light Detection and Ranging) that detects objects by emitting laser light. However, the three-dimensional sensor 11 may also be another non-contact sensor, such as an infrared sensor that detects objects by emitting infrared light or a radar sensor (RADAR: Radio Detection and Ranging) that detects objects by emitting radio waves.

[0030] The output unit 19 outputs the identification result. The output unit 19 is, for example, a monitor, a lamp, a buzzer, etc. The output unit 19 outputs the identification result under the control of the output control unit 29.

[0031] <Controller> The controller 20 identifies objects in the vicinity of the wheel loader 100. The controller 20 is capable of acquiring object detection data from the three-dimensional sensor 11 serving as an object detection sensor. The controller 20 includes a numerical calculation device (processor) such as a CPU (Central Processing Unit).

[0032] FIG. 3 is a block diagram showing a computer system according to the first embodiment. The controller 20 includes a computer system 1000. The computer system 1000 includes a processor 1001 such as a CPU, a main memory 1002 including a nonvolatile memory such as a read-only memory (ROM) and a volatile memory such as a random access memory (RAM), a storage 1003, and an interface 1004 including an input / output circuit. The functions of the controller 20 described above are stored as a program in the storage 1003. The processor 1001 reads the program from the storage 1003, loads it into the main memory 1002, and executes the above-described processing in accordance with the program. The program may be distributed to the computer system 1000 via a network.

[0033] In the embodiment, the controller 20 includes a sensor data acquisition unit 21 , a storage unit 22 , an identification unit 25 , and an output control unit 29 .

[0034] The sensor data acquisition unit 21 acquires object detection data from the three-dimensional sensor 11. The object detection data acquired by the sensor data acquisition unit 21 is, for example, a point cloud made up of a plurality of detection points. Hereinafter, the object detection data will be referred to as point cloud data.

[0035] The storage unit 22 includes at least one of a RAM, a ROM, a flash memory, and a hard disk drive. The storage unit 22 stores data used in the processing of the controller 20.

[0036] The storage unit 22 stores a search range, which is a range in which the object is presumed to exist, and stores a feature pattern of the object for identifying the object.

[0037] The search range is a range for extracting a point cloud for plane extraction used in processing to identify an object from point cloud data, which is object detection data. The point cloud data includes a point cloud of measurement distances and field of view angles of the three-dimensional sensor 11.

[0038] The feature pattern of the object is an image that shows the shape of a flat portion of the object's outer shape. In this embodiment, the feature pattern is a binary image of the side surface of the vessel of a dump truck.

[0039] The identification unit 25 identifies an object from the object detection data of the three-dimensional sensor 11. More specifically, the identification unit 25 estimates a plane that constitutes at least a part of the outline of the object from the point cloud data acquired by the sensor data acquisition unit 21, extracts a point cloud located on the estimated plane to generate a binary image, and identifies the object from the binary image.

[0040] The identification unit 25 projects the point cloud data onto a projection plane, and reconstructs the part of the projection plane where the point cloud density is high into three dimensions, thereby estimating the plane of the object.

[0041] The identification unit 25 detects from the generated binary image a shape that resembles a binary image of a feature pattern that indicates the shape of a plane portion of the pre-stored outline of the object, and identifies the object.

[0042] 4 is a schematic diagram showing the operation of the work vehicle according to the first embodiment. The wheel loader 100 identifies the dump truck 200 to be loaded in order to load the transported goods picked up by the work implement 110. The wheel loader 100 identifies the vessel BE of the dump truck 200 based on the object detection data acquired from the three-dimensional sensor 11.

[0043] The object identification process performed by the identification unit 25 will be described with reference to FIGS. 5 to 8 . FIG. 5 is a diagram illustrating an example of a point cloud representing an object. FIG. 6 is a schematic diagram illustrating an example of the object identification process. FIG. 7 is a diagram illustrating an example of a binary image of the object. FIG. 8 is a diagram illustrating an example of a feature pattern of the object. As shown in FIG. 5 , the point cloud is composed of a plurality of detection points defined on the surface of the object. First, as shown in FIG. 6 , the identification unit 25 estimates a plane constituting the object from the point cloud data. As shown in FIG. 6 ( a1 ), the identification unit 25 extracts a point cloud within a search range from the point cloud data to obtain a point cloud of the identification target. Then, as shown in FIG. 6 ( a2 ), the identification unit 25 projects the plane extraction point cloud onto the xy plane of a coordinate system in which the z-axis is perpendicular to the ground plane. FIG. 6 ( a3 ) shows the point cloud projected onto the xy plane. The point cloud surrounded by a dashed line on the xy plane corresponds to the point cloud on the side of the dump truck, which is surrounded by a dashed line and perpendicular to the xy plane. The point cloud on a plane perpendicular to the xy plane of the dump truck is aggregated onto a single line segment to form a high density. Then, as shown in FIG. 6( a4 ), the identification unit 25 converts the points back to their original three-dimensional positions, leaving only the area surrounded by the dashed line where the point cloud density is high on the xy plane. In this way, the identification unit 25 estimates the plane that constitutes the object by extracting a planar point cloud, which is a point cloud on the plane that constitutes the object. In this example, the planar point cloud is a point cloud on the side surface of the vessel.

[0044] As shown in Fig. 7 , the identification unit 25 generates a binary image based on the extracted plane point cloud. The identification unit 25 generates a binary image by cropping the plane peripheral point cloud, which is a point cloud around the estimated plane, as a point cloud for generating a binary image. The identification unit 25 detects a shape similar to the feature pattern indicating the vessel shape shown in Fig. 8 in the generated binary image. If a similar shape is found, the identification unit 25 identifies the overall shape of the dump truck based on the detected position and the vessel shape of the feature pattern.

[0045] The output control unit 29 controls the output of the identification result by the identification unit 25 via the output unit 19. For example, when an object is identified, the output control unit 29 outputs a video signal that causes an image showing the identified object to be displayed on a monitor. For example, when an object is identified, the output control unit 29 outputs a control signal that causes a lamp to light up. For example, when an object is identified, the output control unit 29 outputs a control signal that causes a buzzer to sound.

[0046] <Processing by Controller> When the target identification process is started during startup of the wheel loader 100, the process shown in Fig. 9 is executed. Fig. 9 is a flowchart showing an example of the process of the target identification method according to the first embodiment. The process of the flowchart shown in Fig. 9 is the target identification process. Note that the processes from step ST19 to step ST21 are not essential processes.

[0047] The controller 20 acquires the point cloud (step ST11). More specifically, the controller 20 receives the point cloud data from the three-dimensional sensor 11 via the sensor data acquisition unit 21. The controller 20 proceeds to step ST12.

[0048] The controller 20 crops the plane extraction point cloud (step ST12). More specifically, the controller 20 extracts, from the point cloud data, a point cloud included in a search range where it is estimated that the target exists, as the plane extraction point cloud using the identification unit 25. The controller 20 proceeds to step ST13.

[0049] The controller 20 downsamples the plane extraction point cloud (step ST13). More specifically, the controller 20 downsamples the plane extraction point cloud using voxel downsampling, which is an example of a point cloud data downsampling method, by the identification unit 25. The controller 20 proceeds to step ST14.

[0050] The controller 20 extracts a point cloud on a plane and estimates a plane (step ST14). More specifically, the controller 20 extracts a point cloud on a plane, which is a point cloud on the vessel side surface, from the plane extraction point cloud downsampled by the identification unit 25, and estimates the vessel side surface, which is a plane. The controller 20 proceeds to step ST15.

[0051] The controller 20 crops the point cloud for generating a binary image (step ST15). More specifically, the controller 20 crops the plane peripheral point cloud, which is the point cloud around the plane estimated by the identification unit 25, as a point cloud for generating a binary image. The controller 20 proceeds to step ST16.

[0052] The controller 20 generates a binary image of the plane peripheral point cloud (step ST16). More specifically, the controller 20 uses the identification unit 25 to generate a binary image on the yz plane of a coordinate system in which the normal vector of the estimated plane coincides with the direction of the x-axis, based on the cropped plane peripheral point cloud. The yz plane is a plane that includes the estimated plane. The controller 20 proceeds to step ST17.

[0053] The controller 20 detects features of the target from the binary image (step ST17). More specifically, the controller 20 detects a shape similar to the feature pattern in the binary image using the recognition unit 25. The controller 20 proceeds to step ST18.

[0054] The controller 20 restores the three-dimensional position (step ST18). More specifically, the controller 20 uses the identification unit 25 to identify the overall position, posture, and shape of the dump truck based on the detected position in the binary image and the vessel shape of the feature pattern. The controller 20 proceeds to step ST19.

[0055] The controller 20 checks whether the dump truck is within the effective field of view of the sensor (step ST19). More specifically, the controller 20 checks, using the identification unit 25, whether the entire position of the dump truck is within the effective field of view of the three-dimensional sensor 11. This is to confirm that the identification performance is guaranteed, since the identification performance decreases around the boundary of the field of view of the three-dimensional sensor 11.

[0056] The controller 20 checks the consistency with the predicted value (step ST20). More specifically, the controller 20 uses the identification unit 25 to compare the predicted position of the target predicted from the previous estimation result with the currently estimated position of the target, and checks whether the difference is equal to or less than a threshold.

[0057] The controller 20 calculates the variance and likelihood of the point cloud classification (step ST21).

[0058] In this way, the controller 20 identifies the vessel of the target dump truck from the point cloud data.

[0059] <Effects> As described above, in this embodiment, an outline plane of an object is estimated from point cloud data, a point cloud located on the estimated plane is extracted to generate a binary image, and the object is identified from the binary image. As a result, this embodiment can reduce the processing load compared to when point cloud data is used directly in the identification process. Since this embodiment reduces the processing load, it is possible to suppress a decrease in identification performance, such as a delay in the identification process. In this way, according to this embodiment, an object can be appropriately identified from point cloud data, which is three-dimensional data.

[0060] In this embodiment, point cloud data is projected onto a projection plane, and a portion of the projection plane with high point cloud density is restored in three dimensions to estimate the target plane. According to this embodiment, noise in the point cloud data can be removed, and the plane can be estimated with high accuracy.

[0061] In this embodiment, the object is identified by detecting a shape from the generated binary image that resembles a binary image of a pre-stored feature pattern. According to this embodiment, the use of binary images can reduce the processing load.

[0062] In this way, according to this embodiment, the vessel of the dump truck can be appropriately identified.

[0063] Second Embodiment A second embodiment will be described using Figures 10 to 17. Figure 10 is a perspective view showing a work vehicle according to the second embodiment. In the second embodiment, the work vehicle 100 is a forklift, which is a type of cargo handling vehicle. The processing in the identification unit 25 of the controller 20 differs from that in the first embodiment. Configurations similar to those in the first embodiment are given the same or corresponding reference numerals, and descriptions thereof will be omitted.

[0064] <Work Vehicle> The forklift 100 performs cargo handling operations. Cargo handling operations include retrieving transported goods that are placed in a predetermined storage location. In this embodiment, the object that the work vehicle identifies is the transported goods. The transported goods are loading platforms or containers on which cargo is loaded. The transported goods are, for example, pallets, skids, or containers. The transported goods have a pair of fork insertion holes. In this embodiment, the object that the work vehicle identifies is, for example, a box-shaped pallet.

[0065] In this embodiment, the feature pattern is a binary image of the fork insertion holes on the front of the pallet.

[0066] As shown in FIG. 10 , the forklift 100 includes a vehicle body 102 , a work implement 104 disposed in front of the vehicle body 102 , wheels 105 supporting the vehicle body 102 , a three-dimensional sensor 11 , and a camera 12 .

[0067] The vehicle body 102 includes a counterweight 120 and a fender 108. The counterweight 120 is disposed at the rear of the vehicle body 102. The counterweight 120 is attached to the rear of the vehicle body 102 to balance the weight of the forklift 100 in the front-to-rear direction when the forklift 100 picks up an object. The fender 108 is disposed at the front of the vehicle body 102. The fenders 108 are disposed on both the left and right sides of the vehicle body 102.

[0068] The work implement 104 performs at least a part of the loading and unloading work. The work implement 104 is disposed in front of the vehicle body 102. The work implement 104 is supported by the vehicle body 102. The work implement 104 has a mast 141, a bracket 142, and a fork 143.

[0069] The mast 141 is supported at the front of the vehicle body 102 so as to be tiltable. The mast 141 is long in the vertical direction. The bracket 142 supports the fork 143. The bracket 142 is supported by the mast 141. The bracket 142 is movable in the vertical direction along the mast 141. The fork 143 supports an object to be transported. The fork 143 is supported by the mast 141 via the bracket 142.

[0070] A pair of forks 143 are provided. The forks 143 include a first fork 143A and a second fork 143B that is disposed to the right of the first fork 143A. The bracket 142 supports the first fork 143A and the second fork 143B.

[0071] The wheels 105 support the vehicle body 102. At least a portion of the wheels 105 is disposed below the vehicle body 102. The wheels 105 include front wheels 105F and rear wheels 105R. The front wheels 105F are disposed forward of the rear wheels 105R. The front wheels 105F are disposed on the left and right sides of the vehicle body 102. The rear wheels 105R are disposed on the left and right sides of the vehicle body 102. Each of the front wheels 105F and the rear wheels 105R is rotatable around a rotation axis. One or both of the front wheels 105F and the rear wheels 105R are steered wheels. In the embodiment, the rear wheels 105R are steered wheels. Note that the front wheels 105F may be steered wheels, or both the front wheels 105F and the rear wheels 105R may be steered wheels.

[0072] In the embodiment, the left-right direction is a direction parallel to the rotation axes of the front wheels 105F and the rear wheels 105R when the forklift 100 travels straight ahead. The up-down direction is a direction perpendicular to the contact surfaces of the front wheels 105F and the rear wheels 105R. The front-rear direction is a direction perpendicular to both the left-right direction and the up-down direction.

[0073] The fenders 108 are positioned so as to cover at least a portion of the front wheels 105F. At least a portion of the fenders 108 are positioned higher than the front wheels 105F. At least a portion of the fenders 108 are positioned rearward of the front wheels 105F. The fenders 108 are positioned on both the left and right sides of the vehicle body 102. The left fender 108 is positioned so as to cover at least a portion of the left front wheel 105F. The right fender 108 is positioned so as to cover at least a portion of the right front wheel 105F.

[0074] 11 is a schematic diagram showing the operation of the work vehicle according to the second embodiment. In order to perform a loading / unloading operation, the forklift 100 identifies the pallet 300 to be unloaded. The forklift 100 identifies the pallet 300 based on the object detection data acquired from the three-dimensional sensor 11.

[0075] The classification processing in the classification unit 25 will be described with reference to Figs. 12 to 16. Fig. 12 is a schematic perspective view illustrating another example of the object classification processing. Fig. 13 is a schematic plan view illustrating another example of the object classification processing. Fig. 14 is a diagram illustrating another example of the object classification processing. Fig. 15 is a diagram illustrating another example of a binary image of the object. Fig. 15 is a diagram illustrating an example of a feature pattern of the object. Fig. 16 is a diagram illustrating an example of a feature pattern of the object.

[0076] The recognition unit 25 estimates the front surface of the palette as a plane, and generates a binary image of the front surface of the palette, in the same manner as in the first embodiment.

[0077] The identification unit 25 moves the passing point cloud that has passed through the front surface of the pallet to the intersection of the line connecting each point with the 3D sensor 11 and the front surface of the pallet, and generates a binary image of the passing point cloud from the moved passing point cloud. As shown in Figures 12 and 13, a point cloud that passes through the estimated front surface of the pallet and is located behind the estimated front surface of the pallet is called a planar passing point cloud. The identification unit 25 calculates the intersection of the line connecting each point of the planar passing point cloud that has passed through the estimated front surface of the pallet to the position of the 3D sensor 11 and the front surface of the pallet. In Figure 13, the estimated front surface of the pallet is indicated by a dashed line. In Figure 13, the point cloud appears above the estimated front surface of the pallet. The point cloud surrounded by an ellipse indicates the point cloud that has passed through the estimated front surface of the pallet. In other words, the point cloud surrounded by an ellipse is the point cloud that has passed through the fork insertion holes of the transported item. The identification unit 25 moves the planar passing point cloud to the position of the calculated intersection, as shown in Figure 14. That is, the identification unit 25 moves the plane passing point cloud onto the front surface of the pallet. Fig. 14(a1) is a diagram showing the intersection of the estimated pallet front surface with a line connecting the point cloud that passes through the pallet front surface and the coordinate system reference of the 3D sensor 11. Fig. 14(a2) is a diagram in which the point cloud is interpolated at the position of the intersection found in (a1). In this way, the identification unit 25 calculates the position where each point of the plane passing point cloud passes through the plane.

[0078] The identification unit 25 generates a binary image from the plane passing point cloud, as shown in FIG. 15 . The identification unit 25 filters the generated binary image with the feature pattern of the fork insertion hole shown in FIG. 16 to calculate the similarity. The identification unit 25 searches the binary image for a pattern similar to the fork insertion hole pattern. In the example shown in FIG. 16 , region P11 is a region where there is a high probability that a fork insertion hole will be located. Region P12 is a region where there is a low probability that a fork insertion hole will be located. Region P13 is between regions P11 and P12 in terms of the probability that a fork insertion hole will be located. C indicates the midpoint of the fork insertion hole.

[0079] 17 is a flowchart showing an example of processing in the object identification method according to the second embodiment. Steps ST31 to ST36 and steps ST38 to ST42 perform the same processing as steps ST11 to ST16 and steps ST17 to ST21 in the flowchart shown in FIG.

[0080] The controller 20 generates a binary image of the plane passing points (step ST37). More specifically, the controller 20 generates a binary image of the plane passing points that have passed through the estimated front surface of the pallet using the identification unit 25.

[0081] In this way, the controller 20 identifies the fork insertion holes of the target pallet from the point cloud data.

[0082] <Effects> As described above, in this embodiment, the cloud of passing points that have passed through the estimated front surface of the pallet are moved to the intersection of the line connecting each point with the 3D sensor 11 and the estimated front surface of the pallet, and a binary image of the passing point cloud is generated from the moved cloud of passing points. In this embodiment, as described above, by projecting the point cloud onto a plane, the points on the plane are aggregated on line segments to achieve high density. In this embodiment, noise is removed by achieving high density. According to this embodiment, fork insertion holes can be identified with high accuracy even when the distance between the 3D sensor 11 and the target is great and the point cloud density is sparse.

[0083] In contrast, with conventional technology, as the distance between the three-dimensional sensor 11 and the object increases, the point cloud density becomes sparse, making it difficult to identify the position of the fork insertion hole from the point cloud.

[0084] In this way, according to this embodiment, the fork insertion holes of the pallet can be properly identified.

[0085] 1...identification system, 11...3D sensor (object detection sensor), 19...output unit, 20...controller, 21...sensor data acquisition unit, 22...memory unit, 25...identification unit, 29...output control unit, 100...wheel loader (work vehicle), 102...vehicle body, 102F...front body, 102R...rear body, 104...traveling device, 104A...power source, 104B...steering device, 106...wheel, 106F...front wheel, 106R...rear wheel, 109...joint mechanism, 110...working machine, 111...boom, 112...bucket, 112B...tip, 113...bell crank, 114...link, 115...boom cylinder, 116...bucket cylinder, 1000...computer system, 1001...processor, 1002...main memory, 1003...storage, 1004...interface.

Claims

1. An identification system for identifying an object from objects present in the vicinity of a work vehicle, comprising: an object detection sensor that is attached to the work vehicle and detects objects present in the vicinity of the work vehicle; and a controller, wherein the controller acquires point cloud data, which is object detection data, from the object detection sensor, estimates a plane that constitutes at least a part of the outline of the object from the point cloud data, extracts a point cloud located on the estimated plane to generate a binary image, and identifies the object from the binary image.

2. The identification system according to claim 1, wherein the controller detects a shape from the generated binary image that resembles a binary image of a characteristic pattern that indicates the shape of a planar portion of the object's pre-stored outline, and identifies the object.

3. The identification system according to claim 2, wherein the object is a vessel of a dump truck, the plane is a side surface of the vessel, and the characteristic pattern indicates a shape of the side surface of the vessel.

4. The identification system of claim 2, wherein the object is a fork insertion hole of a transported object, the plane is the front surface of the transported object, the feature pattern indicates the shape of the fork insertion hole, and the controller moves a group of passing points that have passed through the front surface of the transported object to an intersection with the front surface of the transported object on a straight line connecting each point with the object detection sensor, and generates a binary image of the group of passing points from the moved group of passing points.

5. An identification method for identifying an object from objects present in the vicinity of a work vehicle, comprising the steps of: acquiring point cloud data, which is object detection data, from an object detection sensor attached to the work vehicle and detecting objects present in the vicinity of the work vehicle; estimating a plane that constitutes at least a part of the object's outline from the point cloud data; extracting a point cloud located on the estimated plane to generate a binary image; and identifying the object from the binary image.

6. A work vehicle comprising: a vehicle body; an object detection sensor attached to the vehicle body for detecting objects present around the work vehicle; and a controller, wherein the controller acquires point cloud data, which is object detection data, from the object detection sensor; estimates a plane that constitutes at least a part of the outline of an object from the point cloud data; extracts a point cloud located on the estimated plane to generate a binary image; and identifies the object from the binary image.

Images