Identification system, identification method, and work vehicle

Abstract

To provide an identification system, an identification method, and a work vehicle which can appropriately identify a target from point group data as three-dimensional data.SOLUTION: An identification system 1 includes: a three-dimensional sensor 11 which is attached to a work vehicle so as to detect an object existing around the work vehicle; and a controller 20. The controller 20 acquires point group data as object detection data from the three-dimensional sensor 11, estimates a plane surface forming at least one part of an outside shape of a target from the point group data, extracts point groups positioned on the estimated plane surface so as to generate a binary image, and identifies the target from the binary image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an identification system, an identification method, and a work vehicle. [Background technology]

[0002] For example, in order to perform accurate loading and unloading operations using work vehicles such as dump trucks, wheel loaders, shovels, and forklifts, it is necessary to improve the identification performance of the work target by the work vehicle. As disclosed in Patent Document 1, a technology is known regarding an automatic excavator that automatically performs a series of processes of excavating an excavation target and loading the excavated material into a loading target. In the technology described in Patent Document 1, the relative position of the loading target to the excavator and the relative attitude of the excavator to the loading target are calculated using images captured by a stereo camera. As disclosed in Patent Document 2, a technology is known regarding a loading vehicle that extracts an observation point group from three-dimensional distance data, and when the extracted observation point group includes a feature representing the front of the loading target, extracts the feature and specifies the position, direction, and width of the loading target. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-088625 [Patent Document 2] JP 2016-210586 A Summary of the Invention [Problem to be solved by the invention]

[0004] When performing object identification processing using an image, it may be difficult to identify the object depending on the shooting conditions such as brightness when the image was taken. When performing object identification processing using 3D data of an object detected by a 3D sensor, the processing load becomes large, and it may become difficult to perform calculations in real time.

[0005] An object of the present disclosure is to provide an identification system, an identification method, and a work vehicle that are capable of appropriately identifying an object from point cloud data, which is three-dimensional data. [Means for solving the problem]

[0006] According to an aspect of the present disclosure, there is provided an identification system for identifying an object from objects present in the vicinity of a work vehicle, the system including an object detection sensor attached to a work vehicle and detecting objects present in the vicinity of the work vehicle, and a controller. The controller acquires point cloud data, which is object detection data, from the object detection sensor, estimates a plane constituting at least a part of the outline of the object from the point cloud data, extracts a point cloud located on the estimated plane to generate a binary image, and identifies the object from the binary image.

[0007] According to an aspect of the present disclosure, there is provided an identification method for identifying an object from objects present in the vicinity of a work vehicle, comprising the steps of: acquiring point cloud data, which is object detection data, from an object detection sensor attached to the work vehicle and detecting objects present in the vicinity of the work vehicle; estimating a plane that constitutes at least a part of the outline of the object from the point cloud data; extracting a point cloud located on the estimated plane to generate a binary image; and identifying the object from the binary image.

[0008] According to an aspect of the present disclosure, there is provided a work vehicle including a vehicle body, an object detection sensor attached to the vehicle body and configured to detect objects present around the work vehicle, and a controller. The controller acquires point cloud data, which is object detection data, from the object detection sensor, estimates a plane constituting at least a part of the outline of the object from the point cloud data, extracts a point cloud located on the estimated plane to generate a binary image, and identifies the object from the binary image. Effect of the Invention

[0009] According to aspects of the present disclosure, there is provided an identification system and a work vehicle capable of appropriately identifying an object from point cloud data, which is three-dimensional data. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a side view showing a work vehicle according to a first embodiment. [Diagram 2] FIG. 2 is a functional block diagram showing the identification system according to the first embodiment. [Diagram 3] FIG. 3 is a block diagram showing a computer system according to the first embodiment. [Figure 4] FIG. 4 is a schematic diagram showing the operation of the work vehicle according to the first embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of a point cloud indicating an object. [Figure 6] FIG. 6 is a schematic diagram illustrating an example of the target identification process. [Figure 7] FIG. 7 is a diagram showing an example of a binary image of a target. [Figure 8] FIG. 8 is a diagram showing an example of a feature pattern of a target. [Figure 9] FIG. 9 is a flowchart showing an example of processing of the object identification method according to the first embodiment. [Figure 10] FIG. 10 is a perspective view showing a work vehicle according to the second embodiment. [Figure 11] FIG. 11 is a schematic diagram showing the operation of the work vehicle according to the second embodiment. [Figure 12] FIG. 12 is a schematic perspective view illustrating another example of the object identification process. [Figure 13] FIG. 13 is a schematic plan view illustrating another example of the object identification process. [Figure 14] FIG. 14 is a diagram illustrating another example of the target identification process. [Figure 15] FIG. 15 is a diagram showing another example of a binary image of a target. [Figure 16] FIG. 16 is a diagram showing an example of a target feature pattern. [Figure 17] FIG. 17 is a flowchart showing an example of processing of the object identification method according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings, but the present disclosure is not limited thereto. The components of each embodiment described below can be appropriately combined. In addition, some components may not be used.

[0012] [First embodiment] <Work vehicle> The work vehicle identifies an object from objects present around the work vehicle at a work site. The work vehicle is, for example, a wheel loader or a forklift.

[0013] FIG. 1 is a side view showing a work vehicle according to the first embodiment. In this embodiment, the work vehicle 100 is a wheel loader. The work vehicle 100 may be a manned vehicle operated by a driver, or an unmanned vehicle operated without a driver. The wheel loader 100 performs loading work to load transported objects picked up by a work implement 110.

[0014] In this embodiment, the object to be identified by the wheel loader 100 is the vessel of a dump truck.

[0015] As shown in FIG. 1, the wheel loader 100 includes a vehicle body 102 , a traveling device 104 , and a work machine 110 .

[0016] The vehicle body 102 includes a vehicle front portion 102F and a vehicle rear portion 102R disposed rearward of the vehicle front portion 102F. The vehicle front portion 102F and the vehicle rear portion 102R are connected to each other via a joint mechanism 109 so as to be able to bend.

[0017] The traveling device 104 supports the vehicle body 102 and travels on the ground RS. The traveling device 104 has a power source 104A, a steering device 104B, wheels 106, and a power transmission device 108.

[0018] The power source 104A generates power for driving the wheel loader 100. The power source 104A is, for example, an electric motor.

[0019] The steering device 104B is provided in the joint mechanism 109 in order to steer the wheel loader 100. When the steering device 104B is actuated, the front body part 102F bends relative to the rear body part 102R. When the front body part 102F bends relative to the rear body part 102R, the traveling direction of the wheel loader 100 is adjusted. The steering device 104B is, for example, a hydraulic cylinder.

[0020] The wheels 106 include two front wheels 106F rotatably supported on the vehicle body front portion 102F, and two rear wheels 106R rotatably supported on the vehicle body rear portion 102R.

[0021] In the embodiment, the left-right direction is a direction parallel to the rotation axes of the front wheels 106F and the rear wheels 106R when the wheel loader 100 travels in a straight line. The up-down direction is a direction perpendicular to the ground contact surfaces of the front wheels 106F and the rear wheels 106R. The front-rear direction is a direction perpendicular to both the left-right direction and the up-down direction.

[0022] The power transmission device 108 transmits the driving force of the power source 104A to the wheels 106. The power transmission device 108 changes the output from the power source 104A, thereby accelerating or decelerating the wheel loader 100. The power transmission device 108 switches the rotation direction of the wheels 106, thereby causing the wheel loader 100 to move forward or backward.

[0023] The work implement 110 is supported by the vehicle body front portion 102F. At least a portion of the work implement 110 is disposed forward of the vehicle body front portion 102F. The work implement 110 has a boom 111, a bucket 112, a bell crank 113, a link 114, a boom cylinder 115, and a bucket cylinder 116.

[0024] Boom 111 is rotatably connected to vehicle body front portion 102F. Bell crank 113 is connected to bucket 112 via link 114. Bucket 112 is rotatably connected to the tip of boom 111. Bucket 112 is a working member having tip portion 112B including a cutting edge. Bucket 112 is disposed forward of front tires 106F.

[0025] The boom cylinder 115 is a hydraulic cylinder that generates power to operate the boom 111. The boom cylinder 115 extends and retracts, thereby causing the boom 111 to perform a raising or lowering operation.

[0026] The bucket cylinder 116 is a hydraulic cylinder that generates power to operate the bucket 112. The bucket 112 performs a dumping motion or a tilting motion as the bucket cylinder 116 expands and contracts. The bucket 112 performs a tilting motion, whereby the bucket 112 scoops up the excavated material. The bucket 112 performs a dumping motion, whereby the excavated material held in the bucket 112 is discharged from the bucket 112.

[0027] <Work vehicle identification system> 2 is a functional block diagram showing the identification system according to the first embodiment. The identification system 1 of the wheel loader 100 comprises a three-dimensional sensor (object detection sensor) 11, an output unit 19, and a controller 20. The three-dimensional sensor 11, the output unit 19, and the controller 20 are connected to each other so as to be able to communicate data wirelessly or via wires.

[0028] The three-dimensional sensor 11 is attached to the wheel loader 100. In the embodiment, the three-dimensional sensor 11 is attached to the front of the wheel loader 100. The number of the three-dimensional sensors 11 is not particularly limited, and for example, a plurality of the three-dimensional sensors 11 may be arranged on the wheel loader 100. The three-dimensional sensor 11 is an object detection sensor that detects an object present around the wheel loader 100. In the embodiment, the three-dimensional sensor 11 is an object detection sensor that detects an object present in front of the wheel loader 100. The three-dimensional sensor 11 can detect a range of, for example, about several tens of meters. The three-dimensional sensor 11 transmits the detected object detection data to the controller 20.

[0029] The three-dimensional sensor 11 acquires three-dimensional data of an object. The three-dimensional data of the object includes a point cloud consisting of a plurality of detection points defined on the surface of the object. The point cloud indicates the relative distance and relative position between the three-dimensional sensor 11 and each of the plurality of detection points defined on the surface of the object. An example of the three-dimensional sensor 11 is a laser sensor (LiDAR: Light Detection and Ranging) that detects an object by emitting laser light. Note that the three-dimensional sensor 11 may be another non-contact sensor such as an infrared sensor that detects an object by emitting infrared light or a radar sensor (RADAR: Radio Detection and Ranging) that detects an object by emitting radio waves.

[0030] The output unit 19 outputs the identification result. The output unit 19 is, for example, a monitor, a lamp, a buzzer, etc. The output unit 19 outputs the identification result under the control of the output control unit 29.

[0031] <Controller> The controller 20 identifies objects around the wheel loader 100. The controller 20 is capable of acquiring object detection data from the three-dimensional sensor 11 serving as an object detection sensor. The controller 20 includes a numerical calculation device (processor) such as a CPU (Central Processing Unit).

[0032] 3 is a block diagram showing a computer system according to the first embodiment. The controller 20 includes a computer system 1000. The computer system 1000 includes a processor 1001 such as a CPU, a main memory 1002 including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory), a storage 1003, and an interface 1004 including an input / output circuit. The functions of the controller 20 described above are stored in the storage 1003 as a program. The processor 1001 reads the program from the storage 1003, loads it in the main memory 1002, and executes the above-mentioned processing according to the program. The program may be distributed to the computer system 1000 via a network.

[0033] In the embodiment, the controller 20 includes a sensor data acquisition unit 21, a storage unit 22, a recognition unit 25, and an output control unit 29.

[0034] The sensor data acquisition unit 21 acquires object detection data from the three-dimensional sensor 11. The object detection data acquired by the sensor data acquisition unit 21 is, for example, a point cloud made up of a plurality of detection points. Hereinafter, the object detection data will be referred to as point cloud data.

[0035] The storage unit 22 includes at least one of a RAM, a ROM, a flash memory, and a hard disk drive. The storage unit 22 stores data used in the processing of the controller 20.

[0036] The storage unit 22 stores a search range, which is a range in which the target is assumed to exist, and stores a feature pattern of the target for identifying the target.

[0037] The search range is a range for extracting a plane extraction point cloud used in processing to identify an object from point cloud data, which is object detection data. The point cloud data includes a point cloud of the measurement distance and viewing angle of the three-dimensional sensor 11.

[0038] The feature pattern of the target is an image showing the shape of a flat portion of the target's outer shape. In this embodiment, the feature pattern is a binary image of the side surface of the vessel of a dump truck.

[0039] The identification unit 25 identifies an object from the object detection data of the three-dimensional sensor 11. More specifically, the identification unit 25 estimates a plane that constitutes at least a part of the outline of the object from the point cloud data acquired by the sensor data acquisition unit 21, extracts a point cloud located on the estimated plane to generate a binary image, and identifies the object from the binary image.

[0040] The identification unit 25 projects the point cloud data onto a projection plane, and restores a portion of the projection plane where the point cloud density is high into three dimensions to estimate the plane of the object.

[0041] The identification unit 25 detects, from the generated binary image, a shape that resembles a binary image of a characteristic pattern that indicates the shape of a planar portion of the pre-stored outline of the object, and identifies the object.

[0042] 4 is a schematic diagram showing the operation of the work vehicle according to the first embodiment. The wheel loader 100 identifies the dump truck 200 to which the transported goods picked up by the work implement 110 are to be loaded. The wheel loader 100 identifies the vessel BE of the dump truck 200 based on the object detection data acquired from the three-dimensional sensor 11.

[0043] The object identification process by the identification unit 25 will be described with reference to Figs. 5 to 8. Fig. 5 is a diagram showing an example of a point cloud indicating an object. Fig. 6 is a schematic diagram for explaining an example of the object identification process. Fig. 7 is a diagram showing an example of a binary image of an object. Fig. 8 is a diagram showing an example of a feature pattern of an object. As shown in Fig. 5, the point cloud is composed of a plurality of detection points defined on the surface of an object. First, as shown in Fig. 6, the identification unit 25 estimates a plane constituting the object from the point cloud data. As shown in Fig. 6(a1), the identification unit 25 extracts a point cloud within a search range from the point cloud data to obtain a point cloud of an identification target. Then, as shown in Fig. 6(a2), the identification unit 25 projects the plane extraction point cloud onto the xy plane of a coordinate system in which the z axis is orthogonal to the ground plane. Fig. 6(a3) shows the point cloud projected onto the xy plane. The point cloud surrounded by a dashed line on the xy plane corresponds to the point cloud on the side of the dump truck that is orthogonal to the xy plane and surrounded by a dashed line. The point cloud on the plane perpendicular to the xy plane of the dump truck is aggregated on a single line segment to become dense. Then, as shown in FIG. 6(a4), the identification unit 25 converts the point cloud to the original three-dimensional position, leaving the part surrounded by the dashed line where the point cloud density is high on the xy plane. In this way, the identification unit 25 estimates the plane constituting the object by extracting the plane point cloud, which is the point cloud on the plane constituting the object. In this example, the plane point cloud is the point cloud on the side surface of the vessel.

[0044] As shown in Fig. 7, the identification unit 25 generates a binary image based on the extracted plane point cloud. The identification unit 25 generates a binary image by cropping the plane surrounding point cloud, which is a point cloud around the estimated plane, as a point cloud for generating a binary image. The identification unit 25 detects a shape similar to the characteristic pattern showing the vessel shape shown in Fig. 8 in the generated binary image. When a similar shape is found, the identification unit 25 identifies the overall shape of the dump truck based on the detected position and the vessel shape of the characteristic pattern.

[0045] The output control unit 29 controls the output of the identification result by the identification unit 25 via the output unit 19. For example, when an object is identified, the output control unit 29 outputs a video signal for displaying an image showing the identified object on a monitor. For example, when an object is identified, the output control unit 29 outputs a control signal for turning on a lamp. For example, when an object is identified, the output control unit 29 outputs a control signal for sounding a buzzer.

[0046] <Processing by the controller> When the target identification process is started during startup of the wheel loader 100, the process shown in Fig. 9 is executed. Fig. 9 is a flowchart showing an example of the process of the target identification method according to the first embodiment. The process of the flowchart shown in Fig. 9 is the target identification process. Note that the processes from step ST19 to step ST21 are not essential processes.

[0047] The controller 20 acquires a point cloud (step ST11). More specifically, the controller 20 receives the point cloud data from the three-dimensional sensor 11 via the sensor data acquisition unit 21. The controller 20 proceeds to step ST12.

[0048] The controller 20 crops the plane extraction point cloud (step ST12). More specifically, the controller 20 extracts, from the point cloud data, a point cloud included in a search range in which it is estimated that a target exists, as a plane extraction point cloud, using the identification unit 25. The controller 20 proceeds to step ST13.

[0049] The controller 20 downsamples the plane extraction point cloud (step ST13). More specifically, the controller 20 downsamples the plane extraction point cloud using voxel downsampling, which is an example of a point cloud data downsampling technique, by the discrimination unit 25. The controller 20 proceeds to step ST14.

[0050] The controller 20 extracts the plane point cloud and estimates the plane (step ST14). More specifically, the controller 20 extracts the plane point cloud, which is a point cloud on the vessel side surface, from the plane extraction point cloud downsampled by the identification unit 25, and estimates the vessel side surface, which is a plane. The controller 20 proceeds to step ST15.

[0051] The controller 20 crops the point group for generating a binary image (step ST15). More specifically, the controller 20 crops the plane surrounding point group, which is the point group around the plane estimated by the discrimination unit 25, as a point group for generating a binary image. The controller 20 proceeds to step ST16.

[0052] The controller 20 generates a binary image of the plane peripheral point group (step ST16). More specifically, the controller 20 uses the identification unit 25 to convert the cropped plane peripheral point group into a binary image on the yz plane of a coordinate system in which the normal vector of the estimated plane coincides with the direction of the x-axis. The yz plane is a plane that includes the estimated plane. The controller 20 proceeds to step ST17.

[0053] The controller 20 detects features of the target from the binary image (step ST17). More specifically, the controller 20 detects a shape similar to the feature pattern in the binary image by the discrimination unit 25. The controller 20 proceeds to step ST18.

[0054] The controller 20 restores the three-dimensional position (step ST18). More specifically, the controller 20 identifies the overall position, posture, and shape of the dump truck using the detection position in the binary image and the vessel shape of the feature pattern as references by the identification unit 25. The controller 20 proceeds to step ST19.

[0055] Controller 20 checks whether it is within the effective field of view of the sensor (step ST19). More specifically, controller 20 checks, by identification unit 25, whether the entire position of the dump truck is within the effective field of view of three-dimensional sensor 11. This is to check that the identification performance is guaranteed, since the identification performance decreases around the boundary of the field of view of three-dimensional sensor 11.

[0056] The controller 20 checks the consistency with the predicted value (step ST20). More specifically, the controller 20 compares the predicted position of the target predicted from the previous estimation result with the currently estimated position of the target by the discrimination unit 25, and checks whether the difference is equal to or smaller than a threshold value.

[0057] The controller 20 calculates the variance and likelihood of the point group classification (step ST21).

[0058] In this manner, the controller 20 identifies the vessel of the target dump truck from the point cloud data.

[0059] <Effects> As described above, in this embodiment, the plane of the object's outline is estimated from the point cloud data, a point cloud located on the estimated plane is extracted to generate a binary image, and the object is identified from the binary image. As a result, this embodiment can reduce the processing load compared to when the point cloud data is used directly in the identification process. Since the processing load is reduced in this embodiment, it is possible to suppress a decrease in identification performance, such as a delay in the identification process. In this way, according to this embodiment, the object can be appropriately identified from the point cloud data, which is three-dimensional data.

[0060] In this embodiment, the point cloud data is projected onto a projection plane, and a portion of the projection plane with high point cloud density is restored in three dimensions to estimate the target plane. According to this embodiment, noise in the point cloud data can be removed, and the plane can be estimated with high accuracy.

[0061] In this embodiment, a shape similar to a binary image of a pre-stored feature pattern is detected from the generated binary image to identify the object. According to this embodiment, the processing load can be reduced by using the binary image.

[0062] In this manner, according to this embodiment, the vessel of the dump truck can be appropriately identified.

[0063] [Second embodiment] The second embodiment will be described with reference to Figs. 10 to 17. Fig. 10 is a perspective view showing a work vehicle according to the second embodiment. In the second embodiment, the work vehicle 100 is a forklift, which is a type of loading and unloading vehicle. The processing in the identification unit 25 of the controller 20 differs from that in the first embodiment. Configurations similar to those in the first embodiment are given the same or corresponding reference symbols, and descriptions thereof will be omitted.

[0064] <Work vehicle> The forklift 100 performs loading and unloading operations. The loading and unloading operations include a loading and unloading operation for picking up transported goods placed at a specified storage position. In this embodiment, the object that the work vehicle identifies is the transported goods. The transported goods are loading platforms and containers for carrying luggage. The transported goods are, for example, pallets, skids, and containers. The transported goods have a pair of fork insertion holes. In this embodiment, the object that the work vehicle identifies is, for example, a box-shaped pallet.

[0065] In this embodiment, the feature pattern is a binary image of the fork insertion holes on the front of the pallet.

[0066] As shown in FIG. 10, the forklift 100 includes a vehicle body 102, a work machine 104 disposed in front of the vehicle body 102, wheels 105 supporting the vehicle body 102, a three-dimensional sensor 11, and a camera 12.

[0067] The vehicle body 102 includes a counterweight 120 and a fender 108. The counterweight 120 is disposed at the rear of the vehicle body 102. The counterweight 120 is attached to the rear of the vehicle body 102 in order to balance the weight of the forklift 100 in the front-to-rear direction when the forklift 100 picks up an object to be transported. The fender 108 is disposed at the front of the vehicle body 102. The fenders 108 are disposed on both the left and right sides of the vehicle body 102.

[0068] The work machine 104 performs at least a part of the loading and unloading work. The work machine 104 is disposed in front of the vehicle body 102. The work machine 104 is supported by the vehicle body 102. The work machine 104 has a mast 141, a bracket 142, and a fork 143.

[0069] The mast 141 is supported at the front of the vehicle body 102 so as to be able to tilt. The mast 141 is long in the vertical direction. The bracket 142 supports the fork 143. The bracket 142 is supported by the mast 141. The bracket 142 is able to move in the vertical direction along the mast 141. The fork 143 supports an object to be transported. The fork 143 is supported by the mast 141 via the bracket 142.

[0070] A pair of forks 143 are provided. The forks 143 include a first fork 143A and a second fork 143B disposed to the right of the first fork 143A. The bracket 142 supports the first fork 143A and the second fork 143B.

[0071] The wheels 105 support the vehicle body 102. At least a portion of the wheels 105 is disposed below the vehicle body 102. The wheels 105 have a front wheel 105F and a rear wheel 105R. The front wheel 105F is disposed forward of the rear wheel 105R. The front wheels 105F are disposed on the left and right sides of the vehicle body 102. The rear wheels 105R are disposed on the left and right sides of the vehicle body 102. Each of the front wheel 105F and the rear wheel 105R can rotate around a rotation axis. One or both of the front wheel 105F and the rear wheel 105R are steered wheels. In the embodiment, the rear wheel 105R is the steered wheel. Note that the front wheel 105F may be the steered wheel, or both the front wheel 105F and the rear wheel 105R may be steered wheels.

[0072] In the embodiment, the left-right direction is a direction parallel to the rotation axes of the front wheels 105F and the rear wheels 105R when the forklift 100 travels in a straight line. The up-down direction is a direction perpendicular to the ground contact surfaces of the front wheels 105F and the rear wheels 105R. The front-rear direction is a direction perpendicular to both the left-right direction and the up-down direction.

[0073] The fender 108 is positioned so as to cover at least a portion of the front wheel 105F. At least a portion of the fender 108 is positioned higher than the front wheel 105F. At least a portion of the fender 108 is positioned rearward of the front wheel 105F. The fenders 108 are positioned on both the left and right sides of the vehicle body 102. The left fender 108 is positioned so as to cover at least a portion of the left front wheel 105F. The right fender 108 is positioned so as to cover at least a portion of the right front wheel 105F.

[0074] 11 is a schematic diagram showing the operation of the work vehicle according to the second embodiment. In order to perform loading and unloading work, the forklift 100 identifies the pallet 300 to be unloaded. The forklift 100 identifies the pallet 300 based on the object detection data acquired from the three-dimensional sensor 11.

[0075] The classification process in the classification unit 25 will be described with reference to Figs. 12 to 16. Fig. 12 is a schematic diagram, a perspective view, for explaining another example of the object classification process. Fig. 13 is a schematic diagram, a plan view, for explaining another example of the object classification process. Fig. 14 is a diagram for explaining another example of the object classification process. Fig. 15 is a diagram showing another example of a binary image of the object. Fig. 15 is a diagram showing an example of a feature pattern of the object. The classification unit 25 estimates the front of the palette as a plane and generates a binary image of the front of the palette in the same manner as in the first embodiment.

[0076] The identification unit 25 moves the passing point group that has passed through the front of the pallet to the intersection with the front of the pallet on a straight line connecting each point with the three-dimensional sensor 11, and generates a binary image of the passing point group from the moved passing point group. As shown in Figs. 12 and 13, a point group that passes through the estimated pallet front and is located behind the estimated pallet front is called a plane passing point group. The identification unit 25 calculates the intersection of the straight line that connects each point of the plane passing point group that has passed through the estimated pallet front and the position of the three-dimensional sensor 11 with the pallet front. In Fig. 13, the estimated pallet front is shown by a dashed line. In Fig. 13, the point group appears above the estimated pallet front. The point group surrounded by an ellipse indicates the point group that has passed through the estimated pallet front. In other words, the point group surrounded by an ellipse is the point group that has passed through the fork insertion hole of the transported object. As shown in Fig. 14, the identification unit 25 moves the plane passing point group to the position of the calculated intersection. That is, the identification unit 25 moves the plane passing point cloud onto the front surface of the pallet. Fig. 14(a1) is a diagram showing the intersection between the estimated pallet front surface and a straight line connecting the point cloud that passed through the pallet front surface and the coordinate system reference of the three-dimensional sensor 11. Fig. 14(a2) is a diagram showing the point cloud supplemented at the position of the intersection found in (a1). In this way, the identification unit 25 calculates the position where each point of the plane passing point cloud passed through the plane.

[0077] The identification unit 25 generates a binary image from the plane passing point group as shown in Fig. 15. The identification unit 25 filters the generated binary image with the feature pattern of the fork insertion hole shown in Fig. 16 to calculate the similarity. The identification unit 25 searches for a pattern similar to the fork insertion hole pattern in the binary image. In the example shown in Fig. 16, area P11 is an area where the probability of the fork insertion hole being located is high. Area P12 is an area where the probability of the fork insertion hole being located is low. Area P13 is between areas P11 and P12 in terms of the probability of the fork insertion hole being located. C indicates the midpoint of the fork insertion hole.

[0078] 17 is a flowchart showing an example of the processing of the object identification method according to the second embodiment. Steps ST31 to ST36 and steps ST38 to ST42 are similar to steps ST11 to ST16 and steps ST17 to ST21 in the flowchart shown in FIG.

[0079] The controller 20 generates a binary image of the plane passing points (step ST37). More specifically, the controller 20 generates, by the discrimination unit 25, a binary image of the plane passing points that have passed through the estimated front surface of the pallet.

[0080] In this manner, the controller 20 identifies the fork insertion holes of the target pallet from the point cloud data.

[0081] <Effects> As described above, in this embodiment, the pass point group that passes through the estimated front surface of the pallet is moved to the intersection point with the estimated front surface of the pallet on the line connecting each point with the three-dimensional sensor 11, and a binary image of the pass point group is generated from the moved pass point group. In this embodiment, as described above, by projecting the point group onto a plane, the points on the plane are aggregated on the line segment to become high density. In this embodiment, noise is removed by increasing the density. According to this embodiment, even if the distance between the three-dimensional sensor 11 and the target is far and the point cloud density is sparse, the fork insertion hole can be identified with high accuracy.

[0082] In contrast, in the conventional technology, as the distance between 3D sensor 11 and the target increases, the point cloud density becomes sparse, making it difficult to identify the position of the fork insertion hole from the point cloud.

[0083] In this manner, according to this embodiment, the fork insertion holes of the pallet can be properly identified. [Explanation of symbols]

[0084] 1...identification system, 11...3D sensor (object detection sensor), 19...output section, 20...controller, 21...sensor data acquisition section, 22...memory section, 25...identification section, 29...output control section, 100...wheel loader (work vehicle), 102...body, 102F...front body, 102R...rear body, 104...running gear, 104A...power source, 104B...steering gear, 106...wheel, 106F...front wheel, 106R...rear wheel, 109...joint mechanism, 110...working machine, 111...boom, 112...bucket, 112B...tip, 113...bell crank, 114...link, 115...boom cylinder, 116...bucket cylinder, 000...computer system, 1001...processor, 1002...main memory, 1003...storage, 1004...interface.

Claims

1. An identification system for identifying an object from objects present in the vicinity of a work vehicle, comprising: an object detection sensor attached to the work vehicle and detecting an object present around the work vehicle; A controller; Equipped with The controller: Acquire point cloud data, which is object detection data, from the object detection sensor; estimating a plane constituting at least a part of an outline of an object from the point cloud data; Extracting a point group located on the estimated plane to generate a binary image; identifying the object from the binary image; Identification system.

2. The controller: detecting, from the generated binary image, a shape similar to a binary image of a feature pattern indicating a shape of a plane portion of the pre-stored outline of the object, thereby identifying the object; The identification system of claim 1 .

3. The object is a vessel of a dump truck, the plane is a side surface of the vessel, The characteristic pattern indicates a shape of a side surface of the vessel. The identification system of claim 2 .

4. The object is a fork insertion hole of a transported object, the plane is a front surface of the conveyed object, The characteristic pattern indicates a shape of the fork insertion hole, The controller: a binary image of the passing point cloud is generated from the passing point cloud that has passed through the front surface of the transported object, the binary image being generated from the passing point cloud, the binary image being generated from the passing point cloud that has passed through the front surface of the transported object ... The identification system of claim 2 .

5. 1. A method for identifying an object from objects present in the vicinity of a work vehicle, comprising: Acquire point cloud data, which is object detection data, from an object detection sensor that is attached to the work vehicle and detects objects present around the work vehicle; estimating a plane constituting at least a part of an outline of an object from the point cloud data; Extracting a point group located on the estimated plane to generate a binary image; identifying the object from the binary image; How to identify:

6. A work vehicle, The car body and an object detection sensor attached to the vehicle body and detecting objects present around the work vehicle; A controller; Equipped with The controller: Acquire point cloud data, which is object detection data, from the object detection sensor; estimating a plane constituting at least a part of an outline of an object from the point cloud data; Extracting a point group located on the estimated plane to generate a binary image; identifying the object from the binary image; Work vehicle.

Images