Article detector, article detecting method, and industrial vehicle

Abstract

To provide an article detector, an article detecting method, and an industrial vehicle capable of easily controlling.SOLUTION: An information image generating part 103 generates an information image by converting a camera coordinate system to a palette coordinate system. An arithmetic unit 104 can easily acquire the positional and the posture deviations of an imaging unit 32 if paying attention to a front face 61a in an information image and can easily acquire the positional and the posture deviations of an industrial vehicle 150 based on the positional relationship of the camera coordinate system and the industrial vehicle coordinate system. The information image generating part 103 generates a first information image that a circumference image is projected on a horizontal projection face HF and generates a second information image that the circumference image is projected on a vertical projection face VF. As a result, the arithmetic unit 104 can accurately acquire the deviations of the position and the posture using both information images even if it is hard to detect the deviations of the position and the posture using only one information image.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an article detection device, an article detection method, and an industrial vehicle. [Background technology]

[0002] A known example of a conventional item detection device is the technology described in Patent Document 1. The item detection device described in Patent Document 1 is used to recognize the location of a pallet and the insertion position of the fork when removing and transporting multi-layered pallets using a forklift. This item detection device calculates the position and orientation of the front of the pallet by detecting the position of the pallet, which is the object of loading and unloading, from a characteristic part of the item whose relative relationship to the overall outline of the item is known. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 5-157518 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology of Patent Document 1 is effective when detecting the position of a pallet to be handled after approaching it from the front. However, in recent years, there has been a demand for observing the surroundings from a position away from the pallet, not just from the front, to detect the target pallet and calculate its position and orientation. By having an item detection device determine the position and orientation of the target item before the vehicle approaches the vicinity of the item to be handled, the industrial vehicle can approach the item on a trajectory that ensures smooth transportation. There has also been a demand for not only accurate control of such industrial vehicles, but also ease of doing so.

[0005] SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an article detection device, an article detection method, and an industrial vehicle that can be controlled accurately and easily. [Means for solving the problem]

[0006] An item detection device according to one embodiment of the present invention is an item detection device that detects items transported by an industrial vehicle, and is equipped with an image acquisition unit that acquires an ambient image of the area around the item detection device, an information image creation unit that creates an information image that converts information about the detection target portion of the item based on the ambient image to a state that makes it easier to recognize, and a calculation unit that calculates at least one of the position and attitude of the industrial vehicle based on the information image, wherein the item detection device sets a first coordinate system for the image acquisition unit, sets a second coordinate system at an arbitrary position on the industrial vehicle, and sets a third coordinate system at an arbitrary position on the item, the information image creation unit converts the first coordinate system into the third coordinate system, creates a first information image by projecting the ambient image onto a horizontal projection plane, and creates a second information image by projecting the ambient image onto a vertical projection plane, and the calculation unit calculates at least one of the position and attitude of the industrial vehicle relative to the item based on the positional relationship between the first coordinate system and the second coordinate system and the first information image and the second information image.

[0007] This item detection device includes an image acquisition unit that acquires a surrounding image captured around the item detection device, and an information image creation unit that creates an information image based on the surrounding image, converting information about the target part of the item into a more easily recognizable state. For example, depending on the distance and positional relationship between the item detection device and the item, it may be difficult to directly detect the item from the surrounding image showing the surroundings of the item detection device. In response to this, the information image creation unit can create an information image suitable for detecting the target part of the item based on the surrounding image captured around the item detection device. The item detection device also includes a calculation unit that calculates at least one of the position and orientation of the industrial vehicle based on the information image. In this way, by performing calculations using the information image suitable for detecting the target part of the item, the calculation unit can constantly calculate at least one of the position and orientation of the industrial vehicle relative to the item while the industrial vehicle is approaching the vicinity of the item. Here, the information image creation unit creates the information image by converting the first coordinate system into a third coordinate system. In the converted information image, if there is no deviation in the orientation or position of the photographing unit, the target part of the item is located at a specific position in the information image. On the other hand, if there is a deviation in the posture or position of the photographing unit, the detection target portion in the information image will be shifted from a specific position in the information image. Therefore, by focusing on the detection target portion in the information image, the calculation unit can easily obtain the deviation in the position or posture of the photographing unit, and based on the positional relationship between the first coordinate system and the second coordinate system, the deviation in the position or posture of the industrial vehicle can be easily obtained. In addition, the information image creation unit creates a first information image by projecting the surrounding image onto a horizontal projection plane, and creates a second information image by projecting the surrounding image onto a vertical projection plane. As a result, the calculation unit can accurately obtain the deviation in the position or posture of the industrial vehicle relative to an item that is difficult to detect in only one information image by using both information images. As a result, control can be performed accurately and easily.

[0008] The calculation unit may calculate the attitude and first position of the industrial vehicle based on the amount of deviation between the detection target portion in the first information image and the first reference position in the first information image, and may calculate the second position of the industrial vehicle based on the amount of deviation between the detection target portion in the second information image and the second reference position in the second information image. In this case, the position and attitude of the industrial vehicle can be easily calculated by knowing the amount of deviation between the detection target portion in each information image and the reference position.

[0009] The second coordinate system may be set at the turning center of the industrial vehicle, in which case the calculation unit can easily calculate the operation for updating the position and attitude of the industrial vehicle.

[0010] The article may be a pallet, the detection target portion may be the front surface of the pallet, and the calculation unit may acquire, in the first information image, the amount of deviation between the bottom edge of the front surface and a first reference position, and acquire, in the second information image, the amount of deviation between the center position in the width direction of the front surface and a second reference position. In this case, the information image creation unit can indicate the position of the detection target portion in the first information image and the second information image in a manner that makes its features easily appear.

[0011] The article detection device may further include an operation control unit that controls the industrial vehicle based on at least one of the information on the position and the attitude of the industrial vehicle calculated by the calculation unit. In this case, the operation control unit can control the industrial vehicle to perform a desired operation using the information on at least one of the position and the attitude of the industrial vehicle relative to the article.

[0012] An item detection method according to one embodiment of the present invention is an item detection method for detecting items transported by an industrial vehicle, and includes an image acquisition step for acquiring an ambient image of the area around the industrial vehicle, an information image creation step for creating an information image that converts information about the target portion of the item based on the ambient image to a state that makes it easier to recognize, and a calculation step for calculating at least one of the position and attitude of the industrial vehicle based on the information image, wherein a first coordinate system is set for the image acquisition unit, a second coordinate system is set at an arbitrary position on the industrial vehicle, and a third coordinate system is set at an arbitrary position on the item, and in the information image creation step, the first coordinate system is converted into the second coordinate system, a first information image is created by projecting the ambient image onto a horizontal projection plane, and a second information image is created by projecting the ambient image onto a vertical projection plane, and in the calculation step, at least one of the position and attitude of the industrial vehicle relative to the item is calculated based on the positional relationship between the first information image and the second information image and the first coordinate system and the second coordinate system.

[0013] According to this article detection method, it is possible to obtain the same functions and effects as the above-mentioned article detection device.

[0014] An industrial vehicle according to one aspect of the present invention includes the above-described item detection device.

[0015] This industrial vehicle can provide the same functions and effects as the above-mentioned article detection device. [Effects of the Invention]

[0016] According to the present invention, it is possible to provide an article detection device, an article detection method, and an industrial vehicle that can be easily controlled. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a side view of a forklift equipped with an object detection device according to an embodiment of the present invention. [Figure 2] 1 is a side view of an automatic conveyance device equipped with an object detection device according to an embodiment of the present invention. [Figure 3]FIG. 2 is a block diagram showing the object detection device shown in FIG. 1 and its related components. [Figure 4] FIG. 10 is a schematic plan view showing the operation of the industrial vehicle until it arrives in front of the pallet. [Figure 5] FIG. 2 is a schematic diagram for explaining each coordinate system. [Figure 6] FIG. 2 is a schematic diagram for explaining processing before the industrial vehicle travels. [Figure 7] FIG. 2 is a schematic diagram for explaining processing before the industrial vehicle travels. [Figure 8] FIG. 2 is a schematic diagram for explaining processing before the industrial vehicle travels. [Figure 9] FIG. 2 is a schematic diagram for explaining processing before the industrial vehicle travels. [Figure 10] FIG. 2 is a schematic diagram for explaining processing before the industrial vehicle travels. [Figure 11] 10 is a schematic diagram for explaining processing while the industrial vehicle is traveling, using a first information image projected onto a horizontal projection plane. FIG. [Figure 12] 10 is a schematic diagram for explaining processing while the industrial vehicle is traveling, using a first information image projected onto a horizontal projection plane. FIG. [Figure 13] 10 is a schematic diagram for explaining processing while the industrial vehicle is traveling, using a second information image projected onto a vertical projection plane. FIG. [Figure 14] 10 is a schematic diagram for explaining processing while the industrial vehicle is traveling, using a second information image projected onto a vertical projection plane. FIG. [Figure 15] FIG. 10 is a diagram showing how the position and attitude of the industrial vehicle are updated. [Figure 16] FIG. 10 is a diagram showing how the position and attitude of the industrial vehicle are updated. [Figure 17] FIG. 1 is a flow chart showing the contents of an article detection method. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0019] 1 and 2 are side views showing an example of an industrial vehicle equipped with an article detection device according to an embodiment of the present invention. In the following description, the terms "left" and "right" are used, but these terms correspond to the "left" and "right" when viewed from the rear of the industrial vehicle. The article detection device 100 according to this embodiment is applied to an industrial vehicle 150. An industrial vehicle such as a forklift 50 may be used as the industrial vehicle 150. FIG. 1 illustrates a warehouse reach forklift as the forklift 50. However, the type of forklift 50 is not limited, and industrial vehicles such as counterbalance forklifts and forklifts that can load and unload items onto shelves without turning the vehicle body may also be used. Furthermore, an automatic transport device 151 as shown in FIG. 2 may be used as the industrial vehicle 150. The automatic transport device 151 is an industrial vehicle that transports items in an automated warehouse or the like. These industrial vehicles 150 transport items such as pallets and cardboard boxes.

[0020] In the example shown in FIG. 1, a forklift 50 is illustrated as an industrial vehicle for loading and unloading goods. The forklift 50 includes a vehicle body 51, a photographing unit 32, and an item detection device 100. The forklift 50 includes a moving unit 2 and a loading device 3. The forklift 50 in this embodiment is a reach-type forklift that can be switched between manual driving by a driver sitting in a driver's seat 12 and automatic driving by a control unit 110, which will be described later. Alternatively, the forklift 50 may be capable of fully automatic driving by the control unit 110.

[0021] The moving unit 2 has a pair of left and right reach legs 4 extending forward. Left and right front wheels 5 are rotatably supported on the left and right reach legs 4, respectively. The rear wheel 6 is the only rear wheel and is a drive wheel that also serves as a steering wheel. The rear of the moving unit 2 is a standing-type driver's seat 12. An instrument panel 9 in front of the driver's seat 12 is provided with a loading lever 10 for loading and unloading operations and an accelerator lever 11 for forward and reverse operation. A steering wheel 13 is also provided on the top surface of the instrument panel 9.

[0022] The cargo handling device 3 is provided in front of the moving section 2. When the reach lever of the cargo handling levers 10 is operated, a reach cylinder (not shown) is driven to extend and retract, causing the cargo handling device 3 to move forward and backward along the reach leg 4 within a predetermined stroke range. The cargo handling device 3 also includes a two-stage mast 23, a lift cylinder 24, a tilt cylinder (not shown), and a fork 25. When the lift lever of the cargo handling levers 10 is operated, the lift cylinder 24 is driven to extend and retract, causing the mast 23 to slide and retract in the vertical direction, and the fork 25 rises and falls in conjunction with this.

[0023] The automatic guided vehicle 151 according to the example shown in FIG. 2 includes a vehicle body 152, a photographing unit 32, and an item detection device 100. The vehicle body 152 moves with an item to be transported placed on its upper surface. The vehicle body 152 has drive wheels 153. The vehicle body 152 has a pair of drive wheels 153 on the left and right, and a non-driven front wheel 155. The photographing unit 32 is provided on the front side of the vehicle body 152. The photographing unit 32 is supported by a support member 154 extending upward from the vehicle body 152. This allows the photographing unit 32 to photograph the surroundings of the automatic guided vehicle 151 from a position higher than the vehicle body 152. The item detection device 100 is provided on the vehicle body 152.

[0024] Next, the item detection device 100 according to this embodiment will be described in more detail with reference to FIG. 3. Here, the item detection device 100 provided on a forklift 50 will be described. FIG. 3 is a block diagram showing the item detection device 100 according to this embodiment and its related components. As shown in FIG. 3, the item detection device 100 includes a control unit 110. The control unit 110 of the item detection device 100 is connected to a cargo handling drive system 30 and a traveling drive system 31, and transmits control signals to these. The cargo handling drive system 30 is a drive system that generates a drive force for operating the cargo handling device 3. The traveling drive system 31 is a drive system that generates a drive force for traveling the moving unit 2.

[0025] The control unit 110 is connected to the photographing unit 32 and acquires images photographed by the photographing unit 32. The photographing unit 32 photographs the surroundings of the body 51 of the forklift 50. In the example shown in FIG. 1, the photographing unit 32 is provided on the ceiling of the body 51, but may be provided anywhere as long as it is capable of photographing the surroundings of the body 51. The specific configuration of the photographing unit 32 will be described later. The control unit 110 is connected to the display unit 33 and outputs various image data to the display unit 33. Note that in the case of a forklift 50 that can be fully automatically driven by the control unit 110, the display unit 33, the cargo handling lever 10, the accelerator lever 11, and the steering wheel 13 may not be provided. When the item detection device 100 is employed in the automatic transport device 151, the cargo handling drive system 30 and the display unit 33 are not provided.

[0026] The article detection device 100 is a device that detects articles to be loaded and unloaded. A control unit 110 of the article detection device 100 controls automatic operation of the forklift 50. The control unit 110 detects the article before the forklift 50 approaches the article to be loaded and grasps the position and attitude of the industrial vehicle 150 relative to the loading target portion of the article.

[0027] FIG. 4 is a schematic plan view showing the operation of the industrial vehicle 150 moving to the position of the target pallet 61. The control unit 110 controls the travel of the industrial vehicle 150 so that it approaches the pallet 61 on the shelf 60. At this time, the control unit 110 controls the industrial vehicle 150 so that it approaches the pallet 61 from the side of the shelf 60 from the start position ST (see trajectory TL1). The control unit 110 controls the industrial vehicle 150 so that it turns in front of the pallet 61 (see trajectory TL2). At this time, the industrial vehicle 150 assumes a posture that faces the front surface 60a of the shelf 60 and the front surface 61a of the pallet 61 (the portion to be loaded). Then, the control unit 110 controls the industrial vehicle 150 to approach the front surface 61a of the pallet 61 on the shelf 60 in a straight line and stop at the goal position GL in front of the front surface 61a of the pallet 61. When the industrial vehicle 150 is a forklift 50, the control unit 110 controls the industrial vehicle 150 so that the position where the fork 25 is inserted into the hole in the front surface 61a of the pallet 61 is the goal position GL. To perform this control, the control unit 110 detects the pallet 61 before the industrial vehicle 150 turns, i.e., while the industrial vehicle 150 is traveling on the track TL1, and acquires the position and posture of the industrial vehicle 150 relative to the front surface 61a of the pallet 61. When multiple pallets 61 are arranged on the shelf 60, the control unit 110 determines in advance which pallet 61 will be removed in the current loading / unloading operation and detects the target pallet 61 from the shelf 60. The control unit 110 also detects the position and posture of the detected pallet 61. Based on the detection results, the control unit 110 controls the turning position and turning path of the industrial vehicle 150 so that the industrial vehicle 150 can smoothly approach the front surface 61a of the pallet 61. A similar operation is performed when the automated transport device 151 approaches the target pallet 61. When the industrial vehicle 150 is an automatic conveying device 151, the automatic conveying device 151 sets a goal position GL at a position opposite the front surface 61a of the target pallet 61. In this state, other cargo handling devices or the like transfer the cargo on the pallet 61 onto the automatic conveying device 151.

[0028] The control unit 110 includes an ECU (Electronic Control Unit) that manages the device in an integrated manner. The ECU is an electronic control unit that includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a CAN (Controller Area Network) communication circuit, and the like. The ECU, for example, loads programs stored in the ROM into the RAM and executes the programs loaded into the RAM with the CPU, thereby realizing various functions. The ECU may be composed of multiple electronic units. As shown in FIG. 2, the control unit 110 includes an image acquisition unit 101, a feature plane setting unit 102, an information image creation unit 103, a calculation unit 104, a driving control unit 106, and a storage unit 108.

[0029] The image acquisition unit 101 acquires surrounding images captured around the vehicle body 51 of the industrial vehicle 150. The image acquisition unit 101 acquires surrounding images captured by the photographing unit 32 in chronological order. The photographing unit 32 captures images at predetermined time intervals, capturing images from positions and postures that change from moment to moment while the industrial vehicle 150 is traveling. The industrial vehicle 150 approaches the pallet 61 over time. Therefore, the surrounding images may be images captured by a fisheye camera. That is, the photographing unit 32 may be configured with a fisheye camera. A fisheye camera is a camera that has a typical fisheye lens and can capture images with a wide field of view of approximately 180° using a single eye. Note that the lens of the camera constituting the photographing unit 32 is not limited to a fisheye lens. The photographing unit 32 only needs to have an angle of view large enough to capture images of the pallet 61 from both positions where the industrial vehicle 150 is distant from the pallet 61 and positions where it is close to the pallet 61. That is, the photographing unit 32 may be a wide-field camera that can simultaneously photograph the view in front of and to the side of the industrial vehicle 150. Note that the photographing unit 32 may be a wide-angle camera that can photograph images with a wide field of view. The photographing unit 32 may also be a combination of multiple cameras facing in multiple directions to photograph images with a wide field of view.

[0030] The feature plane setting unit 102 sets a feature plane SF onto which the features of the front surface 61a of the pallet 61 are projected. The feature plane SF is a planar projection surface that is virtually set in three-dimensional space to create an information image. The position and orientation of the feature plane SF are known at the time of setting. In this embodiment, the feature plane setting unit 102 sets horizontal and vertical planes with respect to the front surface 61a of the pallet 61 as the feature planes SF (details will be described later).

[0031] The information image creation unit 103 creates an information image by converting information about the front surface 61a of the pallet 61 based on the surrounding image so that it is easy to recognize. The information image creation unit 103 creates the information image using the feature plane SF. The surrounding image that can be directly acquired from the photographing unit 32 is an image in which the shelves 60 and pallets 61 are shown curved. Furthermore, the sizes of the shelves 60 and pallets 61 in the surrounding image vary depending on the distance from the photographing unit 32. Therefore, it is difficult to directly detect the pallet 61 from the surrounding image. Therefore, the information image creation unit 103 creates an information image that clearly shows information such as the shape and dimensional characteristics of the front surface 61a of the pallet 61 to be loaded, so that it is easy to detect. In this way, the state in which information about the front surface 61a of the pallet 61 is easy to recognize means that the computational load when obtaining this information from the information image by image recognition processing can be lighter than at least the computational load when obtaining this information from the surrounding image by image recognition processing. Furthermore, assuming that the computational load is the same, when the information is acquired from the information image, the information can be acquired more accurately than when it is acquired from the surrounding image. Note that, for creating the feature surface SF and the information image using it, a known method such as that described in JP 2023-124122 A may be used.

[0032] The calculation unit 104 calculates at least one of the position and attitude of the industrial vehicle 150 based on the information image. Here, the "position" and "attitude" of the industrial vehicle 150 include both the three-dimensional position and attitude relative to the pallet 61 at a certain point in time (the position and attitude in the coordinate system of the pallet 61) and the three-dimensional position and attitude of the industrial vehicle in the absolute coordinate system. In this embodiment, the case where the calculation unit 104 calculates the relative position and attitude will be described. Details of the calculation content of the calculation unit 104 will be described later.

[0033] The operation control unit 106 controls the industrial vehicle 150 based on at least one of the information on the position and attitude of the industrial vehicle 150 calculated by the calculation unit 104. For example, the operation control unit 106 controls the industrial vehicle 150 to travel along the trajectories TL1 and TL2 shown in FIG. 4 based on the position and attitude of the industrial vehicle 150 at each time. Here, the item detection device 100 can continue to grasp the current position of the traveling industrial vehicle 150 at each time through the calculations performed by the calculation unit 104. Accordingly, if the industrial vehicle 150 deviates from the trajectories TL1 and TL2, the operation control unit 106 controls the industrial vehicle 150 to eliminate the deviation. A known control method may be adopted to determine how the operation control unit 106 controls the industrial vehicle 150 based on the grasped current position of the industrial vehicle 150. Note that the operation control unit 106 may be configured as a control unit separate from the control unit 110 of the item detection device 100. In this case, the control unit 110 of the article detection device 100 outputs the calculation result to the control unit of the operation control unit 106, and the operation control unit 106 controls the operation of the article detection device 100 based on the calculation result.

[0034] Next, the item detection device 100 according to this embodiment will be described in further detail. Each coordinate system will be described with reference to Fig. 5. Note that in Fig. 5, in order to make each coordinate system easier to see, the description will be given for a case in which an automatic transport device 151 is used as the industrial vehicle 150. The item detection device 100 sets a camera coordinate system X1·Y1·Z1 (first coordinate system) for the photographing unit 32, sets an industrial vehicle coordinate system X2·Y2·Z2 (second coordinate system) at an arbitrary position on the industrial vehicle 150, and sets a pallet coordinate system X3·Y3·Z3 (third coordinate system) at an arbitrary position on the pallet 61.

[0035] The origin of the camera coordinate system X1-Y1-Z1 is set at the center of the image capturing unit 32. The X1 axis is set in the optical axis direction of the image capturing unit 32. The Y1 axis is set in the lateral direction of the image capturing unit 32. The origin of the industrial vehicle coordinate system X2-Y2-Z2 is set at the center of rotation of the industrial vehicle 150. The center of rotation may be set at the center position of the pair of left and right drive wheels 153. The height position of the origin of the industrial vehicle coordinate system X2-Y2-Z2 is not particularly limited, and may be set at the position of the top surface of the vehicle body 152, for example. The X2 axis is set in the front-rear direction of the industrial vehicle 150. The Y2 axis is set in the lateral direction of the forklift 50. The origin of the pallet coordinate system X3-Y3-Z3 is set at the center of the front surface 61a. The X3 axis is set in the depth direction of the pallet 61. The Y3 axis is set in the lateral direction of the pallet 61. The Z1 axis, Z2 axis, and Z3 axis are set in the up-down direction. It should be noted that when the following control is performed, it is assumed that the XY planes are parallel to each other in both coordinate systems.

[0036] The item detection device 100 according to this embodiment measures the position of the pallet 61 in the camera coordinate system over a predetermined time period (several seconds) before the industrial vehicle 150 starts traveling. The item detection device 100 converts the relative position and orientation between the industrial vehicle 150 and the pallet 61 into the position and orientation of the industrial vehicle 150 in the pallet coordinate system X3·Y3·Z3. After the industrial vehicle 150 starts traveling, the item detection device 100 updates the position and orientation of the industrial vehicle 150 based on the amount of positional deviation of the image of the front surface 61a of the pallet 61 on the horizontal projection plane and vertical projection plane as the feature plane SF.

[0037] 6 to 8, the processing content of the item detection device 100 before the industrial vehicle 150 starts moving will be described. Note that in order to make it easier to see the relationship between the coordinate systems, FIGS. 6 to 8 illustrate a case where an automatic guided vehicle 151 is used as the industrial vehicle 150. However, the same explanation also applies to the case of a forklift 50. First, as shown in FIG. 6, the item detection device 100 detects the position and orientation of a pallet 61 in the camera coordinate system X1·Y1·Z1. In this case, the item detection device 100 may employ, as a method for detecting the position and orientation of the pallet 61, a known method as disclosed in Japanese Patent Application Laid-Open No. 2023-124122, a known method as disclosed in Non-Patent Document 1 "Kita, Y., Kita, N., et al. (2019). Detection and localization of pallets on shelves using a wide-angle camera. IEEE 19th International Conference on Advanced Robotics (ICAR).", or a known method as disclosed in Non-Patent Document 2 "Kita, Y., Kita, N., et al. (2022). Localization of pallets on shelves in a warehouse using a wide-angle camera. IEEE 17th International Conference on Advanced Motion Control (AMC)." As a result, the calculation unit 104 can obtain the positional relationship PG1 of the pallet coordinate system X3·Y3·Z3 relative to the origin of the camera coordinate system X1·Y1·Z1, and thereby obtain the orientation of the pallet 61. The positional relationship PG1 is expressed by values on three axes. FIG. 6 shows the attitude of the pallet 61 at a yaw angle θ1 of the X3 axis relative to the X1 axis.

[0038] Next, as shown in FIG. 7, the calculation unit 104 converts the position and orientation of the pallet 61 in the camera coordinate system X1·Y1·Z1 into the position and orientation of the photographing unit 32 in the pallet coordinate system X3·Y3·Z3. As a result, the item detection device 100 can obtain the positional relationship PG2 of the camera coordinate system X1·Y1·Z1 with respect to the pallet coordinate system X3·Y3·Z3, and can obtain the orientation of the photographing unit 32. The positional relationship PG2 is expressed by values on three axes. In FIG. 7, the orientation of the photographing unit 32 is shown by the yaw angle θ2 of the X1 axis with respect to the X3 axis. Note that the values of the positional relationship PG2 and the yaw angle θ2 have the same absolute values as the positional relationship PG1 and the yaw angle θ1, but have opposite signs.

[0039] Next, as shown in Fig. 8, the calculation unit 104 acquires the position and attitude of the industrial vehicle 150 in the pallet coordinate system X3·Y3·Z3 based on the relationship between the camera coordinate system X1·Y1·Z1 and the industrial vehicle coordinate system X2·Y2·Z2 acquired in advance. This allows the operation control unit 106 to generate appropriate travel control commands using an arbitrary control algorithm and start the industrial vehicle 150 traveling.

[0040] The method for pre-acquiring the relationship between the camera coordinate system X1·Y1·Z1 and the industrial vehicle coordinate system X2·Y2·Z2 is not particularly limited, and any known method may be employed. For example, the known method described in Non-Patent Document 3, "Kita, N., Kato, T. (2022). Image Measurement Method for Automatic Insertion of Forks into Inclined Pallet. IEEE 17th International Conference on Control, Automation, Robotics and Vision (ICARCV)," may be employed. That is, as shown in FIG. 9, a panel 40 may be placed on the body 152 of an industrial vehicle 150, and a calibration method such as that described in Non-Patent Document 3 may be employed using the panel 40. The calculation unit 104 aligns the center of the panel 40 with the origin of the industrial vehicle coordinate system X2·Y2·Z2, creates an information image by converting information about the orientation of the vertical and horizontal edges of the panel 40 into an easily recognizable state, and extracts each edge. This allows the calculation unit 104 to obtain the relationship between the camera coordinate system X1·Y1·Z1 and the industrial vehicle coordinate system X2·Y2·Z2 via the panel 40 in the information image.

[0041] As mentioned above, it is assumed that the XY planes are parallel to each other in both coordinate systems. In this embodiment, the industrial vehicle 150 travels on a flat floor, and the pallet 61 is placed on the floor. Therefore, the XY plane of the industrial vehicle coordinate system X2-Y2-Z2 and the XY plane of the pallet coordinate system X3-Y3-Z3 are parallel. Meanwhile, the image capture unit 32 is supported by a support member 154. A special jig may be required to fix the support member 154 vertically. Without a jig, the image capture unit 32 will tilt along with the support member 154. Therefore, as shown in FIG. 10(a), the XY plane may not be parallel to the floor. Therefore, as shown in FIG. 10(b), the item detection device 100 converts the surrounding image captured by the image capture unit 32 into a panoramic image with the Z2 axis of the industrial vehicle coordinate system X2-Y2-Z2 as the central axis, and uses this as the input image. This allows the XY plane of the camera coordinate system X1·Y1·Z1 to be parallel to the XY plane of the industrial vehicle coordinate system X2·Y2·Z2 and the pallet coordinate system X3·Y3·Z3.

[0042] The attitude of the industrial vehicle 150 relative to the pallet 61 at the start position ST (see FIG. 4) of the industrial vehicle 150 is not particularly limited. For example, as shown by the two-dot chain line in FIG. 4, the industrial vehicle 150 may start traveling from a state in which its attitude is misaligned (inclined) with respect to the Y3 axis direction of the pallet 61. In this case, the industrial vehicle 150 may turn on the spot to assume an attitude parallel to the Y3 axis as shown by the solid line in FIG. 4, and then proceed straight ahead. However, at the start position ST, the industrial vehicle 150 may be adjusted from its initial state to the state shown by the solid line.

[0043] Next, the processing contents of the item detection device 100 when the industrial vehicle 150 is traveling will be described. The information image creation unit 103 creates a first information image by projecting a surrounding image onto a horizontal projection plane HF (see FIG. 11) fixed to the pallet coordinate system X3·Y3·Z3. The information image creation unit 103 also creates a second information image by projecting the surrounding image onto a vertical projection plane VF (see FIG. 13) fixed to the pallet coordinate system X3·Y3·Z3. The calculation unit 104 also calculates at least one of the position and posture of the industrial vehicle 150 based on the first information image and the second information image. The processing by the calculation unit 104 may be repeated at a predetermined time cycle from when the industrial vehicle 150 makes a turn on the spot at the start position ST until it reaches the goal position GL.

[0044] The processing details when using the horizontal projection plane HF will be described with reference to Figures 11 and 12. As shown in Figure 11(a), the feature plane setting unit 102 sets the horizontal projection plane HF obtained by translating the XY plane of the pallet coordinate system X3-Y3-Z3 to the height of the bottom surface of the pallet 61 as the feature plane SF. The information image creation unit 103 creates a first information image by projecting the surrounding image captured by the imaging unit 32 onto the horizontal projection plane HF using a known method. The first information image may be, for example, 500 pixels x 500 pixels in size, with each pixel measuring 4 mm on a side.

[0045] The calculation unit 104 calculates the attitude and first position of the image capturing unit 32 (i.e., the attitude and first position of the industrial vehicle 150) based on the amount of deviation between the front surface 61a in the first information image and a first reference position SL1 (see FIG. 12) in the first information image. Here, the first position is a position in the X and Y directions other than the component parallel to the Y3 axis. The calculation unit 104 acquires the amount of deviation between the bottom end 61ax of the front surface 61a and the first reference position SL1 in the first information image. As shown in FIG. 11(b), in the first information image, the image of the bottom end 61ax of the front surface 61a of the pallet 61 is a straight line extending long in the vertical direction. Therefore, the calculation unit 104 detects this by known image processing (for example, extracting edge points and performing a Hough transform).

[0046] Here, as shown in palette 61V in Figure 12, when the position and posture of the photographing unit 32 (i.e., the position and posture of the industrial vehicle 150) does not move, the information image creation unit 103 aligns the feature line FL1 with the vertical first reference position SL1 extending at the center in the left-right direction of the first information image.

[0047] Here, after the industrial vehicle 150 starts traveling, the photographing unit 32 moves from the position and posture at which it was photographed last time. For example, when the industrial vehicle 150 turns on the spot, the photographing unit 32 changes position as well as posture. Therefore, when the information image creation unit 103 creates a first information image in the same way as last time, using the surrounding image of the photographing unit 32 that has moved since last time, the feature line FL1 not only tilts with respect to the first reference position SL1, but also shifts left and right with respect to the first reference position SL1, as shown in FIG. 11(c).

[0048] FIG. 15(a) shows the positional relationship recognized by the calculation unit 104 before updating the position and orientation of the image capturing unit 32 and the industrial vehicle 150. As described above, because the image capturing unit 32 has moved from the position and orientation in the previous image capturing, a deviation occurs in the position and orientation of the pallet 61 in the information image based on the surrounding image captured from that position. In FIG. 15(a), the position of the industrial vehicle 150 recognized by the calculation unit 104 is designated as the recognized position RPG1. For example, if the feature line FL1 is deviated from the first reference position SL1 by an angle θ3 (see FIG. 11(c)), the calculation unit 104 rotates the image capturing unit 32 by minus θ3 around the origin of the pallet coordinate system X3·Y3·Z3, thereby updating the position and orientation of the image capturing unit 32 and the industrial vehicle 150 (see FIG. 15(b)). After the update in FIG. 15(b), the position of the industrial vehicle 150 recognized by the calculation unit 104 is designated as the recognized position RPG2. Note that the calculation unit 104 rotating or moving the image capture unit 32 or the industrial vehicle 150 does not mean moving the actual image capture unit 32 or the industrial vehicle 150, but rather means changing and rewriting the position information of the image capture unit 32 or the industrial vehicle 150 recognized by the calculation unit 104. Next, if the feature line FL1 is shifted from the first reference position SL1 in the X3-axis direction by a shift amount ST1 (see FIGS. 11(c) and 15(b)), the calculation unit 104 updates the positions of the image capture unit 32 and the industrial vehicle 150 by moving the industrial vehicle 150 in the X3-axis direction (see FIG. 16(a)). After the update in FIG. 16(a), the position of the industrial vehicle 150 recognized by the calculation unit 104 is set to the recognized position RPG3. Note that if the shift amount ST1 occupies a pixels in the first information image and one pixel on the horizontal projection plane HF represents 4 mm, the shift amount ST1 is expressed as "4 x a mm."

[0049] Note that movement of the feature line FL1 in the image in the X3-axis direction also occurs due to changes in the height of the image capture unit 32. Therefore, a condition for performing the above update is that the horizontal projection plane HF is set to the correct height. For example, this condition is met when the industrial vehicle 150 is moving on a plane parallel to the bottom surface of the pallet 61. Alternatively, if the vertical movement amount of the image capture unit 32 can be determined by any method, the horizontal projection plane HF can be set to a height that takes this vertical movement amount into account, thereby meeting the condition.

[0050] 11(b), the feature line FL1 overlaps with the first reference position SL1 in the first information image. However, in the first image using the horizontal projection plane HF, the feature line FL1 extends in the Y3-axis direction, and therefore, information corresponding to the deviation amount ST2 of the imaging unit 32 in the Y3-axis direction cannot be obtained.

[0051] Therefore, the calculation unit 104 calculates the deviation of the photographing unit 32 and the industrial vehicle 150 in the Y3 axis direction using a second information image in which the front surface 61a of the pallet 61 is set as a vertical projection plane VF.

[0052] The processing details when using the vertical projection plane VF will be described with reference to Figures 13 and 14. As shown in Figure 13(a), the feature plane setting unit 102 sets the vertical projection plane VF, which is the front surface 61a of the pallet 61 corresponding to the YZ plane of the pallet coordinate system X3-Y3-Z3, as the feature plane SF. The information image creation unit 103 creates a second information image by projecting the surrounding image captured by the imaging unit 32 onto the vertical projection plane VF using a known method.

[0053] The calculation unit 104 calculates the attitude and first position of the photographing unit 32 (i.e., the attitude and position of the industrial vehicle 150 relative to the pallet 61) based on the amount of deviation between the front surface 61a in the first information image and the first reference position SL1 (see FIG. 12) in the first information image. The calculation unit 104 calculates the positions of the photographing unit 32 and the industrial vehicle 150 in the Y3-axis direction based on the amount of deviation between the front surface 61a of the pallet 61 in the second information image and the second reference position SL2 in the second information image. As shown in FIG. 13(b), in the second information image, the calculation unit 104 can extract each edge portion of the front surface 61a of the pallet 61 by a known method. Therefore, the calculation unit 104 can obtain a feature line FL2 at the center position of the front surface 61a in the width direction (Y3-axis direction). The calculation unit 104 may obtain the feature line FL2 by template matching using a template stored in the storage unit 108.

[0054] Here, as shown in the palette 61V in FIG. 14(a), if there is no deviation in the position and posture of the photographing unit 32 (i.e., the position and posture of the industrial vehicle 150), the information image creation unit 103 aligns the feature line FL2 with the vertical second reference position SL2 extending at the center in the left-right direction of the second information image. On the other hand, if there is a deviation in the position of the photographing unit 32 (i.e., the position of the industrial vehicle 150) when creating the next second information image, the feature line FL2 will be deviated from the second reference position SL2, as shown in the palette 61R in FIG. 14(b). The calculation unit 104 can calculate the second position of the photographing unit 32 (i.e., the second position of the industrial vehicle 150) based on the deviation amount ST2 of the feature line FL2 from the second reference position SL2 in the second information image. The second position is a position parallel to the Y3 axis.

[0055] As described above, the first position and orientation of the photographing unit 32 and the industrial vehicle 150, other than the components parallel to the Y3 axis, are updated using the first information image using the horizontal projection plane HF (see FIG. 16(a)). For example, if the feature line FL2 is shifted from the second reference position SL2 in the Y3-axis direction by a shift amount ST2 (see FIGS. 14(b) and 16(a)), the calculation unit 104 updates the positions of the photographing unit 32 and the industrial vehicle 150 by moving the industrial vehicle 150 in the Y3-axis direction (see FIG. 16(b)). After the update in FIG. 16(b), the position of the industrial vehicle 150 recognized by the calculation unit 104 becomes the recognized position RPG4. This update eliminates the misalignment of the pallet 61 in the information image. This completes the update of the position and orientation of the photographing unit 32 and the industrial vehicle 150. Furthermore, the calculation unit 104 can grasp the current positions of the photographing unit 32 and the industrial vehicle 150 based on information about the angle θ3 and information about the deviation amounts ST1 and ST2. By repeating this process while the industrial vehicle 150 is traveling, the item detection device 100 can continue to grasp the current positions of the photographing unit 32 and the industrial vehicle 150 at all times.

[0056] Next, the processing content of the item detection device 100 according to this embodiment will be described with reference to Fig. 17. Fig. 17 is a flow diagram showing the processing content of the item detection device 100. The processing content shown in Fig. 17 is executed by the control unit 110 of the item detection device 100. However, the processing content shown in Fig. 17 is merely an example, and is not limited to this. Note that the processing content from when the industrial vehicle 150 reaches the goal position GL from the start position ST in Fig. 4 will be described here.

[0057] As shown in Fig. 17, the control unit 110 performs initial setting (step S10). In step S10, for example, the control unit 110 sets a camera coordinate system X1·Y1·Z1 (first coordinate system) of the photographing unit 32, sets an industrial vehicle coordinate system X2·Y2·Z2 (second coordinate system) at an arbitrary position on the industrial vehicle 150, and sets a pallet coordinate system X3·Y3·Z3 (third coordinate system) at an arbitrary position on the pallet 61. In addition, the control unit 110 performs the processes described above with reference to Figs. 6 to 10. After step S10 is completed, the control unit 110 starts the industrial vehicle 150 traveling (step S20).

[0058] Once traveling begins, the control unit 110 performs image processing (step S30). In step S30, the information image creation unit 103 creates a first information image by projecting the peripheral image onto a horizontal projection plane HF, calculates the position and attitude of the photographing unit 32 and the industrial vehicle 150 based on the information image of the first information image, and then creates a second information image by projecting the peripheral image onto a vertical projection plane VF. The calculation unit 104 also calculates the position and attitude of the photographing unit 32 and the industrial vehicle 150 based on the first information image and the second information image as described in FIGS. 11 to 14. The calculation unit 104 also updates the position and attitude of the photographing unit 32 and the industrial vehicle 150 as described in FIGS. 15 and 16.

[0059] After updating the position and attitude in step S30, the control unit 110 determines whether or not the industrial vehicle 150 has reached the goal position GL (step S60). If it is determined in step S60 that the industrial vehicle 150 has reached the goal position GL, the processing shown in Fig. 17 ends. On the other hand, if it is determined in step S60 that the industrial vehicle 150 has not reached the goal position GL, the control unit 110 continues the travel of the industrial vehicle 150 (step S70), and repeats the processing again in a predetermined cycle from step S30.

[0060] Next, the actions and effects of the article detection device 100 and the industrial vehicle 150 according to this embodiment will be described.

[0061] The item detection device 100 according to this embodiment includes an image acquisition unit 101 that acquires a surrounding image captured around the item detection device 100, and an information image creation unit 103 that creates an information image by converting information about the front surface 61a of the pallet 61 based on the surrounding image to make it easier to recognize. For example, depending on the distance and positional relationship between the item detection device 100 and the pallet 61, it may be difficult to directly detect an item from an image showing the surroundings of the item detection device 100. It may be difficult to directly identify the pallet 61 from the surrounding image. In response to this, the information image creation unit 103 can create an information image suitable for detecting the front surface 61a of the pallet 61 based on the surrounding image captured around the item detection device 100. Specifically, in the information image, the front surface 61a of the pallet 61 is shown as if photographed from a specific direction and at a constant size regardless of the photographing position. The item detection device 100 also includes a calculation unit 104 that calculates the position and attitude of the industrial vehicle 150 based on the information image. In this way, the calculation unit 104 performs calculations using an information image suitable for detecting the front surface 61a of the pallet 61, and can constantly calculate the position and attitude of the industrial vehicle 150 relative to the front surface 61a while the article detection device 100 approaches the vicinity of the pallet 61. Therefore, the article to be loaded can be detected regardless of its positional relationship with the article.

[0062] Here, the information image creation unit 103 creates an information image by converting the camera coordinate system X1·Y1·Z1 (first coordinate system) into a pallet coordinate system X3·Y3·Z3 (third coordinate system). In the information image converted in this manner, if there is no deviation in the posture or position of the photographing unit 32, the front surface 61a (detection target portion) of the pallet 61 is positioned at a specific position in the information image (see, for example, FIGS. 12(a) and 14(a)). On the other hand, if there is a deviation in the posture or position of the photographing unit 32, the front surface 61a in the information image is deviated from the specific position in the information image (see, for example, FIGS. 12(b) and 14(b)). Therefore, by focusing on the front surface 61a in the information image, the calculation unit 104 can easily obtain the deviation in the position and posture of the photographing unit 32, and can easily obtain the deviation in the position and posture of the industrial vehicle 150 based on the positional relationship between the camera coordinate system X1·Y1·Z1 and the industrial vehicle coordinate system X2·Y2·Z2. Furthermore, the information image creation unit 103 creates a first information image by projecting the surrounding image onto a horizontal projection plane HF, calculates the position and attitude of the photographing unit 32 and the industrial vehicle 150 based on the information image of the first information image, and then creates a second information image by projecting the surrounding image onto a vertical projection plane VF. As a result, by using both information images, the calculation unit 104 can accurately obtain deviations in the position and attitude of the industrial vehicle 150 relative to the pallet 61, which are difficult to detect in just one of the information images. As a result, control can be performed accurately and easily.

[0063] The calculation unit 104 may calculate the attitude of the industrial vehicle 150 and a first position in a direction other than the component parallel to the Y3 axis based on the amount of deviation between the end 61ax of the front surface 61a in the first information image and the first reference position SL1 in the first information image, and may calculate a second position of the industrial vehicle 150 in the Y3 axis direction based on the amount of deviation between the front surface 61a in the second information image and the second reference position SL2 in the second information image. In this case, the position and attitude of the industrial vehicle 150 can be easily calculated by determining the amount of deviation between the front surface 61a in each information image and the reference positions SL1 and SL2.

[0064] The industrial vehicle coordinate system X2·Y2·Z2 may be set at the center of rotation of the industrial vehicle 150. In this case, the calculation unit 104 can easily calculate the operation for updating the position and attitude of the industrial vehicle 150.

[0065] The article is a pallet 61, the detection target portion is the front surface 61a of the pallet 61, and the calculation unit 104 may acquire, in the first information image, the amount of deviation between the bottom end 61ax (characteristic line FL1) of the front surface 61a and the first reference position SL1, and may acquire, in the second information image, the amount of deviation between the center position in the width direction of the front surface 61a (characteristic line FL2) and the second reference position SL2. In this case, the information image creation unit 103 can indicate the position of the detection target portion in the first information image and the second information image in a manner that makes the features easily appear.

[0066] The item detection device 100 may further include an operation control unit 106 that controls the industrial vehicle 150 based on information on at least one of the position and attitude of the industrial vehicle 150 calculated by the calculation unit 104. In this case, the operation control unit 106 can use the information on the position and attitude of the industrial vehicle 150 relative to the pallet 61 to control the industrial vehicle 150 to perform a desired operation.

[0067] The item detection device 100 can detect the position and orientation of the industrial vehicle 150 without using a world coordinate system, and can update the position and orientation of the industrial vehicle 150 at high speed (e.g., every 200 milliseconds) because it only needs to observe the image of the pallet 61 in the surrounding image. Furthermore, the item detection device 100 can be applied even in environments where it is difficult to set a world coordinate system. The position and orientation of the industrial vehicle 150 at the goal position is fixed in the pallet coordinate system (e.g., goal position GL in Figure 4). Furthermore, because the position and orientation of the industrial vehicle 150 in the pallet coordinate system can be obtained based on image measurement, the item detection device 100 can generate travel control commands directly from the measurement values.

[0068] The item detection method of this embodiment is an item detection method for detecting items transported by an industrial vehicle 150, and includes an image acquisition step for acquiring an ambient image of the area around the industrial vehicle 150, an information image creation step for creating an information image that converts information about the detection target portion of the item based on the ambient image to a state that makes it easier to recognize, and a calculation step for calculating at least one of the position and posture of the industrial vehicle 150 based on the information image, in which a first coordinate system is set for the image acquisition unit, a second coordinate system is set at an arbitrary position on the industrial vehicle 150, and a third coordinate system is set at an arbitrary position on the item, and in the information image creation step, the first coordinate system is converted into the second coordinate system, a first information image is created by projecting the ambient image onto a horizontal projection plane HF, and a second information image is created by projecting the ambient image onto a vertical projection plane VF, and in the calculation step, at least one of the position and posture is calculated based on the positional relationship between the first information image and the second information image and the first coordinate system and the second coordinate system.

[0069] According to this article detection method, it is possible to obtain the same functions and effects as those of the above-described article detection device 100.

[0070] The forklift 50 according to this embodiment is equipped with the above-described item detection device 100.

[0071] According to this forklift 50, it is possible to obtain the same functions and effects as those of the above-mentioned item detection device.

[0072] The present invention is not limited to the above-described embodiments.

[0073] For example, the operation of the industrial vehicle is not limited to the above-described operation and can be modified as appropriate. Furthermore, the block configuration of the item detection device is not limited to that shown in FIG. 3 and can be modified as appropriate. Furthermore, the above-described embodiment has been described using an example in which the industrial vehicle 150 is automatically driven, but the present invention may also be adopted in cases in which the driver manually drives the industrial vehicle 150. For example, the attitude and position information of the industrial vehicle 150 acquired by the item detection device 100 may be used for driving assistance.

[0074] [Form 1] An article detection device for detecting an article transported by an industrial vehicle, an image acquisition unit that acquires a surrounding image obtained by capturing an image of the surroundings of the item detection device; an information image creation unit that creates an information image by converting information about the detection target portion of the article into an easily recognizable state based on the surrounding image; a calculation unit that calculates at least one of a position and an attitude of the industrial vehicle based on the information image, the item detection device sets a first coordinate system for the image acquisition unit, sets a second coordinate system at an arbitrary position on the industrial vehicle, and sets a third coordinate system at an arbitrary position on the item; the information image creation unit transforms the first coordinate system into the third coordinate system, creates a first information image by projecting the surrounding image onto a horizontal projection plane, and creates a second information image by projecting the surrounding image onto a vertical projection plane; The calculation unit calculates at least one of the position and the attitude of the industrial vehicle relative to the item based on the positional relationship between the first coordinate system and the second coordinate system, and the first information image and the second information image. [Form 2] The calculation unit calculating an attitude and a first position of the industrial vehicle based on a deviation amount between the detection target portion in the first information image and a first reference position in the first information image; An item detection device as described in form 1, which calculates a second position of the industrial vehicle based on the amount of deviation between the detection target portion in the second information image and a second reference position in the second information image. [Form 3] 3. The object detection device according to claim 1, wherein the second coordinate system is set at a turning center of the industrial vehicle. [Form 4] the object is a pallet, the detection target portion is a front surface of the pallet, The calculation unit acquiring a deviation amount between a bottom edge of the front surface and the first reference position in the first information image; The object detection device according to aspect 2, further comprising: acquiring, in the second information image, an amount of deviation between a center position in the width direction of the front surface and the second reference position. [Form 5] An object detection device according to any one of the first to fourth aspects, further comprising an operation control unit that controls the industrial vehicle based on information on at least one of the position and attitude of the industrial vehicle calculated by the calculation unit. [Form 6] An article detection method for detecting an article transported by an industrial vehicle, comprising: an image acquisition step of acquiring a surrounding image of the surroundings of the industrial vehicle; an information image creation step of creating an information image by converting information about the detection target portion of the article into an easily recognizable state based on the surrounding image; a calculation step of calculating at least one of a position and an attitude of the industrial vehicle based on the information image, setting a first coordinate system of an image acquisition unit, setting a second coordinate system at an arbitrary position of the industrial vehicle, and setting a third coordinate system at an arbitrary position of the article; In the information image creation step, the first coordinate system is transformed into the second coordinate system, a first information image is created by projecting the detection target portion onto a horizontal projection plane, and a second information image is created by projecting the detection target portion onto a vertical projection plane; An object detection method, wherein in the calculation step, at least one of the position and the orientation is calculated based on the first information image and the second information image and the positional relationship between the first coordinate system and the second coordinate system. [Form 7] An industrial vehicle equipped with the article detection device according to any one of the first to fifth aspects. [Explanation of symbols]

[0075] 32...photography unit, 50...forklift (industrial vehicle), 61...pallet (item), 61a...front (area to be loaded), 100...item detection device, 101...image acquisition unit, 103...information image creation unit, 104...calculation unit, 106...operation control unit, 110...control unit, 151...automatic transport device (industrial vehicle).

Claims

1. An article detection device for detecting an article transported by an industrial vehicle, an image acquisition unit that acquires a surrounding image obtained by capturing an image of the surroundings of the item detection device; an information image creation unit that creates an information image by converting information about the detection target portion of the article into an easily recognizable state based on the surrounding image; a calculation unit that calculates at least one of a position and an attitude of the industrial vehicle based on the information image, the article detection device sets a first coordinate system of the image acquisition unit, sets a second coordinate system at an arbitrary position of the industrial vehicle, and sets a third coordinate system at an arbitrary position of the article; the information image creation unit transforms the first coordinate system into the third coordinate system, creates a first information image by projecting the surrounding image onto a horizontal projection plane, and creates a second information image by projecting the surrounding image onto a vertical projection plane; The calculation unit calculates at least one of the position and the attitude of the industrial vehicle relative to the item based on the positional relationship between the first coordinate system and the second coordinate system, and the first information image and the second information image.

2. The calculation unit calculating a posture and a first position of the industrial vehicle based on a deviation amount between the detection target portion in the first information image and a first reference position in the first information image; 2. The object detection device according to claim 1, wherein the second position of the industrial vehicle is calculated based on the amount of deviation between the detection target portion in the second information image and a second reference position in the second information image.

3. The object detection device according to claim 1 , wherein the second coordinate system is set at a turning center of the industrial vehicle.

4. the object is a pallet, the detection target portion is a front surface of the pallet, The calculation unit acquiring a deviation amount between a bottom edge of the front surface and the first reference position in the first information image; The object detection device according to claim 2 , further comprising: acquiring an amount of deviation between a center position in the width direction of the front surface and the second reference position in the second information image.

5. The article detection device according to claim 1 , further comprising an operation control unit that controls the industrial vehicle based on information on at least one of the position and the attitude of the industrial vehicle calculated by the calculation unit.

6. An article detection method for detecting an article transported by an industrial vehicle, comprising: an image acquisition step of acquiring a surrounding image of the surroundings of the industrial vehicle; an information image creation step of creating an information image by converting information about the detection target portion of the article into an easily recognizable state based on the surrounding image; a calculation step of calculating at least one of a position and an attitude of the industrial vehicle based on the information image, setting a first coordinate system of an image acquisition unit, setting a second coordinate system at an arbitrary position of the industrial vehicle, and setting a third coordinate system at an arbitrary position of the article; In the information image creation step, the first coordinate system is transformed into the second coordinate system, a first information image is created by projecting the surrounding image onto a horizontal projection plane, and a second information image is created by projecting the surrounding image onto a vertical projection plane; In the calculation step, at least one of the position and the attitude of the industrial vehicle relative to the item is calculated based on the first information image, the second information image, and the positional relationship between the first coordinate system and the second coordinate system.

7. An industrial vehicle equipped with the article detection device according to any one of claims 1 to 5.

Images