Image generation device, object identification device, and image generation method

Abstract

To provide an image generation system capable of generating an image, which makes it possible to appropriately discriminate an object, despite a limited arithmetic quantity.SOLUTION: An image generation system generates an image which makes it possible to discriminate a predetermined object of detection. The image generation system includes an input unit and a processor. The input unit inputs three-dimensional point group data generated when a sensor detects the surroundings. The processor projects the three-dimensional point group data, which is inputted to the input unit, on a reference plane oriented according to the object of detection, and thus generates a two-dimensional image.SELECTED DRAWING: Figure 3

Description

【Technical Field】 【0001】 This application mainly relates to an image generation device that generates an image for identifying a detection target object. 【Background Art】 【0002】 Patent Document 1 discloses an object detection device that detects a circular object from three-dimensional point cloud data obtained using a laser scanner. The object detection device generates a projection pattern by projecting the three-dimensional point cloud data onto a horizontal plane. The object detection device detects a circular object based on the projection pattern. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 Japanese Patent No. 5464915 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 In Patent Document 1, only the projection of three-dimensional point cloud data onto a horizontal plane is described. Therefore, depending on the shape or orientation of the object, it may be difficult to identify the object from the projection pattern. On the other hand, when directly using the three-dimensional point cloud data to identify the object, the amount of calculation becomes enormous. 【0005】 This application has been made in view of the above circumstances, and its main object is to provide an image generation device that can accurately generate an image for identifying an object with a small amount of calculation. 【Means for Solving the Problems】 【0006】 The problems to be solved by this application are as described above. Next, the means for solving this problem and its effects will be described. 【0007】 According to a first aspect of this application, an image generation device is provided with the following configuration: that is, it generates an image for identifying a predetermined object to be detected. The image generation device comprises an input unit and a processing unit. The input unit receives 3D point cloud data generated by a sensor detecting its surroundings. The processing unit receives the 3D point cloud data input to the input unit. Based Generate a 2D image. The object to be detected is positioned along the detection surface. The processing device determines the position of the detection surface based on the three-dimensional point cloud data input from the sensor. The processing device uses a plane that is the same plane as the detection surface, or a plane parallel to the detection surface, and oriented according to the object to be detected, as a reference plane, and projects the three-dimensional point cloud data onto the reference plane to generate the two-dimensional image. do 。 【0008】 According to a second aspect of this application, an object identification device having the following configuration is provided. That is, the object identification device is The image generation device and, Sensors and equipped The processing device identifies the object to be detected by analyzing the two-dimensional image. 【0009】 According to a third aspect of this application, the following image generation method is provided. That is, the image generation method generates an image for identifying a predetermined object to be detected. The object to be detected is positioned along the detection surface. In the image generation method, 3D point cloud data is generated by detecting the surroundings. The position of the detection surface is determined based on the three-dimensional point cloud data. In image generation methods, A plane identical to the detection surface, or a plane parallel to the detection surface, and The orientation determined according to the object to be detected It is a plane. On the reference plane, The aforementioned 3D point cloud data A 2D image is generated by performing a projection process. [Effects of the Invention] 【0010】 According to this application, it is possible to generate images that can accurately identify objects with a small amount of computation. [Brief explanation of the drawing] 【0011】 [Figure 1] A schematic diagram showing how an AUV inspects a pipeline. [Figure 2] Block diagram of an AUV. [Figure 3] A flowchart illustrating the process of identifying pipelines. [Figure 4] This figure shows a distance image generated by projecting 3D point cloud data onto the same reference plane as the seabed. [Figure 5] Figure showing a binarized image generated based on a distance image. [Figure 6] Figure showing a contour image generated based on the binarized image. [Figure 7] Figure showing a straight line parallel to the longitudinal direction of the pipeline. [Figure 8] Flowchart showing the process of identifying a pipeline and an obstacle. [Figure 9] Figure showing a distance image generated by projecting 3D point cloud data onto a reference plane perpendicular to the front-rear direction. 【Mode for Carrying Out the Invention】 【0012】 Next, embodiments of the present application will be described with reference to the drawings. First, the outline of the AUV 10 will be described with reference to FIGS. 1 and 2. 【0013】 The AUV 10 is an abbreviation for autonomous underwater vehicle. The AUV 10 is an underwater vehicle that autonomously navigates underwater without being operated by a human. The AUV 10 detects the pipeline 100 disposed on the seabed 101 and inspects the pipeline 100. The pipeline 100 is an example of a long object. As shown in FIG. 1, the AUV 10 includes a main body 11, a propulsion device 12, a robotic arm 13, and an inspection tool 14. 【0014】 The main body 11 is provided with a battery and a motor. The propulsion device 12 is, for example, a screw propeller and a rudder. By rotating the screw propeller by the power generated by the motor, the propulsion device 12 generates a propulsion force. Thereby, the AUV 10 can be propelled. Also, by operating the rudder, the AUV 10 can be turned. 【0015】 The robotic arm 13 is provided at the lower part of the main body 11. The robotic arm 13 has a plurality of joints. An actuator is provided at each joint. By driving the actuator, the posture of the robotic arm 13 can be changed. The inspection tool 14 is attached to the tip of the robotic arm 13. The inspection tool 14 has equipment for inspecting the pipeline 100. For example, the equipment that the inspection tool 14 has may be a camera for photographing the pipeline 100, or an inspector for inspecting the degree of deterioration of the anticorrosion treatment of the pipeline 100. 【0016】 As shown in FIGS. 1 and 2, the AUV 10 further includes a control device 15 and an object identification device 20. 【0017】 The control device 15 is a computer including an arithmetic device such as a CPU and a storage device such as an SSD or a flash memory. The control device 15 controls the propulsion device 12 based on the detection result of the pipeline 100. Thereby, the AUV 10 can be navigated along the area where the pipeline 100 is arranged. In addition, the control device 15 further controls the robotic arm 13. Specifically, the control device 15 controls the robotic arm 13 so that the distance between the inspection tool 14 and the pipeline 100 satisfies a predetermined range. 【0018】 The object identification device 20 is a device that detects and identifies an object. In the present embodiment, the pipeline 100 is the detection target. The pipeline 100 is arranged along the seabed 101 which is the detection surface. "Arranged along the seabed 101" means that the pipeline 100 is arranged so as to contact the seabed 101, or the pipeline 100 is arranged at a position where the distance from the seabed 101 is kept constant. Also, the detection target is predetermined. "Predetermined" means that the type or name of the detection target is specified, and further, the shape or approximate position of the detection target is specified in advance. As shown in FIG. 2, the object identification device 2 includes a sensor 21 and an image generation device 22. 【0019】 Sensor 21 is located at the front and bottom of the main body 11. Sensor 21 is a sonar. Sensor 21 transmits sound waves in a planar direction and receives reflected waves to determine the distance to objects in each direction and create 2D point cloud data. Hereinafter, the act of transmitting and receiving sound waves once or in a series of times to detect the surroundings will be referred to as scanning. By performing one scan with sensor 21, 2D point cloud data indicating the positions of objects around AUV10 is obtained. As AUV10 moves, sensor 21 performs multiple scans to obtain 3D point cloud data. The coordinate system of the 3D point cloud data is, for example, the coordinate system of AUV10. The coordinate system of AUV10 is a coordinate system consisting of the front-to-back axis, left-to-right axis, and up-and-down axis, with the orientation of AUV10 as the reference. 【0020】 The image generation device 22 generates an image for identifying the pipeline 100, which is the object to be detected, based on the 3D point cloud data generated by the sensor 21. The image generation device 22 comprises an input unit 22a and a processing unit 22b. The input unit 22a receives the 3D point cloud data generated by the sensor 21. Specifically, the input unit 22a is a signal processing module that receives the 3D point cloud data from the sensor 21 and performs signal processing such as amplification. The input unit 22a may also be a communication module that performs wired or wireless communication with the sensor 21. The processing unit 22b is a computer comprising an arithmetic unit such as a CPU and a storage device such as an SSD or flash memory. The processing unit 22b performs processing such as generating an image for identifying the pipeline 100 by having the arithmetic unit execute a program that has been pre-stored in the storage device. 【0021】 Next, with reference to Figures 3 to 7, the details of the processing performed by the processing unit 22b will be explained. 【0022】 First, the processing unit 22b acquires 3D point cloud data input from the sensor 21 to the input unit 22a (S101). Next, the processing unit 22b analyzes the 3D point cloud data to determine the position of the seabed and determines a reference plane based on the position of the seabed (S102). If there is no object between the sensor 21 and the seabed, the sensor 21 receives reflected waves reflected from the seabed. Therefore, the 3D point cloud data includes a large number of points indicating the seabed. Since the points indicating the seabed form a large surface, the processing unit 22b can identify the points indicating the seabed from the 3D point cloud data. This allows the processing unit 22b to determine the position of the seabed. In this embodiment, the processing unit 22b determines the seabed surface as the reference plane. However, the reference plane may be another surface parallel to the seabed surface, or a surface parallel to the front-to-back axis and left-to-right axis of the coordinate system of the AUV 10. 【0023】 Next, the processing unit 22b projects the 3D point cloud data onto a reference plane to generate a 2D image (S103, Figure 4). At this time, the processing unit 22b varies the brightness of each pixel in the 2D image according to its position in a direction perpendicular to the reference plane. Therefore, the 2D image generated by the processing unit 22b is a depth image. Specifically, the higher an object is located from the seabed, which is the reference plane, the brighter the brightness of each pixel in the 2D image. Therefore, the location where the pipeline 100 exists has a higher brightness compared to the seabed. Since the brightness of the 2D image has three or more levels, the 2D image is a grayscale image, not a binarized image. Note that the brightness of each pixel may be decreased as the distance from the reference plane increases. Alternatively, another drawing parameter, such as hue or saturation, may be varied instead of brightness. 【0024】 Next, the processing unit 22b binarizes the distance image to generate a binarized image (S104, Figure 5). A binarized image is an image drawn with two drawing parameters. In this embodiment, the binarized image is drawn with two colors: white and black. The processing unit 22b sets a threshold value according to the distance from the reference plane and changes the drawing parameters depending on whether the distance is greater than or equal to the threshold value. In this embodiment, the radius of the pipeline 100 is used as the threshold value. Therefore, pixels representing objects located higher than the radius of the pipeline 100 from the seabed are drawn in white, and pixels representing other objects are drawn in black. This makes it possible to clearly define the range in which the pipeline 100 exists. Note that the white and black values ​​may be reversed. Also, the method of defining the threshold value in this embodiment is just one example, and different values ​​may be used. By generating a binarized image, the location of the pipeline 100 becomes clear, and subsequent processing becomes easier. 【0025】 Next, the processing unit 22b generates a contour image from the binarized image (S105, Figure 6). The contour image is an image in which the contours of pixels representing the pipeline 100 are extracted. The process for extracting contours is well known, so it will be briefly explained. That is, the processing unit 22b divides the pixels drawn in white into boundary pixels adjacent to pixels drawn in black and excluded pixels that are not adjacent to pixels drawn in black. Then, it maintains the drawing of boundary pixels in white and converts the excluded pixels from white to black. In this way, an adjacent image can be generated. By generating a contour image, the shape of the pipeline 100 can be highlighted. 【0026】 Next, the processing unit 22b determines the longitudinal direction of the pipeline 100 from the contour image (S106). The process of determining the longitudinal direction from the contour image can be performed, for example, using the Hough transform. The Hough transform is a method for generating a figure that passes through the most feature points of the image. By using the Hough transform, a line segment passing through the contour of the pipeline 100 can be obtained. The direction of the line segment obtained by performing the Hough transform is determined to be the longitudinal direction of the pipeline 100. If multiple line segments are obtained by performing the Hough transform, the longitudinal direction of the pipeline 100 is determined using the average of the directions of each line segment, etc. Note that the longitudinal direction of the pipeline 100 may also be determined using a process other than the Hough transform. 【0027】 Next, the processing unit 22b identifies the center line of the pipeline 100 based on the distance image and the longitudinal direction (S107). Specifically, as shown in Figure 7, the processing unit 22b places a plurality of line segments parallel to the longitudinal direction determined in step S106 on the distance image. The distance between the line segments is preferably less than or equal to the diameter of the pipeline 100, and more preferably less than or equal to 1 / 2, 1 / 5, or 1 / 10 of the diameter of the pipeline 100. Naturally, the distance between the line segments is greater than 0. Next, the processing unit 22b calculates a sum of brightness values ​​by summing the brightness values ​​of the pixels located on each line segment. The processing unit 22b identifies the line segment with the largest sum of brightness values ​​as the center line of the pipeline 100. In this way, the processing unit 22b identifies the pipeline 100 from the data acquired by the sensor 21. 【0028】 The processing unit 22b controls the propulsion system 12 according to the position of the centerline of the pipeline 100 (S108). Specifically, the processing unit 22b controls the propulsion system 12 so that the AUV 10 navigates along the centerline of the pipeline 100. Here, the distance image is an image projected from the 3D point cloud data of the AUV coordinate system. Therefore, the positional relationship between the position on the distance image and the position of the AUV 10 is known. Thus, the processing unit 22b can determine the displacement of the AUV 10 with respect to the centerline of the pipeline 100. The processing unit 22b controls the propulsion system 12 so that the displacement of the left-right axis of the AUV coordinate system approaches zero. 【0029】 The processes described in steps S101 to S108 are repeated at regular intervals. Therefore, even if the AUV 10 is displaced along its left-right axis relative to the pipeline 100, it is corrected so that the displacement along the left-right axis becomes zero. Consequently, the AUV 10 can continue to navigate along the pipeline 100. As a result, the pipeline 100 can be inspected automatically. 【0030】 Furthermore, in the second and subsequent steps S103, the processing unit 22b extracts 3D point cloud data from a specific region from the 3D point cloud data acquired from the sensor 21 and projects the 3D point cloud data from the specific region onto a reference plane. The specific region is the region where the object to be detected is estimated to exist. The processing unit 22b does not project 3D point cloud data from regions other than the specific region onto the reference plane. Therefore, the amount of data handled by the processing unit 22b can be reduced. 【0031】 Specifically, the processing unit 22b defines a specific area by performing the following processing. In this embodiment, the centerline of the pipeline 100 is identified by performing the processing in step S107. It is estimated that the pipeline 100 acquired in the next scan will be located near the identified centerline. Therefore, the processing unit 22b treats the area around the identified centerline as a specific area. For example, if multiple pipelines 100 are installed side by side, the processing unit 22b treats the area including the pipeline 100 identified in step S107, but excluding the adjacent pipeline 100, as a specific area. More specifically, the processing unit 22b defines boundary planes that are parallel to the centerline identified in step S107 and also parallel to the vertical axis. Boundary planes are defined in pairs, left and right, with respect to the identified centerline. The area enclosed by the boundary planes is the specific area. The distance between the boundary planes and the centerline is less than the installation interval of the pipelines 100. By defining a specific area in this way, the possibility of misidentifying adjacent pipelines 100 can be reduced. 【0032】 Note that the process of extracting specific 3D point cloud data from the existing 3D point cloud data is not mandatory and can be omitted. In other words, in subsequent steps S103, all acquired 3D point cloud data may be projected onto the reference plane. 【0033】 Directly using 3D point cloud data to identify pipeline 100 or determine its location requires a massive amount of computation because it necessitates performing various matrix operations on a huge number of points. In contrast, the process of generating a depth image from 3D point cloud data, as in this embodiment, mainly only requires calculating the distance from a point in the 3D coordinate system to the reference plane for each pixel. Therefore, the processing in this embodiment requires less computation compared to directly handling 3D data. Furthermore, the processing performed after generating the depth image is 2D image processing, which also requires less computation compared to directly handling 3D data. As a result, pipeline 100 can be identified and its location determined with less computation compared to directly handling 3D data. 【0034】 The process used to identify the center line of the pipeline 100 in this embodiment is just one example; the center line of the pipeline 100 can also be identified using a different process. For example, a straight line passing through the center of the two line segments obtained in step S106 may be identified as the center line of the pipeline 100. Alternatively, a straight line passing through the center of the area drawn in white on the binarized image may be identified as the center line of the pipeline 100. 【0035】 Furthermore, the distance image or binarized image generated in this embodiment can be used for purposes other than identifying the centerline of the pipeline 100. For example, the distance image or binarized image may be used for recording the position of the pipeline 100 on the seabed. 【0036】 Next, with reference to Figures 8 and 9, we will explain the process of projecting 3D point cloud data onto another reference plane. 【0037】 As described above, the AUV 10 controls the robotic arm 13 so that the distance between the inspection tool 14 and the pipeline 100 is within a predetermined range. However, if there is an obstacle above the pipeline 100 or the pipeline 100 cannot be identified, the control device 15 moves the inspection tool 14 away from the pipeline 100. 【0038】 This helps to suppress damage to the inspection tool 14. The image generation device 22 can also be used in the detection of this type of pipeline 100 and obstacles. The details of this process will be described below. The objects to be detected in this process are the pipeline 100 and obstacles around the pipeline 100. 【0039】 First, the processing unit 22b acquires 3D point cloud data from the sensor 21 (S201). Next, the processing unit 22b determines a reference plane (S202). The reference plane is, for example, a plane perpendicular to the front-to-back direction of the AUV 10, or a plane perpendicular to the longitudinal direction of the pipeline 100. The front-to-back direction of the AUV 10 is one of the coordinate axes of the AUV coordinate system. The longitudinal direction of the pipeline 100 can be determined by the processing in step S106 described above. Preferably, the reference plane is, for example, a predetermined distance forward from the current position of the AUV 10. 【0040】 Next, the processing unit 22b projects the 3D point cloud data onto a reference plane to generate a binarized image (S203). The processing unit 22b draws objects in white if they exist on the reference plane, and in black if they do not exist on the reference plane. Alternatively, a depth image may be generated instead of a binarized image. Next, the processing unit 22b identifies the pipeline 100 from the binarized image (S204). The identification of the pipeline 100 can be done based on the approximate position and size of the pipeline 100. The approximate position and size of the pipeline 100 are known. Alternatively, the position and size of the pipeline 100 can be determined by the process in step S107 described above. Depending on the extent of the presence of obstacles, the pipeline 100 may not be identifiable. 【0041】 Next, the processing unit 22b determines whether or not there are obstacles around the pipeline 100 (S205). Specifically, as shown in the left diagram of Figure 9, the processing unit 22b defines an obstacle detection area above the pipeline 100, and determines that there is an obstacle if an object is present in this area. The center diagram of Figure 9 shows a binarized image when there is an obstacle in the obstacle detection area. The right diagram of Figure 9 shows an example where the obstacle covers the pipeline 100 and the pipeline 100 cannot be detected. Therefore, in step S204, if the processing unit 22b cannot identify the pipeline 100, it determines that there are obstacles around the pipeline 100. 【0042】 Next, the processing unit 22b controls the robot arm 13 based on the obstacle detection result (S206). Specifically, if it determines that there is an obstacle around the pipeline 100, it changes the posture of the robot arm 13 so that the inspection tool 14 does not come into contact with the obstacle. 【0043】 The processes described above, from steps S201 to S206, are repeated at regular intervals. Therefore, when an obstacle appears around the pipeline 100, the inspection tool 14 can be moved away from the obstacle. 【0044】 As described above, the image generation device 22 of this embodiment performs an image generation method to generate an image for identifying a predetermined pipeline 100. The image generation device 22 comprises an input unit 22a and a processing unit 22b. The input unit 22a receives 3D point cloud data generated by the sensor 21 detecting its surroundings. The processing unit 22b generates a 2D image by projecting the 3D point cloud data input to the input unit 22a onto a reference plane oriented according to the pipeline 100. This is Feature 1. 【0045】 By identifying pipeline 100 using a 2D image, the amount of computation can be significantly reduced compared to methods that directly use 3D point cloud data. In particular, since the 2D image is projected onto a reference plane oriented according to the pipeline 100, it is possible to generate an image that accurately identifies pipeline 100 even when using a 2D image. 【0046】 In the image generation device 22 of this embodiment, the processing device 22b generates a distance image as a two-dimensional image by changing the drawing parameters of each pixel according to its position in a direction perpendicular to the reference plane. This is Feature 2. 【0047】 This means that, compared to binarized images, it contains more information, potentially allowing for more accurate identification of the target object. 【0048】 In the image generation device 22 of this embodiment, the object to be detected includes a long pipeline 100. The processing device 22b uses a reference plane that is parallel to the longitudinal direction of the pipeline 100, or a plane that is perpendicular to the longitudinal direction of the pipeline 100, and projects the 3D point cloud data onto the reference plane to generate a 2D image. This is feature 3. 【0049】 This allows the surface that best exhibits the characteristics of elongated objects to be used as the reference plane, enabling accurate identification of the object to be detected. 【0050】 In the image generation device 22 of this embodiment, the pipeline 100, which is the object to be detected, is arranged along the seabed 101, which is the detection surface. The processing device 22b sets the reference plane to be the same plane as the seabed 101, or a plane parallel to the seabed 101, and projects the 3D point cloud data onto the said reference plane to generate a 2D image. The above is Feature 4. 【0051】 This allows the surface where the characteristics of the object to be detected are most easily displayed to be used as the reference plane, thus enabling accurate identification of the object to be detected. 【0052】 In the image generation device 22 of this embodiment, the processing device 22b determines the position of the seabed 101 based on the three-dimensional point cloud data input from the sensor 21. This is the feature 5. 【0053】 This allows the sensor 21 to be used to determine the location of the seabed 101 and to identify the pipeline 100. Therefore, the sensor 21 can be used effectively. 【0054】 In the image generation device 22 of this embodiment, the processing device 22b extracts specific 3D point cloud data from the 3D point cloud data input from the sensor 21 and generates a 2D image by projecting the specific 3D point cloud data onto a reference plane. This is feature 6. 【0055】 This reduces the amount of data handled by the processing unit 22b. 【0056】 The object recognition device 20 of this embodiment comprises an image generation device 22 and a sensor 21. The processing device 22b identifies the pipeline 100 by analyzing the distance image. These are the features 7. 【0057】 In the object identification device 20 of this embodiment, the sensor 21 is mounted on the AUV 10. The processing device 22b identifies the pipeline 100 while the AUV 10 is moving. These are the features 8. 【0058】 This allows for real-time processing based on the identification results of the object identification device 20. 【0059】 In the object recognition device 20 of this embodiment, the sensor 21 is a sensor that generates 2D point cloud data with a single scan. As the AUV 10 moves, the sensor 21 performs multiple scans, thereby generating 3D point cloud data. These are the features 9. 【0060】 This makes it possible to generate 3D point cloud data using inexpensive sensors. 【0061】 In the object identification device 20 of this embodiment, the processing device 22b generates a distance image, which is a two-dimensional image obtained by changing the drawing parameters of each pixel according to its position in a direction perpendicular to the reference plane. The processing device 22b binarizes the distance image according to the drawing parameters and further extracts the contours to generate a contour image. Based on the contour image, the processing device 22b identifies the longitudinal direction of the pipeline 100 and identifies the pipeline 100 as an object to be detected based on the longitudinal direction. The above is feature 10. 【0062】 This allows for accurate identification of pipeline 100. 【0063】 Features 1 through 10 described above can be combined as appropriate, as long as no contradictions arise. For example, feature 3 can be combined with at least one of features 1 and 2. Feature 4 can be combined with at least one of features 1 through 3. Feature 5 can be combined with at least one of features 1 through 4. Feature 6 can be combined with at least one of features 1 through 5. Feature 7 can be combined with at least one of features 1 through 6. Feature 8 can be combined with at least one of features 1 through 7. Feature 9 can be combined with at least one of features 1 through 8. Feature 10 can be combined with at least one of features 1 through 9. 【0064】 Although preferred embodiments of this application have been described above, the above configuration can be modified as follows, for example. 【0065】 The robot arm 13 and inspection tool 14 are not essential components and can be omitted. In this case, the object identification device 20 identifies and stores the location of the pipeline 100. 【0066】 In the above embodiment, the processing unit 22b generates images and identifies objects. Alternatively, the processing unit 22b may only generate images, and other devices may perform object identification. 【0067】 In the above embodiment, the object identification device 20 identifies the pipeline 100 in real time while the AUV 10 is in transit. Alternatively, the object identification device 20 may generate and store a two-dimensional image while the AUV 10 is in transit, and then identify the pipeline 100 after the AUV 10 has arrived at port. 【0068】 In the above embodiment, the object identification device 20 and image generation device 22 mounted on the AUV 10 as a mobile body were described, but they may be mounted on other mobile bodies. For example, they may be mounted on a vehicle, a ship, or an aircraft. Furthermore, the mobile body is not limited to an autonomous mobile body, but may be a mobile body driven by an operator. 【0069】 An example of a vehicle is an autonomous vehicle that travels through work sites such as construction sites or farms. This autonomous vehicle detects a drainage ditch as a target object and takes evasive action. Alternatively, the autonomous vehicle may travel along the drainage ditch as a target object. A drainage ditch is an example of a long object. 【0070】 Another example of a vehicle is an agricultural vehicle. This vehicle detects furrows in a field and drives along them, performing tasks as it goes. The furrows are both the object to be detected and an example of a long object. Crops planted on the furrows may also be added as the object to be detected. In this case, the vehicle can perform agricultural tasks such as spraying pesticides on the crops. 【0071】 An example of an aerial vehicle is a drone. This drone flies along pipes installed in walls or ceilings to inspect them. The pipes are the objects to be detected and are an example of a long object. In addition, air conditioning equipment installed in walls or ceilings may be added as objects to be detected. In this case, the drone can perform maintenance work on the air conditioning equipment. 【0072】 The technical details described using the AUV10 in the above embodiment can be applied to other mobile devices as long as no inconsistencies arise. For example, the process of extracting specific 3D point cloud data from 3D point cloud data input from sensor 21 can be applied to the above agricultural vehicle. Specifically, initially, the processing unit 22b extracts specific 3D point cloud data from the 3D point cloud data input from sensor 21 whose vertical position is less than or equal to the height of the ridge. As a result, the shape of the ridge can be accurately detected because the specific 3D point cloud data does not contain many crops growing on the ridge. Although the height of the ridge is known, the height of the ridge may also be determined based on the 3D point cloud data. Next, the processing unit 22b performs the process shown in the flowchart of Figure 3 to identify the position and longitudinal direction of the ridge. After that, the processing unit 22b determines a boundary plane that is parallel to the longitudinal and vertical directions of the ridge and passes through the center of the width direction of the ridge. The processing unit 22b extracts 3D point cloud data for the region enclosed by the left and right boundary planes, and projects the extracted specific 3D point cloud data onto a reference plane parallel to the side of the vehicle to generate a 2D image. Based on the 2D image, the processing unit 22b identifies the position and shape of the crops. 【0073】 In the above embodiment, the object to be detected includes an elongated body, but the object to be detected may not include an elongated body. In this case, it is preferable to apply a reference plane oriented according to the object to be detected. For example, if the object to be detected has a surface with a distinctive shape, it is preferable to apply a reference plane that is parallel or substantially parallel to the surface with the distinctive shape. 【0074】 In the above embodiment, a sensor 21 that generates 2D point cloud data in a single scan is used, but a sensor that generates 3D point cloud data in a single scan, such as a 3D LIDAR (Laser Imaging Detection and Ranging), may also be used. 【0075】 In the above embodiment, the object recognition device 20 and the image generation device 22 are mounted on a mobile body, but they may also be mounted on equipment other than a mobile body. In this case, it is preferable that the object recognition device 20 includes a sensor that generates 3D point cloud data in a single scan. 【0076】 The flowchart shown in the above embodiment is just one example, and some processes may be omitted, some processes may be modified, or new processes may be added. 【0077】 The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, dedicated processors, integrated circuits, ASICs (Application Specific Integrated Circuits), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuits. In this disclosure, a circuit, unit, or means is hardware that performs the enumerated functions, or hardware programmed to perform the enumerated functions. The hardware may be hardware disclosed herein, or other known hardware that is programmed or configured to perform the enumerated functions. If the hardware is a processor, which is considered a type of circuit, then the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or the processor. [Explanation of symbols] 【0078】 10 AUV (mobile) 11 Main unit 12 Propulsion device 13 Robot Arm 14 Testing Tools 15 Control device 20 Object Identification Device 21 sensors 22 Image generation device 22a Input section 22b Processing Unit

Claims

[Claim 1] In an image generation device that generates images for identifying predetermined objects to be detected, The input unit receives 3D point cloud data generated by the sensor detecting the surroundings, A processing unit that generates a two-dimensional image based on the three-dimensional point cloud data input to the input unit, Equipped with, The object to be detected is arranged along the detection surface. The processing device determines the position of the detection surface based on the three-dimensional point cloud data input from the sensor. The processing device is an image generation device that generates a two-dimensional image by projecting the three-dimensional point cloud data onto a reference plane that is the same plane as the detection surface, or a plane parallel to the detection surface, and has an orientation determined according to the object to be detected. [Claim 2] An image generation apparatus according to claim 1, The processing device is an image generation device that generates a distance image as the two-dimensional image by changing the drawing parameters of each pixel according to its position in a direction perpendicular to the reference plane. [Claim 3] An image generation apparatus according to claim 1, The processing device is an image generation device that extracts specific three-dimensional point cloud data from the three-dimensional point cloud data input from the sensor and generates a two-dimensional image by projecting the specific three-dimensional point cloud data onto the reference plane. [Claim 4] The image generation apparatus according to Claim 1, The aforementioned sensor and, Equipped with, The processing device is an object identification device that identifies the object to be detected by analyzing the two-dimensional image. [Claim 5] The object identification device according to claim 4, The aforementioned sensor is mounted on a mobile device. The processing device is an object identification device that identifies the object to be detected while the moving body is in motion. [Claim 6] The object identification device according to claim 5, The aforementioned sensor is a sensor that generates two-dimensional point cloud data in a single scan, An object recognition device that generates three-dimensional point cloud data by having the sensor perform multiple scans while the moving object is in motion. [Claim 7] The object identification device according to claim 4, The objects to be detected include elongated objects, The processing device generates a distance image, which is a two-dimensional image, in which the drawing parameters of each pixel are changed according to the position in a direction perpendicular to the reference plane. The processing device generates a contour image by binarizing the distance image according to the drawing parameters and further extracting contours. The processing apparatus is an object identification device that identifies the longitudinal direction of the elongated object based on the contour image and identifies the elongated object as a target object based on the longitudinal direction. [Claim 8] In an image generation method for generating images to identify predetermined objects to be detected, The object to be detected is arranged along the detection surface. By detecting the surroundings, 3D point cloud data is generated. Based on the three-dimensional point cloud data, the position of the detection surface is determined. An image generation method that generates a two-dimensional image by projecting the three-dimensional point cloud data onto a reference plane which is the same plane as the detection surface, or a plane parallel to the detection surface, and which is oriented according to the object to be detected.

Images