Measurement system and measurement method

Abstract

To obtain a technology having high versatility related to an identification target, in which the identification target can be imaged from different viewpoints and from the front.SOLUTION: A photographic metrology system comprises identification targets 122 and 128 that are identified by being photographed and cameras 111 and 113 that photograph the identification targets. The identification targets 122 and 128 have a first side and a second side facing in different directions, and an identification code is displayed on the first side and the second side. The cameras 111 and 113 are oriented in different directions, the first face being able to face camera 111 and the second face being able to face camera 113.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to three-dimensional photogrammetry using a target. [Background technology]

[0002] Various sensing technologies are being researched with the aim of improving the efficiency of agricultural work. One such technology is the 3D photometry of crops. In 3D photometry, targets are used to identify correspondences between stereo images and to orient cameras. For example, Patent Document 1 describes a target used for 3D photometry in farm fields. Furthermore, Patent Documents 2 and 3 describe a three-dimensional target with multiple identification surfaces. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2021-139749 [Patent Document 2] Japanese Patent Application Laid-Open No. 2002-354506 [Patent Document 3] Japanese Patent Application Laid-Open No. 2003-42726 Summary of the Invention [Problem to be solved by the invention]

[0004] In three-dimensional photo measurement in farm fields, it is not practical to take pictures by aiming at each subject. A practical method is to have a work robot equipped with a camera patrol the field and take pictures.

[0005] When a mobile robot is used to photograph crops in a field, it is necessary to use a camera attached to the mobile robot to photograph a large number of identification targets placed in the field in an identifiable state. Incidentally, the identification targets used for orientation display a graphic for detecting the target's center in the image and a graphic representing the identification information. These identification targets should be photographed as closely as possible to the front. This is because photographing them from an oblique angle distorts the graphic, reducing the accuracy of reading the identification code and the center position.

[0006] Therefore, it is important to point the mobile robot's camera directly at the target to be identified when taking the photograph. On the other hand, when considering the creation of a 3D model based on stereophotography, it is necessary to take photographs from many different viewpoints. Therefore, it is required to photograph the target to be identified from different viewpoints and from the front.

[0007] Furthermore, there is a demand for identification targets that are not limited to a single identification method but can be identified by a variety of methods.

[0008] In this context, an object of the present invention is to provide a technology relating to a highly versatile identification target. [Means for solving the problem]

[0009] The present invention provides Multiple plants placed in the field Identify targets and a mobile body that can move within the field; and a first camera and a second camera that are provided on the mobile body and facing in different directions; The aforementioned Multiple The identification targets are oriented in different directions. The identification code is displayed. The first and second sides the first camera can face directly onto each of the first surfaces of the plurality of identification targets having a normal direction in a first direction, and the second camera can face directly onto each of the second surfaces of the plurality of identification targets having a normal direction in a second direction different from the first direction, and while the moving body is moving, the first camera can intermittently photograph each of the first surfaces of the plurality of identification targets arranged at intervals in the moving direction of the moving body, and the second camera intermittently photographs each of the second surfaces of the plurality of identification targets arranged at intervals in the moving direction of the moving body. This is a measurement system that can do this.

[0010] In the present invention, the same identification code is displayed on the first surface and the second surface. There are The following aspects can be mentioned.

[0011] In the present invention, the identification target may have a polyhedron structure, the first surface being one surface of the polyhedron, and the second surface being another surface of the polyhedron.

[0012] In the present invention, the identification target has the shape of a truncated pyramid, and the first surface, the second surface, and another surface having an identification code are formed using multiple inclined surfaces that make up the truncated pyramid and the smaller of the two parallel surfaces that make up the truncated pyramid.

[0013] In the present invention, before The first camera Any of the plurality of identification targets In a first state in which the first surface is viewed from the front, the second camera A plurality of said identification targets The second surface Both of When the moving object is moved from the first state, the first camera A plurality of said identification targets The first surface Both of The second camera is not directly in front of the object. Any of the plurality of identification targets An example of such a state is when the second surface is captured from the front.

[0014] In the present invention 、 The multiple identification targets may include targets of different sizes and / or shapes, and the differences in size and / or shape of the identification targets may be detected by a means for measuring three-dimensional information, thereby identifying the identification targets.

[0015] The present invention is an identification target having a three-dimensional structure on which an identification code is displayed, the size and / or shape of the three-dimensional structure corresponding to the identification information of the identification code, and enabling detection of the identification information by the identification code and detection of the identification information by the size and / or shape of the three-dimensional structure.

[0016] The present invention provides Multiple plants placed in the field Identify targets and a first camera and a second camera provided on a mobile body that can move within the field and facing in different directions; A measurement method using the Multiple The identification targets are oriented in different directions. The identification code is displayed. The first and second sides the first camera is capable of facing directly to each of the first surfaces of the plurality of identification targets having a first direction as a normal direction, and the second camera is capable of facing directly to each of the second surfaces of the plurality of identification targets having a second direction different from the first direction as a normal direction, and while the moving body is moving, the first camera is used to intermittently photograph each of the first surfaces of the plurality of identification targets arranged at intervals in the moving direction of the moving body, and the second camera is used to intermittently photograph each of the second surfaces of the plurality of identification targets arranged at intervals in the moving direction of the moving body. It is a measurement method. [Effects of the Invention]

[0017] The present invention provides a highly versatile technology for identifying targets. [Brief explanation of the drawings]

[0018] [Figure 1] 1A and 1B are conceptual diagrams of an embodiment. [Figure 2] 1A and 1B are perspective views of an identification target. [Figure 3] 1A and 1B are diagrams showing an example of a code display for identification purposes. [Figure 4] 10 is a flowchart illustrating an example of a processing procedure. [Figure 5] 10A, 10B, and 10C are perspective views showing other examples of identification targets. DETAILED DESCRIPTION OF THE INVENTION

[0019] 1. First embodiment (composition) Figure 1 shows a measurement system deployed in a farm field 100. Figure 1(A) is a front view of a work robot (mobile robot) 110, and Figure 1(B) is a side view of the work robot 110.

[0020] A farm field 100 is provided with shelves 101, and a crop 102 (for example, grapes) is cultivated using the shelves 101. The cultivated crop is not limited to grapes, and may be fruits or vegetables other than grapes.

[0021] A work robot 110, which is a mobile body equipped with a camera, moves on the ground 120 of a farm field 100. The work robot 110 is equipped with a drive means such as a motor and moves autonomously using wheels. The work robot 110 moves along a predetermined path. Of course, it is also possible for the work robot 110 to be configured to autonomously set its own movement path. It is also possible for the robot to move using caterpillars, rails, or wires instead of wheels. A UAV can also be used as the mobile body equipped with a camera.

[0022] 1, the work robot 110 is equipped with multiple cameras. Camera 111 faces diagonally upward at 45 degrees in front of the vehicle 110, camera 112 faces vertically upward, camera 113 faces diagonally upward at 45 degrees behind the vehicle, camera 114 faces diagonally upward at 45 degrees to the left in the direction of travel (positive direction of the X-axis), and camera 115 faces diagonally upward at 45 degrees to the right in the direction of travel (positive direction of the X-axis).

[0023] In this example, each camera repeatedly captures still images. The interval between captures is set so that the nth and n+1th captured images overlap while the work robot 110 is moving. When capturing video, the captured image is cut out from the overlapping images.

[0024] 1 shows identification targets 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, and 131 arranged on a shelf 101. Each identification target has a different identification marking, but all have the same shape.

[0025] The positions of at least some of the identified targets are known. The coordinate system used to describe the positions may be an absolute coordinate system or a local coordinate system. The absolute coordinate system is the coordinate system used in GNSS and maps.

[0026] In this example, three rows of identification targets are arranged extending in the X-axis direction along the movement path of work robot 110. The central row of these three rows of identification targets is made up of identification targets 121, 122, 123, 124, 125, 126, 127, 128, and 129 shown in Figure 1(B). Similarly, the row of identification targets extending in the X-axis direction to which identification target 130 belongs, and the row of identification targets extending in the X-axis direction to which identification target 131 belongs are also arranged.

[0027] 1, all the identification targets have the same shape, but the identification data displayed is different. The shapes of the identification targets will be explained below. Here, the identification target 122 will be explained as a representative.

[0028] An identification target 122 is shown in Figure 2. Figures 2(A) and 2(B) show the state as seen from different viewpoints.

[0029] The identification target 122 has the shape of an upside-down truncated quadrangular pyramid. Fig. 2 shows inclined surfaces 122A, 122B, 122C, and 122D and a bottom surface 122E of the identification target 122. The inclined surfaces 122A, 122B, 122C, and 122D and the bottom surface 122E serve as code display surfaces on which the same identification code is displayed.

[0030] Sloped surface 122A is a plane facing diagonally downward at an angle of 45° to the positive X-axis. Sloped surface 122C is a plane facing diagonally downward at an angle of 45° to the negative X-axis on the opposite side of slope 122A in the X-axis direction.

[0031] Sloped surface 122B is a plane facing diagonally downward at an angle of 45° along the negative Y-axis. Sloped surface 122D is a plane facing diagonally downward at an angle of 45° along the positive Y-axis, opposite to slope 122B in the Y-axis direction. Bottom surface 122E is a plane facing vertically downward (in the negative Z-axis direction).

[0032] The inclined surface 122C is set in a direction that allows it to face the camera 111. That is, in FIG. 1, the camera 111 is set so that its optical axis faces in a direction diagonally upward at 45° on the positive X-axis. In the state of FIG. 1(B), the camera 111 and the inclined surface 122C face each other directly, that is, the inclined surface 122C is set so that it faces diagonally downward at 45° on the negative X-axis so that the camera 111 photographs the inclined surface 122C from the front. This is also true for the corresponding inclined surfaces of the identification targets 121, 123 to 129.

[0033] Using a similar concept, inclined surface 122A is set in an orientation that allows it to face camera 113 directly. That is, in FIG. 1, camera 113 is set so that its optical axis faces in a direction diagonally upward at 45° in the negative direction of the X-axis. In the state shown in FIG. 1(B), camera 113 and inclined surface 122A do not face each other directly, but by moving work robot 110 in the positive direction of the X-axis, a situation can be achieved in which camera 113 can photograph inclined surface 112 from the front. To achieve this situation, inclined surface 122A is set so that it faces in a direction diagonally downward at 45° in the positive direction of the X-axis. The same applies to the inclined surfaces corresponding to identification targets 121, 123 to 129.

[0034] The orientation of the lower surface 122E is set so that it faces the camera 112 facing vertically upward when the camera 112 is positioned vertically downward. This is the same for the lower surfaces corresponding to the identification targets 121, 123 to 129.

[0035] The inclined surface of the identification target 130 facing diagonally downward at 45° along the negative Y axis is oriented so that it can directly face the camera 114 facing diagonally upward at 45° along the positive Y axis. This is also true for the other identification targets (not shown) that are included in the row of the identification target 130 and lined up in the X axis direction. The inclined surface of the identification target 131 that faces diagonally downward at 45° along the positive Y axis is arranged so that it can directly face the camera 115 that is facing diagonally upward at 45° along the negative Y axis. This is also true for the other identification targets (not shown) that are included in the row of the identification target 131 and lined up in the X axis direction.

[0036] In the above explanation, it has been explained that the orientation of the code display surface of each identification target is set to match the orientation of the camera, but it can also be considered that the orientation of the camera is set to match the orientation of the code display surface. Alternatively, it can be considered that the orientations of the code display surface and the camera are set to achieve the above-mentioned orientation relationship. Furthermore, by attaching a code to the top surface of this inverted trapezoidal target, it can also be recognized from the air by a UAV or the like.

[0037] The same code display is placed on each of the five code display surfaces of each identification target. Figure 3(A) shows an example of a code display. In this code display, the center of the code display in the image is recognized by detecting the center of the central circle and the center of curvature of the arc in the captured image of the code display, and the position and length of the arc are detected, allowing the identification information indicated by the code display to be recognized.

[0038] The identification targets 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, and 131 have code display surfaces facing five different directions, and the code display surfaces can be photographed from the front from five directions. In other words, the targets are easy to identify from multiple directions.

[0039] As mentioned above, if the code display in Figure 3 is photographed from an oblique angle, the shape of the figure will be distorted, increasing the possibility of erroneous detection of the center position and problems with reading the identification information. The identification targets 121 to 131 can be viewed directly from five directions, making identification from multiple directions easy.

[0040] In the system shown in Fig. 1, any of the cameras facing in various directions can photograph the code display surface of the identification target from the front or a position close to the front. For example, in the state shown in Fig. 1(B), one slope of identification target 122 faces directly toward camera 111. Also, the bottom surface of identification target 125 faces directly toward camera 112. Also, one slope of identification target 128 faces directly toward camera 113.

[0041] Here, when work robot 110 moves in the positive direction of the X axis, it deviates from the above-mentioned facing state. However, with further movement, one slope of identification target 121 faces camera 111, the bottom surface of identification target 124 faces camera 112, and one slope of identification target 127 faces camera 113.

[0042] In this way, cameras 111, 112, and 113 intermittently, but not always, face the code display surface of the identification target while working robot 110 is moving. Even if they do not face the target completely, a situation as close to a direct face as possible can be created for as long as possible.

[0043] The same can be said for the cameras 114 and 115. Similar to the identification targets 121, 122, 123, 124, 125, 126, 127, 128, and 129, a plurality of identification targets are also arranged in the X-axis direction of the identification targets 130 and 131.

[0044] Now, let's focus on identification target 130. In this case, when work robot 110 moves in the positive direction of the X-axis from a state in which camera 114 is directly facing one of the slopes of identification target 130, identification target 130 moves out of view of camera 114, but soon the next identification target will come into view in front of camera 114. The same is true for camera 115.

[0045] 1(B) in which code display surface 122D faces camera 111, and code display surface 122A faces camera 113. When work robot 110 moves in the positive direction of the X axis from the state in FIG. 1(B) in which code display surface 122D faces camera 111, code display surface 122A faces camera 113.

[0046] In this way, while working robot 110 is moving, each camera can capture an image of the identification target from the front as many times as possible.

[0047] There is no limit to the number of cameras, but the more cameras you have, the more complete the image capture will be. The same goes for identification targets.

[0048] One method for moving the camera is to use a manually operated cart. Another method is for workers to carry multiple cameras and take pictures while they are moving. In this case, multiple cameras are fixed to the worker's head (for example, a helmet) or work vest, facing in multiple directions, and the worker simultaneously takes pictures in multiple directions while moving.

[0049] (Example of processing) An example of a procedure for obtaining identification information of an identification target from a captured image will be described below. Fig. 4 is a flowchart showing an example of the processing procedure.

[0050] First, photographs are taken within the field (step S101). For example, the work robot 110 in FIG. 1 is moved thoroughly within the field, photographing with the multiple cameras it is equipped with. At this time, when focusing on a certain camera, the movement speed and photographing interval of the work robot 110 are adjusted so that an image photographed at a first timing and an image photographed at the following second timing partially overlap. The images photographed at the first timing and the second timing become the stereo images used for stereophotography.

[0051] Next, image data of the multiple captured images taken in step S101 is received by an image processing device (step S102). The image processing device is configured, for example, by a PC (personal computer), and performs processing related to the detection of identification targets from the captured images (image recognition) and sfm (structure from motion). The sfm processing includes the extraction of feature points from the captured images, the identification of correspondences between multiple images that make up a stereo image, orientation using identification targets, the generation of point cloud data related to objects captured in the captured images, and the creation of a three-dimensional model based on the point cloud data.

[0052] The image processing device that has received the image data performs the following process. First, it detects the code indication (see, for example, FIG. 3) displayed on the code indication surface of the identification target from each captured image (step S103). If multiple different code indications are captured in one captured image, each code indication is detected.

[0053] When multiple identical code indications are captured in a single captured image, the reliability of code detection is improved. For example, suppose a first code indication and a second code indication with the same content are detected, but some of the information that differs from each other is missing. In this case, by interpolating the missing information, more reliable code content detection is possible. It also reduces false detection of code content.

[0054] Furthermore, priority is given to detecting code displays photographed from the front as much as possible. Whether or not the image was photographed from the front is determined by evaluating the distortion of the image. For example, in the case of Figure 3(A), if the image was not photographed from the front, the arc will be distorted and will become a curved line rather than an arc. By selecting a code display that is closer to an arc, it is possible to select a code display photographed from a closer front. For example, in the case of Figure 3(B), if the image was not photographed from the front, there will be deviation from the rectangular shape. By selecting a code display that is closer to a rectangle, it is possible to select a code display photographed from a closer front.

[0055] In the case of Figure 3(A), if the image is not taken from the front, there will be inconsistencies in the positions of the centers of curvature of the multiple arcs, and inconsistencies between the positions of the centers of curvature of the arcs and the center positions of the circles. One way to select a code display that is taken from a more frontal position is to select a code display with the least degree of inconsistency.

[0056] Alternatively, it is possible to evaluate the reliability of a target for which multiple identical codes have been detected and register it as a point on a map, or if the target has already been registered as a point on a map, to use the coordinate values ​​of that point as the coordinates of the detected target.

[0057] Once the code display is detected, the code content (center position and identification information) is acquired (step S104). Next, feature points are extracted from each captured image (step S105), and further, correspondences between stereo images are identified (step S106).

[0058] Next, orientation is performed using the code displayed in the stereo images to determine the position and orientation of the camera (step S107). Next, the three-dimensional positions of feature points in the stereo images are calculated using the forward intersection method to obtain point cloud data (step S108). Next, a three-dimensional model of the subject is created based on the point cloud data obtained in step S108 (step S109).

[0059] In the above process, there is a high probability that many of the captured images will be taken from the front of the code display, so erroneous detection or detection failure of the code display can be reduced.

[0060] Although the above example shows an example of stereo measurement, it is also possible to use a marker such as that shown in Figure 3(B) to perform measurement from a single image using the single photo orientation method. In this case, single photo orientation can be performed by detecting multiple feature points on the target and providing the condition that the target is placed on a plane. In this case, in order to utilize the plane condition, it is even more important that the targets are facing each other directly than in the stereo method.

[0061] Figure 3(B) shows an example of an Ar marker as a code display. The Ar marker is composed of multiple rectangular shapes. The corners of each rectangular shape are extracted from the captured image as feature points. The positions and combinations of the rectangular shapes are predetermined, and the identification information is read based on this. In addition, the positional relationships of the corners of each rectangular shape are known, and by providing the condition that they are arranged on a plane, the position and orientation of the camera can be calculated using single-photo orientation.

[0062] Specifically, the position and orientation of the camera that photographed the code display surface relative to the code display surface are calculated by the backward intersection method, with the corners of the rectangular shape being set as known points.

[0063] 2. Second embodiment In addition to the camera facing the code display, multiple cameras may be placed with their optical axes slightly offset from the optical axis of the camera. Although the camera with the slightly offset optical axis does not face the code display of the identification target exactly, it can capture many images that are close to being directly facing the code display.

[0064] In this case, even if the positional relationship between the moving object and the identification target is somewhat approximate, one of the multiple cameras that are oriented similarly can be directly or nearly so facing the identification surface (code display surface) of the identification target. The ground in a farm field is not necessarily horizontal or flat, and it is possible that the moving object carrying the camera may tilt. Furthermore, when a camera is attached to a person, the direction of the camera's optical axis will not be constant. In such cases, by arranging multiple cameras with slightly offset optical axis directions, one of them will be directly facing (or nearly facing) the identification surface. This makes it possible to obtain a captured image that is directly facing the identification surface with a high probability.

[0065] As the cameras 111 to 115 in FIG. 1, a stereo camera using two or more cameras can also be used.

[0066] 3. Third embodiment It is also possible to use different identification codes on the multiple code display surfaces of an identification target. For example, it is possible to use different codes on all or some of the inclined surfaces 122A, 122B, 122C, and 122D and the bottom surface 122E in FIG. 2. This makes it possible to determine the direction in which a moving object is facing relative to the target. This allows for more reliable navigation and map creation when there is no map or when creating a map, by utilizing these determinations.

[0067] 4. Fourth Embodiment The orientation of the camera and the orientation of the identification target may be set so that, when viewed in the direction of travel of the mobile body (work robot 110) on which the camera is mounted, the code display surface faces directly in the upper right direction and / or the code display surface faces directly in the upper left direction.

[0068] 5. Fifth Embodiment Other examples of the shape of the identification target are shown in Fig. 5. Fig. 5(A) shows an identification target 400 having code display surfaces 401, 402, 403, 404, and 405. The code display surfaces 401 to 405 are set to face in different directions, for example, every 30°.

[0069] Figure 5(B) shows an identification target 410 having code display surfaces 411 and 412. Figure 5(C) shows an identification target 420 having code display surfaces 421, 422, and 423. Note that there is another code display surface of the same shape on the back side of code display surface 422, but it is hidden and not visible in the figure.

[0070] 6. Sixth Embodiment In addition to identifying targets by taking images, it is also possible to identify targets by using a means capable of acquiring three-dimensional information about the target, such as a Laider or depth camera.

[0071] Means for obtaining three-dimensional information of an object include stereo cameras, Laider, ToF cameras, cameras that obtain depth information using projected light such as dot patterns, radar, and combinations of two or more of these.

[0072] There are several types of identification targets. The first type is a type that identifies each identification target by the difference in size and / or shape. For example, Laider can evaluate the size and shape of the measurement target, so it can detect the difference in size and shape of the identification target.

[0073] For example, by preparing different shapes corresponding to the identification code, it becomes possible to use camera images in combination with sensors that capture three-dimensional shapes, such as Laider, which will greatly improve the efficiency of the process of unifying the three-dimensional space captured by each sensor and the three-dimensional space captured by images on the same map.

[0074] In the second form, the truncated pyramidal identification target shown in FIG. 2 is combined with a rectangular parallelepiped, and identification is performed based on the difference in length (dimension in the Z-axis direction) of this rectangular parallelepiped.

[0075] The third type is a method in which the code display is formed as a protruding convex or recessed (groove) shape, and the display content is detected by measuring the uneven shape using a laser scan.

[0076] The fourth method is to form the code marking and the non-code marking from materials that have significantly different absorptance (or reflectance) for the wavelength of the laser light used by the laser scanner. In this case, the code marking can be recognized by the intensity of the reflected light of the laser light used for measurement. It is also possible to combine two or more of the first to fourth methods.

[0077] A specific example is shown below. As a simple example, let's assume that a square, pentagonal, and hexagonal pyramid are used as identification targets, with three different sizes. In this case, nine different codes (identification information) can be handled by combining three different shapes and three different sizes.

[0078] In this case, the code content of the code display surface of each identification target is set to correspond to the nine types of code information. Note that the display content on the multiple code surfaces of each identification target is the same. The display format uses the codes shown in Figure 3(A) or 3(B), for example.

[0079] For example, suppose the code content is numbers 1 to 9. In this case, the first identification target has a shape and size corresponding to number 1 and is provided with a code display corresponding to number 1. The second identification target has a shape and size corresponding to number 2 and is provided with a code display corresponding to number 2. The same applies to the third to ninth identification targets.

[0080] In this case, it is possible to detect the code from the image captured by the camera, and to detect the code based on the size and shape detection by a means of acquiring three-dimensional information of the object, such as Lidar or a depth camera. For example, in the above case, the identification target number 1 is identified by taking a picture with the camera, and on the other hand, the identification target number 1 is identified by detecting the size and shape corresponding to the number 1 with Lidar or a depth camera. In other words, the same identification target can be identified using different observation means (measurement means).

[0081] This makes it possible to identify common identification targets in both data, even when the data is measured using different measurement methods, such as image data captured using a moving camera and Lidar measurement data, and to easily determine the relationship between the two data.

[0082] For example, it is easy to integrate a 3D model based on SFM and a 3D model based on Lidar, or to integrate 3D data obtained by a depth camera into a 3D model based on SFM and an image captured by the work robot 110.

[0083] For example, suppose there is a 3D model (first 3D model) of a farm field created by sfm based on images captured by work robot 110. On the other hand, suppose there is a 3D model (second 3D model) of a part of the farm field measured by Lidar.

[0084] Here, when integrating the first 3D model and the second 3D model, it is necessary to identify the correspondence between the first 3D model and the second 3D model, or to align the first 3D model and the second 3D model.

[0085] This task requires the work of finding (searching for) common parts between both 3D models. There are various methods for searching for common parts between 3D models, but they have problems such as an increase in the amount of calculations and incorrect matching, making them impractical.

[0086] When the above-described identification target is used, it is possible to know in which captured image, among the captured images used to create the first 3D model, the identification target detected in the second 3D model appears, making it easy to know which part of the first 3D model the second 3D model corresponds to.

[0087] If it is possible to know which part of the first 3D model corresponds to which second 3D model, the first 3D model can be easily integrated with the second 3D model.

[0088] A specific example will be described below. For example, suppose that photography is performed using the work robot 110 shown in Fig. 1 and a 3D model of the farm field 100 (a 3D model of the entire farm field) is created using SFM (see, for example, the processing in Fig. 4). On the other hand, suppose that measurements are performed using Lidar on the fruits of some crops in the farm field 100 and a 3D model of the fruits (a 3D model of the fruits) is obtained.

[0089] Assume that an identification target of this embodiment is located near the object to be measured by Lidar, and that at least one identification target of this embodiment is included in the laser scan data (laser scan point cloud) that forms the basis of the "3D model of the object." In this case, a 3D model of the nearby identification target is also obtained along with the "3D model of the object." In other words, the data for the "3D model of the object" also includes the 3D model of the identification target.

[0090] In this case, code information (identification information) based on the size and / or shape of the identification target is extracted from the laser scan data that forms the basis of the "real 3D model." Let's say the extracted code information of the identification target is "code number 5," for example.

[0091] In this case, a search is made for an image containing the identification target with "code number 5" from the group of captured images that form the basis of the 3D model of the entire farm field. This identifies the correspondence between the "real 3D model" obtained through Lidar measurement and the image captured by the camera mounted on the work robot 110. Normally, there will be multiple captured images that contain this "code number 5." This is because the capture conditions for the images captured by the camera mounted on the work robot 110 are set so that multiple images with overlapping capture ranges can be obtained in order to perform SFM.

[0092] Here, the 3D model created by SFM based on the multiple captured images for which the above correspondences have been identified includes a 3D model of the identification target with "code number 5." Therefore, the 3D model of the identification target with "code number 5" can be identified in both the "3D model of the field" and the "3D model of the fruit."

[0093] For example, by overlapping the 3D model portion of the identification target "Code No. 5" in the "3D model of the field" with the 3D model portion of the identification target "Code No. 5" in the "3D model of the fruit," rotating one relative to the other and translating it if necessary, and then zooming in and out, the state in which the two 3D models related to the identification targets overlap is searched for.

[0094] This search identifies the correspondence between the "3D model of the field" and the "3D model of the fruit." In other words, it identifies which parts of the "3D model of the field" correspond to the "3D model of the fruit." Once the correspondence between the two 3D models has been identified, the "3D model of the field" and the "3D model of the fruit" can be integrated.

[0095] At this time, the exterior orientation elements (position and attitude) of the Lidar in the coordinate system describing the "3D model of the field" are also determined.

[0096] It is also possible to provide discrimination by utilizing only one of differences in size and shape as discrimination targets.

[0097] 7. Seventh Embodiment 1 shows a configuration in which the identification target is placed facing downward, but it can also be placed facing horizontally (for example, the code display surface 122E in FIG. 2 is placed facing horizontally) or facing vertically upward (upside down from FIG. 2). In this case, the direction of the optical axis of the camera is set to match the orientation of the code display surface.

[0098] 8. Eighth Embodiment The structure of the identification target can be a polygonal truncated pyramid (n-sided truncated pyramid) such as a triangular truncated pyramid, a pentagonal truncated pyramid, or a hexagonal truncated pyramid, in addition to a square truncated pyramid.

[0099] 8. Eighth Embodiment In another embodiment, these identification targets can be placed on the ground. By placing them on the ground, it becomes possible to handle, for example, outdoor-grown cabbage, lettuce, strawberries, and other plants. In this case, the same as in the first embodiment can be achieved by placing a camera on the moving body facing downward or diagonally downward instead of upward.

[0100] 9. Ninth embodiment If the identification target 122 in Figure 2 is placed upside down, it becomes a target suitable for shooting from a mobile object moving through the air. In this case, identification from various directions is also possible. Examples of mobile objects in this case include UAVs, robots that move through the air using wires, and robots that move on rails placed in high places. In this case, multiple cameras facing various directions, including downward, are placed on the mobile object moving through the air.

[0101] 10. Tenth embodiment It is also possible to prepare two identification targets 122 as shown in Figure 2 and join the wider parallel surfaces together to form an identification target. In this case, identification is possible from both below and above. In this case, for example, an image is taken from below by a work robot traveling on the ground as shown in Figure 1, and from above by a UAV. If this identification target is used by laying it on its side, identification is possible from both the left and right sides. [Explanation of symbols]

[0102] 100...field, 101...shelf, 102...crop, 111...camera, 112...camera, 113...camera, 114...camera, 115...camera, 120...ground, 121...identification target, 121A...code display surface, 121B...code display surface, 121C...code display surface, 121D...code display surface, 121E...code display surface, 122...identification target, 123...identification target, 124...identification target, 125...identification target, 126...identification target, 127...identification target, 128...identification target, 129...identification target, 130...identification target, 131...identification target.

Claims

1. A method of detecting a plurality of identification targets arranged in a field; a mobile body that can move within the farm field; a first camera and a second camera provided on the moving body and facing in different directions; Equipped with the plurality of identification targets each have a first surface and a second surface facing in different directions and displaying an identification code; the first camera can face directly onto each of the first surfaces of the plurality of identification targets, the normal direction of which is a first direction, and the second camera can face directly onto each of the second surfaces of the plurality of identification targets, the normal direction of which is a second direction different from the first direction; A measurement system in which, while the moving body is moving, the first camera is capable of intermittently photographing each of the first surfaces of a plurality of the identification targets arranged at intervals in the direction of movement of the moving body, and the second camera is capable of intermittently photographing each of the second surfaces of the plurality of the identification targets arranged at intervals in the direction of movement of the moving body.

2. The measurement system according to claim 1 , wherein the first surface and the second surface have the same identification code displayed thereon.

3. the identification target has a polyhedral structure; the first face is one face of the polyhedron, The measurement system according to claim 1 , wherein the second surface is another surface of the polyhedron.

4. the identification target has a truncated pyramid shape; The measurement system of claim 3, wherein the first surface, the second surface, and another surface having an identification code are formed using the multiple inclined surfaces that form the truncated pyramid and the smaller surface of the two parallel surfaces that form the truncated pyramid.

5. In a first state in which the first camera captures the first surface of any of the plurality of identification targets in a frontal direction, the second camera does not capture any of the second surfaces of the plurality of identification targets in a frontal direction; A measurement system as described in any one of claims 1 to 4, wherein by moving the moving body from the first state, a second state is reached in which the first camera does not capture any of the first surfaces of the multiple identification targets directly in front, and the second camera captures the second surface of one of the multiple identification targets directly in front.

6. The plurality of identification targets include targets of different sizes and / or shapes; A measurement system according to any one of claims 1 to 5, wherein the difference in size and / or shape of the identification target is detected by a means for measuring three-dimensional information, thereby identifying the identification target.

7. A measurement method using a plurality of identification targets arranged in a field, and a first camera and a second camera mounted on a mobile body movable within the field and facing in different directions, comprising: the plurality of identification targets each have a first surface and a second surface facing in different directions and displaying an identification code; the first camera can face directly onto each of the first surfaces of the plurality of identification targets, the normal direction of which is a first direction, and the second camera can face directly onto each of the second surfaces of the plurality of identification targets, the normal direction of which is a second direction different from the first direction; A measurement method in which, while the moving body is moving, the first camera is used to intermittently photograph each of the first surfaces of a plurality of the identification targets arranged at intervals in the direction of movement of the moving body, and the second camera is used to intermittently photograph each of the second surfaces of the plurality of the identification targets arranged at intervals in the direction of movement of the moving body.

Images