Method and device for determining the position and position of a marking

Abstract

A method and apparatus for determining the position and pose of a marker having encoded information includes the step of acquiring an image of the marker by a stereo camera. The center of the marker is determined, and then the position of the marker is determined based on the center of the marker. A plurality of vertices on the mark around the center of the mark are then determined. Using the plurality of vertices, pitch, roll, and heading of the marker are determined. The attitude of the marker is determined based on the pitch, roll and heading of the marker. Methods and / or devices for determining the position and pose of a marker may be used in various applications to determine the position and pose of an object in which the marker is located.

Description

Technical Field

[0001] The present disclosure relates generally to methods and apparatus for position determination, and more particularly, to methods and apparatus for determining the position and pose of a marker. Background Art

[0002] For various reasons, it is usually necessary to determine the position and attitude of an object. For example, it may be necessary to know the position and attitude of an agricultural sprayer towed behind a tractor to determine the position of the sprayed liquid in the field, thereby determining whether it complies with local or national regulations. Machines such as agricultural sprayers can be towed behind a vehicle such as a tractor. The towed machine is typically connected to the tractor in a rotatable manner via a hitch, which enables the towed machine to pivot behind the tractor. Therefore, the position and attitude of the tractor are different from the position and attitude of the towed machine, and determining the position and attitude of the tractor may not be enough to determine, for example, whether it complies with regulations. Electronic devices can be placed on the towed machine and other types of machines to determine the position and attitude of a specific machine. However, it may be expensive to install electronic devices on each machine. In addition, the user may need to know the position of multiple machines at the same time, which will require the use of multiple electronic devices, which is even more expensive. Summary of the invention

[0003] In one embodiment, a method and apparatus for determining the position and attitude of a marker includes the step of acquiring an image of the marker by a stereo camera. The marker has encoded information. Based on the image, the position of the marker relative to the local coordinate system of the stereo camera is determined. Based on the position of the marker, the position of the object relative to the local coordinate system of the stereo camera is determined. In one embodiment, the encoded information identifies the object and the encoded information may also identify the position of the marker on the object. In one embodiment, the positioning of the stereo camera in a global coordinate system is determined, and the positioning of the object relative to the global coordinate system may be determined based on the position of the object relative to the local coordinate system and the positioning of the stereo camera in the global coordinate system. In one embodiment, the center of the marker is determined, and then the position of the marker is determined based on the center of the marker. Then, multiple vertices on the marker around the center of the marker are determined. Using the multiple vertices, the pitch, roll, and heading of the marker are determined. Based on the pitch, roll, and heading of the marker, the attitude of the marker is determined.

[0004] The method and apparatus for determining the position and attitude of a marker can be used in various applications to determine the position and attitude of an object on which the marker is located. Applications of the method or apparatus include: determining the position and attitude of a bulldozer blade, determining the position and attitude of an implement towed behind a tractor, controlling the movement of a vehicle, determining the position and attitude of a vehicle, and parking a vehicle based on the position and attitude of the marker. The positioning of a stereo camera can be used to determine the position of the marker, and based on the positioning of the stereo camera, determine the position of the object or machine on which the marker is located. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 A system positioned on a vehicle for determining the position and attitude of a marker is shown according to one embodiment;

[0006] Figure 2 shows a marking according to one embodiment;

[0007] Figure 3 shows a coordinate system of a stereo camera, a video sensor of the stereo camera, and a 2D projection of a marker on the sensor according to an embodiment;

[0008] Figure 4 A flow chart of a method for determining the position and posture of a marker according to one embodiment is shown;

[0009] Figure 5 Shows Figure 4 Application of the method, wherein the position and attitude of a blade of a construction machine are determined;

[0010] Figure 6 Shows Figure 4 Application of the method, wherein the position and attitude of the towed device are determined;

[0011] Figure 7 Shows Figure 4 Application of the method of claim 1, wherein the movement of a vehicle is controlled based on the position and attitude of a marker in an environment;

[0012] Figure 8 Shows Figure 4 Application of the method of claim 1, wherein markers are placed on an automatic vehicle or autonomous vehicle to determine the position and attitude of the vehicle; and

[0013] Fig. 9 Shows Figure 4Application of the method wherein markers are placed in parking spaces to allow an automated vehicle or autonomous vehicle to identify and locate a designated parking space. DETAILED DESCRIPTION

[0014] The method for determining the position and posture of a marker utilizes components including a stereo camera, which is used to capture images of markers positioned within the camera field of view (FOV). Each marker in the marker has a unique geometry, color, and size, which makes the marker easy to find and highlight in the left camera frame and the right camera frame of the stereo camera. The unique geometry of each marker consists of encoded information. In one embodiment, the encoded information can be used to identify the marker. Using triangulation, the three-dimensional (3D) position and posture of the marker positioned in the coordinate system of the stereo camera can be determined. The term "position" used herein refers to the position of the marker or object in the local coordinate system of the stereo camera (described in detail below). Markers can be placed on objects to determine the position and posture of the corresponding markers placed on these objects. Autonomous vehicles or machines (also referred to as autonomous vehicles or machines) can be controlled based on the position and posture of the markers located in the environment where the autonomous vehicle operates.

[0015] Figure 1An embodiment of a system 100 for determining the position and posture of a marker is shown, in which a stereo camera 102 is mounted on a vehicle 104. In one embodiment, the stereo camera 102 has two or more lenses, each of which has a separate image sensing device. In various embodiments, the stereo camera 102 can be a Stereolabs ZED 2i, Framos D435e / D455e, or StereoCam 3D stereo camera (2MP). Each lens is arranged to generate a view slightly different from the view from the other lenses. The difference in view can be used to determine the position of the object captured in the image relative to the stereo camera. The stereo camera 102 communicates with a controller 106, and the controller 106 receives images from the stereo camera 102. In one embodiment, the stereo camera 102 communicates with the controller 106 via an Ethernet or USB3 connector. In various embodiments, the controller 106 can be a rugged embedded system driven by NVIDIA Jetson AGX / NX Xavier or Orin. The controller 106 also communicates with a navigation system 108, which provides positioning and orientation information to the controller 106. In one embodiment, the navigation system 108 is a global navigation satellite system (GNSS). The navigation system 108 can be other types of positioning and orientation determination systems, such as systems using triangulation or other positioning determination methods. The term "location" as used herein refers to the positioning of a marker, object, machine, or camera in a coordinate system larger than the local coordinate system of a stereo camera. The larger coordinate system can be a global coordinate system. The global coordinate system can be any type of global coordinate system, such as a spherical coordinate system or an elliptical coordinate system (e.g., World Geodetic System 1984 (WGS84)) using a geodetic reference. The controller 106 also communicates with a vehicle control system 110, which monitors and controls the operation of the vehicle 104. In one embodiment, the controller 106 communicates with the vehicle control system 110 via one of an RS232 connector, CAN, or Ethernet. In one embodiment, the controller 106 includes a processor 1004 that controls the overall operation of the controller 106 by executing computer program instructions that define such operations. The computer program instructions may be stored in a storage device 1012 or other computer-readable medium (e.g., a disk, CD ROM, etc.) and loaded into the memory 1010 when the computer program instructions are needed to be executed. Figure 4The method steps (described in detail below) may be defined by computer program instructions stored in the memory 1010 and / or the storage device 1012, and may be controlled by the processor 1004 executing the computer program instructions. For example, the computer program instructions may be implemented as a computer executable code programmed by a person skilled in the art to execute the method by Figure 4 Thus, by executing the computer program instructions, the processor 1004 is able to perform the Figure 4 The controller 106 also includes one or more network interfaces 1006 for communicating with other devices via a network. The controller 106 also includes input / output devices 1008 (e.g., a display, keyboard, mouse, speaker, buttons, etc.) for enabling a user to interact with the controller 106. Those skilled in the art will recognize that an actual controller implementation may also include other components, and for illustrative purposes, this description of the controller 106 is a high-level representation of some of the components of such a controller.

[0016] In one embodiment, the controller 106 analyzes images received from the stereo camera 102 to identify markers such as the marker 200 displayed in the images captured by the stereo camera 102 and within the camera's field of view. In one embodiment, the controller 106 determines the position and pose of the marker relative to the stereo camera based on the images.

[0017] Figure 2 Depicted is a tag 200 in the local coordinate system of the stereo camera 102. In one embodiment, the tag 200 has a unique shape and the tag 200 is encoded with information. In one embodiment, the encoded information identifies the object where the tag 200 is located and the encoded information can also identify the position where the tag is located on the object. In one embodiment, the tag 200 can be a two-dimensional barcode drawn directly on the object or printed on a material (such as an adhesive decal that can be applied to the object). In one embodiment, each tag can contain up to 12 bits of encoded information. In one embodiment, the tag 200 is an AprilTag target. AprilTag is a system using a tag (i.e., AprilTag target) encoded with information, which is captured using one or more cameras. In one embodiment, one of the square tag 36h11 (composed of 587 unique tags) and the circular tag 49h12 (composed of 65535 unique tags) of AprilTag is used. The position and posture of the AprilTag target can be determined based on the image of the AprilTag target captured using one or more cameras. like Figure 2As shown, the marker 200 has four vertices mP1, mP2, mP3 and mP4, which together form a plane square positioned around the center of the marker. In one embodiment, each of the four vertices is positioned at the corner of the plane square. These four vertices are used to define the three axes of the marker 200. Axis Xm is defined as a line formed by vertices mP3 and mP4. Axis Ym is defined as a line formed by vertices mP1 and mP4. Axis Zm is orthogonal to axis Xm and axis Ym and axis Zm intersects with vertex mP4. The rotation around each of these axes is defined as follows. Roll is defined as a clockwise rotation around axis Zm. Pitch is defined as a clockwise rotation around axis Xm. Heading is defined as a clockwise rotation around the Ym axis.

[0018] Figure 3 A local coordinate system 300 of a stereo camera 102 according to an embodiment is shown. The stereo camera 102 is positioned at a point O where the X-axis, Y-axis, and Z-axis intersect. Point O is defined as the upper left corner of the left video sensor 302 of the stereo camera 102. The horizontal axis X is orthogonal to the vertical axis Y, and both the X-axis and the Y-axis are located in the plane of the left video sensor and they are both orthogonal to the Z-axis. In one embodiment, the position and posture of the marker 200 are determined relative to the local coordinate system 300 of the stereo camera 102. The local coordinate system 300 allows the position of a marker or object relative to the stereo camera 102 to be determined without using a larger coordinate system, such as a global coordinate system. The local coordinate system can be used to describe the position of any point relative to the stereo camera 102. If the position and orientation of the stereo camera 102 in a large coordinate system (such as a global coordinate system) is known, the position of an object with a known position in the local coordinate system can be determined for the large coordinate system.

[0019] The parameters of the stereo camera can affect various aspects of marker detection. The maximum detection / recognition distance from a marker of a specific size to the camera depends on the angular resolution of the camera. The better the angular resolution of the camera, the greater the maximum distance between the camera and the marker, so that the marker can be detected. In one embodiment, the camera exposure mode uses a global shutter mode to avoid spatial distortion of the marker shape that can move quickly relative to the camera. The width of the base of the stereo camera can affect the accuracy of marker position determination. In one embodiment, the use of sub-pixel resolution requires a base width of 10 centimeters when the distance between the camera and the marker reaches 6 meters. When the distance between the camera and the marker reaches 10 meters, a 15-centimeter base can be used and the centimeter-level marker position determination accuracy can be maintained. The angular resolution of the camera typically depends on the lens characteristics (e.g., field of view) and sensor resolution of the camera. The method described herein can achieve millimeter-level precision and centimeter-level accuracy (limited by stereo camera calibration) for positioning. The method can provide sub-degree accuracy for marker posture determination.

[0020] Figure 4 A flow chart of a method 400 for determining the position and pose of a marker 200 is shown. In one embodiment, the controller 106 performs the steps of the method 400. At step 402, an image of the marker 200 is acquired by the stereo camera 102. In one embodiment, the marker includes encoded information, and the image acquired by the stereo camera 102 includes two frames, one frame for each lens of the stereo camera 102.

[0021] At steps 402a and 402b, the tag 200 is identified and a plurality of vertices of the 2D projection 304 of the tag 200 are determined for the left frame and the right frame of the stereo camera 102, respectively. In one embodiment, various techniques may be used to perform tag identification and vertex determination, such as techniques related to identification and vertex determination of AprilTags. In one embodiment, an AprilTag detector is used to detect the AprilTag.

[0022] At step 404, the vertices mP1, mP2, mP3, and mP4 of the marker 200 in the camera local coordinate system may be calculated using triangulation based on the camera characteristics (e.g., stereo base and calibration data) and the information about the 2D vertices of the marker projection 304 for the two frames calculated during the previous step. mP[0] is the x-coordinate value of the local camera system 300, mP[1] is the y-coordinate value of the local camera system 300, and mP[2] is the z-coordinate value of the local camera system 300. Thus, index 0 in the array is the x-coordinate, index 1 is the y-coordinate, and index 2 is the z-coordinate.

[0023] At step 406, the center cP of the marker 200 in the camera local coordinate system is determined. In one embodiment, the following equation is used to determine Figure 3 The calculation is performed at the center of the marker 200 in the coordinate system 300 shown in:

[0024] cP[0]=(mP1[0]+mP2[0]+mP3[0]+mP4[0]) / 4

[0025] cP[1]=(mP1[1]+mP2[1]+mP3[1]+mP4[1])4

[0026] cP[2]=(mP1[2]+mP2[2]+mP3[2]+mP4[2]) / 4.

[0027] Among them, mP1, mP2, mP3 and mP4 are calculated at step 404.

[0028] At step 408, the pitch, roll, and heading of the marker 200 are determined. In one embodiment, the pitch, roll, and heading of the marker 200 are determined based on the vertices mP1, mP2, mP3, and mP4 (calculated at step 404).

[0029] The roll of the marker 200 is expressed in degrees and in one embodiment is determined using all four vertices as follows:

[0030] roll1=atan2(mP2[1]-mP1[1], mP2[0]-mP1[0])

[0031] roll2=atan2(mP3[1]-mP4[1], mP3[0]-mP4[0])

[0032] roll=(roll1+roll2) / 2.

[0033] Among them, mP1, mP2, mP3 and mP4 are calculated at step 404.

[0034] In one embodiment, before calculating the heading of the marker 200, the intermediate 3D coordinates iP1, iP2, iP3 and iP4 of each of the four vertices of the marker 200 are calculated. In one embodiment, the intermediate 3D coordinates are calculated as follows:

[0035] iP1[0]=mP1[0]*cos(roll)+mP1[1]*sin(roll)

[0036] iP1[1]=-mP1[0]*sin(roll)+mP1[1]*cos(roll)

[0037] iP1[2]=mP1[2].

[0038] iP2[0]=mP2[0]*cos(rOll)+mP2[1]*sin(roll)

[0039] iP2[1]=-mP2[0]*sin(roll)+mP2[1]*cos(roll)

[0040] iP2[2]=mP2[2].

[0041] iP3[0]=mP3[0]*cos(roll)+mP3[1]*sin(roll)

[0042] iP3[1]=-mP3[0]*sin(roll)+mP3[1]*cos(roll)

[0043] iP3[2]=mP3[2].

[0044] Among them, mP1, mP2, mP3 and mP4 are calculated at step 404.

[0045] The heading of the marker 200 is expressed as an angle, and in one embodiment the heading of the marker 200 is determined as follows:

[0046] heading1=-atan2(iP2[2]-iP1[2], iP2[0]-iP1[0])

[0047] heading 2=-atan2(iP3[2]-iP4[2], iP3[0]-iP4[0])

[0048] heading=(heading 1+heading 2) / 2.

[0049] In one embodiment, before calculating the pitch of the marker 200, the intermediate 3D coordinates jP1, jP2, jP3 and jP4 of each of the four vertices of the marker 200 are calculated. In one embodiment, the intermediate 3D coordinates are calculated as follows:

[0050] jP1[0]=iP1[0]*cos(heading)-iP1[2]*sin(heading)

[0051] jP1[1]=iP1[1]

[0052] jP1[2]=iP1[0]*sin(heading)+iP1[2]*cos(heading).

[0053] jP2[0]=iP2[0]*cos(heading)-iP2[2]*sin(heading)

[0054] jP2[1]=iP2[1]

[0055] jP2[2]=iP2[0]*sin(heading)+iP2[2]*cos(heading).

[0056] jP3[0]=iP3[0]*cos(heading)-iP3[2]*sin(heading)

[0057] jP3[1]=iP3[1]

[0058] jP3[2]=iP3[0]*sin(heading)+iP3[2]*cos(heading).

[0059] jP4[0]=iP4[0]*cos(heading)-iP4[2]*sin(heading)

[0060] jP4[1]=iP4[1]

[0061] jP4[2]=iP4[0]*sin(heading)+iP4[2]*cos(heading).

[0062] Among them, iP1, iP2, iP3 and iP4 are calculated at the previous step.

[0063] The pitch of the marker 200 is expressed in degrees and in one embodiment is determined as follows:

[0064] pitch1=atan2(jP1[2]-jP4[2], jP1[1]-jP4[1])

[0065] pitch2=atan2(jP2[2]-jP3[2], jP2[1]-jP3[1])

[0066] pitch=(pitch1+pitch2) / 2.

[0067] At step 410, the position and orientation of the stereo camera 102 is determined. In one embodiment, the position and orientation of the stereo camera 102 is determined based on information received by the controller 106 from the navigation system 108. In one embodiment, the navigation system 108 determines the position and orientation of the stereo camera 102 and transmits this data to the controller 106. In one embodiment, the data is the longitude coordinates, latitude coordinates, altitude coordinates, and orientation angles (pitch, roll, and heading) of the vehicle 104, thereby identifying the position and orientation of the navigation system 108. Based on the position and orientation of the stereo camera 102 on the vehicle 104 relative to the navigation system 108, the position and orientation of the stereo camera 102 is determined. In one embodiment, the relative position and orientation of the stereo camera 102 relative to the navigation system 108 is stored in the controller 106, and the controller 106 can determine the position and orientation of the stereo camera 102 based on the position information received from the navigation system 108.

[0068] At step 412, the position of the marker 200 in the global coordinate system is determined. In one embodiment, the position of the marker 200 is determined based on the position of the center of the marker 200 as determined at step 406. The position of the center of the marker 200 in the coordinate system 300 of the stereo camera 102 is known. Since the position and orientation of the stereo camera 102 are known from step 410, and the position of the center of the marker 200 relative to the stereo camera 102 is known, the position of the marker 200 can also be determined. At the same step, the attitude of the marker 200 is determined based on the pitch, roll and heading of the marker 200 determined in step 408. Since the position and orientation of the stereo camera 102 are known from step 410, and the attitude of the marker 200 relative to the stereo camera 102 is known from step 408, the attitude of the marker 200 in the global coordinate system can also be determined. It should be noted that method 400 is a non-contact method for determining the position and attitude of a marker.

[0069] The determined position and posture of the marker can be used to determine the position and posture of the object where one or more markers are located. The method 400 for determining the position and posture of the marker 200 can be used in various applications. For example, the application can use one or more markers to determine the position and / or posture of the object where the marker is located. Several such applications are described below.

[0070] Figure 5An application of the method 400 for determining the position and attitude of a marker is shown, wherein the marker is positioned on a construction machine. A bulldozer 500 has a blade 504 that can be used for leveling operations and other surface finishing operations. Markers 502a and 502b are arranged on the blade 504, which is in the field of view 506 of a stereo camera 508 positioned on the bulldozer 500. The position and attitude of the blade 504 in the local coordinate system of the stereo camera 102 can be determined based on the position and attitude of the markers 502a and 502b positioned on the blade 504 using the techniques described above. The position and attitude of the blade 504 in the world coordinate system can be determined based on the positioning of the bulldozer 500 and the position and attitude of the blade 504 in the local coordinate system. In one embodiment, the stereo camera 102 can be positioned on a mast that is arranged to have a moving marker positioned within the camera field of view. The non-contact nature of method 400 allows the three-dimensional position and orientation of the machine and machine implements to be calculated, the volume of material prisms (e.g., soil prismatic structures) to be estimated to prevent them from passing through the blade, and the speed of the tracked vehicle to be determined for comparison with the speed of the vehicle's tracks to calculate a coefficient that characterizes the degree of slippage of the vehicle when the vehicle is moving under different conditions (e.g., heavy loads).

[0071] Figure 6 An application of the method 400 for determining the position and attitude of a marker is shown, where the marker is positioned on a towed implement 602. The tractor 600 is towing the towed implement 602, which is configured to perform an agricultural operation. The position and attitude of the towed implement 602 is known based on the position and attitude of markers 604a and 604b positioned in the field of view 606 of a stereo camera 608 mounted on the tractor 600. In one embodiment, the method described herein can be used to estimate the height of the towed implement 602. Therefore, operations that require monitoring the height of the marker, such as the depth of a plow sunk into the ground, can be determined.

[0072] Figure 7An application of method 400 for determining the position and attitude of a marker is shown, wherein the movement of a vehicle 700 (e.g., an asphalt paver) is controlled based on markers located in an environment in which the vehicle is operating. The vehicle 700 is shown as traveling through a tunnel having markers 704a, 704b, 704c, 704d located on a fixed object in the environment (e.g., one wall of the tunnel) and markers 706a, 706b, 706c, and 706d located on an opposite fixed object in the environment (e.g., the opposite wall of the tunnel). The markers are detected by one or more stereo cameras 708a and 708b (e.g., one stereo camera pointing to one side of the vehicle 700 and one stereo camera pointing to the opposite side of the vehicle 700). The markers 704b and 704c are shown as being located within a field of view 702a of a stereo camera 708a pointing to one side of the vehicle 700. Markers 706b and 706c are shown as being positioned within the field of view 702b of a stereo camera 708b pointing to the opposite side of the vehicle 700. In one embodiment, the positioning of each marker is based on the field of view of each stereo camera, so that at least one marker on each tunnel wall is positioned within the field of view of the corresponding stereo camera when the stereo camera moves. In one embodiment, the position of the marker relative to the starting point of the tunnel is encoded in the shape of the marker. For example, the marker ID reflects the distance (meters) of the marker relative to the beginning of the tunnel. The position and posture of the marker positioned within the field of view of the stereo camera are used to determine the position and posture of the vehicle relative to the marker based on the position and posture of the stereo camera. The movement (e.g., speed and direction) of the vehicle is controlled based on the positioning of the determined vehicle and the positioning of the marker. Therefore, the method can be used to control the movement and operation of the autonomous vehicle, or / and when the GNSS system cannot be used (e.g., when the GNSS signal cannot be received in the tunnel), the method can be used to control the movement and operation of the autonomous vehicle. In one embodiment, the markers may be arranged around the periphery of an area (e.g., on the walls around a room) to determine the location of the machine within the area. For example, if the position of one or more markers in a local coordinate system is known, the local coordinates of the machine within the room may be determined by triangulation.

[0073] Figure 8Shown is the application of the method 400 for determining the position and attitude of a mark, wherein the mark is positioned on a plurality of automatic vehicles or autonomous vehicles. The mark positioned on each vehicle allows other vehicles to identify and determine the position and attitude of each vehicle. Combine harvester 800 is shown as having a mark 802, which allows other autonomous vehicles or automatic vehicles to identify this mark 802 and locate the position and orientation of this mark 802. When automatic vehicles or autonomous vehicles are used as a group, this type of information can be used for a collision avoidance system.

[0074] Fig. 9 An application of method 400 for determining the position and pose of a marker is shown, wherein the marker is placed at a parking spot where an automated vehicle or autonomous vehicle is to be parked (e.g., the parking spot is drawn or painted on the parking space). Tractor 900 has a stereo camera 908, which is used to capture images of the marker. Markers 902 and 904 are used to identify vehicle parking spaces P9 and P10, respectively. Stereo camera 908 of tractor 900 captures images of the markers positioned within the field of view 906 of the stereo camera. Markers 902 and 904 identify their respective parking spaces and allow tractor 900 to move into the designated space based on the position and pose of the respective markers. In one embodiment, the autonomous vehicle approaches 910 parking space P9, thereby attempting to minimize both the distance between the camera and marker 902 and the roll angle for marker 902.

[0075] The foregoing detailed description should be understood to be illustrative and exemplary in all respects, rather than restrictive, and the scope of the inventive concept disclosed herein should be interpreted in accordance with the full scope permitted by patent law. It should be understood that the embodiments shown and described herein are only used to illustrate the principles of the inventive concept, and that various modifications may be implemented by those skilled in the art without departing from the scope and spirit of the inventive concept. Various other feature combinations may be implemented by those skilled in the art without departing from the scope and spirit of the inventive concept.

Claims

1. A method, the method include: Acquire an image of a marker on an object from a stereo camera, wherein the marker includes encoded information; Based on the image, determining a position of the marker relative to a local coordinate system of the stereo camera; and Based on the position of the marker, the position of the object relative to the local coordinate system of the stereo camera is determined.

2. The method according to claim 1, in, The encoded information identifies the object.

3. The method according to claim 2, in, The encoded information identifies the location of the marker on the object.

4. The method according to claim 1, further comprising: include: Determining the position and orientation of the stereo camera in a global coordinate system; as well as The location of the object in the global coordinate system is determined based on the position of the object relative to the local coordinate system and the location of the stereo camera in the global coordinate system.

5. The method according to claim 1, further comprising: include: determining a center of the marker based on the image, Wherein, the position of the marker relative to the local coordinate system of the stereo camera is also determined based on the center of the marker.

6. The method according to claim 1, further comprising: include: determining a roll of the marker; determining a heading of the mark; determining the elevation of the marker; as well as The attitude of the marker is determined based on the pitch of the marker, the roll of the marker, and the heading of the marker.

7. The method according to claim 6, further comprising: include: A pose of the object is determined based on the pose of the marker.

8. The method according to claim 6, further comprising: include: A plurality of vertices surrounding the center of the marker are determined, Wherein, the posture of the marker is also determined based on the multiple vertices.

9. The method according to claim 3, in, The position of the object is also determined based on the position of the marker on the object.

10. The method according to claim 1, in, The stereo camera is attached to a machine and the object is an appliance attached to the machine.

11. The method according to claim 10, in, The machine is a bulldozer and the implement is a blade.

12. The method according to claim 10, in, The machine is a tractor and the object is a towed machine.

13. The method according to claim 1, in, The stereo camera is attached to a first vehicle and the object is a second vehicle.

14. A method, the method comprising: include: acquiring an image of a marker from a stereo camera attached to a machine, the marker being located in an environment in which the machine is located and the marker including encoded information; Determining, based on the image, a position of the marker relative to a local coordinate system of the stereo camera; as well as Based on the position of the marker relative to the local coordinate system of the stereo camera and the encoded information, the positioning of the machine relative to a global coordinate system is determined.

15. The method according to claim 14, in, The encoded information identifies the location of the marker in the environment relative to the global coordinate system.

16. The method according to claim 15, in, The machine is an asphalt paver and the marking is located on a fixed object in the environment.

17. The method according to claim 15, in, The machine is a vehicle and the marker is located on a stationary object.

18. The method of claim 15, in, The machine is a first vehicle and the indicia is located on a second vehicle.

19. A device, wherein the device include: Stereo camera; A controller in communication with the stereo camera, the controller being configured to perform the following operations, the operations comprising: acquiring an image of a marker on an object from the stereo camera, wherein the marker includes encoded information; determining a position of the marker relative to a local coordinate system of the stereo camera based on the image; and The position of the object relative to the local coordinate system of the stereo camera is determined based on the position of the marker.

20. The device according to claim 19, in, The encoded information identifies the object.

21. The device according to claim 20, in, The encoded information identifies the location of the marker on the object.

22. The device according to claim 21, in, The determination of the position of the object is also based on the position of the marker on the object.

23. The apparatus according to claim 19, wherein the operation further comprises: include: Determining the positioning of the stereo camera in a global coordinate system; as well as The location of the object in the global coordinate system is determined based on the position of the object relative to the local coordinate system and the location of the stereo camera in the global coordinate system.

24. The apparatus according to claim 19, wherein the operation further comprises: include: determining a roll of the marker; determining a heading of the mark; determining the elevation of the marker; as well as The attitude of the marker is determined based on the pitch of the marker, the roll of the marker, and the heading of the marker.

Images