A target positioning method and device

Abstract

An embodiment of the present invention discloses a target positioning method and device. The method includes: determining image position information of a spare ground traffic marker in a road image based on a preset object detection model and the road image, where the road image is an image collected by an image acquisition device set for a target to be positioned; determining the top-down projection position information of the spare ground traffic marker in the ground coordinate system by using the internal parameter information of the image acquisition device, the first conversion relationship between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be positioned, and the image position information of the spare ground traffic marker; obtaining the initial pose information of the target to be positioned; and determining the target pose information of the target to be positioned based on the initial pose information, a preset ground traffic marker map, the device projection model of the image acquisition device, the first conversion relationship, and the top-down projection position information, so as to improve the accuracy and robustness of the positioning result.

Description

Technical Field

[0001] The present invention relates to the technical field of positioning, and more specifically, to a target positioning method and device. Background Art

[0002] Currently, in the autonomous positioning solutions of target objects such as autonomous driving vehicles and robots, generally: positioning is performed based on the visual information collected by the image acquisition device set on the target object. In the above process of positioning based on visual information, generally: the position information of elements such as street lamp poles and traffic signs in the visual information, and the initial pose information determined by the target object through an odometer are used to determine the map elements matching the street lamp poles and traffic signs in the visual information from a scene map preset with the map position information of such traffic elements, and then, the position information of the street lamp poles and traffic signs and the map position information of their matching map elements in the scene image are used to determine the more accurate pose information of the autonomous driving vehicle or the robot.

[0003] However, traffic elements such as the above-mentioned street lamp poles and traffic signs are generally sparse in the actual traffic scene, which to a certain extent affects the accuracy and robustness of the positioning result of the target object in the above positioning process. Summary of the Invention

[0004] The present invention provides a target positioning method and device to improve the accuracy and robustness of the positioning result. The specific technical solutions are as follows:

[0005] In a first aspect, an embodiment of the present invention provides a target positioning method, the method includes:

[0006] Based on a preset object detection model and a road image, determine the image position information of spare ground traffic markers in the road image, where the road image is an image collected by an image acquisition device set on a target to be located;

[0007] Using the internal parameter information of the image acquisition device, the first conversion relationship between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be located, and the image position information of the spare ground traffic markers, determine the top-down projection position information of the spare ground traffic markers in the ground coordinate system, where the ground coordinate system is a three-dimensional rectangular coordinate system with the plane where the horizontal axis and the vertical axis are located parallel to the ground;

[0008] Obtain the initial pose information of the target to be located;

[0009] Determine the target pose information of the target to be located based on the initial pose information, the preset ground traffic marker map, the device projection model of the image acquisition device, the first conversion relationship, and the top-down projection position information, where the preset ground traffic marker map includes: each map ground traffic marker and its map pose information.

[0010] Optionally, the step of determining the target pose information of the target to be located based on the initial pose information, the preset ground traffic marker map, the internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information includes:

[0011] Based on the initial pose information, determine the local map corresponding to the initial pose information from the preset ground traffic marker map;

[0012] Use the initial pose information, the map ground traffic markers and their map pose information in the local map, the internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information to determine the target pose information of the target image to be located.

[0013] Optionally, the initial pose information is the pose information in the world coordinate system, and the map pose information of each map ground traffic marker is the pose information in the world coordinate system;

[0014] The step of using the initial pose information, the map ground traffic markers and their map pose information in the local map, the internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information to determine the target pose information of the target image to be located includes:

[0015] Use the initial pose information, the second conversion relationship between the device coordinate system and the coordinate system where the target to be located is located, the device projection model of the image acquisition device, and the map pose information of each map ground traffic marker in the local map to determine the mapping position information of each map ground traffic marker in the road image;

[0016] Based on the mapping position information of each map ground traffic marker in the road image, the internal reference information of the image acquisition device, and the first conversion relationship, determine the top-down mapping position information of each map ground traffic marker in the ground coordinate system;

[0017] Use the top-down mapping position information of each map ground traffic marker and the top-down projection position information of the backup ground traffic marker to determine the target pose information of the target to be located.

[0018] Optionally, the step of determining the top-down projection position information of the backup ground traffic marker in the ground coordinate system by using the internal parameter information of the image acquisition device, the first conversion relationship between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be located, and the image position information of the backup ground traffic marker includes:

[0019] Using the internal parameter information of the image acquisition device and the image position information of the backup ground traffic marker to determine the device position information of the backup ground traffic marker in the device coordinate system of the image acquisition device;

[0020] Using the device position information of the backup ground traffic marker and the first conversion relationship between the device coordinate system and the ground coordinate system corresponding to the target to be located to determine the top-down projection position information of the backup ground traffic marker in the ground coordinate system.

[0021] Optionally, the step of obtaining the initial pose information of the target to be located includes:

[0022] Obtaining the sensor data collected by each of the other sensors provided on the target to be located;

[0023] Based on the sensor data collected by each of the other sensors, determining the initial pose information of the target to be located.

[0024] Optionally, the backup ground traffic marker includes at least one of a lane line, a parking space, a stop line, a zebra crossing, and a traffic indication arrow.

[0025] In a second aspect, an embodiment of the present invention provides a target positioning device, and the device includes:

[0026] A first determination module, configured to determine the image position information of the backup ground traffic marker in the road image based on a preset object detection model and the road image, where the road image is an image collected by an image acquisition device provided on the target to be located;

[0027] A second determination module, configured to use the internal parameter information of the image acquisition device, the first conversion relationship between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be located, and the image position information of the backup ground traffic marker to determine the top-down projection position information of the backup ground traffic marker in the ground coordinate system, where the ground coordinate system is a three-dimensional rectangular coordinate system in which the plane of the horizontal axis and the vertical axis is parallel to the ground;

[0028] An acquisition module, configured to acquire the initial pose information of the target to be located;

[0029] A third determination module, configured to determine target pose information of the target to be located based on the initial pose information, a preset ground traffic marker map, a device projection model of the image acquisition device, the first conversion relationship, and the top-down projection position information, where the preset ground traffic marker map includes: various map ground traffic markers and their map pose information.

[0030] Optionally, the third determination module includes:

[0031] A first determination unit, configured to determine a local map corresponding to the initial pose information from the preset ground traffic marker map based on the initial pose information;

[0032] A second determination unit, configured to determine target pose information of the target image to be located by using the initial pose information, map ground traffic markers and their map pose information in the local map, internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information.

[0033] Optionally, the initial pose information is pose information in a world coordinate system, and the map pose information of each map ground traffic marker is pose information in the world coordinate system;

[0034] The second determination unit is specifically configured to determine mapping position information of each map ground traffic marker in the road image by using the initial pose information, a second conversion relationship between the device coordinate system and the coordinate system where the target to be located is located, the device projection model of the image acquisition device, and the map pose information of each map ground traffic marker in the local map;

[0035] Based on the mapping position information of each map ground traffic marker in the road image, the internal reference information of the image acquisition device, and the first conversion relationship, determine top-down mapping position information of each map ground traffic marker in the ground coordinate system;

[0036] Determine the target pose information of the target to be located by using the top-down mapping position information of each map ground traffic marker and the top-down projection position information of the spare ground traffic marker.

[0037] Optionally, the second determination module is specifically configured to determine device position information of the spare ground traffic marker in the device coordinate system of the image acquisition device by using the internal reference information of the image acquisition device and the image position information of the spare ground traffic marker;

[0038] Using the device position information of the spare ground traffic marker and the first conversion relationship between the device coordinate system and the ground coordinate system corresponding to the target to be located, determine the top-down projection position information of the spare ground traffic marker in the ground coordinate system.

[0039] Optionally, the obtaining module is specifically configured to obtain sensor data collected by each other sensor provided by the target to be located;

[0040] Based on the sensor data collected by each of the other sensors, determine the initial pose information of the target to be located.

[0041] Optionally, the spare ground traffic marker includes at least one of a lane line, a parking space, a stop line, a zebra crossing, and a traffic indication arrow.

[0042] As can be seen from the above, a target positioning method and device provided by an embodiment of the present invention determine the image position information of a spare ground traffic marker in a road image based on a preset object detection model and the road image, where the road image is an image collected by an image acquisition device provided by the target to be located; using the internal parameter information of the image acquisition device, the first conversion relationship between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be located, and the image position information of the spare ground traffic marker, determine the top-down projection position information of the spare ground traffic marker in the ground coordinate system, where the ground coordinate system is a three-dimensional rectangular coordinate system in which the plane where the horizontal axis and the vertical axis are located is parallel to the ground; obtain the initial pose information of the target to be located; based on the initial pose information, a preset ground traffic marker map, the device projection model of the image acquisition device, the first conversion relationship, and the top-down projection position information, determine the target pose information of the target to be located, where the preset ground traffic marker map includes each map ground traffic marker and its map pose information.

[0043] Applying the embodiment of the present invention, denser ground traffic markers can be used as the basis for positioning the target to be located, which can improve the accuracy and robustness of the positioning result to a certain extent. And determine the top-down projection position information of the spare ground traffic marker in the ground coordinate system, and then use this top-down projection position information and the map ground traffic markers and their map pose information in the preset ground traffic marker map to determine the target pose information of the target to be located, which can eliminate the problem of objects appearing larger when closer and smaller when farther away in the ground traffic markers caused by the imaging mechanism of the image acquisition device to a certain extent, and can eliminate the influence of height information on the matching of ground traffic markers and subsequent pose calculation, and improve the accuracy and robustness of the determined target pose information of the target to be located, that is, the positioning result to a certain extent. Of course, any product or method implementing the present invention does not necessarily need to achieve all the above advantages at the same time.

[0044] The innovative points of the embodiments of the present invention include:

[0045] 1. Denser ground traffic markers can be used as the basis for positioning the target to be located, which can improve the accuracy and robustness of the positioning result to a certain extent. And by using the top-down projection position information in the ground coordinate system to determine the target pose information, the problem of objects appearing larger when closer and smaller when farther away in the ground traffic markers caused by the imaging mechanism of the image acquisition device can be eliminated to a certain extent, and the influence of height information on the matching of ground traffic markers and subsequent pose calculation can be eliminated, so as to improve the accuracy and robustness of the target pose information of the target to be located, that is, the positioning result, to a certain extent.

[0046] 2. The ground traffic markers on the map are converted from the coordinate system corresponding to the map to the image coordinate system, and then from the image coordinate system to the ground coordinate system to obtain the top-down mapping position information of the ground traffic markers on the map. Then, the top-down mapping position information of the ground traffic markers on the map and the top-down projection position information of the spare ground traffic markers are used for residual optimization to determine the target pose information of the target to be located, so as to eliminate the influence of height information on the matching of ground traffic markers and the calculation of pose information, and improve the accuracy and robustness of the determined target pose information, that is, the positioning result. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0048] Figure 1 It is a schematic flowchart of a target positioning method provided by an embodiment of the present invention;

[0049] Figure 2 It is a schematic structural diagram of a target positioning device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0051] It should be noted that the terms "include" and "have" in the embodiments of the present invention and the accompanying drawings, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0052] The present invention provides a target positioning method and device to improve the accuracy and robustness of the positioning result. The embodiments of the present invention will be described in detail below.

[0053] Figure 1 It is a schematic flowchart of a target positioning method provided by an embodiment of the present invention. The method may include the following steps:

[0054] S101: Based on a preset object detection model and a road image, determine the image position information of the standby ground traffic markers in the road image.

[0055] Among them, the road image is an image collected by an image acquisition device set for the target to be positioned. The target to be positioned can be an autonomous driving vehicle or a robot. The target to be positioned is provided with at least one image acquisition device, which can collect images of the environment where the target to be positioned is located. The target to be positioned is also provided with various other sensors for assisting in the positioning of the target to be positioned. The other sensors may include, but are not limited to, wheel speed sensors, IMUs (Inertial Measurement Units), GPSs (Global Positioning Systems), and GNSSs (Global Navigation Satellite Systems), etc.

[0056] The target positioning method provided by the embodiments of the present invention can be applied to any electronic device with computing capabilities, and the electronic device can be a terminal or a server. In one implementation, the functional software for implementing the target positioning method can exist in the form of a separate client software or in the form of a plugin of the current relevant client software, which is acceptable.

[0057] In one implementation, the target to be located is an autonomous vehicle. Correspondingly, the electronic device can be an in-vehicle device, which is set on the target to be located. The electronic device can directly obtain the road images collected by the image acquisition device set on the target to be located and the sensor data collected by other sensors set thereon. The electronic device can also be a non-vehicle-mounted device. Correspondingly, the electronic device can be connected to the in-vehicle device in the target to be located to obtain the road images collected by the image acquisition device in the target to be located, the sensor data collected by other sensors set on the target to be located, or the initial pose information of the target to be located determined by the in-vehicle device based on the sensor data collected by other sensors.

[0058] In one case, there are multiple image acquisition devices set on the target to be located. The multiple image acquisition devices can take panoramic pictures of the environment where the target to be located is located to obtain multiple road images of the environment around the target to be located. The road information of the environment where the target to be located is located is included in the road images. Correspondingly, the electronic device can execute the target positioning method provided in the embodiments of the present invention for each road image to obtain a positioning result with high accuracy.

[0059] After the electronic device obtains the road image, it detects the road image based on a preset object detection model to determine the ground traffic markers in the road image as the standby ground traffic markers, and determines the image position information of each standby ground traffic marker in the road image.

[0060] The preset object detection model is a neural network model pre-trained based on sample images marked with ground traffic markers. The training process thereof can refer to the training process of the neural network model in the related art and will not be elaborated herein. Among them, the ground traffic markers include standby traffic markers and subsequent map traffic markers.

[0061] In one implementation manner of the present invention, the standby ground traffic markers include at least one of lane lines, parking spaces, stop lines, zebra crossings, and traffic indication arrows.

[0062] S102: Use the internal reference information of the image acquisition device, the first conversion relationship between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be located, and the image position information of the standby ground traffic markers to determine the top-down projection position information of the standby ground traffic markers in the ground coordinate system.

[0063] Among them, the ground coordinate system is a three-dimensional rectangular coordinate system in which the plane where the horizontal axis and the vertical axis are located is parallel to the ground.

[0064] The electronic device pre-stores, in its local or connected storage device, the internal parameter information of the image acquisition device set for the target to be located, as well as the position conversion relationship, i.e., the first conversion relationship, between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be located.

[0065] Among them, the internal parameter information of the image acquisition device may include: the focal length fx in the horizontal axis direction in the image coordinate system of the image acquisition device, the focal length fy in the vertical axis direction in the image coordinate system of the image acquisition device. Generally, the two are equal; the position information of the principal point of the image in the imaging plane of the image acquisition device, and the axis tilt parameter s of the image coordinate system.

[0066] In this step, the electronic device can determine the device position information of the spare ground traffic marker in the device coordinate system of the image acquisition device based on the internal parameter information of the image acquisition device and the image position information of the spare ground traffic marker; furthermore, using the first conversion relationship and the device position information of the spare ground traffic marker, determine the position information of the spare ground traffic marker in the ground coordinate system as the top-down projection position information. The ground coordinate system is a three-dimensional rectangular coordinate system, the plane where its horizontal axis and vertical axis are located is parallel to the ground, and the vertical axis is perpendicular to the ground and upward.

[0067] Among them, in the top-down projection position information of the spare ground traffic marker in the ground coordinate system, the coordinate value of the vertical axis may not be included, that is, there is no height information, which can, to a certain extent, eliminate the influence of height information on the subsequent matching of the spare ground traffic marker and the determination of the target pose information and positioning result of the target to be located, and improve the positioning accuracy.

[0068] S103: Obtain the initial pose information of the target to be located.

[0069] Among them, the initial pose information includes: the initial position information and attitude information of the target to be located in the preset three-dimensional rectangular coordinate system. In one case, the preset three-dimensional rectangular coordinate system is the world coordinate system.

[0070] In one implementation, the electronic device can directly obtain the initial pose information of the target to be located, where the initial pose information is the pose information determined by any combination of the sensor data collected by other sensors of the target to be located and / or the road image. For example: it can be: the initial pose information is determined by the combination of the road image and the sensor data collected by the IMU, the initial pose information is determined by the combination of the road image and the sensor data collected by the GNSS, the initial pose information is determined by the combination of the road image and the sensor data collected by the wheel speed sensor, and the initial pose information is determined by the combination of the sensor data collected by the wheel speed sensor, the sensor data collected by the IMU, and the sensor data collected by the GNSS, etc.

[0071] S104: Determine the target pose information of the target to be located based on the initial pose information, the preset ground traffic marker map, the device projection model of the image acquisition device, the first conversion relationship, and the top-down projection position information.

[0072] Among them, the preset ground traffic marker map includes: each map ground traffic marker and its map pose information. Each of these map ground traffic markers exists in the preset ground traffic marker map in the form of a three-dimensional point cloud. The map ground traffic markers include, but are not limited to: lane lines, parking spaces, stop lines, zebra crossings, traffic indicator arrows, etc.

[0073] The preset ground traffic marker map is pre-stored in the local electronic device or the connected storage device. After obtaining the initial pose information of the target to be located, in one implementation, the electronic device can directly determine, based on the initial pose information, a local area near the location of the target to be located from the preset ground traffic marker map as the local map. Among them, the local map includes the map ground traffic markers. Furthermore, use the initial pose information, the map pose information of the map ground traffic markers in the local map and their map pose information, the internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information to determine the target pose information of the target image to be located. Specifically, it can be: match the map ground traffic markers in the local map with the spare ground traffic markers, and use the map pose information of the mutually matching map ground traffic markers, the top-down projection position information of the spare ground traffic markers, the initial pose information, the internal reference information of the image acquisition device, and the first conversion relationship to determine the target pose information of the target image to be located.

[0074] In another embodiment of the present invention, the initial pose information is the pose information in the world coordinate system, and the map pose information of each map ground traffic marker is the pose information in the world coordinate system;

[0075] The step of using the initial pose information, the map pose information of the map ground traffic markers in the local map and their map pose information, the internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information to determine the target pose information of the target image to be located may include the following steps 011-013:

[0076] 011: Use the initial pose information, the second conversion relationship between the device coordinate system and the coordinate system where the target to be located is located, the device projection model of the image acquisition device, and the map pose information of each map ground traffic marker in the local map to determine the mapping position information of each map ground traffic marker in the road image.

[0077] 012: Determine the top-down mapping position information of each map ground traffic marker in the ground coordinate system based on the mapping position information of each map ground traffic marker in the road image, the internal reference information of the image acquisition device, and the first conversion relationship.

[0078] 013: Use the top-down mapping position information of each map ground traffic marker and the top-down projection position information of the spare ground traffic marker to determine the target pose information of the target to be located.

[0079] In this implementation, the electronic device can use the initial pose information, the second conversion relationship between the device coordinate system and the coordinate system where the target to be located is located, the device projection model of the image acquisition device, and the map pose information of each map ground traffic marker in the local map to convert each map ground traffic marker in the local map from the world coordinate system to the coordinate system where the target to be located, and then to the device coordinate system of the image acquisition device, and then to the image coordinate system where the road image is located, so as to determine the mapping position information of each map ground traffic marker in the road image.

[0080] Subsequently, use the mapping position information of each map ground traffic marker in the road image, the internal reference information of the image acquisition device, and the first conversion relationship to convert each map ground traffic marker from the image coordinate system to the device coordinate system of the image acquisition device, and then to the ground coordinate system, so as to determine the top-down mapping position information of each map ground traffic marker in the ground coordinate system to eliminate the height information of the map ground traffic marker.

[0081] Furthermore, use the top-down mapping position information of each mutually matching map ground traffic marker and the top-down projection position information of the spare ground traffic marker to construct an objective function; adjust the initial pose information based on the function value of the objective function until the function value of the objective function reaches the preset convergence condition, and determine the adjusted initial pose information in this case, that is, the target pose information of the target to be located. Among them, the preset convergence condition can be: the sum of the position residuals between each mutually matching map ground traffic marker and the spare ground traffic marker is not higher than the preset residual threshold.

[0082] In one case, the ground coordinate system coincides with the coordinate system where the target to be located is located. In another case, the translated ground coordinate system coincides with the coordinate system where the target to be located is located.

[0083] Among them, in the case where the ground coordinate system coincides with the coordinate system where the target to be located is located, the mapping position information of the map ground traffic marker in the road image can be determined by the following formula (1):

[0084]

[0085] Among them, represents the top-down mapping position information of the i-th map ground traffic marker in the ground coordinate system, T wb represents the position conversion relationship between the coordinate system where the target to be located is located and the world coordinate system, that is, the initial pose information; T bc represents the position conversion relationship between the device coordinate system and the coordinate system where the target to be located is located, that is, the second conversion relationship; represents the map pose information of the i-th map ground traffic marker; Π(·) represents determining the mapping position information of the map ground traffic marker in the road image, that is, using the initial pose information, the second conversion relationship, and the device projection model of the image acquisition device to map the ground traffic marker from the world coordinate system to the image coordinate system where the road image is located; Ω(·) represents determining the top-down mapping position information of the map ground traffic marker in the ground coordinate system, that is, using the internal parameter information of the image acquisition device and the first conversion relationship to project the ground traffic marker from the image coordinate system to the ground coordinate system; among them, the value range of i is [1, N], and N represents the number of map ground traffic markers.

[0086] The process of determining the target pose information of the target to be located can be represented by the following formula (2):

[0087]

[0088] Among them, I(·) represents the energy value function of each traffic marker. Among them, the closer the point in the traffic marker is to the center position of the traffic marker, the greater the energy value of the point determined by using this energy value function. This energy value function can be a Gaussian blur function. represents the top-down projection position information of the backup ground traffic marker matched with the i-th map ground traffic marker. This traffic marker includes the backup traffic marker and the map ground traffic marker in the local image. f(T wb ) represents the objective function. When the function value of this f(T wb ) reaches the preset convergence condition, the adjusted initial pose information at this time is determined as the target pose information.

[0089] By applying the embodiments of the present invention, denser ground traffic markers can be used as the basis for positioning the target to be located, which can improve the accuracy and robustness of the positioning result to a certain extent. Moreover, by using the top-down projection position information in the ground coordinate system to determine the target pose information, the problem of objects appearing larger when closer and smaller when farther away in the ground traffic markers caused by the imaging mechanism of the image acquisition device can be eliminated to a certain extent, and the influence of height information on the matching of ground traffic markers and subsequent pose calculation can be eliminated, thereby improving the accuracy and robustness of the target pose information, i.e., the positioning result, of the target to be located to a certain extent.

[0090] In another embodiment of the present invention, step S102 may include the following steps 021-022:

[0091] 021: Determine the device position information of the spare ground traffic marker in the device coordinate system of the image acquisition device by using the internal parameter information of the image acquisition device and the image position information of the spare ground traffic marker.

[0092] 023: Determine the top-down projection position information of the spare ground traffic marker in the ground coordinate system by using the device position information of the spare ground traffic marker and the first conversion relationship between the device coordinate system and the ground coordinate system corresponding to the target to be located.

[0093] In this implementation, the electronic device can convert each spare ground traffic marker from the image coordinate system of the road image to the device coordinate system of the image acquisition device based on the internal parameter information of the image acquisition device and the image position information of the spare ground traffic marker, that is, determine the device position information of the spare ground traffic marker in the device coordinate system. Furthermore, based on the device position information of the spare ground traffic marker and the first conversion relationship between the device coordinate system and the ground coordinate system corresponding to the target to be located, the spare ground traffic marker is converted from the device coordinate system to the ground coordinate system to determine the top-down projection position information of the spare ground traffic marker in the ground coordinate system.

[0094] Among them, when converting each spare ground traffic marker from the image coordinate system of the road image by using the internal parameter information of the image acquisition device, the height information of each spare ground traffic marker, that is, the vertical axis coordinate system in the device coordinate system, is not determined. Subsequently, the top-down projection position information of the determined spare ground traffic marker in the ground coordinate system does not include the height information of the spare ground traffic marker, that is, the vertical axis coordinate system in the ground coordinate system, which can eliminate the influence of height information on the matching between traffic markers and the determination of target pose information to a certain extent. And it can solve the problem of objects appearing larger when closer and smaller when farther away in the ground traffic markers caused by the imaging mechanism of the image acquisition device, ensuring the accurate determination of the target pose information.

[0095] In another embodiment of the present invention, step S103 may include the following steps 031-032:

[0096] 031: Obtain the sensor data collected by each other sensor provided by the target to be located.

[0097] 032: Based on the sensor data collected by each other sensor, determine the initial pose information of the target to be located.

[0098] In this implementation, the electronic device can directly obtain the sensor data collected by each other sensor provided by the target to be located. The data of the sensor data collected by each other sensor corresponding to the road image is the data collected in the same acquisition cycle as the road image.

[0099] Furthermore, use any combination of the sensor data collected by each other sensor or the combination with the road image to determine the initial pose information of the target to be located. The process of determining the initial pose information can refer to the process of determining the initial pose information of the target in the related art, which will not be elaborated here.

[0100] Other sensors may include but are not limited to: wheel speed sensors, IMUs, GPSs, and GNSSs, etc.

[0101] Corresponding to the above method embodiment, an embodiment of the present invention provides a target positioning device, as Figure 2 shown, the device may include:

[0102] A first determination module 210, configured to determine the image position information of the spare ground traffic marker in the road image based on a preset object detection model and the road image, where the road image is: an image collected by an image acquisition device provided by the target to be located;

[0103] A second determination module 220, configured to use the internal parameter information of the image acquisition device, the first conversion relationship between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be located, and the image position information of the spare ground traffic marker to determine the top-down projection position information of the spare ground traffic marker in the ground coordinate system, where the ground coordinate system is a three-dimensional rectangular coordinate system in which the plane where the horizontal axis and the vertical axis are located is parallel to the ground;

[0104] An acquisition module 230, configured to acquire the initial pose information of the target to be located;

[0105] A third determination module 240, configured to determine target pose information of the target to be located based on the initial pose information, a preset ground traffic marker map, a device projection model of the image acquisition device, the first conversion relationship, and the top-down projection position information, where the preset ground traffic marker map includes: each map ground traffic marker and its map pose information

[0106] By applying the embodiments of the present invention, denser ground traffic markers can be used as the basis for positioning the target to be located, which can improve the accuracy and robustness of the positioning result to a certain extent. And by using the top-down projection position information in the ground coordinate system to determine the target pose information, the problem of objects appearing larger when closer and smaller when farther away in the ground traffic markers caused by the imaging mechanism of the image acquisition device can be eliminated to a certain extent, and the influence of height information on the matching of ground traffic markers and subsequent pose calculation can be eliminated, so as to improve the accuracy and robustness of the target pose information of the target to be located, that is, the positioning result, to a certain extent.

[0107] In another embodiment of the present invention, the third determination module 240 includes:

[0108] A first determination unit (not shown in the figure), configured to determine a local map corresponding to the initial pose information from the preset ground traffic marker map based on the initial pose information;

[0109] A second determination unit (not shown in the figure), configured to determine target pose information of the target image to be located by using the initial pose information, the map ground traffic markers and their map pose information in the local map, the internal parameter information of the image acquisition device, the first conversion relationship, and the top-down projection position information.

[0110] In another embodiment of the present invention, the initial pose information is pose information in the world coordinate system, and the map pose information of each map ground traffic marker is pose information in the world coordinate system;

[0111] The second determination unit is specifically configured to determine the mapping position information of each map ground traffic marker in the road image by using the initial pose information, the second conversion relationship between the device coordinate system and the coordinate system where the target to be located is located, the device projection model of the image acquisition device, and the map pose information of each map ground traffic marker in the local map;

[0112] Based on the mapping position information of each map ground traffic marker in the road image, the internal parameter information of the image acquisition device, and the first conversion relationship, determine the top-down mapping position information of each map ground traffic marker in the ground coordinate system;

[0113] Determine the target pose information of the target to be located by using the top-down mapping position information of each ground traffic marker on the map and the top-down projection position information of the backup ground traffic marker.

[0114] In another embodiment of the present invention, the second determination module 220 is specifically configured to determine the device position information of the backup ground traffic marker in the device coordinate system of the image acquisition device by using the internal parameter information of the image acquisition device and the image position information of the backup ground traffic marker;

[0115] Determine the top-down projection position information of the backup ground traffic marker in the ground coordinate system by using the device position information of the backup ground traffic marker and the first conversion relationship between the device coordinate system and the ground coordinate system corresponding to the target to be located.

[0116] In another embodiment of the present invention, the acquisition module 230 is specifically configured to acquire the sensor data collected by each other sensor provided on the target to be located;

[0117] Determine the initial pose information of the target to be located based on the sensor data collected by each of the other sensors.

[0118] In another embodiment of the present invention, the backup ground traffic marker includes at least one of a lane line, a parking space, a stop line, a zebra crossing, and a traffic indication arrow.

[0119] The above system and device embodiments correspond to the system embodiment and have the same technical effects as the method embodiment. For specific descriptions, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For specific descriptions, please refer to the method embodiment section and will not be elaborated here. Those of ordinary skill in the art can understand that the drawings are only schematic diagrams of one embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present invention.

[0120] Those of ordinary skill in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be correspondingly changed and located in one or more devices different from this embodiment. The modules in the above embodiment can be combined into one module, or further split into multiple sub-modules.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target positioning method, characterized in that, the method includes: Based on a preset object detection model and a road image, determining the image position information of spare ground traffic markers in the road image, where the road image is an image collected by an image acquisition device set for the target to be positioned; Using the internal reference information of the image acquisition device, the first conversion relationship between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be positioned, and the image position information of the spare ground traffic markers, determining the top-down projection position information of the spare ground traffic markers in the ground coordinate system, including: using the internal reference information of the image acquisition device and the image position information of the spare ground traffic markers to determine the device position information of the spare ground traffic markers in the device coordinate system of the image acquisition device, and using the device position information of the spare ground traffic markers and the first conversion relationship between the device coordinate system and the ground coordinate system corresponding to the target to be positioned to determine the top-down projection position information of the spare ground traffic markers in the ground coordinate system, where the ground coordinate system is a three-dimensional rectangular coordinate system with the plane where the horizontal axis and the vertical axis are located parallel to the ground; Obtaining the initial pose information of the target to be positioned; Based on the initial pose information, a preset ground traffic marker map, the device projection model of the image acquisition device, the first conversion relationship, and the top-down projection position information, determining the target pose information of the target to be positioned, where the preset ground traffic marker map includes: various map ground traffic markers and their map pose information; The step of determining the target pose information of the target to be positioned based on the initial pose information, a preset ground traffic marker map, the internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information includes: Based on the initial pose information, determining a local map corresponding to the initial pose information from the preset ground traffic marker map; Using the initial pose information, the map ground traffic markers and their map pose information in the local map, the internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information, determining the target pose information of the target image to be positioned.

2. The method according to claim 1, characterized in that, the initial pose information is the pose information in the world coordinate system, and the map pose information of each map ground traffic marker is the pose information in the world coordinate system; The step of using the initial pose information, the map ground traffic markers and their map pose information in the local map, the internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information to determine the target pose information of the target image to be positioned includes: Determine the mapping position information of each map ground traffic marker in the road image by using the initial pose information, the second conversion relationship between the device coordinate system and the coordinate system where the target to be located is located, the device projection model of the image acquisition device, and the map pose information of each map ground traffic marker in the local map; Based on the mapping position information of each map ground traffic marker in the road image, the internal parameter information of the image acquisition device, and the first conversion relationship, determine the top-down mapping position information of each map ground traffic marker in the ground coordinate system; Use the top-down mapping position information of each map ground traffic marker and the top-down projection position information of the spare ground traffic marker to determine the target pose information of the target to be located.

3. The method according to any one of claims 1-2, characterized in that, The step of obtaining the initial pose information of the target to be located includes: Obtain the sensor data collected by each other sensor provided by the target to be located; Based on the sensor data collected by each of the other sensors, determine the initial pose information of the target to be located.

4. The method according to any one of claims 1-2, characterized in that, The spare ground traffic markers include at least one of lane lines, parking spaces, stop lines, zebra crossings, and traffic indication arrows.

5. A target positioning device, characterized in that, The device includes: A first determination module, configured to determine the image position information of the spare ground traffic marker in the road image based on a preset object detection model and the road image, where the road image is an image collected by an image acquisition device provided by the target to be located; A second determination module, configured to use the internal parameter information of the image acquisition device, the first conversion relationship between the device coordinate system of the image acquisition device and the ground coordinate system corresponding to the target to be located, and the image position information of the spare ground traffic marker to determine the top-down projection position information of the spare ground traffic marker in the ground coordinate system, where the ground coordinate system is a three-dimensional rectangular coordinate system in which the plane of the horizontal axis and the vertical axis is parallel to the ground; the second determination module is specifically configured to use the internal parameter information of the image acquisition device and the image position information of the spare ground traffic marker to determine the device position information of the spare ground traffic marker in the device coordinate system of the image acquisition device; use the device position information of the spare ground traffic marker and the first conversion relationship between the device coordinate system and the ground coordinate system corresponding to the target to be located to determine the top-down projection position information of the spare ground traffic marker in the ground coordinate system; An acquisition module, configured to acquire the initial pose information of the target to be located; A third determination module, configured to determine the target pose information of the target to be located based on the initial pose information, a preset ground traffic marker map, the device projection model of the image acquisition device, the first conversion relationship, and the top-down projection position information, where the preset ground traffic marker map includes: each map ground traffic marker and its map pose information; The third determination module includes: A first determination unit, configured to determine a local map corresponding to the initial pose information from the preset ground traffic marker map based on the initial pose information; A second determination unit, configured to determine the target pose information of the target image to be located by using the initial pose information, the map ground traffic markers and their map pose information in the local map, the internal reference information of the image acquisition device, the first conversion relationship, and the top-down projection position information.

6. The device according to claim 5, wherein, The initial pose information is the pose information in the world coordinate system, and the map pose information of each map ground traffic marker is the pose information in the world coordinate system; The second determination unit is specifically configured to determine the mapping position information of each map ground traffic marker in the road image by using the initial pose information, the second conversion relationship between the device coordinate system and the coordinate system where the target to be located is located, the device projection model of the image acquisition device, and the map pose information of each map ground traffic marker in the local map; Based on the mapping position information of each map ground traffic marker in the road image, the internal reference information of the image acquisition device, and the first conversion relationship, determine the top-down mapping position information of each map ground traffic marker in the ground coordinate system; Use the top-down mapping position information of each map ground traffic marker and the top-down projection position information of the backup ground traffic marker to determine the target pose information of the target to be located.

Images