A semantic positioning initialization method and device, a terminal device, and a storage medium

Abstract

The application relates to the technical field of semantic positioning, in particular to a semantic positioning initialization method and device, a terminal device and a storage medium, the method comprises the following steps: acquiring a semantic frame for reflecting lane line image data, determining positioning data of a vehicle based on the semantic frame, and determining state information of the positioning data; if the state information is RTK floating point solution, projecting position information corresponding to the positioning data onto each lane center line, and determining initialization candidate points of each lane; acquiring lane line point cloud corresponding to the semantic frame, projecting the lane line point cloud to the position of the initialization candidate points, and performing point-line matching with lane lines of a map to obtain a matching result; determining a semantic positioning initialization point based on the matching result, wherein the semantic positioning initialization point is used for vehicle positioning. The application matches the semantic positioning result by using lane lines and a map, is favorable for realizing positioning initialization, and vehicle positioning is more accurate.

Description

Technical Field

[0001] This invention relates to the field of semantic positioning technology, and in particular to a semantic positioning initialization method, apparatus, terminal device, and storage medium. Background Technology

[0002] Vehicles rely on high-precision integrated inertial navigation systems (INS) for positioning while traveling on highways. However, this positioning function only works properly in open, unobstructed areas with strong satellite signals. When a vehicle passes through tunnels or is surrounded by bridges that obstruct the signal, the long-term trajectory calculations by the INS while waiting for GPS to recover a fixed solution inevitably lead to the accumulation of errors in both the lateral and longitudinal directions, resulting in a final positioning deviation from the map. Therefore, it is clear that current positioning methods relying solely on high-precision integrated INS cannot guarantee accurate positioning.

[0003] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a semantic positioning initialization method, device, terminal equipment and storage medium to address the above-mentioned defects of the prior art, so as to solve the problem that the positioning effect cannot be guaranteed by relying solely on a high-precision combined inertial navigation system in the prior art.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a semantic localization initialization method, wherein the method includes: Obtain semantic frames that reflect lane line image data, determine vehicle positioning data based on the semantic frames, and determine the status information of the positioning data; If the state information is a dynamic floating-point solution, then the position corresponding to the positioning data is projected onto the center line of each lane, and the initial candidate point for each lane is determined. Obtain the lane line point cloud corresponding to the semantic frame, project the lane line point cloud onto the position of the initialized candidate point, and perform point-line matching with the lane lines on the map to obtain the matching result; Based on the matching results, a semantic positioning initialization point is determined, wherein the semantic positioning initialization point is used for vehicle positioning.

[0006] Based on the aforementioned technical means, the embodiments of this application can achieve dynamic floating-point deinitialization, enabling rapid reinitialization even without accurate GPS location information, ensuring that the vehicle does not miss a lane during lane-level positioning. Furthermore, the embodiments of this application utilize the semantic positioning results obtained by matching lane lines with high-precision maps to assist the integrated inertial navigation system in achieving positioning initialization, ensuring that lane-level positioning remains in the correct lane, resulting in more accurate vehicle positioning.

[0007] In one embodiment of this application, determining the location data based on the semantic frame includes: Obtain the timestamp of the semantic frame; Based on the timestamp, the location data corresponding to the timestamp is obtained from the data cache queue.

[0008] Based on the above technical means, the embodiments of this application can obtain the positioning data corresponding to the timestamp of the semantic frame from the data cache queue, such as the positioning data of the most recent frame, which is beneficial to ensure the real-time and accuracy of positioning.

[0009] In one embodiment of this application, if the state information is a dynamic floating-point solution, then projecting the location information corresponding to the positioning data onto the center line of each lane and determining the initial candidate point for each lane includes: Construct a binary search tree for each of the lane centerlines. For each binary search tree, the nearest neighbor search is performed using the position information of the dynamic floating-point solution as the reference point to determine the two closest consecutive points; The position information of the dynamic floating-point solution is projected onto the line connecting two consecutive points to obtain the projection point; The initial candidate point for each lane is determined based on the projection point.

[0010] Based on the above technical means, this application embodiment constructs a binary search tree for the center line of each lane and determines the initial candidate point based on the binary search tree, which is beneficial for quickly determining the initial candidate point.

[0011] In one embodiment of this application, the step of projecting the lane line point cloud onto the location of the initialized candidate point and performing point-line matching with the lane lines on the map to obtain a matching result includes: Based on the timestamp of the semantic frame, the pose of the semantic frame corresponding to each of the initialization candidate points is determined by using a trajectory estimation method. The lane line point cloud is projected onto the semantic frame pose to obtain the projected lane line point cloud. The projected lane line point cloud is matched with the lane lines of the map to obtain the matching result, wherein the matching result includes: the number of successful point-to-line matching, the point-to-line matching success rate, and the median error between points and lines.

[0012] Based on the above technical means, the embodiments of this application use point-line matching to filter initialization candidate points so as to select initialization candidate points that meet the conditions, which helps to ensure accurate positioning.

[0013] In one embodiment of this application, determining the semantic localization initialization point based on the matching result includes: Obtain preset filtering conditions, wherein the filtering conditions are: the number of successful point-line matching in the matching results reaches a first preset value, the point-line matching success rate reaches a second preset value, and the median error from point to line is less than a third preset value; Based on the filtering conditions and the matching results, the initialization candidate points are filtered to determine the initialization candidate points that meet the conditions. Boundary line verification is performed on the initial candidate points that meet the conditions to obtain the semantic positioning initial point.

[0014] Based on the above technical means, the embodiments of this application perform boundary line verification on the initialization candidate points that meet the conditions, so as to further filter out the best candidate points, thereby making the semantic positioning initialization points obtained in subsequent steps more accurate.

[0015] In one embodiment of this application, the step of performing boundary line verification on the qualified initialization candidate points to obtain the semantic localization initialization point includes: For each eligible initial candidate point, calculate the lateral distance between the eligible initial candidate point and the left and right road boundary lines on the map. The lateral distance is compared with the reference lateral distances of the road boundary lines on the left and right sides perceived by the forward-looking vehicle camera, and the boundary line distance between the boundary line perceived by the forward-looking vehicle camera and the map boundary line is determined for the same side boundary line. If the distance between the boundary lines on both sides is less than the fourth preset value, it is determined that both boundary lines have passed the verification. The initialization candidate points that meet the conditions and whose boundary lines on both sides have passed the verification are used as the semantic positioning initialization points.

[0016] Based on the above technical means, in this application embodiment, for each qualified initial candidate point, the boundary lines on both sides are verified. Only when both boundary lines pass the verification can the corresponding qualified initial candidate point be determined as the best candidate point, which is beneficial to improving the positioning effect.

[0017] In one embodiment of this application, the method further includes: If the state information is a fixed solution, then the location corresponding to the positioning data is used as the positioning initialization point.

[0018] Based on the above technical means, in this embodiment of the application, when the status information of the positioning data is a fixed solution, the location corresponding to the positioning data is taken as the positioning initialization point to ensure the smooth progress of positioning initialization.

[0019] Secondly, embodiments of the present invention also provide a semantic positioning initialization device, wherein the device includes: The state determination module is used to acquire semantic frames that reflect lane line image data, determine vehicle positioning data based on the semantic frames, and determine the state information of the positioning data. The alternative point determination module is used to project the location information corresponding to the positioning data onto the center line of each lane if the status information is a dynamic floating-point solution, and determine the initial alternative point for each lane. The point-line matching module is used to obtain the lane line point cloud corresponding to the semantic frame, project the lane line point cloud onto the position of the initial candidate point, and perform point-line matching with the lane lines of the map to obtain the matching result. The result determination module is used to determine the semantic positioning initialization point based on the matching result, wherein the semantic positioning initialization point is used for vehicle positioning.

[0020] In one embodiment of this application, the state determination module includes: A timestamp acquisition unit is used to acquire the timestamp of the semantic frame; The data acquisition unit is used to acquire the location data corresponding to the timestamp from the data cache queue based on the timestamp.

[0021] In one embodiment of this application, the alternative point determination module includes: Binary search tree construction unit, used to construct a binary search tree for each of the lane centerlines. The nearest neighbor search unit is used to perform a nearest neighbor search for each of the binary search trees, using the position information of the dynamic floating-point solution as a reference point, to determine the two closest consecutive points. A position projection unit is used to project the position information of the dynamic floating-point solution onto the line connecting two consecutive points to obtain a projection point. The alternative point determination unit is used to determine the initial alternative point for each lane based on the projection point.

[0022] In one embodiment of this application, the dot-line matching module includes: The trajectory estimation unit is used to determine the pose of the semantic frame corresponding to each of the initialization candidate points by using trajectory estimation based on the timestamp of the semantic frame. A point cloud projection unit is used to project the lane line point cloud onto the semantic frame pose to obtain the projected lane line point cloud. The point-line matching unit is used to match the projected lane line point cloud with the lane lines of the map to obtain the matching result, wherein the matching result includes: the number of successful point-line matching, the point-line matching success rate, and the median error between points and lines.

[0023] In one embodiment of this application, the result determination module includes: The condition acquisition unit is used to acquire preset filtering conditions, wherein the filtering conditions are: the number of successful point-line matching in the matching results reaches a first preset value, the point-line matching success rate reaches a second preset value, and the median error from point to line is less than a third preset value. The filtering execution unit is used to filter the initialization candidate points based on the filtering conditions and the matching results, and determine the initialization candidate points that meet the conditions. The boundary verification unit is used to verify the boundary lines of the initialization candidate points that meet the conditions, so as to obtain the semantic positioning initialization point.

[0024] In one embodiment of this application, the boundary verification unit includes: The distance calculation sub-unit is used to calculate the lateral distance between each eligible initial candidate point and the left and right road boundary lines on the map. The boundary analysis subunit is used to compare the lateral distance with the reference lateral distances of the road boundary lines on the left and right sides sensed by the forward-looking vehicle camera, and to determine the boundary line distance between the boundary line sensed by the forward-looking vehicle camera and the map boundary line on the same side. The verification passed subunit is used to determine that both boundary lines have passed the verification if the distance between the boundary lines on both sides is less than the fourth preset value. The positioning point determination sub-unit is used to select qualified initialization candidate points that have passed the verification of both boundary lines as the semantic positioning initialization points.

[0025] In one embodiment of this application, the apparatus further includes: The positioning point determination module is used to take the position corresponding to the positioning data as the positioning initialization point if the state information is a fixed solution.

[0026] Thirdly, embodiments of the present invention also provide a terminal device, wherein the terminal device includes a memory, a processor, and a semantic location initialization program stored in the memory and executable on the processor. When the processor executes the semantic location initialization program, it implements the steps of the semantic location initialization method of any of the above schemes.

[0027] Fourthly, embodiments of the present invention also provide a computer-readable storage medium, wherein a semantic location initialization program is stored on the computer-readable storage medium, and when the semantic location initialization program is executed by a processor, it implements the steps of the semantic location initialization method described in any of the above schemes.

[0028] Beneficial Effects: Compared with existing technologies, this invention provides a semantic localization initialization method. This invention analyzes the state information of vehicle positioning data. If the state information is a dynamic floating-point solution, the location corresponding to the positioning data is projected onto the center lines of each lane, and an initialization candidate point is determined for each lane. Then, the lane line point cloud is projected onto the location of the initialization candidate point, and point-line matching is performed with the lane lines on the map to obtain the matching result. Finally, based on the matching result, the semantic localization initialization point is determined. It is evident that this invention can achieve RTK floating-point solution initialization, and can quickly re-initialize even without accurate GPS location information, ensuring that the vehicle does not miss a lane during lane-level positioning.

[0029] Furthermore, this invention utilizes the semantic positioning results of lane lines matched with high-precision maps to assist the integrated inertial navigation system in achieving positioning initialization, ensuring that lane-level positioning remains in the correct lane, and making vehicle positioning more accurate. Attached Figure Description

[0030] Figure 1 A flowchart illustrating a specific implementation of the semantic localization initialization method provided in this embodiment of the invention; Figure 2 This is a functional principle diagram of the semantic positioning initialization device provided in an embodiment of the present invention; Figure 3 A schematic diagram of a terminal device provided in an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0032] Because existing technologies rely solely on high-precision integrated inertial navigation systems for positioning, the positioning effect cannot be guaranteed. Therefore, this embodiment provides a semantic positioning initialization method. This embodiment analyzes the state information of the vehicle's positioning data, which can be GPS (Global Positioning System) data. If the state information is a dynamic floating-point solution, specifically an RTK (Real-time kinematic) floating-point solution, the location corresponding to the positioning data is projected onto the center lines of each lane, and an initialization candidate point is determined for each lane. Then, the lane line point cloud is projected onto the location of the initialization candidate point, and point-line matching is performed with the lane lines on the map to obtain the matching result. Finally, based on the matching result, the semantic positioning initialization point is determined. Thus, this embodiment can achieve RTK floating-point solution initialization, and can quickly re-initialize even without accurate GPS location information, ensuring that the vehicle does not miss a lane during lane-level positioning.

[0033] The semantic localization initialization method of this embodiment can be applied to terminal devices or cloud devices. When the method of this embodiment is applied to a terminal device, the terminal device can be an in-vehicle terminal, such as an in-vehicle central control computer. Alternatively, the terminal device can also be a user's mobile terminal, such as a mobile phone, which can be connected to the in-vehicle terminal to receive data transmitted by the in-vehicle terminal and perform corresponding analysis and processing.

[0034] Specifically, such as Figure 1 As shown in the figure, the semantic localization initialization method of this embodiment includes the following steps: Step S100: Obtain a semantic frame that reflects lane line image data, determine the vehicle's positioning data based on the semantic frame, and determine the status information of the positioning data.

[0035] This embodiment first obtains a semantic frame, which is lane-line related image data captured by an onboard forward-looking camera and obtained after semantic segmentation. Based on the lane-line image data reflected in this semantic frame, specifically, this embodiment can fit the lane-line image data to a single-frame lane line using a preset cubic curve equation, thereby obtaining the semantic frame. The cubic curve equation is: , in, For the constant term of the cubic curve parameters, For the linear term of the cubic curve parameter, For the quadratic term of the cubic curve parameter, For cubic curve parameters, the cubic term, Let this be the longitudinal distance from the lane line point cloud to the vehicle body. It is the lateral distance from the lane line point cloud to the vehicle body.

[0036] In one implementation, this embodiment includes the following steps when acquiring vehicle location data: Step S101: Obtain the timestamp of the semantic frame; Step S102: Based on the timestamp, obtain the location data corresponding to the timestamp from the data cache queue.

[0037] To ensure timely and accurate positioning, this embodiment uses the latest GPS data. First, it obtains the timestamp of the semantic frame. Then, based on this timestamp, it retrieves the positioning data corresponding to the timestamp from a cache queue. For example, it retrieves the GPS data of the most recent frame from the cache queue, which reflects the vehicle's current positioning. In this embodiment, the semantic frames obtained after semantic segmentation of the lane line image data are stored in the cache queue, along with the corresponding GPS data. This facilitates timely retrieval of the required data. Furthermore, retrieving the most recent GPS frame helps ensure real-time and accurate positioning.

[0038] Next, this embodiment can further determine the status information of the positioning data. In this embodiment, the status information of the positioning data reflects the status of the solution obtained by the positioning system when solving for the location information. For example, if the status information of the positioning data is a dynamic floating-point solution, specifically an RTK floating-point solution, it means that the positioning system has not yet obtained a fixed solution when solving for the location information, and the positioning accuracy is relatively low, between centimeter and meter level; while if the status information of the positioning data is a fixed solution, specifically an RTK fixed solution, it means that the positioning system has obtained a correct solution when solving for the location information, and the positioning progress is relatively high.

[0039] Step S200: If the status information is a dynamic floating-point solution, then project the position corresponding to the positioning data onto the center line of each lane, and determine the initial candidate point for each lane.

[0040] In order to achieve the goal of quick re-initialization even without accurate GPS location information, when the status information of the positioning data is a dynamic floating-point solution (i.e., RTK floating-point solution), the position corresponding to the positioning data can be projected onto the center line of each lane, and then the initialization candidate point of each lane can be determined.

[0041] In one implementation, this embodiment includes the following steps when determining the initialization candidate point: Step S201: Construct a binary search tree for each lane centerline. Step S202: For each binary search tree, perform nearest neighbor search using the position information of the RTK floating-point solution as the reference point to determine the two closest consecutive points; Step S203: Project the position information of the RTK floating-point solution onto the line connecting two consecutive points to obtain the projection point; Step S204: Determine the initialization candidate point for each lane based on the projection point.

[0042] Specifically, this embodiment first constructs a binary search tree for each lane centerline. Then, for each binary search tree, this embodiment performs a nearest neighbor search using the position information of the RTK floating-point solution as a reference point to determine the two closest consecutive points. These two consecutive points are the points on the left and right sides of the reference point, respectively. This embodiment can project the position information of the RTK floating-point solution onto the line connecting the two consecutive points to obtain a projection point. This projection point is the initialization candidate point. This embodiment can perform a nearest neighbor search on the binary search tree of each lane centerline, thereby determining the initialization candidate point for each lane. By constructing a binary search tree for each lane centerline and determining the initialization candidate point based on this binary search tree, this embodiment facilitates the rapid determination of the initialization candidate point.

[0043] Step S300: Obtain the lane line point cloud corresponding to the semantic frame, project the lane line point cloud onto the position of the initial candidate point, and perform point-line matching with the lane lines on the map to obtain the matching result.

[0044] In this embodiment, after determining the location of the initial candidate point, the lane line point cloud can be projected onto the location of the initial candidate point, and point-line matching can be performed with the lane lines on the map. It is then determined whether the lane lines projected from the lane line point cloud based on the initial candidate point can be successfully matched with the lane lines on the map, thereby obtaining the matching result.

[0045] In one implementation, this embodiment includes the following steps when performing point-line matching: Step S301: Based on the timestamp of the semantic frame, determine the semantic frame pose corresponding to each of the initialization candidate points by using a trajectory estimation method; Step S302: Project the lane line point cloud onto the semantic frame pose to obtain the projected lane line point cloud; Step S303: Match the projected lane line point cloud with the lane lines of the map to obtain the matching result, wherein the matching result includes: the number of successful point-line matching, the point-line matching success rate, and the median error between points and lines.

[0046] Specifically, the inertial measurement unit in the vehicle-mounted integrated inertial navigation system of this embodiment can measure acceleration and angular velocity. Combined with the vehicle's wheel speed pulse signals, it can obtain the vehicle's accurate speed in real time. Then, the position and attitude of the vehicle body at the initial candidate point are calculated through trajectory extrapolation. In practical applications, the trajectory extrapolation formula is as follows: , , and These are the translations at times n-1 and n, respectively. and These are the rotation matrices at times n-1 and n, respectively. It is the vehicle speed at time n-1. It is the time interval between two frames of data. This is the angular acceleration at time n-1. In this embodiment, the vehicle pose at the initial moment is set as the identity matrix. The vehicle inertial navigation data is used to calculate the trajectory from the starting point using the above formula. Based on the vehicle velocity and the vehicle angular acceleration, the translational and rotational changes of the vehicle between two adjacent frames are obtained, and the pose at the current moment is calculated. Based on this, this embodiment can perform the above-mentioned trajectory calculation for each initialization candidate point to obtain the semantic frame pose corresponding to different initialization points. This embodiment uses point-line matching to filter the initialization candidate points in order to select the initialization candidate points that meet the conditions, which helps to ensure accurate positioning.

[0047] Next, in this embodiment, the lane line point cloud is projected onto the semantic frame pose to obtain the projected lane line point cloud. Finally, in this embodiment, the projected lane line point cloud is matched with the lane lines of the map to obtain the matching result. In this embodiment, the matching result includes: the number of successful point-to-line matches, the point-to-line matching success rate, and the median error between points and lines. That is, in this embodiment, the projected lane line point cloud is then matched with the lane lines of the map to determine the number of successful point-to-line matches, the point-to-line matching success rate, and the median error between points and lines.

[0048] Step S400: Based on the matching result, determine the semantic positioning initialization point, wherein the semantic positioning initialization point is used for vehicle positioning.

[0049] In one implementation, step S400 of this embodiment specifically includes the following steps: Step S401: Filter the initial candidate points based on preset filtering conditions to determine the initial candidate points that meet the conditions. The filtering conditions are: the number of successful point-line matching in the matching results reaches a first preset value, the point-line matching success rate reaches a second preset value, and the median error from point to line is less than a third preset value. Step S402: Perform boundary line verification on the initialization candidate points that meet the conditions to obtain the semantic positioning initialization points.

[0050] After obtaining the matching results, this embodiment can filter the initialization candidate points based on the matching results to determine the semantic positioning initialization points, so that the vehicle can be positioned based on the semantic positioning initialization points. Specifically, this embodiment sets filtering conditions, which are: the number of successful point-to-line matches reaches a first preset value, the point-to-line matching success rate (the ratio of the number of successful point-to-line matches to the total number of points) reaches a second preset value, and the median error from point to line is less than a third preset value. For example, the first preset value is 50, the second preset value is 80%, and the third preset value is 0.8 meters. If the number of successful point-to-line matches in the matching results reaches 50, the point-to-line matching success rate reaches 80%, and the median error from point to line is less than 0.8 meters, then the initialization candidate points that meet the above conditions can be determined as qualified initialization candidate points. That is to say, if the matching results meet the above filtering conditions, then the initialization candidate points that are being matched can be determined as qualified initialization candidate points; if the matching results do not meet the above filtering conditions, then the initialization candidate points that are being matched are filtered out. Next, this embodiment performs boundary line verification on the qualified initialization candidate points to obtain the semantic localization initialization point. This embodiment performs boundary line verification on the qualified initialization candidate points to further filter out the best candidate points, thereby making the semantic localization initialization points obtained in subsequent steps more accurate.

[0051] Specifically, in this embodiment, for each eligible candidate point, the lateral distance between the eligible candidate point and the left and right road boundary lines on the map is calculated. Then, these lateral distances are compared with the reference lateral distances of the left and right road boundary lines sensed by the forward-looking vehicle camera, respectively, to determine the boundary line distance between the boundary line sensed by the forward-looking vehicle camera and the map boundary line on the same side. If the boundary line distances on both sides are less than a fourth preset value, for example, if the fourth preset value is less than 1.5 meters, then both boundary lines are considered to have passed verification, and the eligible candidate point with both boundary lines passing verification is used as the semantic positioning initialization point. Positioning initialization is then performed based on this semantic positioning initialization point for vehicle positioning. In other words, this embodiment verifies both boundary lines separately; only when both boundary lines pass verification can the corresponding eligible candidate point be determined as the optimal candidate point, which helps improve the positioning effect.

[0052] In another implementation, if only one side of the boundary line passes the verification and the state information of the CPS data is a fixed RTK solution, then the lateral distance between the qualified candidate point and the left and right road boundary lines on the map is further verified. If the lateral distance does not exceed 1.3 meters, then the qualified candidate point can also be determined as the best candidate point (i.e., the determined semantic positioning initialization point), that is, the positioning initialization is successful. If the lateral distance exceeds 1.3 meters, then the positioning initialization is considered to have failed, and the above initialization process needs to be repeated until it succeeds.

[0053] In addition, in other implementations, in order to ensure the correctness of initialization, this embodiment requires that the best alternative points (i.e. the determined semantic positioning initialization points) determined for a consecutive preset number of frames (e.g., 15 consecutive frames) are all located in the same lane. If so, the positioning initialization is considered successful. Otherwise, the positioning initialization fails. The above process is repeated until the positioning initialization is successful.

[0054] In one implementation, if the state information of the positioning data is a fixed solution, specifically an RTK fixed solution, it means that the positioning system obtains the correct solution when solving for the location information. In this embodiment, the location corresponding to the positioning data can be used as the positioning initialization point to directly perform vehicle positioning, so as to ensure the smooth progress of positioning initialization.

[0055] Based on the above embodiments, the present invention also provides a semantic positioning initialization device, such as... Figure 2As shown, the semantic localization initialization device 100 includes: a state determination module 10, a candidate point determination module 20, a point-line matching module 30, and a result determination module 40. Specifically, the state determination module 10 is used to acquire a semantic frame reflecting lane line image data, determine vehicle positioning data based on the semantic frame, and determine the state information of the positioning data. The candidate point determination module 20 is used to, if the state information is an RTK floating-point solution, project the position corresponding to the positioning data onto the center line of each lane and determine the initial candidate point for each lane. The point-line matching module 30 is used to acquire the lane line point cloud corresponding to the semantic frame, project the lane line point cloud onto the position of the initial candidate point, and perform point-line matching with the lane lines on the map to obtain a matching result. The result determination module 40 is used to determine a semantic localization initialization point based on the matching result, wherein the semantic localization initialization point is used for vehicle localization.

[0056] In one embodiment of this application, the state determination module 10 includes: A timestamp acquisition unit is used to acquire the timestamp of the semantic frame; The data acquisition unit is used to acquire the location data corresponding to the timestamp from the data cache queue based on the timestamp.

[0057] In one embodiment of this application, the alternative point determination module 20 includes: Binary search tree construction unit, used to construct a binary search tree for each of the lane centerlines. The nearest neighbor search unit is used to perform a nearest neighbor search for each of the binary search trees, using the position information of the RTK floating-point solution as a reference point, to determine the two closest consecutive points. The position projection unit is used to project the position information of the RTK floating-point solution onto the line connecting two consecutive points to obtain the projection point. The alternative point determination unit is used to determine the initial alternative point for each lane based on the projection point.

[0058] In one embodiment of this application, the dot-line matching module 30 includes: The trajectory estimation unit is used to determine the pose of the semantic frame corresponding to each of the initialization candidate points by using trajectory estimation based on the timestamp of the semantic frame. A point cloud projection unit is used to project the lane line point cloud onto the semantic frame pose to obtain the projected lane line point cloud. The point-line matching unit is used to match the projected lane line point cloud with the lane lines of the map to obtain the matching result, wherein the matching result includes: the number of successful point-line matching, the point-line matching success rate, and the median error between points and lines.

[0059] In one embodiment of this application, the result determination module 40 includes: The condition acquisition unit is used to acquire preset filtering conditions, wherein the filtering conditions are: the number of successful point-line matching in the matching results reaches a first preset value, the point-line matching success rate reaches a second preset value, and the median error from point to line is less than a third preset value. The filtering execution unit is used to filter the initialization candidate points based on the filtering conditions and the matching results, and determine the initialization candidate points that meet the conditions. The boundary verification unit is used to verify the boundary lines of the initialization candidate points that meet the conditions, so as to obtain the semantic positioning initialization point.

[0060] In one embodiment of this application, the boundary verification unit includes: The distance calculation sub-unit is used to calculate the lateral distance between each eligible initial candidate point and the left and right road boundary lines on the map. The boundary analysis subunit is used to compare the lateral distance with the reference lateral distances of the road boundary lines on the left and right sides sensed by the forward-looking vehicle camera, and to determine the boundary line distance between the boundary line sensed by the forward-looking vehicle camera and the map boundary line on the same side. The verification passed subunit is used to determine that both boundary lines have passed the verification if the distance between the boundary lines on both sides is less than the fourth preset value. The positioning point determination sub-unit is used to select qualified initialization candidate points that have passed the verification of both boundary lines as the semantic positioning initialization points.

[0061] In one embodiment of this application, the semantic localization initialization device 100 further includes: The positioning point determination module is used to take the position corresponding to the positioning data as the positioning initialization point if the status information is an RTK fixed solution.

[0062] The working principle of each module in the semantic localization initialization device 100 of this embodiment is the same as that of each step in the above method embodiment, and will not be repeated here.

[0063] The semantic positioning initialization device 100 according to this embodiment can realize RTK floating-point deinitialization, and can quickly reinitialize even without accurate GPS location information, ensuring that the vehicle does not miss the correct lane during lane-level positioning. Furthermore, this embodiment utilizes the semantic positioning results of lane lines matched with high-precision maps to assist the integrated inertial navigation system in achieving positioning initialization, ensuring that lane-level positioning remains in the correct lane, resulting in more accurate vehicle positioning.

[0064] Figure 3 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. The terminal device may include: a memory 301, a processor 302, and a computer program stored in the memory 301 and executable on the processor 302. When the processor 302 executes the program, it implements the semantic positioning initialization method provided in the above embodiment.

[0065] Furthermore, the terminal equipment also includes: Communication interface 303 is used for communication between memory 301 and processor 302.

[0066] The memory 301 is used to store computer programs that can run on the processor 5302.

[0067] The memory 301 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0068] If the memory 301, processor 302, and communication interface 303 are implemented independently, then the communication interface 303, memory 301, and processor 302 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0069] In practical implementation, if the memory 301, processor 302, and communication interface 303 are integrated on a single chip, then the memory 301, processor 302, and communication interface 303 can communicate with each other through an internal interface. The processor 302 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0070] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the semantic location initialization method described above.

[0071] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0072] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0073] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0074] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0075] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0076] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0077] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0078] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A semantic localization initialization method, characterized in that, The method includes: Obtain semantic frames that reflect lane line image data, determine vehicle positioning data based on the semantic frames, and determine the status information of the positioning data; If the state information is a dynamic floating-point solution, then the position corresponding to the positioning data is projected onto the center line of each lane, and the initial candidate point for each lane is determined. Obtain the lane line point cloud corresponding to the semantic frame, project the lane line point cloud onto the position of the initialized candidate point, and perform point-line matching with the lane lines on the map to obtain the matching result; Based on the matching results, a semantic positioning initialization point is determined, wherein the semantic positioning initialization point is used for vehicle positioning; If the state information is a dynamic floating-point solution, then the location information corresponding to the positioning data is projected onto the center line of each lane, and an initial candidate point for each lane is determined, including: Construct a binary search tree for each of the lane centerlines. For each binary search tree, the nearest neighbor search is performed using the position information of the dynamic floating-point solution as the reference point to determine the two closest consecutive points; The position information of the dynamic floating-point solution is projected onto the line connecting two consecutive points to obtain the projection point; The initial candidate point for each lane is determined based on the projection point; The step of determining the semantic localization initialization point based on the matching result includes: Obtain preset filtering conditions, wherein the filtering conditions are: the number of successful point-line matching in the matching results reaches a first preset value, the point-line matching success rate reaches a second preset value, and the median error from point to line is less than a third preset value; Based on the filtering conditions and the matching results, the initialization candidate points are filtered to determine the initialization candidate points that meet the conditions. Boundary line verification is performed on the initial candidate points that meet the conditions to obtain the semantic positioning initial point.

2. The semantic localization initialization method according to claim 1, characterized in that, The determination of vehicle location data based on the semantic frame includes: Obtain the timestamp of the semantic frame; Based on the timestamp, the location data corresponding to the timestamp is obtained from the data cache queue.

3. The semantic localization initialization method according to claim 1, characterized in that, The step of projecting the lane line point cloud onto the location of the initialized candidate point and performing point-line matching with the lane lines on the map to obtain the matching result includes: Based on the timestamp of the semantic frame, the pose of the semantic frame corresponding to each of the initialization candidate points is determined by using a trajectory estimation method. The lane line point cloud is projected onto the semantic frame pose to obtain the projected lane line point cloud. The projected lane line point cloud is matched with the lane lines of the map to obtain the matching result, wherein the matching result includes: the number of successful point-to-line matching, the point-to-line matching success rate, and the median error between points and lines.

4. The semantic localization initialization method according to claim 1, characterized in that, The step of performing boundary line verification on the qualified initialization candidate points to obtain the semantic localization initialization points includes: For each eligible initial candidate point, calculate the lateral distance between the eligible initial candidate point and the left and right road boundary lines on the map. The lateral distance is compared with the reference lateral distances of the road boundary lines on the left and right sides perceived by the forward-looking vehicle camera, and the boundary line distance between the boundary line perceived by the forward-looking vehicle camera and the map boundary line is determined for the same side boundary line. If the distance between the boundary lines on both sides is less than the fourth preset value, it is determined that both boundary lines have passed the verification. The initialization candidate points that meet the conditions and whose boundary lines on both sides have passed the verification are used as the semantic positioning initialization points.

5. The semantic localization initialization method according to claim 4, characterized in that, The method further includes: If the state information is a fixed solution, then the location corresponding to the positioning data is used as the positioning initialization point.

6. A semantic localization initialization device, characterized in that, The apparatus is used to implement the steps of the semantic localization initialization method according to any one of claims 1-5, and the apparatus comprises: The state determination module is used to acquire semantic frames that reflect lane line image data, determine vehicle positioning data based on the semantic frames, and determine the state information of the positioning data. The alternative point determination module is used to project the position corresponding to the positioning data onto the center line of each lane if the state information is a dynamic floating-point solution, and determine the initial alternative point for each lane. The point-line matching module is used to obtain the lane line point cloud corresponding to the semantic frame, project the lane line point cloud onto the position of the initial candidate point, and perform point-line matching with the lane lines of the map to obtain the matching result. The result determination module is used to determine the semantic positioning initialization point based on the matching result, wherein the semantic positioning initialization point is used for vehicle positioning.

7. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a semantic location initialization program stored in the memory and executable on the processor. When the processor executes the semantic location initialization program, it implements the steps of the semantic location initialization method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a semantic location initialization program, which, when executed by a processor, implements the steps of the semantic location initialization method as described in any one of claims 1-5.

Images