Memory parking positioning method and related device

Abstract

The invention discloses a memory parking positioning method and a related device. According to the method, the driving track of the vehicle in the target scene is acquired, the semantic map and the feature point map are constructed based on the driving track, the semantic map and the feature point map are constructed based on the pose state information of the vehicle, and then the memory parking operation in the target scene is performed in response to the vehicle. The position and posture state information of the current vehicle is obtained, the memory parking operation is used for indicating the vehicle to park according to the driving track, the semantic information of the current vehicle is matched with the semantic information in the semantic map, and the feature point information of the current vehicle is matched with the feature point information in the feature point map. And finally, initialized positioning is performed on the vehicle based on a matching result, so that the vehicle enters an initialized state of memory parking. According to the invention, accurate and efficient initialization positioning of the vehicle in memory parking is realized.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a memory parking positioning method and related devices. Background Art

[0002] With the development of society, the current autonomous driving technology is gradually becoming important in daily life. The autonomous driving technology includes the function of memory parking, which can realize automatic parking in and out of the vehicle without human intervention.

[0003] When performing memory parking, the vehicle must first be initialized and positioned. In related technologies, the method for initializing and positioning the vehicle requires a large amount of computing resources, and the accuracy of the initialization positioning is low.

[0004] Therefore, in order to accurately and efficiently initialize and locate a memory parking vehicle, a memory parking positioning method is currently in urgent need. Summary of the invention

[0005] In view of this, an embodiment of the present application provides a memory parking positioning method and related devices, aiming to achieve accurate and efficient initialization positioning of a memory parked vehicle.

[0006] In a first aspect, an embodiment of the present application provides a memory parking positioning method, the method comprising:

[0007] Acquire a driving trajectory of a vehicle in a target scene, and construct a semantic map and a feature point map based on the driving trajectory, wherein the semantic map and the feature point map are constructed based on the posture state information of the vehicle;

[0008] In response to the vehicle performing a memory parking operation in the target scene, obtaining current posture state information of the vehicle, wherein the memory parking operation is used to instruct the vehicle to park according to the driving trajectory, and the posture state information is obtained based on semantic information and feature point information;

[0009] Matching the semantic information of the current vehicle with the semantic information in the semantic map, and matching the feature point information of the current vehicle with the feature point information in the feature point map;

[0010] The vehicle is initially positioned based on the matching result, so that the vehicle enters an initialization state of memory parking.

[0011] Optionally, constructing a semantic map and a feature point map based on the driving trajectory includes:

[0012] Extracting semantic information from the surround view data generated in the driving trajectory using a surround view camera, inputting the extracted semantic information and vehicle body information into an error state Kalman filter (ESKF) model, and constructing the semantic map using the vehicle posture state information output by the ESKF model, wherein the vehicle body information includes inertial sensor IMU information and wheel encoder information;

[0013] The feature point information of the front surround view data generated in the driving trajectory is extracted using a front surround view camera, the extracted feature point information and the vehicle body information are input into the ESKF model, and the feature point map is constructed using the posture state information of the vehicle output by the ESKF model.

[0014] Optionally, obtaining the driving trajectory of the vehicle in the target scene includes:

[0015] Acquiring global navigation satellite system (GNSS) information and vehicle body information of the vehicle, wherein the vehicle body information includes inertial sensor (IMU) information;

[0016] Performing noise calibration on the IMU to obtain initialized IMU information, and performing coordinate system initialization on the GNSS to obtain initialized GNSS information, thereby completing an initialization operation on the vehicle;

[0017] Acquire the driving trajectory of the vehicle that completes the initialization operation in the target scene.

[0018] Optionally, in response to the vehicle performing a memory parking operation in the target scene, obtaining the current posture state information of the vehicle includes:

[0019] In response to the vehicle performing a memory parking operation in the target scene, acquiring semantic information and feature point information of the current vehicle;

[0020] Based on the semantic information and the feature point information, the current position and posture state information of the vehicle is obtained.

[0021] Optionally, the initializing the positioning of the vehicle based on the matching result so that the vehicle enters an initialization state of memory parking includes:

[0022] When the matching result shows that the semantic information of the vehicle matches the first semantic information in the semantic map, and the feature point information of the vehicle matches the first feature point information in the feature point map, determining the first position state information of the vehicle based on the first semantic information and the first feature point information;

[0023] The vehicle is initialized and positioned based on the first position state information so that the vehicle enters an initialization state of memory parking.

[0024] Optionally, the acquiring of the driving trajectory of the vehicle in the target scene and constructing a semantic map and a feature point map based on the driving trajectory includes:

[0025] Set preset distance;

[0026] A target driving trajectory of the vehicle in a target scene is obtained, and a semantic map and a feature point map are constructed based on the target driving trajectory, wherein the trajectory distance of the target driving trajectory is the preset distance.

[0027] Optionally, the method further includes:

[0028] At least one operation of adding, deleting, modifying and searching is performed on the acquired driving trajectory.

[0029] In a second aspect, an embodiment of the present application provides a memory parking positioning device, the device comprising: a construction module, an acquisition module, a matching module and a positioning module;

[0030] The construction module is used to obtain the driving trajectory of the vehicle in the target scene, and to construct a semantic map and a feature point map based on the driving trajectory, wherein the semantic map and the feature point map are constructed based on the posture state information of the vehicle;

[0031] The acquisition module is used to acquire the current posture state information of the vehicle in response to the vehicle performing a memory parking operation in the target scene, wherein the memory parking operation is used to instruct the vehicle to park according to the driving trajectory;

[0032] The matching module is used to match the semantic information of the current vehicle with the semantic information in the semantic map, and to match the feature point information of the current vehicle with the feature point information in the feature point map;

[0033] The positioning module is used to initialize the positioning of the vehicle based on the matching result, so that the vehicle enters an initialization state of memory parking.

[0034] In a third aspect, the present application provides an electronic device, the device comprising: a processor, a memory, and a system bus;

[0035] The processor and the memory are connected via the system bus;

[0036] The memory is used to store one or more programs, and the one or more programs include instructions, which, when executed by the processor, enable the processor to execute the method described in the first aspect.

[0037] In a fourth aspect, an embodiment of the present application provides a computer storage medium, in which a code is stored. When the code is executed, a device executing the code implements any method described in the first aspect.

[0038] The present application provides a memory parking positioning method and related devices. When executing the method, firstly, the driving trajectory of the vehicle in the target scene is obtained, and a semantic map and a feature point map are constructed based on the driving trajectory. The semantic map and the feature point map are constructed based on the posture state information of the vehicle. Then, in response to the vehicle performing a memory parking operation in the target scene, the posture state information of the current vehicle is obtained. The memory parking operation is used to instruct the vehicle to park according to the driving trajectory, and the semantic information of the current vehicle is matched with the semantic information in the semantic map, and the feature point information of the current vehicle is matched with the feature point information in the feature point map. Finally, the vehicle is initialized and positioned based on the matching result, so that the vehicle enters the initialization state of memory parking. In this way, by obtaining the driving trajectory of the vehicle in the target scene and constructing the corresponding semantic map and feature point map based on the driving trajectory, when the vehicle needs to perform a memory parking operation in the target scene, the posture state information of the current vehicle can be obtained, and the initialization positioning of the vehicle can be achieved based on the matching of the posture state information with the semantic information and feature point information in the semantic map. The implementation process is simple and avoids the waste of a large amount of computing resources. In addition, the use of semantic maps and feature point maps at the same time is conducive to enhancing the accuracy of initial positioning in scenes with weak GNSS signals. And because the semantic information and feature point information of the entire driving trajectory are recorded in the semantic map and feature point map, it is possible to determine the initial positioning position by matching the posture state information regardless of the current position of the vehicle, which ensures the accuracy of the initial positioning of the vehicle during the memory parking operation to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flowchart of a method for memorizing parking positioning provided in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of trajectory acquisition provided in an embodiment of the present application;

[0042] Figure 3 A flow chart of trajectory acquisition provided in an embodiment of the present application;

[0043] Figure 4 A schematic diagram of updating a map list provided in an embodiment of the present application;

[0044] Figure 5 A schematic diagram of initialization positioning provided in an embodiment of the present application;

[0045] Figure 6 A flowchart of initialization positioning provided in an embodiment of the present application;

[0046] Figure 7 A schematic diagram of the structure of a memory parking positioning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0049] In the study of related technologies, it was found that with the development of society, the memory parking function in autonomous driving technology has higher and higher requirements for the accuracy of initialization positioning. In order to achieve memory parking without human assistance, the autonomous driving system needs to provide more accurate positioning to ensure the safety and reliability of the vehicle in various complex environments. When performing memory parking, the vehicle needs to be initialized and positioned first, and the method of initializing and positioning the vehicle in related technologies requires a lot of computing resources, and the accuracy of initialization positioning is low.

[0050] Based on this, the present application proposes a memory parking positioning method and related devices. The simultaneous use of semantic maps and feature point maps is conducive to enhancing the accuracy of initialization positioning in scenes with weak GNSS signals. And because the semantic information and feature point information of the entire driving trajectory are recorded in the semantic map and feature point map, it is possible to determine the initialization positioning position by matching the posture state information regardless of the current position of the vehicle, which ensures the accuracy of the initialization positioning of the vehicle when performing memory parking operations to a certain extent.

[0051] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] Figure 1 A flowchart of a method for memorizing parking positioning provided in an embodiment of the present application is provided in Figure 1 As shown, a memory parking positioning method provided in an embodiment of the present application includes:

[0053] S11: Acquire the driving trajectory of the vehicle in the target scene, and construct a semantic map and a feature point map based on the driving trajectory.

[0054] The driving trajectory of the vehicle in the target scene is a trajectory for memory parking that needs to be predetermined in order to achieve memory parking, so that when the vehicle needs to perform memory parking in the target scene later, the driving trajectory is used as the parking trajectory to complete the memory parking. The driving trajectory may include multiple driving trajectories, and there may be differences between different driving trajectories. The target scene may be an outdoor parking lot, an indoor parking lot (such as an underground parking lot), etc., which is not limited here.

[0055] The aforementioned S11 mentions "obtaining the driving trajectory of the vehicle in the target scenario". The method of obtaining the driving trajectory requires initializing the vehicle, and then obtaining the driving trajectory based on the initialized vehicle. The method may be: first obtaining the global navigation satellite system GNSS information and vehicle body information of the vehicle, wherein the vehicle body information includes the inertial sensor IMU information. Then, the IMU is noise calibrated to obtain the initialized IMU information, and the coordinate system of the GNSS is initialized to obtain the initialized GNSS information, thereby completing the initialization operation of the vehicle. Finally, the driving trajectory of the vehicle that has completed the initialization operation in the target scenario is obtained.

[0056] That is, the GNSS information and body information of the vehicle are obtained, and the body information may include but is not limited to the inertial sensor IMU information and the wheel encoder information. When the GNSS signal is normal, the IMU is noise calibrated to obtain the initialized IMU information, and the GNSS coordinate system is initialized to obtain the initialized GNSS information, thereby completing the initialization operation of the vehicle. After the initialization operation is completed, the driving trajectory of the vehicle in the target scene can be obtained. Among them, the inertial sensor (Inertial Measurement Unit, IMU), also known as the inertial measurement unit, is a sensor mainly used to detect and measure acceleration and rotational angular velocity.

[0057] In the embodiment of the present application, it is necessary to solve the posture state information of the vehicle based on the GNSS data. When initializing the GNSS, the longitude and latitude need to be converted into a rectangular coordinate system to provide a coordinate origin for the solution process and to use the position information obtained by GNSS in the ESKF (Error State Kalman Filter) model to update the ESKF model state. That is, when obtaining the driving trajectory of the vehicle in the target scene, the ESKF and GNSS information can be combined for real-time positioning, and the vehicle state can be estimated by combining the vehicle body information (IMU information), wherein the IMU data is subjected to noise calibration and inertial solution, and the high-frequency dynamic information provided by the IMU is obtained, and the GNSS provides relatively low-frequency single-precision absolute position information. By fusing IMU information and GNSS information, the ESKF model can achieve high-precision positioning and following of the vehicle.

[0058] By initializing the vehicle, the vehicle will not be disturbed by existing data, ensuring that the acquired driving trajectory is adaptable, which is conducive to direct use of the driving trajectory when the vehicle needs to perform memory parking.

[0059] It should be noted that in the embodiment of the present application, the data generated by the wheel encoder of the vehicle can also be obtained, and the speed of the data can be resolved to provide the ESKF model with the speed observation of the vehicle.

[0060] The aforementioned S11 also mentions "constructing a semantic map and a feature point map based on the driving trajectory". The semantic map and the feature point map are constructed based on the posture state information of the vehicle. In one possible implementation, the method is: using a surround-view camera to extract semantic information from the surround-view data generated in the driving trajectory, combining the extracted semantic information with the posture state information of the vehicle output by the ESKF model, and constructing the semantic map. Using a front surround-view camera to extract feature point information from the front surround data generated in the driving trajectory, combining the extracted feature point information with the posture state information of the vehicle output by the ESKF model, and constructing the feature point map.

[0061] The ESKF (Error-State Kalman Filter) model refers to the Error State Kalman Filter model, which is a technology used in autonomous systems such as robots and drones to filter and estimate variables such as position, velocity, acceleration, and attitude. In the embodiment of the present application, the ESKF model needs to be used in combination with the data obtained or generated in the driving trajectory to determine the vehicle's posture state information.

[0062] In the embodiment of the present application, the vehicle's position status information refers to the position information and posture information of the vehicle itself with a memory parking function, wherein the position information includes longitude, latitude and altitude, and the posture information includes roll angle, pitch angle and heading angle.

[0063] In the embodiment of the present application, the vehicle is equipped with a surround view camera, wherein the surround view camera includes four surround view cameras of front, back, left and right. In the process of acquiring the driving trajectory, the surround view camera will process the data generated in the process, and build a map in combination with the posture state information of the vehicle output by the ESKF model. Specifically, the surround view camera is used to extract semantic information from the surround view data generated in the process of forming the driving trajectory, and the extracted semantic information is input into the error state Kalman filter ESKF model in combination with the vehicle body information, and the posture state information of the vehicle output by the ESKF model is used to build a semantic map (that is, the semantic information and the vehicle body information are input into the ESKF model as input data, so that the ESKF model outputs the corresponding vehicle posture state information according to the input data to build a semantic map). The front surround view camera is used to extract feature point information from the front surround view data (including but not limited to ORB (Oriented FAST and Rotated BRIEF) features) generated in the process of forming the driving trajectory, and the extracted feature point information is input into the ESKF model in combination with the vehicle body information, and the posture state information of the vehicle output by the ESKF model is used to build a feature point map. It should be noted that the number of the front surround view camera and the surround view camera can be one or more, which is not limited here. For example, there can be 4 surround view cameras and 1 front surround view camera. The feature point information may include ORB feature information or optical flow information, etc. The vehicle body information may include but is not limited to inertial sensor IMU information, wheel encoder information, and high-precision global positioning sensor RTK (Real-time kinematic) information.

[0064] In actual application scenarios, there is a state machine inside the vehicle. When the state machine issues the "start memorizing the path" command, feature point information can be extracted through the front surround camera, and semantic information can be extracted through the surround camera. The semantic information can include point cloud information and XML documents. The point cloud information can include ground signs (such as lane lines, objects, etc.), and the XML document can include information about roads and storage locations. Then, semantic maps and feature point maps are constructed respectively based on the extracted semantic information and feature point information. When the state machine issues the "end memorizing the path" command, the driving trajectory is saved and the process ends.

[0065] Figure 2 A schematic diagram of trajectory acquisition provided in an embodiment of the present application, see Figure 2As shown, the position of GNSS (Global Navigation Satellite System) data is solved to provide the coordinate origin and GNSS observation in ESKF (Error State Kalman Filter); the speed of wheel encoder data is solved to provide speed observation for ESKF; the noise calibration and inertial solution of IMU data are performed to provide state prediction for ESKF to achieve ESKF state update. Visual feature points are obtained through the front surround camera, and after the front surround camera is initialized, the pose is solved using the acquired feature point information. The front surround camera is used to provide feature observation for ESKF. Visual semantics are obtained through the surround camera, and the pose is solved using the acquired semantic information. The surround camera is used to provide semantic observation for ESKF. When it is judged that there is an observation, the ESKF observation is updated and then the posterior pose is calculated, the acquired vehicle state is cleared, and the pose is output to obtain the memory trajectory. The output pose and feature point information are combined to perform pose solution to obtain a feature map (i.e., feature point map), and the output pose and semantic information are combined to perform pose solution to obtain a semantic map.

[0066] It should also be noted that in the embodiment of the present application, in the process of obtaining the driving trajectory of the vehicle in the target scene and constructing the semantic map and the feature point map based on the driving trajectory, the distance of the driving trajectory can be preset. When the distance of the driving trajectory of the vehicle in the target scene reaches a preset value, the semantic map and the feature point map can be directly constructed. In a possible implementation, the method is: first setting a preset distance, then obtaining the target driving trajectory of the vehicle in the target scene, and constructing a semantic map and a feature point map based on the target driving trajectory, and the trajectory distance of the target driving trajectory is the preset distance.

[0067] The preset distance can be set by those skilled in the art according to actual conditions and application scenarios, and is not limited here. For example, the preset distance can be 300 meters. That is, when obtaining the driving trajectory at this time, the target driving trajectory with a trajectory distance of the preset distance is obtained.

[0068] This method of acquiring driving trajectories can avoid unlimited acquisition of driving trajectories, which causes the memorized driving trajectories to occupy a large amount of storage space. At the same time, it can avoid a large amount of waste of computing power resources in the process of acquiring driving trajectories, which is conducive to improving the efficiency of acquiring driving trajectories.

[0069] Figure 3 A flow chart of trajectory acquisition provided in an embodiment of the present application is shown in FIG. Figure 3As shown, first input the GNSS (Global Navigation Satellite System) and vehicle information to determine whether the initialization is successful. If so, perform ESKF (Error State Kalman Filter) trajectory extrapolation to determine whether to start memorizing the path. If so, obtain visual semantics (i.e. semantic information) and features (i.e. feature information), and solve the pose (i.e. pose state information). When it is determined that the memory distance is met, save the feature map and semantic map. If not, repeat the above steps. When the initialization is unsuccessful, determine whether the GNSS signal is normal. If so, perform IMU (Inertial Sensor) noise calibration and coordinate initialization. If the GNSS signal is abnormal, re-enter the GNSS and vehicle information. During the ESKF trajectory extrapolation process, it can be determined whether to end the path memory. If so, save the memory trajectory and end the process.

[0070] S12: In response to the vehicle performing a memory parking operation in the target scene, obtaining current posture state information of the vehicle.

[0071] Among them, the memory parking operation is used to instruct the vehicle to perform automatic parking according to the driving trajectory. The driving trajectory is the driving trajectory of the vehicle in the target scene obtained in the aforementioned S11, and the posture state information is obtained based on the semantic information and feature point information.

[0072] In a possible implementation method, the implementation method of step S12 is: first, in response to the vehicle performing a memory parking operation in the target scene, obtaining the semantic information and feature point information of the current vehicle, and then obtaining the current posture state information of the vehicle based on the semantic information and the feature point information.

[0073] That is, when the vehicle needs to perform a memory parking operation in the target scene, the current semantic information and feature point information of the vehicle will be obtained. Similar to the above introduction, the semantic information can be obtained by semantic extraction of the surround view data obtained by the surround view camera, and the feature point information can be obtained by feature extraction of the front surround view data obtained by the front surround view camera. According to the obtained semantic information and feature point information, the ESKF model can be updated to obtain the corresponding current vehicle posture state information.

[0074] S13: Matching the semantic information of the current vehicle with the semantic information in the semantic map, and matching the feature point information of the current vehicle with the feature point information in the feature point map.

[0075] The semantic map and the feature point map are constructed based on the driving trajectory of the vehicle in the aforementioned step S11.

[0076] S14: Initializing the positioning of the vehicle based on the matching result, so that the vehicle enters an initialization state of memory parking.

[0077] The method for initializing the positioning of the vehicle mentioned in step S14 may be: first, when the matching result shows that the semantic information of the vehicle matches the first semantic information in the semantic map, and the feature point information of the vehicle matches the first feature point information in the feature point map, the first position state information of the vehicle is determined based on the first semantic information and the first feature point information. Then, the vehicle is initialized and positioned based on the first position state information, so that the vehicle enters the initialization state of memory parking.

[0078] That is, the semantic information of the current vehicle is matched with the semantic information in the semantic map, and the first semantic information in the semantic map is determined to be matched, and the feature point information of the current vehicle is matched with the feature point information in the feature point map, and the first feature point information in the feature point map is determined to be matched. At this time, the first position state information of the vehicle can be determined based on the first semantic information and the first feature point information, and then the vehicle is initialized and positioned based on the first position state information, so that the vehicle enters the initialization state of memory parking.

[0079] It should be noted that in an embodiment of the present application, at least one operation of adding, deleting, modifying and searching can be performed on the driving trajectory obtained in step S11. After acquisition, the driving trajectory can be stored in a module, in which a map list is created, and the correspondence between the map origin and the map name is stored in the map list. In the stage of driving trajectory acquisition in step S11, the module will store the acquired driving trajectory. In step S13, the appropriate driving trajectory can be determined from the map list based on the determined current vehicle posture state information. It should be noted that at least one operation of adding, deleting, modifying and searching the driving trajectory can be performed automatically by the module, and can also be performed by the user of the vehicle, which is not limited here.

[0080] Figure 4 A schematic diagram of updating a map list provided in an embodiment of the present application, see Figure 4 As shown, when loading the map list, obtaining the vehicle position (i.e., the current vehicle position status information), obtaining the state machine command, and determining whether to perform path (i.e., driving trajectory) maintenance, if so, you can add a path list, delete a path list, modify a path list, and search a path list to update the map list. If you do not perform path maintenance, reload the map list.

[0081] Figure 5 A schematic diagram of initialization positioning provided in an embodiment of the present application, see Figure 5As shown in the figure, the position of GNSS (Global Navigation Satellite System) data is solved to provide the coordinate origin and GNSS observation in ESKF (Error State Kalman Filter); the speed of wheel encoder data is solved to provide speed observation for ESKF; the noise calibration and inertial solution of IMU data are performed to provide state prediction for ESKF to achieve ESKF prediction update. When there is an observation, ESKF observation updates and calculates the posterior pose (i.e., the pose state information of the current vehicle), then clears the acquired state and outputs the pose (i.e., pose state information). The output pose state information is feature matched with the visual feature points (i.e., feature point information) in the feature map (i.e., feature point map) to complete the rough positioning of the visual feature points. The output pose state information is semantically matched with the visual semantics (i.e., semantic information) in the semantic map to complete the matching of semantic information. Based on the matching results, the pose of the vehicle is solved, the visual pose is obtained, and the visual observation is transmitted to ESKF.

[0082] Figure 6 A flowchart of initialization positioning provided for an embodiment of the present application, first load the map and path, then input the GNSS (Global Navigation Satellite System) and vehicle body information, and determine whether the initialization is successful. If so, perform ESKF (Error State Kalman Filter) trajectory extrapolation to determine whether to start cruising (turn on the memory parking operation). If so, obtain visual semantics (i.e. semantic information) and features (i.e. feature information), match the map (i.e. feature point map and semantic map) to solve the pose, and output the pose. Determine whether the vehicle is under control. If so, end the initialization process. If not, re-enter the GNSS and vehicle body information, and repeat the above operation. When the above-mentioned initialization is unsuccessful, determine whether the GNSS signal is normal. If so, perform IMU (inertial sensor) noise calibration and coordinate initialization. If the GNSS signal is abnormal, re-enter the GNSS and vehicle body information, and repeat the above operation. If it is determined that cruising has not started, it is also necessary to re-enter the GNSS and vehicle body information, and repeat the above operation.

[0083] It should be noted that, in general, in ground parking lots, since the GNSS signal is relatively strong, positioning initialization can be performed well without visual observation updates (i.e., semantic information matching of the semantic map to achieve ESKF updates), so visual updates are mainly for positioning initialization scenarios with weak GNSS signals in underground parking lots. As the vehicle goes deeper into the underground parking lot, the IMU's posture prediction error will become larger and larger. When it reaches the memory starting point, it is difficult to determine the specific position of the vehicle in the memory path (i.e., driving trajectory). At this time, the vehicle's posture state information is updated with the help of the visual map (i.e., semantic map), so that the vehicle can be accurately positioned at the position in the memory path, preparing for subsequent vehicle control and cruising.

[0084] In this embodiment, a method for memory parking positioning is proposed. The method first obtains the driving trajectory of the vehicle in the target scene, and constructs a semantic map and a feature point map based on the driving trajectory. The semantic map and the feature point map are constructed based on the posture state information of the vehicle. Then, in response to the vehicle performing a memory parking operation in the target scene, the posture state information of the current vehicle is obtained. The memory parking operation is used to instruct the vehicle to park according to the driving trajectory, and the semantic information of the current vehicle is matched with the semantic information in the semantic map, and the feature point information of the current vehicle is matched with the feature point information in the feature point map. Finally, the vehicle is initialized and positioned based on the matching result, so that the vehicle enters the initialization state of memory parking. In this way, by obtaining the driving trajectory of the vehicle in the target scene and constructing the corresponding semantic map and feature point map based on the driving trajectory, when the vehicle needs to perform a memory parking operation in the target scene, the posture state information of the current vehicle can be obtained, and the initialization positioning of the vehicle can be achieved based on the matching of the posture state information with the semantic information and feature point information in the semantic map. The implementation process is simple and avoids the waste of a large amount of computing resources. In addition, the use of semantic maps and feature point maps at the same time is conducive to enhancing the accuracy of initial positioning in scenes with weak GNSS signals. And because the semantic information and feature point information of the entire driving trajectory are recorded in the semantic map and feature point map, it is possible to determine the initial positioning position by matching the posture state information regardless of the current position of the vehicle, which ensures the accuracy of the initial positioning of the vehicle during the memory parking operation to a certain extent.

[0085] Figure 7 A schematic diagram of the structure of a memory parking positioning device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, a memory parking positioning device specifically includes: a construction module 100, an acquisition module 200, a matching module 300 and a positioning module 400;

[0086] The construction module 100 is used to obtain a driving trajectory of a vehicle in a target scene, and to construct a semantic map and a feature point map based on the driving trajectory, wherein the semantic map and the feature point map are constructed based on the posture state information of the vehicle;

[0087] The acquisition module 200 is used to acquire the current posture state information of the vehicle in response to the vehicle performing a memory parking operation in the target scene, wherein the memory parking operation is used to instruct the vehicle to park according to the driving trajectory;

[0088] The matching module 300 is used to match the semantic information of the current vehicle with the semantic information in the semantic map, and to match the feature point information of the current vehicle with the feature point information in the feature point map;

[0089] The positioning module 400 is used to initialize the positioning of the vehicle based on the matching result, so that the vehicle enters an initialization state of memory parking.

[0090] In a possible implementation, the building module 100 is used to:

[0091] Extracting semantic information from the surround view data generated in the driving trajectory using a surround view camera, inputting the extracted semantic information and vehicle body information into an error state Kalman filter (ESKF) model, and constructing the semantic map using the vehicle posture state information output by the ESKF model;

[0092] The feature point information of the front surround view data generated in the driving trajectory is extracted using a front surround view camera, the extracted feature point information and the vehicle body information are input into the ESKF model, and the feature point map is constructed using the posture state information of the vehicle output by the ESKF model.

[0093] In a possible implementation, the building module 100 is used to:

[0094] Acquiring global navigation satellite system (GNSS) information and vehicle body information of the vehicle, wherein the vehicle body information includes inertial sensor (IMU) information;

[0095] Performing noise calibration on the IMU to obtain initialized IMU information, and performing coordinate system initialization on the GNSS to obtain initialized GNSS information, thereby completing an initialization operation on the vehicle;

[0096] Acquire the driving trajectory of the vehicle that completes the initialization operation in the target scene.

[0097] In a possible implementation, the acquisition module 200 is used to:

[0098] In response to the vehicle performing a memory parking operation in the target scene, acquiring semantic information and feature point information of the current vehicle;

[0099] Based on the semantic information and the feature point information, the current position and posture state information of the vehicle is obtained.

[0100] In a possible implementation, the positioning module 400 is used to:

[0101] When the matching result shows that the semantic information of the vehicle matches the first semantic information in the semantic map, and the feature point information of the vehicle matches the first feature point information in the feature point map, determining the first position state information of the vehicle based on the first semantic information and the first feature point information;

[0102] The vehicle is initialized and positioned based on the first position state information so that the vehicle enters an initialization state of memory parking.

[0103] In a possible implementation, the building module 100 is used to:

[0104] Set preset distance;

[0105] A target driving trajectory of the vehicle in a target scene is obtained, and a semantic map and a feature point map are constructed based on the target driving trajectory, wherein the trajectory distance of the target driving trajectory is the preset distance.

[0106] In a possible implementation, the device further includes an operation module, wherein the operation module is configured to:

[0107] At least one operation of adding, deleting, modifying and searching is performed on the acquired driving trajectory.

[0108] In this embodiment, a device for memory parking positioning is proposed, which includes: a construction module, an acquisition module, a matching module and a positioning module. The construction module is used to obtain the driving trajectory of the vehicle in the target scene, and to construct a semantic map and a feature point map based on the driving trajectory. The semantic map and the feature point map are constructed based on the posture state information of the vehicle; the acquisition module is used to respond to the vehicle performing a memory parking operation in the target scene, and obtain the posture state information of the current vehicle. The memory parking operation is used to instruct the vehicle to park according to the driving trajectory; the matching module is used to match the semantic information of the current vehicle with the semantic information in the semantic map, and match the feature point information of the current vehicle with the feature point information in the feature point map; the positioning module is used to initialize the vehicle based on the matching result, so that the vehicle enters the initialization state of memory parking. In this way, by acquiring the driving trajectory of the vehicle in the target scene, and constructing the corresponding semantic map and feature point map based on the driving trajectory, when the vehicle needs to perform a memory parking operation in the target scene, the posture state information of the current vehicle can be obtained, and the initialization positioning of the vehicle can be achieved based on the posture state information and the semantic information and feature point information in the semantic map. The implementation process is simple and avoids the waste of a large amount of computing resources. In addition, the use of semantic maps and feature point maps at the same time is conducive to enhancing the accuracy of initial positioning in scenes with weak GNSS signals. And because the semantic information and feature point information of the entire driving trajectory are recorded in the semantic map and feature point map, it is possible to determine the initial positioning position by matching the posture state information regardless of the current position of the vehicle, which ensures the accuracy of the initial positioning of the vehicle during the memory parking operation to a certain extent.

[0109] The flow chart and block diagram in the accompanying drawings illustrate the possible implementation architecture, function and operation of the device and method according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the module, the program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0110] The embodiments of the present application also provide corresponding devices and computer-readable storage media for implementing the solutions provided by the embodiments of the present application.

[0111] The device includes a memory and a processor, the memory is used to store instructions or codes, and the processor is used to execute the instructions or codes, so that the device executes a memory parking positioning method described in any embodiment of the present application.

[0112] In practical applications, the computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0113] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0114] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0115] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0116] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0117] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A memory parking positioning method, characterized in that: The method comprises: Acquire a driving trajectory of a vehicle in a target scene, and construct a semantic map and a feature point map based on the driving trajectory, wherein the semantic map and the feature point map are constructed based on the posture state information of the vehicle; In response to the vehicle performing a memory parking operation in the target scene, obtaining current posture state information of the vehicle, wherein the memory parking operation is used to instruct the vehicle to park according to the driving trajectory, and the posture state information is obtained based on semantic information and feature point information; Matching the semantic information of the current vehicle with the semantic information in the semantic map, and matching the feature point information of the current vehicle with the feature point information in the feature point map; The vehicle is initially positioned based on the matching result, so that the vehicle enters an initialization state of memory parking.

2. The method according to claim 1, characterized in that The constructing of a semantic map and a feature point map based on the driving trajectory includes: Extracting semantic information from the surround view data generated in the driving trajectory using a surround view camera, inputting the extracted semantic information and vehicle body information into an error state Kalman filter (ESKF) model, and constructing the semantic map using the vehicle posture state information output by the ESKF model, wherein the vehicle body information includes inertial sensor IMU information and wheel encoder information; The feature point information of the front surround view data generated in the driving trajectory is extracted using a front surround view camera, the extracted feature point information and the vehicle body information are input into the ESKF model, and the feature point map is constructed using the posture state information of the vehicle output by the ESKF model.

3. The method according to claim 1, characterized in that The obtaining of the driving trajectory of the vehicle in the target scene includes: Acquiring global navigation satellite system (GNSS) information and vehicle body information of the vehicle, wherein the vehicle body information includes inertial sensor (IMU) information; Performing noise calibration on the IMU to obtain initialized IMU information, and performing coordinate system initialization on the GNSS to obtain initialized GNSS information, thereby completing an initialization operation on the vehicle; Acquire the driving trajectory of the vehicle that completes the initialization operation in the target scene.

4. The method according to claim 1, characterized in that: In response to the vehicle performing a memory parking operation in the target scene, obtaining current posture state information of the vehicle includes: In response to the vehicle performing a memory parking operation in the target scene, acquiring semantic information and feature point information of the current vehicle; Based on the semantic information and the feature point information, the current position and posture state information of the vehicle is obtained.

5. The method according to claim 1, characterized in that The initializing positioning of the vehicle based on the matching result so that the vehicle enters an initialization state of memory parking includes: When the matching result shows that the semantic information of the vehicle matches the first semantic information in the semantic map, and the feature point information of the vehicle matches the first feature point information in the feature point map, determining the first position state information of the vehicle based on the first semantic information and the first feature point information; The vehicle is initialized and positioned based on the first position state information so that the vehicle enters an initialization state of memory parking.

6. The method according to claim 1, characterized in that The obtaining of the driving trajectory of the vehicle in the target scene and constructing a semantic map and a feature point map based on the driving trajectory include: Set preset distance; A target driving trajectory of the vehicle in a target scene is obtained, and a semantic map and a feature point map are constructed based on the target driving trajectory, wherein the trajectory distance of the target driving trajectory is the preset distance.

7. The method according to claim 1, characterized in that The method further comprises: At least one operation of adding, deleting, modifying and searching is performed on the acquired driving trajectory.

8. A memory parking positioning device, characterized in that: The device comprises: a construction module, an acquisition module, a matching module and a positioning module; The construction module is used to obtain the driving trajectory of the vehicle in the target scene, and to construct a semantic map and a feature point map based on the driving trajectory, wherein the semantic map and the feature point map are constructed based on the posture state information of the vehicle; The acquisition module is used to acquire the current posture state information of the vehicle in response to the vehicle performing a memory parking operation in the target scene, wherein the memory parking operation is used to instruct the vehicle to park according to the driving trajectory; The matching module is used to match the semantic information of the current vehicle with the semantic information in the semantic map, and to match the feature point information of the current vehicle with the feature point information in the feature point map; The positioning module is used to initialize the positioning of the vehicle based on the matching result, so that the vehicle enters an initialization state of memory parking.

9. An electronic device, characterized in that: The device includes: a processor, a memory, and a system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the memory parking positioning method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an implementation program for implementing the memory parking positioning method, and when the implementation program for implementing the memory parking positioning method is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images