Vehicle positioning method, device and equipment

Abstract

The invention provides a vehicle positioning method, device and equipment, and the method comprises the steps: determining the initial layer height of a first flat layer based on the sensor data of a target vehicle if the target vehicle runs to the boundary position of a ramp and the first flat layer in the cross-layer running process of the target vehicle; determining whether an associated floor of the first floor exists or not based on the initial floor height and the stored reference floor height of each floor; if not, the initial floor height is stored as the reference floor height of the first flat layer, and if yes, the reference floor height of the associated flat layer is stored as the reference floor height of the first flat layer; if the target vehicle runs on the first flat layer, correcting the current floor height determined by the sensor data based on the reference floor height of the first flat layer to obtain a target false floor height; and selecting target map data matched with the target pseudo-layer height from the established map data, and positioning the target vehicle based on the target map data. According to the technical scheme, when the target vehicle runs across layers, accurate positioning of the vehicle can be automatically completed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mapping positioning, and in particular to a vehicle positioning method, device and equipment. BACKGROUND

[0002] Memory parking, as one of the important applications in the field of intelligent driving, can realize functions such as vehicle autonomous cruising and vehicle autonomous parking. Memory parking refers to, after a driver manually drives a vehicle to travel a parking route, an intelligent driving system can assist the driver to drive the vehicle from a starting point of a set route to a terminal point of the set route and park in a parking space that has been memorized by the intelligent driving system according to a "memory route".

[0003] Mapping positioning technology is one of the key technologies to realize memory parking, and the mapping positioning technology includes two stages of teaching mapping and real-time positioning. In the teaching mapping stage, a driver drives a vehicle, an intelligent driving system records a motion trajectory of the vehicle in a teaching process as a teaching trajectory, and observes and scans a surrounding scene for mapping. In the real-time positioning stage, once the vehicle enters an already mapped area, the intelligent driving system can automatically complete positioning of the vehicle and provide accurate position information for an automatic driving function of the vehicle.

[0004] However, in a multi-layer parking lot or an underground parking lot, vehicle cross-layer travel is inevitable. When the vehicle travels cross-layer, the intelligent driving system cannot automatically complete accurate positioning of the vehicle and cannot provide accurate position information. SUMMARY

[0005] The present application provides a vehicle positioning method, which comprises:

[0006] In the process of cross-layer travel of a target vehicle, if the target vehicle travels to an intersection position of a ramp and a first flat layer, an initial layer height of the first flat layer is determined based on sensor data of the target vehicle;

[0007] Based on the initial layer height and a reference layer height of each flat layer stored, it is determined whether there is an associated flat layer of the first flat layer, a difference between the reference layer height of the associated flat layer and the initial layer height is less than a first threshold value; if not, the initial layer height is stored as the reference layer height of the first flat layer, and if yes, the reference layer height of the associated flat layer is stored as the reference layer height of the first flat layer;

[0008] If the target vehicle travels on the first flat layer, a current layer height determined by sensor data is corrected based on the reference layer height of the first flat layer to obtain a target pseudo-layer height of the first flat layer;

[0009] Target map data matched with the target pseudo-layer height is selected from already mapped map data, a difference between a calibrated layer height of the target map data and the target pseudo-layer height is less than a second threshold value.

[0010] positioning the target vehicle based on the target map data.

[0011] The application provides a vehicle positioning device, which comprises:

[0012] A determining module is configured to, during the target vehicle driving across layers, determine an initial layer height of a first flat layer based on sensor data of the target vehicle if the target vehicle drives to a junction position of the first flat layer and a slope; determine whether there is an associated flat layer of the first flat layer based on the initial layer height and a reference layer height of each flat layer stored in the memory, the difference between the reference layer height of the associated flat layer and the initial layer height being less than a first threshold value; if not, store the initial layer height as the reference layer height of the first flat layer, and if yes, store the reference layer height of the associated flat layer as the reference layer height of the first flat layer.

[0013] A correcting module is configured to, if the target vehicle drives on the first flat layer, correct a current layer height determined based on sensor data based on the reference layer height of the first flat layer to obtain a target pseudo layer height of the first flat layer.

[0014] A selecting module is configured to select target map data matching the target pseudo layer height from built map data, the difference between a calibrated layer height of the target map data and the target pseudo layer height being less than a second threshold value.

[0015] A positioning module is configured to position the target vehicle based on the target map data.

[0016] The application provides a vehicle terminal device, which comprises a processor and a machine readable storage medium, the machine readable storage medium stores machine executable instructions which can be executed by the processor; the processor is used to execute the machine executable instructions to realize the vehicle positioning method.

[0017] The application provides a vehicle, which comprises:

[0018] A camera is configured to acquire surround view image data and send the surround view image data to a processor.

[0019] An IMU sensor is configured to acquire acceleration information and angular velocity information and send the acceleration information and the angular velocity information to the processor.

[0020] A wheel speed sensor is configured to acquire wheel speed and send the wheel speed to the processor.

[0021] The processor is used to realize the vehicle positioning method based on the surround view image data, the acceleration information, the angular velocity information and the wheel speed.

[0022] The vehicle comprises a vehicle carrying an intelligent driving system.

[0023] The vehicle comprises a vehicle carrying an autonomous unmanned system.

[0024] The application provides a machine-readable storage medium storing machine executable instructions capable of being executed by a processor; wherein the processor is configured to execute the machine executable instructions to implement the vehicle positioning method.

[0025] The application provides a computer program product comprising a computer program, which, when executed by a processor, implements the vehicle positioning method.

[0026] As can be seen from the above technical solutions, in the embodiment of the application, during the cross-layer driving of the target vehicle, if the target vehicle drives on the first flat layer, the target map data can be selected based on the target pseudo-floor height of the first flat layer, and the target vehicle is positioned based on the target map data, that is, the target vehicle is positioned based on the target pseudo-floor height to filter accurate map data, so that when the target vehicle drives across layers, the accurate positioning of the vehicle can be automatically completed, and accurate position information is provided for the automatic driving function of the vehicle. For positioning in similar scenarios of different floors, the target pseudo-floor height is used to estimate the floor height of the floor where the target vehicle is located, and the initial positioning pose is determined in combination with the map and visual observation, so as to ensure that the initial positioning is accurate and effective, and false matching can be effectively prevented.

[0027] For example, during the driving of the target vehicle on the first flat layer, the error of the sensor data accumulated is getting larger and larger (the sensor data is obtained by recursion, and as the recursion distance increases, the error will gradually increase), therefore, the error of the current floor height determined by the sensor data is getting larger and larger. Based on this, the current floor height determined by the sensor data is corrected based on the reference floor height of the first flat layer to obtain the target pseudo-floor height, and the accuracy of the target pseudo-floor height is higher than that of the current floor height, so that when the target map data is selected based on the target pseudo-floor height, accurate and reliable target map data can be selected, thereby improving the accuracy of positioning the target vehicle. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1A is a flowchart of the vehicle positioning method in an embodiment of the application;

[0029] Figure 1B is a flowchart of the map construction method in an embodiment of the application;

[0030] Figure 2 is a sensor diagram of the demonstration vehicle / target vehicle in an embodiment of the application;

[0031] Figure 3 is a schematic diagram of a mapping positioning process in an embodiment of the present application;

[0032] Figure 4 is a schematic diagram of a mapping process in an embodiment of the present application;

[0033] Figure 5 is a schematic diagram of a double-threshold ramp detection in an embodiment of the present application;

[0034] Figure 6 is a schematic diagram of a pseudo-layer height algorithm in an embodiment of the present application;

[0035] Figure 7 is a schematic diagram of a real-time positioning process in an embodiment of the present application;

[0036] Figure 8 is a schematic diagram of a vehicle positioning device in an embodiment of the present application;

[0037] Figure 9 is a hardware structure diagram of a vehicle terminal device in an embodiment of the present application. DETAILED DESCRIPTION

[0038] A vehicle positioning method is provided in the embodiments of the present application, which can be applied to a vehicle terminal device, and the vehicle terminal device is deployed on a target vehicle. The vehicle terminal device supports an intelligent driving system, and is used to realize intelligent driving of the target vehicle. For example, the target vehicle can be a vehicle carrying an intelligent driving system (such as a common vehicle carrying an intelligent driving system or an automatic parking system, and the intelligent driving system is used to assist a driver to complete vehicle positioning). Alternatively, the target vehicle can also be a vehicle carrying an autonomous unmanned driving system (i.e., a vehicle that does not need a driver to travel, and the autonomous unmanned driving system alone completes vehicle positioning), such as a robot, a logistics vehicle, or an unmanned passenger vehicle, and the type of the target vehicle is not limited.

[0039] Referring to Figure 1A , a schematic diagram of a flowchart of the method is shown, and the method can include:

[0040] In step 101, during the cross-layer driving of the target vehicle, if the target vehicle drives to the intersection position of the ramp and the first flat layer, the initial layer height of the first flat layer is determined based on the sensor data of the target vehicle.

[0041] Step 102, based on the initial layer height and the stored reference layer height of each flat layer, determine whether there is an associated flat layer of the first flat layer, the difference between the reference layer height of the associated flat layer and the initial layer height is less than a first threshold value; if not, the initial layer height of the first flat layer can be stored as the reference layer height of the first flat layer, and if yes, the reference layer height of the associated flat layer is stored as the reference layer height of the first flat layer.

[0042] Step 103, if the target vehicle is running on the first flat layer, the current layer height determined by the sensor data is corrected based on the reference layer height of the first flat layer to obtain the target pseudo layer height of the first flat layer.

[0043] Step 104, select the target map data matched with the target pseudo layer height from the built map data, and the difference between the calibrated layer height of the target map data and the target pseudo layer height is less than a second threshold value.

[0044] Step 105, based on the target map data, the target vehicle is positioned.

[0045] The process of determining the intersection position of the slope and the first flat layer where the target vehicle runs to, comprises: determining the pitch angle corresponding to the sensor data of each time of the target vehicle; if the pitch angle corresponding to the sensor data of the first time reaches a first angle threshold, search backward from the first time to a second time, the pitch angle corresponding to the sensor data of the second time reaches a second angle threshold; the second angle threshold is less than the first angle threshold; the position of the target vehicle at the second time is determined as the intersection position of the slope and the first flat layer.

[0046] Determining the pitch angle corresponding to the sensor data of each time of the target vehicle, comprises: if the sensor data includes longitudinal acceleration, determining the pitch angle corresponding to the sensor data based on the longitudinal acceleration and the wheel acceleration of the target vehicle; wherein the pitch angle corresponding to the sensor data is determined by the following formula: Or, if the sensor data includes lateral acceleration, longitudinal acceleration and skyward acceleration, the pitch angle corresponding to the sensor data is determined based on the lateral acceleration, longitudinal acceleration, skyward acceleration and wheel acceleration of the target vehicle; wherein the pitch angle corresponding to the sensor data is determined by the following formula: θ pitch represents the pitch angle, a lon represents the longitudinal acceleration, a vel represents the wheel acceleration, g represents the gravitational acceleration, a lat represents the lateral acceleration, a up represents the skyward acceleration.

[0047] For example, if the target vehicle is driving on the first flat layer, the positioning of the target vehicle based on the target map data includes: when the target vehicle is initially positioned, obtaining the surround view image data of the target vehicle, determining the visual feature point feature of the current position based on the surround view image data; and based on the visual feature point feature of the current position and the visual feature point feature of the plurality of trajectory points in the target map data, the target vehicle is initially positioned. Alternatively, when the target vehicle is follow-up positioned, the surround view image data of the target vehicle is obtained, and the visual semantic feature of the current position is determined based on the surround view image data; and based on the visual semantic feature of the current position and the visual semantic feature of the plurality of trajectory points in the target map data, the target vehicle is follow-up positioned.

[0048] For example, when the target vehicle is driving on the first flat layer, the first flat layer can have lane lines, speed bumps and other special markings. The visual feature point feature refers to the detection and description of key points in image processing. These key points have unique properties in the image, such as corner points and edges, and can represent certain specific regions or objects in the image. Based on this, the visual feature point feature of lane lines, speed bumps and other special markings can be extracted, such as using the Difference of Gaussian (DoG) to extract the visual feature point feature, and the extraction method of the visual feature point feature is not limited.

[0049] When the target vehicle is driving on the first flat layer, the first flat layer can have lane lines, speed bumps and other special markings. The visual semantic feature refers to the semantic description of an object in an image. For example, the position of the lane line in the image is identified, and the lane line is labeled as the visual semantic of the position. The position of the speed bump in the image is identified, and the speed bump is labeled as the visual semantic of the position. In summary, the visual semantic feature can identify the position of the special marking in the image and label the visual semantic of the position, which is not limited.

[0050] For example, if the target vehicle is driving on a slope, the first map data matching the slope can also be selected from the built map data. On this basis, based on the first map data matching the slope, when the target vehicle is initially positioned or follow-up positioned, the surround view image data of the target vehicle is obtained, and the visual feature point feature of the current position is determined based on the surround view image data; and based on the visual feature point feature of the current position and the visual feature point feature of the plurality of trajectory points in the first map data, the target vehicle is initially positioned or follow-up positioned. That is, the initial positioning or follow-up positioning is realized by using the visual feature point feature.

[0051] It can be seen from the above technical solutions that, in the embodiment of the application, if the target vehicle travels on the first flat layer during the cross-layer travel of the target vehicle, the target map data can be selected based on the target pseudo-floor height of the first flat layer, and the target vehicle is positioned based on the target map data, that is, the target vehicle is positioned based on the accurate map data selected based on the target pseudo-floor height. In this way, when the target vehicle travels across layers, the accurate positioning of the vehicle can be automatically completed, and accurate position information is provided for the automatic driving function of the vehicle. For positioning in similar scenarios of different floors, the floor height of the floor where the target vehicle is located is estimated using the target pseudo-floor height, and the initial positioning pose is determined in combination with the map and visual observation to ensure accurate and effective initial positioning, which can effectively prevent mismatching.

[0052] For example, during the travel of the target vehicle on the first flat layer, the error of the current floor height determined by the sensor data becomes larger and larger (the sensor data is recursively obtained, and as the recursive distance increases, the error gradually increases). Therefore, the current floor height determined by the sensor data is corrected based on the reference floor height of the first flat layer to obtain the target pseudo-floor height, and the accuracy of the target pseudo-floor height is higher than that of the current floor height. When the target map data is selected based on the target pseudo-floor height, accurate and reliable target map data can be selected, thereby improving the accuracy of positioning the target vehicle.

[0053] In the embodiment of the application, a map construction method is proposed, which can be applied to a vehicle terminal device, and the vehicle terminal device is deployed on a teaching vehicle. The vehicle terminal device supports an intelligent driving system, and is used to realize intelligent driving of the teaching vehicle. For example, the teaching vehicle can be a vehicle carrying an intelligent driving system (such as a common vehicle carrying an intelligent driving system or an automatic parking system, and the intelligent driving system is used to assist the driver to complete vehicle positioning). Alternatively, the teaching vehicle can also be a vehicle carrying an autonomous unmanned driving system (i.e., a vehicle that does not need a driver to travel, and the autonomous unmanned driving system alone completes vehicle positioning), such as a robot, a logistics vehicle, or an unmanned passenger vehicle. The type of the teaching vehicle is not limited.

[0054] Referring to Figure 1B As shown in the figure, the method can include the following steps:

[0055] In step 111, during the cross-layer travel of the teaching vehicle, if the teaching vehicle travels to the intersection position of the slope and the second flat layer, the initial floor height of the second flat layer is determined based on the sensor data of the teaching vehicle.

[0056] Step 112, based on the initial layer height and the stored reference layer height of each flat layer, determine whether there is an associated flat layer of the second flat layer, the difference between the reference layer height of the associated flat layer and the initial layer height is less than the first threshold value; if not, the initial layer height of the second flat layer can be stored as the reference layer height of the second flat layer, and if yes, the reference layer height of the associated flat layer is stored as the reference layer height of the second flat layer.

[0057] Step 113, if the teaching vehicle is driving on the second flat layer, the current layer height determined based on the sensor data is corrected based on the reference layer height of the second flat layer to obtain the calibrated layer height of the second flat layer.

[0058] Step 114, establish the map data of the teaching vehicle on the second flat layer, and associate the established map data of the second flat layer with the calibrated layer height based on the calibrated layer height of the second flat layer.

[0059] The process of determining the intersection position of the slope and the second flat layer where the teaching vehicle drives to, comprises: determining the pitch angle corresponding to the sensor data of each time of the teaching vehicle; if the pitch angle corresponding to the sensor data of the third time reaches the first angle threshold, searching backward from the third time to the fourth time, the pitch angle corresponding to the sensor data of the fourth time reaches the second angle threshold; the second angle threshold is less than the first angle threshold; the position of the teaching vehicle at the fourth time is determined as the intersection position of the slope and the second flat layer.

[0060] Illustratively, determining the pitch angle corresponding to the sensor data of each time of the teaching vehicle can include but is not limited to: if the sensor data includes longitudinal acceleration, determining the pitch angle corresponding to the sensor data based on the longitudinal acceleration and the wheel acceleration of the teaching vehicle; or, if the sensor data includes lateral acceleration, longitudinal acceleration and skyward acceleration, determining the pitch angle corresponding to the sensor data based on the lateral acceleration, longitudinal acceleration, skyward acceleration and wheel acceleration of the teaching vehicle.

[0061] Illustratively, if the teaching vehicle is driving on the second flat layer, establishing the map data of the teaching vehicle on the second flat layer can include: if the driving distance of the teaching vehicle from the preset initial position is less than the preset distance threshold, obtaining the surround view image data of the teaching vehicle, determining the visual feature point feature of the current position based on the surround view image data; based on the visual feature point feature of the current position, establishing the map data of the teaching vehicle on the second flat layer, which is used for initial positioning of the target vehicle. Or, if the driving distance of the teaching vehicle from the preset initial position is not less than the preset distance threshold, obtaining the surround view image data of the teaching vehicle, determining the visual semantic feature of the current position based on the surround view image data; based on the visual semantic feature of the current position, establishing the map data of the teaching vehicle on the second flat layer, which is used for following positioning of the target vehicle.

[0062] For example, if the teaching vehicle is driving on a slope, map data of the teaching vehicle on the slope can also be established. For example, establishing the map data of the teaching vehicle on the slope can include but is not limited to: if the driving distance of the teaching vehicle from the preset initial position is less than the preset distance threshold, the surround view image data of the teaching vehicle is acquired, the visual feature point features of the current position are determined based on the surround view image data; the map data of the teaching vehicle on the slope is established based on the visual feature point features of the current position, which can be used for initial positioning of the target vehicle. Alternatively, if the driving distance of the teaching vehicle from the preset initial position is not less than the preset distance threshold, the surround view image data of the teaching vehicle is acquired, the visual feature point features of the current position are determined based on the surround view image data; the map data of the teaching vehicle on the slope is established based on the visual feature point features of the current position, which can be used for follow-up positioning of the target vehicle.

[0063] The above technical solutions of the embodiments of the present application are described below in combination with specific application scenarios.

[0064] The mapping and positioning technology includes two stages of teaching mapping and real-time positioning. In the teaching mapping stage, the driver drives the vehicle, the intelligent driving system records the motion trajectory of the vehicle in the teaching process as a teaching trajectory, and observes the scanning of the surrounding scene for mapping. In the real-time positioning stage, once the vehicle enters the built map area, the intelligent driving system can automatically complete the positioning of the vehicle, and provide accurate position information for the automatic driving function of the vehicle. However, in a multi-story parking lot or an underground parking lot, when the vehicle crosses the layers, the intelligent driving system cannot automatically complete the accurate positioning of the vehicle, and cannot provide accurate position information.

[0065] For example, the similarity of scenes in different floors is high, which can cause mis-matching in initial positioning, and then cause the inability to automatically complete the accurate positioning of the vehicle. Cross-layer means passing through a slope, and the visual semantic information of the slope is often not rich enough, which can cause the inability to automatically complete the accurate positioning of the vehicle. The computing resources of the vehicle terminal device are limited, which cannot construct a complex map and cannot use a positioning algorithm with large resource consumption. Therefore, the accurate positioning of the vehicle cannot be automatically completed based on the complex map and the positioning algorithm.

[0066] In view of the above finding, an embodiment of the present application proposes a vehicle positioning method, which can realize a cross-layer memory parking mapping positioning scheme in the case of limited computing resources of a vehicle terminal device. For the positioning problem of similar scenes in different floors, the pseudo-floor height is used to estimate the floor height of the vehicle, and the initial positioning pose is determined in combination with the map and visual observation, which can effectively prevent false matching. For the problem of insufficient features in the ramp scene, the IMU (Inertial Measurement Unit) gravity constraint is used to calculate the pitch angle, and a multi-threshold method is used to improve the accuracy of ramp detection. In the ramp scene, more feature points are used to complete mapping and positioning. In the flat scene, a visual semantic with higher precision and efficiency is used to complete mapping and positioning.

[0067] For example, the mapping and positioning technology can include two stages of demonstration mapping and real-time positioning. For convenience, the vehicle in the demonstration mapping stage is referred to as a demonstration vehicle, and the vehicle in the real-time positioning stage is referred to as a target vehicle. The demonstration vehicle and the target vehicle can be the same, and the demonstration vehicle and the target vehicle can also be different.

[0068] The demonstration vehicle / target vehicle can include a vehicle terminal device, and the vehicle positioning is realized by the vehicle terminal device. In addition to the vehicle terminal device, referring to Figure 2 As shown, the demonstration vehicle / target vehicle further includes the following sensors.

[0069] Surround-view camera: taking four cameras as an example, the four cameras are left camera, right camera, front camera and rear camera. Of course, the number of cameras can be more or less. For example, two front and rear cameras. Or a camera with 8 lenses on the top of the vehicle. The surround-view camera is used to provide visual observation of the environment around the vehicle. For example, surround-view image data can be collected by the camera and sent to the vehicle terminal device.

[0070] Positioning sensor: the positioning sensor can be a GPS (Global Positioning System) sensor, and the positioning sensor can also be a Beidou sensor, which is not limited. Taking the GPS sensor as an example. The positioning sensor is used to provide satellite positioning signals, and can output the absolute pose of the demonstration vehicle / target vehicle, that is, the absolute pose of the demonstration vehicle / target vehicle can be sent to the vehicle terminal device.

[0071] IMU sensor / wheel speed sensor. The IMU sensor can be a 3-axis IMU sensor or a 6-axis IMU sensor, which is used to provide motion information of the demonstration vehicle / target vehicle, such as acceleration information and angular velocity information, that is, the acceleration information and angular velocity information of the demonstration vehicle / target vehicle can be sent to the vehicle terminal device.

[0072] The wheel speed sensor is used to provide motion information of the teaching vehicle / target vehicle, such as detecting the wheel speed of the teaching vehicle / target vehicle, and sending the wheel speed of the teaching vehicle / target vehicle to the terminal device.

[0073] For example, the data output by the IMU sensor (such as acceleration information and angular velocity information) and the data output by the wheel speed sensor (such as wheel speed) can be referred to as sensor data, i.e., vehicle motion data.

[0074] For example, referring to Figure 3 As shown, a schematic diagram of the mapping positioning process is shown.

[0075] For the mapping positioning process, the input data is the data provided by the camera (such as surround view image data around the teaching vehicle / target vehicle), the data provided by the GPS sensor (such as the absolute pose of the teaching vehicle / target vehicle), the data provided by the IMU sensor (such as acceleration information and angular velocity information of the teaching vehicle / target vehicle), and the data provided by the wheel speed sensor (such as wheel speed of the teaching vehicle / target vehicle).

[0076] After obtaining the input data, the terminal device can perform data preprocessing on the input data. In the data preprocessing process, time synchronization and invalid data filtering can be performed. Time synchronization refers to associating data at the same time, such as associating surround view image data, absolute pose, acceleration information, angular velocity information, and wheel speed at the same time. Invalid data filtering refers to removing invalid data, redundant data, etc.

[0077] In the mapping mode, the terminal device can implement teaching mapping based on the input data to obtain map data. In the positioning mode, the terminal device can implement real-time positioning based on the input data and the map data to obtain a positioning result, which can represent the real-time position of the target vehicle. Before implementing real-time positioning, teaching mapping for the same section has been completed, i.e., teaching mapping is before real-time positioning.

[0078] In the mapping mode, the terminal device comprehensively utilizes the observation information of each sensor, extracts and generates necessary elements in the scene, and completes map construction after fusion processing. In the positioning mode, the terminal device combines the built map and the observation information of each sensor to calculate the vehicle position and attitude at the current time.

[0079] First, the teaching mapping process. After the driver clicks the start mapping button, the teaching vehicle starts the teaching mapping function. Referring to Figure 4 As shown, a flowchart of the teaching mapping process is shown. The teaching mapping process can include six processes: odometer calculation, ramp detection, pseudo-layer height estimation, mapping mode selection, feature point extraction and fusion, and semantic extraction and fusion. These processes are described below.

[0080] 1. Odometer calculation. For the odometer calculation process, the input data can be sensor data (e.g. acceleration information and angular velocity information output by the IMU sensor, and wheel speed output by the wheel speed sensor), and the output data can be odometer data, which can include the pose of the vehicle, i.e. the position and the attitude, and the attitude at least includes the pitch angle, i.e. the pitch angle corresponding to the teaching vehicle.

[0081] In the odometer calculation process, based on the sensor data (e.g. acceleration information and angular velocity information output by the IMU sensor, and wheel speed output by the wheel speed sensor), the relative pose of the teaching vehicle at different times (i.e. odometer data) can be calculated in a recursive manner, which can include the pitch angle.

[0082] For a 3-axis IMU sensor, the lateral acceleration, longitudinal acceleration and heading angular velocity of the teaching vehicle can be output, and there is no direct measurement of the pitch angular velocity and roll angular velocity. For a 6-axis IMU sensor, although the output data also includes the skyward acceleration, pitch angular velocity and roll angular velocity, after a long time of recursion, the pitch angle and roll angle calculated by the odometer will gradually diverge, which is not conducive to slope detection.

[0083] Therefore, in this embodiment, gravity constraint is introduced to provide observation of the pitch angle and roll angle, and to suppress the divergence of the pitch angle and roll angle. The gravity constraint can provide absolute pitch angle and roll angle observation, effectively suppress the divergence of these two degrees of freedom, and ensure that the pitch angle calculated by the odometer is consistent with the actual value. When implementing slope detection and pseudo-layer height estimation based on the pitch angle, the feasibility of slope detection and pseudo-layer height estimation is ensured.

[0084] In one possible implementation, if the sensor data includes longitudinal acceleration, the pitch angle corresponding to the teaching vehicle can be determined based on the longitudinal acceleration and the wheel acceleration of the teaching vehicle. For example, for a 3-axis IMU sensor, the sensor data output by the IMU sensor can include longitudinal acceleration, and therefore the pitch angle and roll angle corresponding to the teaching vehicle can be determined using the following formula (1).

[0085]

[0086] In one possible implementation, if the sensor data includes lateral acceleration, longitudinal acceleration and skyward acceleration, the pitch angle corresponding to the teaching vehicle can be determined based on the lateral acceleration, longitudinal acceleration, skyward acceleration and wheel acceleration of the teaching vehicle. For example, for a 6-axis IMU sensor, the sensor data output by the IMU sensor can include lateral acceleration, longitudinal acceleration and skyward acceleration, and therefore the pitch angle and roll angle corresponding to the teaching vehicle can be determined using the following formula (2).

[0087]

[0088] In formulas (1) and (2), θ pitch θ represents the pitch angle corresponding to the teaching vehicle. roll This indicates the roll angle corresponding to the teaching vehicle. lon a represents longitudinal acceleration. lat a represents lateral acceleration. up These three accelerations represent the axial acceleration, which can be determined by sensor data output from the IMU sensor.

[0089] a vel This represents wheel acceleration, which can be determined from the wheel speed data output by the wheel speed sensor. For example, the wheel speed can be determined based on the wheel speed data, and the wheel acceleration can be obtained by differentiating the wheel speeds.

[0090] g represents gravitational acceleration, which can be a fixed acceleration value.

[0091] 2. Slope Detection. For the slope detection process, the input data can be the pitch angle of the teaching vehicle, and the output data can be the slope detection result, which indicates whether it is a slope or a level.

[0092] For example, during ramp detection, if the pitch angle of the teaching vehicle is greater than a threshold, the location of the teaching vehicle can be considered a ramp. However, since pitch angle estimation is affected by noise and fluctuates, the single-threshold detection method has poor stability and is prone to false ramp detection.

[0093] To address this issue, this embodiment employs a dual-threshold ramp detection method. See [link / reference] Figure 5 The diagram illustrates a dual-threshold ramp detection method. When the pitch angle reaches threshold B for multiple consecutive frames, the teaching vehicle is determined to have entered the ramp. Using this time period as a baseline, all moments exceeding threshold A are searched forward and backward, and the teaching vehicle's position at these moments is considered a ramp. This dual-threshold ramp detection method effectively filters out false ramp detections caused by pitch angle estimation errors, thus accurately detecting ramps.

[0094] In a possible implementation, based on the calculation of the odometer, the pitch angle corresponding to the sensor data at each time of the teaching vehicle can be obtained. On this basis, if the pitch angle corresponding to the sensor data at the third time reaches the first angle threshold (such as threshold B), the fourth time is searched backward from the third time, and the pitch angle corresponding to the sensor data at the fourth time reaches the second angle threshold (such as threshold A), that is, the fourth time is the first time after the third time when the pitch angle is equal to the second angle threshold. The fifth time is searched forward from the third time, and the pitch angle corresponding to the sensor data at the fifth time reaches the second angle threshold, that is, the fifth time is the first time before the third time when the pitch angle is equal to the second angle threshold.

[0095] On this basis, the position of the teaching vehicle between the fifth time and the fourth time can be a ramp position. Assuming that the position of the teaching vehicle at the fifth time is recorded as position 5, and the position of the teaching vehicle at the fourth time is recorded as position 4, the positions between position 5 and position 4 are ramp positions.

[0096] For example, the first angle threshold (such as threshold B) can be configured according to experience, and the second angle threshold (such as threshold A) can be configured according to experience, and the second angle threshold is less than the first angle threshold.

[0097] Obviously, all ramp positions can be obtained by using the above-mentioned method. In a multi-story parking lot or an underground parking lot, when the teaching vehicle travels across layers, in addition to the ramp positions, the remaining positions are flat layer positions. Therefore, the positions between two ramp positions can be regarded as flat layer positions. For example, the positions between position 5 and position 4 are ramp positions, and the positions between position 3 and position 2 are ramp positions. When the teaching vehicle travels from position 4 to position 3, the positions between position 4 and position 3 are flat layer positions, and position 4 is the junction position of the ramp and the flat layer (i.e., the starting position of the flat layer), and position 3 is the junction position of the ramp and the flat layer (i.e., the end position of the flat layer). Similarly, when the teaching vehicle travels from position 3 to position 4, the positions between position 3 and position 4 are flat layer positions, and position 3 is the junction position of the ramp and the flat layer (i.e., the starting position of the flat layer), and position 4 is the junction position of the ramp and the flat layer (i.e., the end position of the flat layer).

[0098] In summary, in the ramp detection process, the ramp detection result can be obtained, which can indicate whether each position is a ramp position or a flat layer position. For the junction position of the ramp and the flat layer, the ramp detection result can also indicate that the position is the junction position of the ramp and the flat layer.

[0099] 3. Pseudo-layer height estimation. For the pseudo-layer height estimation process, the input data can be sensor data, ramp detection results and GPS data, and the output data can be a pseudo-layer height, which is a calibration layer height, i.e., a calibration pseudo-layer height.

[0100] Exemplarily, in the outdoor scene, the GPS sensor can output the absolute height of the location where the teaching vehicle is located. Once entering the indoor environment, the GPS signal will be blocked by the building and disappear, and the GPS sensor cannot output the absolute height of the location where the teaching vehicle is located. Thereafter, the absolute height of the location where the teaching vehicle is located can be determined based on the sensor data. However, the sensor data is recursively obtained to determine the absolute height of the location where the teaching vehicle is located, and the height error will gradually increase as the recursive distance increases.

[0101] For example, referring to FIG. 1, a height variation diagram before correction is shown. After the GPS signal disappears, the height of the teaching vehicle can be recursively obtained based on the sensor data. As the recursive distance increases, the height gradually diverges, and after the teaching vehicle returns to the ground, a large height error has appeared compared with the actual value. Figure 6 FIG. 2 is a diagram for correcting the recursive error of the flat layer height. The height of the flat layer position is corrected by using the ramp detection result, that is, the height of the flat layer does not change, so that the height error is only caused by the ramp recursive error. Obviously, compared with FIG. 1, part of the height error can be corrected.

[0102] For example, referring to FIG. 1, a height variation diagram before correction is shown. After the GPS signal disappears, the height of the teaching vehicle can be recursively obtained based on the sensor data. As the recursive distance increases, the height gradually diverges, and after the teaching vehicle returns to the ground, a large height error has appeared compared with the actual value. Figure 6 For example, referring to FIG. 1, a height variation diagram before correction is shown. After the GPS signal disappears, the height of the teaching vehicle can be recursively obtained based on the sensor data. As the recursive distance increases, the height gradually diverges, and after the teaching vehicle returns to the ground, a large height error has appeared compared with the actual value.

[0103] Figure 6 For example, based on the ramp detection result, it is determined that the positions A to B belong to the same flat layer, and the position A is the starting position of the flat layer. Then, based on the height x of the position A, the height of all positions from the position A to the position B is set to the height x, that is, the height of all positions of the flat layer is the height x. In this way, the height of all positions of the flat layer does not change, that is, the height is not changed based on the sensor data. Obviously, after the above processing, the height error caused by the sensor data in the flat layer can be corrected.

[0104] For example, referring to FIG. 1, a height variation diagram before correction is shown. After the GPS signal disappears, the height of the teaching vehicle can be recursively obtained based on the sensor data. As the recursive distance increases, the height gradually diverges, and after the teaching vehicle returns to the ground, a large height error has appeared compared with the actual value.

[0105] For example, referring to FIG. 1, a height variation diagram before correction is shown. After the GPS signal disappears, the height of the teaching vehicle can be recursively obtained based on the sensor data. As the recursive distance increases, the height gradually diverges, and after the teaching vehicle returns to the ground, a large height error has appeared compared with the actual value. Figure 6 For example, referring to FIG. 1, a height variation diagram before correction is shown. After the GPS signal disappears, the height of the teaching vehicle can be recursively obtained based on the sensor data. As the recursive distance increases, the height gradually diverges, and after the teaching vehicle returns to the ground, a large height error has appeared compared with the actual value.

[0106] ​For example, the height x1 of the -1 floor can be recorded when the teaching vehicle passes the -1 floor for the first time, and the height x2 of the -1 floor can be recorded when the teaching vehicle passes the -1 floor for the second time. If the absolute value of the difference between the height x1 and the height x2 is less than the height difference threshold, it indicates that the height x1 and the height x2 correspond to the same floor.

[0107] For example, the height x1 of the -1 floor can be recorded when the teaching vehicle passes the -1 floor for the first time, and the height x2 of the -1 floor can be recorded when the teaching vehicle passes the -1 floor for the second time. If the absolute value of the difference between the height x1 and the height x2 is less than the height difference threshold, it indicates that the height x1 and the height x2 correspond to the same floor. Figure 6 FIG. 4 of the fourth aspect is a schematic diagram of the -1 floor height correction and the ground height association. The floor will be corrected according to the height of the first pass, and after the height of the -1 floor is corrected, the height difference of the two passes of the ground will also be less than the height difference threshold, so the ground height will also be associated.

[0108] For example, the height x1 of the -1 floor can be recorded when the teaching vehicle passes the -1 floor for the first time, and the height x2 of the -1 floor can be recorded when the teaching vehicle passes the -1 floor for the second time. If the absolute value of the difference between the height x1 and the height x2 is less than the height difference threshold, it indicates that the height x1 and the height x2 correspond to the same floor.

[0109] For example, the height x1 of the -1 floor can be recorded when the teaching vehicle passes the -1 floor for the first time, and the height x2 of the -1 floor can be recorded when the teaching vehicle passes the -1 floor for the second time. If the absolute value of the difference between the height x1 and the height x2 is less than the height difference threshold, it indicates that the height x1 and the height x2 correspond to the same floor. Figure 6 FIG. 5 of the fifth aspect is a schematic diagram of the ground height correction. After the ground height is associated, the ground height will also be corrected according to the height of the first pass, and thus the height error between the same floors is eliminated.

[0110] Of course, the present application is not limited to a two-floor floor scenario, and for an indoor cross-slope floor scenario, the above-mentioned way of calculating the pseudo-floor height can be used to eliminate the sensor data cumulative error between the same floors.

[0111] When the above-mentioned correction is used, the corrected floor height is not completely consistent with the true height, so the corrected floor height can be referred to as a pseudo-floor height. For example, from the principle of the pseudo-floor height algorithm, it can be seen that the sensor data recursive error is not corrected when the corresponding floor ramp is passed for the first time, but the error is relatively small and will not affect the accuracy of the teaching mapping process and the real-time positioning process.

[0112] For example, based on the principle of the pseudo-floor height algorithm, the following way can be used to determine the calibration floor height:

[0113] If the GPS signal is valid, the GPS sensor can output the absolute height of the position where the teaching vehicle is located, so if the teaching vehicle drives to a flat floor, the reference floor height of the flat floor can be determined based on the height output by the GPS sensor, and the reference floor height of the flat floor is stored. This process is not limited.

[0114] If the GPS signal is invalid, then if the teaching vehicle drives to a flat floor, it can include:

[0115] Step S11, during the process of the teaching vehicle crossing layers, if the teaching vehicle drives to the intersection position of the slope and the second flat layer, the initial layer height of the second flat layer is determined based on the sensor data of the teaching vehicle. For convenience, the flat layer in the teaching mapping process is recorded as the second flat layer, which can be any flat layer.

[0116] For example, when the GPS signal is effective, the GPS sensor can output the absolute height of the position where the teaching vehicle is located, and when the GPS signal is invalid, the absolute height when the GPS signal is invalid is recorded as height k1. In the subsequent process, the height of the position where the teaching vehicle is located is determined based on the sensor data on the basis of height k1.

[0117] For example, from the beginning of the GPS signal being invalid, it is assumed that the height of the teaching vehicle is determined to decrease by height k2 based on the sensor data, and the height of the current position is k1-k2, and so on.

[0118] It is assumed that height k1 is the height of a flat layer, and when the teaching vehicle drives from the flat layer to another flat layer, it will pass through a slope, and the height of the slope can be counted. For example, based on the slope detection result, the starting position of the slope and the ending position of the slope can be obtained, and based on the sensor data at the starting position of the slope and the sensor data at the ending position of the slope, the height of the slope can be determined, which is recorded as height k3. Based on this, if the teaching vehicle drives to the intersection position of the slope and the flat layer, the initial layer height of the flat layer is determined, that is, k1-k3.

[0119] It is assumed that the reference layer height (definition see subsequent steps) of a flat layer is k4, and when the teaching vehicle drives from the flat layer to another flat layer, it will pass through a slope, and based on the sensor data at the starting position of the slope and the sensor data at the ending position of the slope, the height of the slope can be determined, which is recorded as height k5. Based on this, if the teaching vehicle drives to the intersection position of the slope and the flat layer, the initial layer height of the flat layer is determined as k4-k5. If the teaching vehicle drives from the flat layer to another flat layer, the initial layer height of the flat layer is determined as k4+k5.

[0120] In summary, during the process of the teaching vehicle crossing layers, if the teaching vehicle drives to the intersection position of the slope and the second flat layer, the initial layer height of the second flat layer can be determined, and this process is not limited.

[0121] Step S12, based on the initial layer height of the second flat layer and the reference layer height of each flat layer stored, it is determined whether there is an associated flat layer of the second flat layer, the difference between the reference layer height of the associated flat layer and the initial layer height of the second flat layer is less than a first threshold value, and the first threshold value can be configured according to experience, and this is not limited.

[0122] For example, after obtaining the reference floor height of a certain floor, the reference floor height of the floor can be stored. Based on this, after obtaining the initial floor height of the second floor, the difference between the initial floor height of the second floor and the reference floor height of each floor can be calculated, such as the absolute value of the difference between the initial floor height and the reference floor height of each floor.

[0123] If the difference between the reference floor height of a certain floor and the initial floor height of the second floor is less than the first threshold, the floor is taken as the associated floor of the second floor. If the difference between the reference floor height of all floors and the initial floor height of the second floor is not less than the first threshold, there is no associated floor of the second floor.

[0124] For example, the difference between the initial floor height of the second floor and the reference floor height of each floor is calculated, and the minimum difference is found. If the minimum difference is less than the first threshold, the floor corresponding to the minimum difference is taken as the associated floor of the second floor. If the minimum difference is not less than the first threshold, there is no associated floor of the second floor.

[0125] Step S13, if there is no associated floor of the second floor, the initial floor height of the second floor is stored as the reference floor height of the second floor. If there is an associated floor of the second floor, the reference floor height of the associated floor is stored as the reference floor height of the second floor. At this point, the reference floor height is stored for the second floor.

[0126] Step S14, if the teaching vehicle is driving on the second floor, the current floor height determined based on the sensor data is corrected based on the reference floor height of the second floor to obtain the calibrated floor height of the second floor.

[0127] For example, during the driving of the teaching vehicle on the second floor, the current floor height of the current position of the teaching vehicle can be determined based on the sensor data. On this basis, the current floor height of the current position can be corrected based on the reference floor height of the second floor, that is, the reference floor height of the second floor is taken as the output floor height of the current position. Obviously, the output floor height of each position of the second floor is the reference floor height of the second floor. In order to distinguish conveniently, the floor height of each position of the second floor is recorded as the calibrated floor height, that is, the second floor corresponds to the same calibrated floor height. The accuracy of the calibrated floor height of the second floor is better than the current floor height determined by the sensor data.

[0128] 4. Mapping mode selection. For the mapping mode selection process, the input data can be the ramp detection result, and the output data is the mapping mode, such as the feature point extraction and fusion mode, the semantic extraction and fusion mode.

[0129] For example, in order to solve the problem of insufficient visual semantics in the ramp scene, a scene-based mapping mode can be used. For example, a preset initial position can be pre-configured, which can be the mapping starting point, can be specified by the driver, and can be any position, which is not limited.

[0130] If the driving distance of the teaching vehicle from the preset initial position (i.e., the distance the vehicle has traveled) is less than a preset distance threshold (which can be configured according to experience), the mapping mode is the feature point extraction and fusion mode, that is, the feature point extraction and fusion mode is used for mapping regardless of whether the current position is a ramp or a flat layer, that is, visual feature point mapping is used to ensure that the visual feature point initial positioning function is used for the initial road section of mapping.

[0131] If the driving distance of the teaching vehicle from the preset initial position is not less than the preset distance threshold, then: if the current position of the teaching vehicle is a ramp, the mapping mode is the feature point extraction and fusion mode, and the feature point extraction and fusion mode is used for mapping on the ramp, that is, visual feature point mapping is used at the ramp position to make full use of visual feature information such as wall texture to improve positioning accuracy. If the current position of the teaching vehicle is a flat layer, the mapping mode is the semantic extraction and fusion mode, and the semantic extraction and fusion mode is used for mapping on the flat layer, that is, visual semantic mapping is used at the flat layer position to improve positioning efficiency and accuracy.

[0132] 5. Feature point extraction and fusion. For the feature point extraction and fusion process, the input data can be sensor data and surround view image data (provided by a camera), and the output data can be map data.

[0133] For example, if the teaching vehicle is driving on a second flat layer (which can be any flat layer), if the driving distance of the teaching vehicle from the preset initial position is less than the preset distance threshold, that is, the driving distance is within the preset range, surround view image data of the teaching vehicle is obtained, and visual feature point features of the current position are determined based on the surround view image data. Map data of the teaching vehicle on the second flat layer is established based on the visual feature point features of the current position, and the map data is used for initial positioning of the target vehicle. The mapping process is not limited.

[0134] For example, if the teaching vehicle is driving on a ramp, if the driving distance of the teaching vehicle from the preset initial position is less than the preset distance threshold, surround view image data of the teaching vehicle is obtained, and visual feature point features of the current position are determined based on the surround view image data. Map data of the teaching vehicle on the ramp is established based on the visual feature point features of the current position, and the map data is used for initial positioning of the target vehicle.

[0135] For example, if the teaching vehicle is driving on a ramp, if the driving distance of the teaching vehicle from the preset initial position is not less than the preset distance threshold, surround view image data of the teaching vehicle is obtained, and visual feature point features of the current position are determined based on the surround view image data. Map data of the teaching vehicle on the ramp is established based on the visual feature point features of the current position, and the map data is used for follow-up positioning of the target vehicle.

[0136] In summary, if the driving distance of the teaching vehicle from the preset initial position is less than the preset distance threshold, regardless of whether it is on a flat layer or a ramp, the map data is established using the visual feature point features, and the map data of this process is used for initial positioning of the target vehicle. In addition, if the teaching vehicle is driving on a ramp, regardless of whether the driving distance is less than the preset distance threshold, the map data is established using the visual feature point features.

[0137] 6. Semantic extraction and fusion. For the semantic extraction and fusion process, the input data can be sensor data and surround view image data (provided by a camera), and the output data can be map data.

[0138] For example, if the teaching vehicle is driving on a second flat layer (the second flat layer can be any flat layer), if the driving distance of the teaching vehicle from the preset initial position is not less than the preset distance threshold, the surround view image data of the teaching vehicle can be obtained, and the visual semantic features of the current position can be determined based on the surround view image data. The map data of the teaching vehicle on the second flat layer is established based on the visual semantic features of the current position, and this map data is used for follow-up positioning of the target vehicle, and the mapping process is not limited.

[0139] For example, for visual feature point mapping and visual semantic mapping, image single-frame feature extraction is first performed (visual feature point mapping extracts image feature points, and visual semantic mapping extracts image semantic information), then the pose of the teaching vehicle is determined based on sensor data, and the extracted features are fused based on the pose to form the corresponding map. Neither the visual feature point mapping nor the visual semantic mapping is limited.

[0140] Of course, in order to improve the smoothness of the positioning switching moment, the mapping method selection does not need to be strictly divided according to the ramp and the flat layer, and can be appropriately extended to allow the visual feature point map and the visual semantic map to overlap to some extent.

[0141] For example, for the map data of the teaching vehicle on the second flat layer (established based on visual feature point features or based on visual semantic features), a calibrated layer height can also be set for the built map data of the second flat layer, that is, the built map data of the second flat layer is associated with the calibrated layer height of the second flat layer.

[0142] For example, the calibrated layer height is used to filter map data. That is, for each trajectory point of the teaching vehicle on the second flat layer, the trajectory point can correspond to the calibrated layer height of the second flat layer.

[0143] In summary, after the mapping task is completed, corresponding map data can be output, which can include but is not limited to: the last frame of valid GPS position coordinates before the mapping starts, the teaching trajectory (each trajectory point includes the calibrated height), the visual feature point map, and the visual semantic map.

[0144] Second, real-time positioning process. When the target vehicle enters the range of the built map, the positioning mode can be started. In the positioning process, two processes such as initial positioning and follow-up positioning can be involved. The initial positioning refers to positioning after the target vehicle enters the map without knowing the pose, and the positioning accuracy can reach the meter level. The follow-up positioning refers to positioning with known pose, and the positioning accuracy can reach the centimeter level.

[0145] Referring to Figure 7 FIG. 2 shows a flowchart of the real-time positioning process. The real-time positioning process can include processes such as odometer calculation, slope detection, pseudo height estimation, feature point initial positioning, predicted pose calculation, map selection and cutting, feature point follow-up positioning, and semantic follow-up positioning. The following describes these processes.

[0146] 1. Odometer calculation. For the odometer calculation process, the input data is sensor data (such as acceleration information and angular velocity information output by the IMU sensor, and wheel speed output by the wheel speed sensor), and the output data can be odometer data, which includes the pitch angle, i.e., the pitch angle corresponding to the target vehicle.

[0147] In one possible implementation, if the sensor data includes longitudinal acceleration, the pitch angle corresponding to the target vehicle can be determined based on the longitudinal acceleration and the wheel acceleration of the target vehicle. For example, for a 3-axis IMU sensor, the sensor data output by the IMU sensor can include longitudinal acceleration, and therefore, the pitch angle and roll angle corresponding to the target vehicle can be determined using the above formula (1).

[0148] In one possible implementation, if the sensor data includes lateral acceleration, longitudinal acceleration, and skyward acceleration, the pitch angle corresponding to the target vehicle can be determined based on the lateral acceleration, longitudinal acceleration, skyward acceleration, and wheel acceleration of the target vehicle. For example, for a 6-axis IMU sensor, the sensor data output by the IMU sensor can include lateral acceleration, longitudinal acceleration, and skyward acceleration, and therefore, the pitch angle and roll angle corresponding to the target vehicle can be determined using the above formula (2).

[0149] 2. Slope detection. For the slope detection process, the input data can be the pitch angle corresponding to the target vehicle, and the output data can be the slope detection result, which indicates the slope or flat layer.

[0150] In a possible implementation, based on the calculation of the odometer, the pitch angle corresponding to the sensor data of each moment of the target vehicle can be obtained. On this basis, if the pitch angle corresponding to the sensor data of the first moment reaches a first angle threshold (such as threshold B), the second moment is searched backward from the first moment, and the pitch angle corresponding to the sensor data of the second moment reaches a second angle threshold (such as threshold A), that is, the second moment is the first moment behind the moment when the pitch angle is equal to the second angle threshold. The sixth moment is searched forward from the first moment, and the pitch angle corresponding to the sensor data of the sixth moment reaches the second angle threshold, that is, the sixth moment is the first moment before the moment when the pitch angle is equal to the second angle threshold. On this basis, the position of the target vehicle between the sixth moment and the second moment can be a ramp position.

[0151] Obviously, all ramp positions can be obtained by using the above method. In addition to the ramp position, the remaining positions are flat positions, and the positions between two ramp positions can be regarded as flat positions.

[0152] In summary, in the ramp detection process, a ramp detection result can be obtained, which can indicate whether each position is a ramp position or a flat position. For the junction position of the ramp and the flat, the ramp detection result can also indicate that the position is the junction position of the ramp and the flat.

[0153] 3. Pseudo-layer height estimation. For the pseudo-layer height estimation process, the input data can be sensor data, a ramp detection result, and GPS data, and the output data can be a pseudo-layer height, denoted as a target pseudo-layer height.

[0154] For example, if the GPS signal is invalid, when the target vehicle travels to a flat, it can include:

[0155] Step S21, during the cross-layer driving of the target vehicle, if the target vehicle travels to the junction position of the ramp and the flat, the initial layer height of the flat is determined based on the sensor data of the target vehicle. In order to distinguish conveniently, the flat in the real-time positioning process is denoted as a first flat, and the first flat can be any flat.

[0156] For the way of determining the initial layer height of the first flat, refer to step S11, which will not be repeated here.

[0157] Step S22, based on the initial layer height of the first flat and the reference layer height of each flat stored, it is determined whether there is an associated flat of the first flat, the difference between the reference layer height of the associated flat and the initial layer height of the first flat is less than a first threshold, and the first threshold can be configured according to experience, which is not limited.

[0158] For example, after obtaining the reference layer height of a certain flat layer, the reference layer height of the flat layer can be stored. Based on this, after obtaining the initial layer height of the first flat layer, the difference between the initial layer height of the first flat layer and the reference layer height of each flat layer is calculated. If the difference between the reference layer height of a certain flat layer and the initial layer height of the first flat layer is less than the first threshold value, the flat layer is taken as the associated flat layer of the first flat layer. If the difference between the reference layer height of all flat layers and the initial layer height of the first flat layer is not less than the first threshold value, there is no associated flat layer of the first flat layer.

[0159] Step S23, if there is no associated flat layer of the first flat layer, the initial layer height of the first flat layer is stored as the reference layer height of the first flat layer. If there is an associated flat layer of the first flat layer, the reference layer height of the associated flat layer is stored as the reference layer height of the first flat layer. Thus, the reference layer height is stored for the first flat layer.

[0160] Step S24, if the target vehicle is driving on the first flat layer, the current layer height determined by the sensor data is corrected based on the reference layer height of the first flat layer to obtain the target pseudo layer height of the first flat layer.

[0161] For example, during the driving of the target vehicle on the first flat layer, the current layer height of the current position of the target vehicle can be determined based on the sensor data. On this basis, the current layer height of the current position can be corrected based on the reference layer height of the first flat layer, that is, the reference layer height of the first flat layer is taken as the output layer height of the current position. Obviously, the output layer height of each position of the first flat layer is the reference layer height of the first flat layer. In order to distinguish conveniently, the output layer height of each position of the first flat layer is recorded as the target pseudo layer height, that is, the first flat layer corresponds to the same target pseudo layer height. The accuracy of the target pseudo layer height is higher than that of the current layer height determined by the sensor data.

[0162] 4, Feature point initial positioning. For the feature point initial positioning process, the input data can be the pseudo layer height estimation result, the map data and the surround view image data, the pseudo layer height estimation result can include the target pseudo layer height of the first flat layer, and the output data can be the initial positioning pose of the target vehicle.

[0163] For example, the memory parking function does not limit the number of teaching mapping, that is, the target vehicle can download and store multiple maps, so that one or more built map data can be selected from all built map data based on the nearest valid GPS position (recorded as GPS position A) corresponding to the current position, and the distance between the GPS position of the built map data and the GPS position A is less than a preset distance threshold. Based on the selected one or more built map data, the initial positioning can be realized in the following way.

[0164] In the initial positioning of the target vehicle, if the target vehicle is running on a slope (the current position is a slope based on the slope detection result), the first map data matching the slope can be selected from all the built map data, so as to perform the initial positioning of the target vehicle based on the first map data. In the initial positioning process, the initial positioning is realized by using the visual feature point feature, and the first map data is established based on the visual feature point feature.

[0165] For example, the surround view image data of the target vehicle can be acquired, and the visual feature point feature of the current position is determined based on the surround view image data. If the first map data includes the visual feature point features of multiple track points, the initial positioning of the target vehicle is performed based on the visual feature point feature of the current position and the visual feature point features of the multiple track points, that is, the track point of the current position of the target vehicle is determined from the multiple track points.

[0166] In the initial positioning of the target vehicle, if the target vehicle is running on a first flat layer (the current position is a first flat layer based on the slope detection result), the target map data matching the target pseudo-layer height of the first flat layer (the target pseudo-layer height is determined based on the pseudo-layer height estimation result) can be selected from all the built map data, so as to perform the initial positioning of the target vehicle based on the target map data. In the initial positioning process, the initial positioning is realized by using the visual feature point feature, and the target map data is established based on the visual feature point feature.

[0167] For example, in the map construction process, each built map data corresponds to a calibrated layer height, therefore, the difference between the target pseudo-layer height of the first flat layer and the calibrated layer height of each built map data is calculated, and then the built map data with a difference less than a second threshold value is selected, so as to screen the built map data and find the built map data most matching the target pseudo-layer height as the target map data. Obviously, the difference between the calibrated layer height of the target map data and the target pseudo-layer height of the first flat layer is less than the second threshold value. In this way, in the initial positioning of the target vehicle based on the target map data, an accurate and reliable positioning result can be obtained.

[0168] For example, the surround view image data of the target vehicle can be acquired, and the visual feature point feature of the current position is determined based on the surround view image data. If the target map data includes the visual feature point features of multiple track points, the initial positioning of the target vehicle is performed based on the visual feature point feature of the current position and the visual feature point features of the multiple track points, that is, the track point of the current position of the target vehicle is determined from the multiple track points.

[0169] In summary, whether the target vehicle is on a slope or a first flat layer, the map data (first map data or target map data) can be matched with the visual feature point observation of the current position, so as to obtain the initial value of the pose of the target vehicle, and the initial positioning process of the target vehicle is not limited.

[0170] In the above process, pseudo floor height can be used for screening, even if the visual feature points in similar scenes of different floors have small differences, the pseudo floor height can be used to screen the map data of the corresponding floor, so as to distinguish the map data in the initial positioning process, avoid false matching, and successfully locate.

[0171] 5. Predictive pose calculation. After the initial positioning of the target vehicle is successful, the follow-up positioning is started. The first predictive pose of the follow-up positioning is provided by the initial positioning. The subsequent predictive pose is calculated based on the relative pose estimated by the odometry based on the last positioning pose. This process is not limited.

[0172] 6. Map selection and extraction, feature point follow-up positioning, and semantic follow-up positioning.

[0173] For example, when the target vehicle is being positioned, if the target vehicle is driving on a ramp, based on the map selection and extraction function, the first map data matching the ramp can be selected from all the built map data, so that the target vehicle is positioned based on the first map data.

[0174] Based on the first map data matching the ramp, based on the feature point follow-up positioning function, the surround view image data of the target vehicle is obtained, and the visual feature point features of the current position are determined based on the surround view image data. If the first map data includes visual feature point features of multiple track points, the target vehicle can be positioned based on the visual feature point features of the current position and the visual feature point features of the multiple track points.

[0175] For example, when the target vehicle is being positioned, if the target vehicle is driving on a first flat layer, based on the map selection and extraction function, the target map data matching the target pseudo floor height of the first flat layer can be selected from all the built map data, so that the target vehicle is positioned based on the target map data.

[0176] Based on the target map data matching the target pseudo floor height, based on the semantic follow-up positioning function, the surround view image data of the target vehicle is obtained, and the visual semantic features of the current position are determined based on the surround view image data. If the target map data includes visual semantic features of multiple track points, the target vehicle can be positioned based on the visual semantic features of the current position and the visual semantic features of the multiple track points.

[0177] For example, after the predictive pose is known, it can be queried from the map data whether a visual feature point map or a visual semantic map is built near the predictive pose, and the visual feature point follow-up positioning or the visual semantic follow-up positioning is selected accordingly, and finally a high-precision positioning pose is output. This positioning process is not limited.

[0178] From the above technical solutions, in the embodiment of the application, during the cross-layer driving of the target vehicle, the target vehicle can be positioned based on the target pseudo-floor height to accurately screen the map data, and the accurate positioning of the vehicle is automatically completed, thereby providing accurate position information for the automatic driving function of the vehicle. For positioning in different floor scenarios, the floor height of the floor where the target vehicle is located is estimated using the target pseudo-floor height, and the initial positioning pose is determined in combination with the map and visual observation, thereby ensuring accurate and effective initial positioning and effectively preventing mismatching. The driver drives the vehicle to perform teaching, and the map construction is automatically completed. After the teaching is completed, real-time positioning can be performed to provide high-precision pose for automatic driving. No high-cost sensors such as laser radars are required, the computing power requirement is not high, and the autonomous cross-layer memory parking function can be realized. The scene-based mapping and positioning scheme can solve the positioning problem caused by the lack of rich visual semantics on the ramp. In the ramp scene, visual feature points are used for mapping and positioning to fully utilize non-ground visual feature information to improve the robustness of positioning. In the flat floor scene, visual semantics are used for mapping and positioning to ensure high-precision output of positioning and further reduce the dependence on computing resources.

[0179] Based on the same application concept as the above method, the embodiment of the application proposes a vehicle positioning device, as shown in Figure 8 The device can include:

[0180] The determination module 81 is configured to, during the cross-layer driving of the target vehicle, if the target vehicle drives to the junction position of the ramp and the first flat floor, determine the initial floor height of the first flat floor based on the sensor data of the target vehicle; determine whether there is an associated flat floor of the first flat floor based on the initial floor height and the reference floor height of each flat floor stored, the difference between the reference floor height of the associated flat floor and the initial floor height being less than a first threshold value; if not, store the initial floor height as the reference floor height of the first flat floor, and if yes, store the reference floor height of the associated flat floor as the reference floor height of the first flat floor.

[0181] The correction module 82 is configured to, if the target vehicle drives on the first flat floor, correct the current floor height determined based on the sensor data based on the reference floor height of the first flat floor to obtain the target pseudo-floor height of the first flat floor.

[0182] The selection module 83 is configured to select target map data matched with the target pseudo-floor height from the built map data, the difference between the calibrated floor height of the target map data and the target pseudo-floor height being less than a second threshold value.

[0183] The positioning module 84 is configured to position the target vehicle based on the target map data.

[0184] For example, the determining module 81 is specifically configured to determine the pitch angle corresponding to the sensor data of each time point of the target vehicle when determining the intersection position of the slope and the first flat layer. If the pitch angle corresponding to the sensor data of a first time point reaches a first angle threshold, the second time point is searched backward from the first time point, and the pitch angle corresponding to the sensor data of the second time point reaches a second angle threshold. The second angle threshold is smaller than the first angle threshold. The position of the target vehicle at the second time point is determined as the intersection position of the slope and the first flat layer.

[0185] For example, the determining module 81 is specifically configured to determine the pitch angle corresponding to the sensor data of each time point of the target vehicle. If the sensor data includes longitudinal acceleration, the pitch angle corresponding to the sensor data is determined based on the longitudinal acceleration and the wheel acceleration of the target vehicle. The pitch angle corresponding to the sensor data is determined by the following formula: Or, if the sensor data includes lateral acceleration, longitudinal acceleration and skyward acceleration, the pitch angle corresponding to the sensor data is determined based on the lateral acceleration, the longitudinal acceleration, the skyward acceleration and the wheel acceleration of the target vehicle. The pitch angle corresponding to the sensor data is determined by the following formula: θ pitch represents the pitch angle, a lon represents the longitudinal acceleration, a vel represents the wheel acceleration, g represents the gravitational acceleration, a lat represents the lateral acceleration, a up represents the skyward acceleration.

[0186] For example, if the target vehicle is driving on the first flat layer, the positioning module 84 is specifically configured to determine the visual feature points of the current position based on the surround view image data of the target vehicle when positioning the target vehicle based on the target map data. The target vehicle is initially positioned based on the visual feature points of the current position and the visual feature points of the plurality of track points in the target map data. Or, when the target vehicle is positioned following, the visual semantic features of the current position are determined based on the surround view image data of the target vehicle. The target vehicle is positioned following based on the visual semantic features of the current position and the visual semantic features of the plurality of track points in the target map data.

[0187] Exemplarily, if the target vehicle is driving on a slope, the selection module 83 selects first map data matching the slope from the built map data, and the positioning module 84 is further configured to acquire surround view image data of the target vehicle when performing initial positioning or follow-up positioning on the target vehicle, determine visual feature point features of a current position based on the surround view image data, and perform initial positioning or follow-up positioning on the target vehicle based on the visual feature point features of the current position and visual feature point features of a plurality of track points in the first map data.

[0188] Exemplarily, the determination module 81 is further configured to, during the process in which the teaching vehicle is driving across layers, if the teaching vehicle drives to a junction position of a slope and a second flat layer, determine an initial layer height of the second flat layer based on sensor data of the teaching vehicle, determine whether there is an associated flat layer of the second flat layer based on the initial layer height of the second flat layer and reference layer heights of each flat layer stored, the reference layer height of the associated flat layer having a difference from the initial layer height less than a first threshold value, if not, store the initial layer height as the reference layer height of the second flat layer, and if yes, store the reference layer height of the associated flat layer as the reference layer height of the second flat layer, and the correction module 82 is further configured to, if the teaching vehicle is driving on the second flat layer, correct a current layer height determined based on sensor data based on the reference layer height of the second flat layer to obtain a calibrated layer height of the second flat layer, and the vehicle positioning apparatus further comprises an establishment module configured to establish map data of the teaching vehicle on the second flat layer, and associate the built map data of the second flat layer with the calibrated layer height based on the calibrated layer height of the second flat layer.

[0189] Exemplarily, if the teaching vehicle is driving on the second flat layer, the establishment module is specifically configured to, if a driving distance of the teaching vehicle from a preset initial position is less than a preset distance threshold value, acquire surround view image data of the teaching vehicle, determine visual feature point features of a current position based on the surround view image data, and establish map data of the teaching vehicle on the second flat layer based on the visual feature point features of the current position, the map data being used for initial positioning of a target vehicle; or, if the driving distance of the teaching vehicle from the preset initial position is not less than the preset distance threshold value, acquire surround view image data of the teaching vehicle, determine visual semantic features of a current position based on the surround view image data, and establish map data of the teaching vehicle on the second flat layer based on the visual semantic features of the current position, the map data being used for follow-up positioning of a target vehicle.

[0190] Based on the same application concept as the above method, an embodiment of the present application proposes a vehicle, comprising:

[0191] a camera configured to acquire surround view image data and transmit the surround view image data to a processor;

[0192] an IMU sensor configured to acquire acceleration information and angular velocity information and transmit the acceleration information and the angular velocity information to the processor;

[0193] a wheel speed sensor configured to acquire wheel speed and transmit the wheel speed to the processor;

[0194] the processor is configured to implement the vehicle positioning method of the above examples of the present application based on the surround view image data, the acceleration information, the angular velocity information, and the wheel speed;

[0195] In some embodiments, the vehicle comprises a vehicle carrying an intelligent driving system.

[0196] In some embodiments, the vehicle comprises a vehicle carrying an autonomous unmanned driving system.

[0197] Based on the same application concept as the above method, the embodiments of the present application provide a vehicle terminal device, as shown in Figure 9 the vehicle terminal device comprises a processor 91 and a machine readable storage medium 92, the machine readable storage medium 92 stores machine executable instructions that can be executed by the processor 91; the processor 91 is configured to execute the machine executable instructions to implement the vehicle positioning method disclosed in the above examples of the present application.

[0198] Based on the same application concept as the above method, the embodiments of the present application further provide a machine readable storage medium, the machine readable storage medium stores a plurality of computer instructions, and the computer instructions can implement the vehicle positioning method disclosed in the above examples of the present application when executed by a processor.

[0199] The machine readable storage medium can be any electronic, magnetic, optical, or other physical storage device, and can contain or store information such as executable instructions, data, etc. For example, the machine readable storage medium can be a RAM (Random Access Memory), a volatile memory, a non-volatile memory, a flash memory, a storage drive (such as a hard disk drive), a solid state disk, any type of storage disk (such as an optical disk, a DVD, etc.), or similar storage medium, or a combination thereof.

[0200] Based on the same application concept as the above method, the embodiments of the present application further provide a computer program product, which can include a computer program, and the computer program can implement the vehicle positioning method disclosed in the above examples of the present application when executed by a processor.

[0201] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code thereon for use by a computer or an instruction execution system. Program code embodied on a computer-usable storage medium can be transmitted using any

[0202] The above description is embodied only by the embodiments of the present application, and is not intended to limit the present application. Various modifications and changes can be made by those skilled in the art to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of claims of the present application.

Claims

1. A vehicle positioning method characterized by, The method comprises: During the target vehicle crossing a layer, if the target vehicle drives to a junction position of a ramp and a first flat layer, determining an initial layer height of the first flat layer based on sensor data of the target vehicle; Based on the initial layer height and the reference layer height of each flat layer stored, determining whether there is an associated flat layer of the first flat layer, the difference between the reference layer height of the associated flat layer and the initial layer height being less than a first threshold value; if not, storing the initial layer height as the reference layer height of the first flat layer, and if yes, storing the reference layer height of the associated flat layer as the reference layer height of the first flat layer; If the target vehicle drives on the first flat layer, correcting a current layer height determined based on sensor data based on the reference layer height of the first flat layer to obtain a target pseudo-layer height of the first flat layer; Selecting target map data matching the target pseudo-layer height from built map data, the difference between the target map data and the target pseudo-layer height being less than a second threshold value; Positioning the target vehicle based on the target map data.

2. The method of claim 1, wherein the process of determining that the target vehicle drives to the junction position of the ramp and the first flat layer comprises: Determining a pitch angle corresponding to sensor data of each time of the target vehicle; If the pitch angle corresponding to the sensor data of the first time reaches a first angle threshold value, searching backward from the first time to a second time, the pitch angle corresponding to the sensor data of the second time reaching a second angle threshold value; wherein the second angle threshold value is less than the first angle threshold value; Determining the position of the target vehicle at the second time as the junction position of the ramp and the first flat layer.

3. The method of claim 1, wherein the determining the pitch angle corresponding to the sensor data of each time of the target vehicle comprises: If the target vehicle drives on the first flat layer, the positioning the target vehicle based on the target map data comprises: When performing initial positioning of the target vehicle, acquiring surround view image data of the target vehicle, determining visual feature points of a current position based on the surround view image data; and performing initial positioning of the target vehicle based on the visual feature points of the current position and visual feature points of a plurality of track points in the target map data; If the sensor data comprises longitudinal acceleration, a pitch angle corresponding to the sensor data is determined based on the longitudinal acceleration and wheel acceleration of the target vehicle; wherein the pitch angle corresponding to the sensor data is determined by using the following formula: Alternatively, if the sensor data comprises lateral acceleration, longitudinal acceleration and skyward acceleration, a pitch angle corresponding to the sensor data is determined based on the lateral acceleration, the longitudinal acceleration, the skyward acceleration and wheel acceleration of the target vehicle; wherein the pitch angle corresponding to the sensor data is determined by using the following formula: θ pitch denotes the pitch angle, a lon denotes the longitudinal acceleration, a vel denotes the wheel acceleration, g denotes the gravitational acceleration, a lat denotes the lateral acceleration, a up denotes the skyward acceleration.

4. The method of claim 1, wherein, Or, when performing follow-up positioning of the target vehicle, acquiring surround view image data of the target vehicle, determining visual semantic features of a current position based on the surround view image data; and performing follow-up positioning of the target vehicle based on the visual semantic features of the current position and visual semantic features of a plurality of track points in the target map data. If the target vehicle drives on a ramp, selecting first map data matching the ramp from built map data, the method further comprising: ​ 5. The method of claim 1, wherein, ​ In the initial positioning or follow-up positioning of the target vehicle, surround view image data of the target vehicle is acquired, and visual feature point features of a current position are determined based on the surround view image data; and the target vehicle is positioned based on the visual feature point features of the current position and visual feature point features of a plurality of track points in the first map data.

6. The method according to any one of claims 1 to 5, characterized in that, Before the target map data matching the target pseudo-layer height is selected from the built map data, the method further comprises: In the process of the teaching vehicle driving across layers, if the teaching vehicle drives to a junction position of a slope and a second flat layer, an initial layer height of the second flat layer is determined based on sensor data of the teaching vehicle; Based on the initial layer height of the second flat layer and the reference layer height of each flat layer stored, it is determined whether there is an associated flat layer of the second flat layer, and the difference between the reference layer height of the associated flat layer and the initial layer height is less than a first threshold value; if not, the initial layer height is stored as the reference layer height of the second flat layer, and if yes, the reference layer height of the associated flat layer is stored as the reference layer height of the second flat layer; If the teaching vehicle drives on the second flat layer, a current layer height determined based on sensor data is corrected based on the reference layer height of the second flat layer to obtain a calibrated layer height of the second flat layer; The map data of the teaching vehicle on the second flat layer is established, and the built map data of the second flat layer is associated with the calibrated layer height based on the calibrated layer height of the second flat layer.

7. The method of claim 6, wherein, If the teaching vehicle drives on the second flat layer, the establishment of the map data of the teaching vehicle on the second flat layer comprises: If the driving distance of the teaching vehicle from a preset initial position is less than a preset distance threshold, surround view image data of the teaching vehicle is acquired, and visual feature point features of a current position are determined based on the surround view image data; and the map data of the teaching vehicle on the second flat layer is established based on the visual feature point features of the current position, and the map data is used for the initial positioning of a target vehicle; or If the driving distance of the teaching vehicle from a preset initial position is not less than a preset distance threshold, surround view image data of the teaching vehicle is acquired, and visual semantic features of a current position are determined based on the surround view image data; and the map data of the teaching vehicle on the second flat layer is established based on the visual semantic features of the current position, and the map data is used for the follow-up positioning of a target vehicle.

8. A vehicle positioning apparatus characterized by comprising: The device comprises: The determining module is configured to, in the process of the target vehicle driving across layers, if the target vehicle drives to a junction position of a slope and a first flat layer, determine an initial layer height of the first flat layer based on sensor data of the target vehicle; and determine whether there is an associated flat layer of the first flat layer based on the initial layer height and the reference layer height of each flat layer stored, and the difference between the reference layer height of the associated flat layer and the initial layer height is less than a first threshold value; if not, store the initial layer height as the reference layer height of the first flat layer, and if yes, store the reference layer height of the associated flat layer as the reference layer height of the first flat layer; The correction module is configured to correct a current height determined based on sensor data, if the target vehicle is in the first flat layer, based on a reference height of the first flat layer, to obtain a target pseudo height of the first flat layer; The selection module is configured to select target map data matched with the target pseudo height from the built map data, and a difference between a calibrated height of the target map data and the target pseudo height is less than a second threshold value; The positioning module is configured to position the target vehicle based on the target map data.

9. A vehicle terminal device, characterized by comprising: The method comprises: a processor and a machine readable storage medium, the machine readable storage medium stores machine executable instructions which can be executed by the processor; The processor is configured to execute the machine executable instructions to implement the method of any one of claims 1-7.

10. A vehicle characterized by comprising: The method comprises: a camera configured to acquire surround view image data and send the surround view image data to a processor; an IMU sensor configured to acquire acceleration information and angular velocity information and send the acceleration information and the angular velocity information to the processor; a wheel speed sensor configured to acquire wheel speed and send the wheel speed to the processor; The processor is configured to implement the method of any one of claims 1-7 based on the surround view image data, the acceleration information, the angular velocity information and the wheel speed; The vehicle comprises a vehicle carrying an intelligent driving system; or The vehicle comprises a vehicle carrying an autonomous unmanned driving system.

Images