A method and device for positioning wheel slots in a battery swap station and an autonomous driving vehicle

Abstract

The present application provides a method, device and autonomous driving vehicle for positioning a wheel well at a battery swap station, comprising: obtaining a panoramic image of a target vehicle in a battery swap station scene; if the panoramic image contains a target pixel point of the battery swap station wheel well category, determining the wheel well point cloud of the target pixel point in the vehicle body coordinate system based on the pixel coordinates of the target pixel point in the panoramic image; determining the target wheel well length and target wheel well width based on the wheel well point cloud; if the difference between the target wheel well length and the preset wheel well length is less than a length threshold, and the difference between the target wheel well width and the preset wheel well width is less than a width threshold, converting the wheel well point cloud into a target wheel well map point in a map coordinate system, determining the target posture corresponding to the target vehicle based on the target wheel well map point and the calibrated wheel well map point; if the difference between the target posture and the calibrated posture is less than the posture threshold, determining that the target vehicle is located in the wheel well of the battery swap station. Through the technical solution of the present application, it is possible to accurately locate whether the vehicle is located in the wheel well of the battery swap station.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a method and device for positioning a wheel well in a battery swap station, and an autonomous driving vehicle. Background Art

[0002] A battery swap station is a facility that centrally stores, charges, and distributes a large number of batteries through a centralized charging station, and provides battery swap services for electric vehicles. Alternatively, a battery swap station is a facility that integrates battery charging, logistics coordination, and battery swap services. Through a battery swap station, electric vehicles can meet their driving needs by simply swapping batteries without having to recharge. A battery swap station is a facility that separates electric vehicles from their batteries for recharging.

[0003] To perform a battery swap at a battery swap station, the electric vehicle must be driven to a designated location. For example, the station may have a wheel bay (called a "battery bay") where the driver must drive the electric vehicle to the bay, park it accurately, and then swap the battery.

[0004] However, for self-driving electric vehicles, there is no suitable solution for how to drive the electric vehicle to the wheel slot of the battery swap station and how to detect whether the electric vehicle is in the wheel slot of the battery swap station. Summary of the Invention

[0005] The present application provides a method for positioning a wheel well in a battery swap station, the method comprising:

[0006] Obtain a panoramic image of the target vehicle in the battery swap station scene;

[0007] If the panoramic image contains a target pixel of the wheel well category of the battery swap station, then based on the pixel coordinates of the target pixel in the panoramic image, determine the wheel well point cloud of the target pixel in the vehicle body coordinate system, where the vehicle body coordinate system takes the center of the rear wheel of the target vehicle as the coordinate origin;

[0008] Determining a target wheel groove length and a target wheel groove width based on the wheel groove point cloud;

[0009] If the difference between the target wheel well length and the preset wheel well length is less than a length threshold, and the difference between the target wheel well width and the preset wheel well width is less than a width threshold, converting the wheel well point cloud into a target wheel well map point in a map coordinate system, and determining a target pose corresponding to the target vehicle based on the target wheel well map point and the calibrated wheel well map point in the map coordinate system;

[0010] If the difference between the target posture and the calibrated posture is less than the posture threshold, it is determined that the target vehicle is located in the wheel well of the battery swap station; wherein the calibrated posture is the posture of the calibrated vehicle when it is located in the wheel well of the battery swap station.

[0011] The present application provides a positioning device for a wheel groove of a battery swap station, the device comprising:

[0012] An acquisition module is used to acquire a panoramic image of the target vehicle in the battery swap station scene;

[0013] a determination module configured to, if a target pixel point of the wheel well category of the battery swap station exists in the panoramic image, determine a wheel well point cloud of the target pixel point in a vehicle body coordinate system based on the pixel coordinates of the target pixel point in the panoramic image, wherein the vehicle body coordinate system has the center of the rear wheel of the target vehicle as the coordinate origin; and determine a target wheel well length and a target wheel well width based on the wheel well point cloud;

[0014] a processing module, configured to, if a difference between the target wheel well length and the preset wheel well length is less than a length threshold, and a difference between the target wheel well width and the preset wheel well width is less than a width threshold, convert the wheel well point cloud into a target wheel well map point in a map coordinate system, and determine a target pose corresponding to the target vehicle based on the target wheel well map point and the calibrated wheel well map points in the map coordinate system;

[0015] A positioning module is used to determine that the target vehicle is located in the wheel well of the battery swap station if the difference between the target posture and the calibrated posture is less than a posture threshold; wherein the calibrated posture is the posture of the calibrated vehicle when it is located in the wheel well of the battery swap station.

[0016] The present application provides an autonomous driving vehicle, comprising: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; wherein the processor is used to execute the machine-executable instructions to implement the above-mentioned method for positioning the wheel slot of a battery swap station.

[0017] As can be seen from the above technical solutions, in the embodiment of the present application, a vehicle self-positioning method based on the prior size of the wheel slot of the battery swap station is proposed for self-driving electric vehicles. Combined with visual semantic positioning, it can achieve a very high positioning accuracy in the battery swap station, so that the vehicle can be driven to the wheel slot of the battery swap station, and it can detect whether the vehicle is located in the wheel slot of the battery swap station. Based on the actual size of the wheel slot of the battery swap station, by continuously comparing the difference between the measured size and the actual size of the wheel slot of the battery swap station, it is accurately determined whether the vehicle is located on the battery swap platform. If it is already located on the battery swap platform, the accurate vehicle positioning posture is obtained by matching the wheel slots. By comparing the difference between the positioning posture and the marked posture, it can be determined whether the vehicle is accurately in place, with very high positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments of the present application or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings of the embodiments of the present application.

[0019] Figure 1 This is a flow chart of a method for positioning a wheel well in a battery swap station in one embodiment of the present application;

[0020] Figure 2 It is a structural diagram of an automatic battery replacement system in one embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a sensor configuration of an autonomous driving vehicle in one embodiment of the present application;

[0022] Figure 4 is a block diagram of an autonomous driving system in one embodiment of the present application;

[0023] Figure 5 This is a flow chart of a method for positioning a wheel well in a battery swap station in one embodiment of the present application;

[0024] Figure 6A is a schematic diagram of the positional relationship of three coordinate systems in one embodiment of the present application;

[0025] Figure 6B is a schematic diagram of a visualization result of a map in one embodiment of the present application;

[0026] Figure 7 This is a flow chart of a method for positioning a wheel well in a battery swap station in one embodiment of the present application;

[0027] Figure 8 This is a structural schematic diagram of a positioning device for a wheel slot in a battery swap station in one embodiment of the present application;

[0028] Figure 9 This is a hardware structure diagram of an autonomous driving vehicle in one embodiment of the present application. DETAILED DESCRIPTION

[0029] The terms used in the embodiments of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms "a," "the," and "the" used in this application and claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to any or all possible combinations of one or more associated listed items.

[0030] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" used may also be interpreted as "at the time of" or "when" or "in response to determining".

[0031] In the embodiment of the present application, a method for locating a wheel well in a battery swap station is proposed. The method can be applied to an autonomous driving system. The autonomous driving system is deployed on a target vehicle. The target vehicle can be any autonomous driving vehicle that needs to have its battery replaced. Figure 1 FIG. 5 is a flow chart of the method, which includes:

[0032] Step 101: Acquire a panoramic image of a target vehicle in a battery swap station scenario.

[0033] Step 102: If the panoramic image contains a target pixel of the wheel well category at the battery swap station, determine the wheel well point cloud of the target pixel in the vehicle coordinate system based on the pixel coordinates of the target pixel in the panoramic image. The vehicle coordinate system may have the center of the rear wheel of the target vehicle as the coordinate origin.

[0034] Step 103: Determine a target wheel groove length and a target wheel groove width based on the wheel groove point cloud.

[0035] Step 104: If the difference between the target wheel well length and the preset wheel well length is less than the length threshold, and the difference between the target wheel well width and the preset wheel well width is less than the width threshold, the wheel well point cloud is converted into a target wheel well map point in the map coordinate system, and the target posture corresponding to the target vehicle is determined based on the target wheel well map point and the calibrated wheel well map point in the map coordinate system.

[0036] Step 105: If the difference between the target posture and the calibrated posture is less than the posture threshold, it is determined that the target vehicle is located in the wheel well of the battery swap station; wherein the calibrated posture is the posture of the calibrated vehicle when it is located in the wheel well of the battery swap station.

[0037] Exemplarily, after obtaining a panoramic image of the target vehicle in a battery swap station scenario, semantic segmentation can be performed on each pixel in the panoramic image to obtain a semantic segmentation map corresponding to the panoramic image. The resolution of the semantic segmentation map is the same as that of the panoramic image, and the semantic segmentation map includes the category corresponding to each pixel in the panoramic image. Based on the semantic segmentation map, it is determined whether the panoramic image contains target pixels with the battery swap station wheel well category; if at least M pixels in the semantic segmentation map correspond to the battery swap station wheel well category, where M is a positive integer, the panoramic image contains target pixels with the battery swap station wheel well category.

[0038] Exemplarily, based on the pixel coordinates of the target pixel point in the panoramic image, determining the wheel well point cloud of the target pixel point in the vehicle body coordinate system may include but is not limited to: based on the pixel coordinates of the target pixel point in the panoramic image, the length and height of the panoramic image, and the mapping relationship between the calibrated pixels and physical dimensions, the pixel coordinates can be converted into reference coordinates in the central coordinate system, and the central coordinate system can have the center of the target vehicle as the coordinate origin. Based on the reference coordinates and the external parameter relationship from the calibrated central coordinate system to the vehicle body coordinate system, the reference coordinates can be converted into the three-dimensional coordinates corresponding to the target pixel point in the vehicle body coordinate system. The wheel well point cloud is determined based on the three-dimensional coordinates corresponding to each target pixel point in the vehicle body coordinate system, wherein the wheel well point cloud may include the three-dimensional coordinates corresponding to each target pixel point in the vehicle body coordinate system.

[0039] Exemplarily, determining the target well length and target well width based on the well point cloud may include, but is not limited to: if the well point cloud includes multiple three-dimensional coordinates, obtaining the maximum x-direction value, minimum x-direction value, maximum y-direction value, and minimum y-direction value of the multiple three-dimensional coordinates. Based on this, the target well length may be determined based on the maximum x-direction value and the minimum x-direction value; and the target well width may be determined based on the maximum y-direction value and the minimum y-direction value.

[0040] Exemplarily, converting the wheel well point cloud into target wheel well map points in a map coordinate system and determining a target pose corresponding to the target vehicle based on the target wheel well map points and calibrated wheel well map points in the map coordinate system may include, but is not limited to, determining a ground landmark outline point cloud for a reference pixel of a ground landmark category in the panoramic image in the vehicle body coordinate system based on the pixel coordinates of the reference pixel in the panoramic image. Based on the acquired initial pose, converting the ground landmark outline point cloud into target ground landmark map points in the map coordinate system, performing a nearest neighbor search on the target ground landmark map points and the calibrated ground landmark map points in the map coordinate system to obtain a matching relationship between point pairs. Based on the matching relationship between the point pairs, a matching algorithm may be used to determine a reference pose corresponding to the target vehicle. Based on the reference pose, converting the wheel well point cloud into target wheel well map points in the map coordinate system. Performing a nearest neighbor search on the target wheel well map points and the calibrated wheel well map points to obtain a matching relationship between point pairs; based on the matching relationship between the point pairs, a matching algorithm may be used to determine the target pose corresponding to the target vehicle.

[0041] Exemplarily, the difference between the target posture and the marked posture is less than the posture threshold, which may include: the distance between the target posture and the marked posture is less than a preset distance threshold, and the angle difference between the target posture and the marked posture is less than a preset angle threshold. After determining that the target vehicle is located in the wheel well of the battery swap station, a query message can be sent to the battery swap station to inquire whether it is located in the wheel well of the battery swap station; if a successful response is received from the battery swap station, the successful response is used to confirm that the target vehicle is located in the wheel well of the battery swap station, then the automatic driving mode can be exited, and the battery swap station will perform automatic battery swapping on the target vehicle, that is, automatically replace the battery.

[0042] Exemplary methods for obtaining calibrated wheel well map points in a map coordinate system may include, but are not limited to: obtaining a sample image of a sample vehicle in a battery swap station scenario; if the sample image contains a sample pixel point of the wheel well category at the battery swap station, determining a wheel well point cloud of the sample pixel point in the vehicle body coordinate system based on the pixel coordinates of the sample pixel point in the sample image, and determining a sample wheel well length and a sample wheel well width based on the wheel well point cloud. If the difference between the sample wheel well length and a preset wheel well length is less than a length threshold, and the difference between the sample wheel well width and a preset wheel well width is less than a width threshold, converting the wheel well point cloud into a calibrated wheel well map point in the map coordinate system, and storing the calibrated wheel well map point.

[0043] For example, after the sample vehicle arrives at the wheel well of the battery swap station, the corresponding marked posture of the sample vehicle when it is in the wheel well of the battery swap station can also be obtained and stored.

[0044] As can be seen from the above technical solutions, in the embodiment of the present application, a vehicle self-positioning method based on the prior size of the wheel slot of the battery swap station is proposed for self-driving electric vehicles. Combined with visual semantic positioning, it can achieve a very high positioning accuracy in the battery swap station, so that the vehicle can be driven to the wheel slot of the battery swap station, and it can detect whether the vehicle is located in the wheel slot of the battery swap station. Based on the actual size of the wheel slot of the battery swap station, by continuously comparing the difference between the measured size and the actual size of the wheel slot of the battery swap station, it is accurately determined whether the vehicle is located on the battery swap platform. If it is already located on the battery swap platform, the accurate vehicle positioning posture is obtained by matching the wheel slots. By comparing the difference between the positioning posture and the marked posture, it can be determined whether the vehicle is accurately in place, with very high positioning accuracy.

[0045] The above technical solutions of the embodiments of the present application are described below in conjunction with specific application scenarios.

[0046] To perform a battery swap at a battery swap station, the electric vehicle must be driven to a designated location. For example, the station may have a wheel bay (called a "battery bay") where the driver must drive the electric vehicle to the bay, park it accurately, and then swap the battery.

[0047] However, for self-driving electric vehicles, there is no suitable solution for how to drive the electric vehicle to the wheel slot of the battery swap station and how to detect whether the electric vehicle is located in the wheel slot of the battery swap station. For example, there is a metal roof blocking the battery swap station and lacks GPS (Global Positioning System) signals, so GNSS (Global Navigation Satellite System) cannot be used to obtain accurate vehicle positioning. The building materials in the battery swap station are highly repetitive and have weak textures, and it is difficult to obtain accurate positioning inside the battery swap station based on positioning based on visual feature points. Due to the above reasons, it is impossible to drive the electric vehicle to the wheel slot of the battery swap station and it is impossible to accurately detect whether the electric vehicle is located in the wheel slot of the battery swap station.

[0048] In response to the above findings, an embodiment of the present application proposes a vehicle self-positioning method based on the prior size of the wheel well of the battery swap station. Combined with visual semantic positioning, it can achieve very high positioning accuracy within the battery swap station, and achieve a positioning accuracy of less than 0.1 meters in the horizontal and vertical directions within the battery swap station, meeting the needs of determining whether the vehicle has accurately reached the wheel well, so that the vehicle can be driven to the wheel well of the battery swap station, and can detect whether the vehicle is located in the wheel well of the battery swap station.

[0049] See also Figure 2The figure shows a schematic diagram of the structure of an automatic battery swapping system. The automatic battery swapping system may include a battery swapping station, a cloud platform server, a map server, and an autonomous vehicle. The battery swapping station may include a battery swapping platform, guide rails, a battery pack management platform, and a battery transport cart. A battery swapping station wheel well is located on the battery swapping platform. The wheel well may include a V-shaped front wheel well and a flat rear wheel well. The V-shaped front wheel well is used to longitudinally clamp the front wheels of the vehicle, thereby positioning the vehicle. A laser sensor is placed at the bottom of the V-shaped front wheel well, with one side emitting laser light and the other side receiving laser light. If the receiving sensor fails to receive the laser signal, it indicates that the vehicle has arrived and the battery swapping service can be initiated.

[0050] An autonomous driving vehicle is a vehicle with an autonomous driving function (such as a vehicle that needs to replace the battery, such as an electric vehicle, etc.) and the autonomous driving vehicle has an AVBS (Automatic Valet Battery Service) function. In this embodiment, the autonomous driving vehicle can be referred to as a target vehicle.

[0051] When an autonomous vehicle enters a map range area near a battery swap station, the autonomous vehicle may request the map server to download or update a map of the area where the battery swap station is located (a map of the battery swap station scene). Alternatively, the autonomous vehicle may check whether a map of the area where the battery swap station is located is already stored inside the vehicle. If not, the autonomous vehicle may request the map server to download a map of the area where the battery swap station is located.

[0052] After obtaining a map of the area where the battery swap station is located, the autonomous vehicle uses its body sensors to perform initial positioning on the map. If the initial positioning is successful and the path planning is successful, autonomous driving is activated, controlling the autonomous vehicle to automatically drive to the battery swap platform at the battery swap station and subsequently automatically complete the battery swap.

[0053] When an autonomous vehicle enters a battery swap station, it uses a positioning algorithm to determine its position and posture in real time. When the distance between its current position and the wheel well position in the map is less than a threshold, it is confirmed that the autonomous vehicle has reached its destination. At this point, the autonomous vehicle can request an automatic battery swap from the battery swap station through the cloud platform server. After verifying that the vehicle model, frame number, and vehicle position status meet the requirements (the wheels are located in the wheel wells at the battery swap station), the battery swap station will begin responding to the battery swap request, executing the battery swap action, and completing the battery swap.

[0054] The map server stores maps of all mapped areas where battery swap stations are located. These maps include the location of the battery swap stations and other map elements of the parking lots / charging stations where they are located, such as berths, lane markings, arrows, speed bumps, curbs, wheel wells, and more. Furthermore, the map may include lane-level road network topology, which illustrates the connections between lanes. This map can be autonomously constructed by AVBS-enabled vehicles or constructed, stored, and maintained by high-precision map vendors.

[0055] Autonomous driving vehicles need to be equipped with sensors that can support autonomous driving functions. These sensors may include but are not limited to at least one of the following: cameras (such as surround-view cameras, front-view cameras, etc.), ultrasonic radars, GPS, IMUs (Inertial measurement units), and wheel speed meters.

[0056] See also Figure 3 The figure shows a schematic diagram of the sensor configuration. This is just an example of the sensor location. There is no restriction on the sensor location and it can be arranged according to the sensor model and vehicle structure.

[0057] See also Figure 3 As shown, an autonomous vehicle may include sensors such as a forward-looking camera, a surround-view camera, an ultrasonic radar, a GPS, and an IMU, which can collect information around the autonomous vehicle.

[0058] See also Figure 3 As shown, an autonomous driving vehicle may include an HMI (Human Machine Interface), which is a medium for interaction and information exchange between the system and the user.

[0059] See also Figure 3 As shown, the autonomous vehicle may include a communication module, through which the autonomous vehicle communicates with the battery swap station and the cloud platform server (referred to as the cloud server). For example, the communication module communicates with the battery swap station via near-field communication, and the communication module communicates with the cloud server via 4G, 5G, Wi-Fi, etc.

[0060] See also Figure 3 As shown, an autonomous vehicle may include an autonomous driving computing unit, also known as an autonomous driving system. The autonomous driving computing unit may include a vehicle-side mapping module, a vehicle-side perception module, a control module, a vehicle-side positioning module, a decision-making and planning module, and an application subsystem.

[0061] See also Figure 4As shown in Figure 1, it is a block diagram of an autonomous driving system. The autonomous driving system may include an algorithm subsystem and an application subsystem. Among them, the algorithm subsystem may include a mapping and positioning module (i.e., a vehicle-side mapping module and a vehicle-side positioning module). The mapping and positioning module may obtain sensor data (such as sensor data collected by GPS, IMU, wheel speed meter, surround view camera, etc.), and implement mapping and positioning functions based on sensor data and maps. The algorithm subsystem may include a vehicle-side perception module. The vehicle-side perception module may obtain sensor data (such as sensor data collected by surround view camera, ultrasonic radar, etc.), and implement perception functions based on sensor data. The algorithm subsystem may include a decision planning module. The decision planning module implements decision planning functions based on maps. The algorithm subsystem may include a control module. The control module implements control of the vehicle chassis.

[0062] The application subsystem can include UI / UE, communication modules, etc., and there is no restriction on the function of this application subsystem. In addition, the autonomous driving system can also include the vehicle chassis, such as ECU, actuators, etc.

[0063] In the embodiments of the present application, a map construction method and a map-based method for positioning wheel slots at a battery swap station may be involved. With respect to the map construction method, it can be applied to a sample vehicle (the autonomous driving vehicle used in the map construction process is referred to as a sample vehicle), such as a mapping and positioning module applied to the sample vehicle. With respect to the map-based method for positioning wheel slots at a battery swap station, it can be applied to a target vehicle (the autonomous driving vehicle used in the positioning process of wheel slots at a battery swap station is referred to as a target vehicle), such as a mapping and positioning module applied to the target vehicle.

[0064] See also Figure 5 FIG. 1 is a flow chart of a map construction method, which may include:

[0065] Step 501: Trigger and start the vehicle-side self-map building process.

[0066] For example, after entering the battery swap station scene, the user can select the starting point of the sample vehicle and trigger the vehicle-side self-map building process through the vehicle HMI or voice interaction.

[0067] Step 502: Obtain a sample image of a sample vehicle in a battery swap station scenario.

[0068] For example, a user drives a sample vehicle from a starting point to a battery swap platform at a battery swap station. During the driving process, sample images of the sample vehicle in the battery swap station scene can be periodically collected. For example, based on the sensor data collected by multiple surround-view cameras, the sensor data collected by multiple surround-view cameras can be spliced ​​together to form a sample image through methods such as IPM (Inverse Perspective Mapping).

[0069] The sample image can be a 360-degree panoramic image, also known as a bird's-eye view. The physical scale represented by each pixel in the sample image is fixed. For example, the field of view of the sample image is related to the field of view of the surround-view camera surrounding the sample vehicle, such as a field of view of 12 meters by 12 meters.

[0070] Step 503: Perform semantic segmentation on each pixel in the sample image to obtain a semantic segmentation map (also called a semantic bird's-eye view map) corresponding to the sample image. The resolution of the semantic segmentation map is the same as that of the sample image, and the semantic segmentation map includes the category corresponding to each pixel in the sample image.

[0071] For example, the sample image can be input into a semantic segmentation network (there is no restriction on the structure and training process of the semantic segmentation network, as long as the semantic segmentation function can be realized), and the semantic segmentation network performs semantic segmentation on each pixel in the sample image to obtain a semantic segmentation map corresponding to the sample image.

[0072] After obtaining the semantic segmentation map, the semantic segmentation map includes the category corresponding to each pixel in the sample image, and the category may include but is not limited to at least one of the following: berth, lane line, arrow, speed bump, zebra crossing, wall column, curb, wheel groove, etc. Of course, the above are just a few examples and are not limited to this. Among them, if the category corresponding to the pixel point is berth, it means that the physical position of the pixel point is in the berth; if the category corresponding to the pixel point is lane line, it means that the physical position of the pixel point is in the lane line; if the category corresponding to the pixel point is wheel groove, it means that the physical position of the pixel point is in the wheel groove.

[0073] Step 504: Obtain the calibrated ground marker map points and store the ground marker map points.

[0074] For example, based on sensor data (such as sensor data collected by surround view cameras, IMU, GPS, ultrasonic radar, etc.), SLAM (Simultaneous Localization and Mapping) technology can be used to obtain the mapping pose of the sample vehicle at each moment. Construction pose Indicates the pose of the sample vehicle in the map coordinate system, and the pose is mapped to it There is no restriction on how to obtain it.

[0075] The posture in this embodiment refers to the position and posture of the sample vehicle in a certain coordinate system. The posture is a 3D posture, including coordinates x, y, z, and attitude angles roll, pitch, and yaw.

[0076] For example, ground markings may include but are not limited to parking spaces, lane lines, arrows, speed bumps, zebra crossings, etc., and there is no restriction on the type of ground markings. For example, taking the example of a lane line as the ground marking, the lane line map point can be obtained, that is, the lane line map point can be calibrated. The lane line map point represents the three-dimensional point cloud of the lane line in the map coordinate system, and the three-dimensional point cloud can be maintained by multi-frame accumulation. For example, in order to obtain the lane line map point, the following method can be used:

[0077] See also Figure 6A As shown, this embodiment involves the following three coordinate systems, namely the image coordinate system I (also called the pixel coordinate system) of the sample image, the center coordinate system F, and the vehicle coordinate system B. Among them, the coordinate origin of the image coordinate system I is located at the upper left corner of the field of view boundary, with the positive direction of the x-axis facing right and the positive direction of the y-axis facing downward. The coordinate origin of the center coordinate system F is located at the center of the field of view boundary, with the positive direction of the x-axis facing right, the positive direction of the y-axis facing forward, and the z-axis being orthogonal to the x-axis and the y-axis to form a right-handed system. The coordinate origin of the center coordinate system F is the center of the sample vehicle. The coordinate origin of the vehicle coordinate system B is located at the center of the rear wheels of the sample vehicle (i.e., the center point of the two rear wheels), with the positive direction of the x-axis facing forward, the positive direction of the y-axis facing left, and the z-axis being orthogonal to the x-axis and the y-axis to form a right-handed system.

[0078] Based on the semantic segmentation map corresponding to the sample image, the category corresponding to each pixel in the sample image is determined. For the pixel whose category is the ground sign category (such as the lane line category), the pixel coordinates of the pixel in the image coordinate system I are The pixel coordinates can be converted to Convert to the reference coordinates under the central coordinate system F The reference coordinates can be converted to Converted to three-dimensional coordinates in vehicle coordinate system B Three-dimensional coordinates The collection is a three-dimensional point cloud of multiple pixel points in the map coordinate system, and this three-dimensional point cloud is recorded as a ground landmark map point, which needs to be stored.

[0079]

[0080]

[0081] In formula (1), w represents the width of the sample image, h represents the height of the sample image, and α represents the mapping relationship between pixels and physical size, that is, the physical scale represented by each pixel of the sample image. The value of α is configured based on experience and is not restricted. Obviously, based on the width of the sample image, the height of the sample image, and the mapping relationship α between pixels and physical size, the pixel coordinates can be Convert to reference coordinates

[0082] In formula (2), It represents the external parameter relationship from the center coordinate system F to the vehicle body coordinate system B. This external parameter relationship is pre-calibrated, and there is no restriction on the calibration method of this external parameter relationship. Represents the mapping pose of the sample vehicle. Assuming that the sample image is the sample image at time A, then is the mapping pose of the sample vehicle at time A. Obviously, based on the external parameter relationship from the center coordinate system to the vehicle body coordinate system Mapping pose of sample vehicles The reference coordinates Converted to three-dimensional coordinates in vehicle coordinate system B Among them, multiple three-dimensional coordinates under the vehicle coordinate system B It can be a ground marker map point in the map coordinate system.

[0083] Step 505: Based on the semantic segmentation map corresponding to the sample image, determine whether the sample image contains sample pixels of the wheel well category at a battery swap station. If not, return to step 502 and wait for the next cycle to continue acquiring sample images of the sample vehicle in the battery swap station scenario. If yes, proceed to step 506.

[0084] Exemplarily, if the category corresponding to at least M pixels in the semantic segmentation map is the battery swap station wheel slot category, and M is a positive integer, then the sample image has sample pixels of the battery swap station wheel slot category.

[0085] For example, assuming that M is 1, when there is at least one pixel in the semantic segmentation map whose corresponding category is the battery swap station wheel slot category, it means that the sample image has a sample pixel of the battery swap station wheel slot category.

[0086] For another example, assuming that M is 10, when there are at least 10 pixels in the semantic segmentation map whose corresponding categories are the battery swap station wheel slot categories, it means that the sample image has sample pixels of the battery swap station wheel slot category.

[0087] Step 506: If the sample image contains sample pixel points (such as multiple sample pixel points) of the battery swap station wheel groove category, the wheel groove point cloud of the sample pixel point in the vehicle body coordinate system is determined based on the pixel coordinates of the sample pixel point in the sample image, and the sample wheel groove length and sample wheel groove width are determined based on the wheel groove point cloud.

[0088] For each sample pixel, the pixel coordinates can be converted to reference coordinates in the central coordinate system based on the pixel coordinates of the sample pixel in the sample image, the length and height of the sample image, and the mapping relationship between pixels and physical dimensions. Based on the reference coordinates and the external parameter relationship between the central coordinate system and the vehicle body coordinate system, the reference coordinates can be converted to the three-dimensional coordinates of the sample pixel in the vehicle body coordinate system.

[0089] For example, the pixel coordinates of the sample pixel in the sample image are marked as The pixel coordinates can be converted to Convert to reference coordinates in the central coordinate system The reference coordinates can be converted to Converted to three-dimensional coordinates in the vehicle coordinate system

[0090] After obtaining the three-dimensional coordinates of each sample pixel point in the vehicle body coordinate system, the wheel well point cloud is determined based on the three-dimensional coordinates of these sample pixel points, and the wheel well point cloud includes the three-dimensional coordinates of multiple sample pixel points in the vehicle body coordinate system. The sample wheel well length and sample wheel well width can be determined based on the wheel well point cloud. For example, the wheel well point cloud includes the three-dimensional coordinates of multiple sample pixel points in the vehicle body coordinate system, and the maximum value x in the x direction among the multiple three-dimensional coordinates is obtained. max , minimum value x in the x direction min , maximum value y in the y direction max and the minimum value y in the y direction min Based on the maximum value x in the x direction max and the minimum value x in the x direction min Determine the sample wheel groove length L, L = x max -x min The sample wheel groove width W is determined based on the maximum value and minimum value in the y direction, W=y max -y min .

[0091] Step 507: Determine whether the difference between the sample wheel groove length and the preset wheel groove length is less than a length threshold, and determine whether the difference between the sample wheel groove width and the preset wheel groove width is less than a width threshold.

[0092] If yes, execute step 508; if not, clear the wheel groove point cloud, return to step 502, wait for the next cycle, and continue to obtain sample images of the sample vehicle in the battery swap station scene.

[0093] Exemplarily, the physical dimensions of the wheel groove of the battery swap station are known a priori, that is, the preset wheel groove length and preset wheel groove width of the wheel groove of the battery swap station are pre-configured, and the preset wheel groove length and preset wheel groove width are both known values.

[0094] A first difference between the sample wheel well length and the preset wheel well length can be calculated, and a second difference between the sample wheel well width and the preset wheel well width can be calculated. If the absolute value of the first difference is less than a length threshold (which can be configured based on experience), and the absolute value of the second difference is less than a width threshold (which can be configured based on experience), then the physical size priors for the wheel well at the battery swap station are met, and step 508 can be executed. If the absolute value of the first difference is not less than the length threshold, and / or the absolute value of the second difference is not less than the width threshold, then the physical size priors for the wheel well at the battery swap station are not met. For example, the sample vehicle is located on a non-planar surface, and the ranging error is large, i.e., the sample wheel well length and sample wheel well width have large errors. In this case, the wheel well point cloud is cleared, and steps 502-506 are repeated.

[0095] Step 508: Convert the wheel well point cloud of the sample pixel points in the vehicle coordinate system into calibrated wheel well map points in the map coordinate system, and store the calibrated wheel well map points. The calibrated wheel well map points represent the three-dimensional coordinates of the wheel well at the battery swap station in the map coordinate system, and these three-dimensional coordinates are labeled as wheel well map points.

[0096] For example, if the difference between the sample wheel groove length and the preset wheel groove length is less than the length threshold, and the difference between the sample wheel groove width and the preset wheel groove width is less than the width threshold, it means that the sample vehicle has arrived at the battery swap station platform. At this time, the wheel groove point cloud can be converted into wheel groove map points under the map coordinate system, that is, the wheel groove point cloud in the vehicle body coordinate system is converted into wheel groove map points under the map coordinate system. There is no restriction on this conversion process.

[0097] After obtaining the wheel groove map point in the map coordinate system, the wheel groove map point can be marked for the wheel groove of the battery swap station, that is, the wheel groove map point is stored, and thus the wheel groove map point is recorded as a calibrated wheel groove map point.

[0098] Step 509: After the sample vehicle arrives at the wheel slot of the battery swap station, obtain the corresponding marked pose (i.e., mapping pose) of the sample vehicle when it is in the wheel slot of the battery swap station, and store this marked pose.

[0099] For example, when a user drives a sample vehicle to a wheel slot at a battery swap station and meets the required battery swap requirements, the user can use various methods, such as the vehicle's HMI or voice interaction, to mark the sample vehicle's arrival at the wheel slot at the battery swap station. Based on this, the corresponding mapping pose of the sample vehicle at the wheel slot at the battery swap station can be obtained. This mapping pose represents the pose of the sample vehicle in the map coordinate system, and there are no restrictions on how this mapping pose is obtained. After obtaining this mapping pose, it is stored as the marked pose.

[0100] For example, a map of the battery swap station scene can also be generated based on sensor data during the sample vehicle driving process (from the starting point to the wheel slot of the battery swap station), and there is no restriction on this map construction process.

[0101] To sum up, during the map construction process, calibrated ground marker map points, calibrated wheel groove map points and calibrated postures can be stored. The calibrated ground marker map points, calibrated wheel groove map points and calibrated postures can be stored in the map or in other spaces, and there is no restriction on this.

[0102] For example, the visualization results of the map can be seen in Figure 6B As shown, the map can include berths, arrows, lane lines, speed bumps, curbs, columns, and wheel wells (such as front and rear wheel wells) near the mapping trajectory. Each element can be described using the element category and its precise coordinates in the map coordinate system. The geometric shapes of map elements are represented by vector elements such as polygons and polylines, which can be used for subsequent positioning.

[0103] The above process is the step of building a map on the vehicle side. It can be understood that the battery swap station scene can be an indoor environment or an outdoor environment. If the battery swap station scene is an outdoor environment, the GPS signal can be received during the mapping process. Therefore, the map coordinate system can be bound to the GPS coordinate system. When you come to the battery swap station next time, the map can be pushed according to the GPS coordinates in the map and the GPS coordinates of the current vehicle.

[0104] See also Figure 7 FIG. 1 is a flow chart of a wheel slot positioning method for a battery swap station, the method comprising:

[0105] Step 701: trigger the target vehicle to start the fully automatic valet battery replacement service function.

[0106] For example, the target vehicle can continuously check whether there is a mapped battery swap station near the GPS location (such as 100 meters). When it is found that the battery swap station map exists for a period of time (such as 5 seconds), the user can be reminded that the AVBS function can be used through the vehicle HMI or voice interaction.

[0107] The user can enter the AVBS main interface through HMI buttons or voice interaction. If the target vehicle does not have a map yet, the map server will be requested to list maps near the target vehicle, and the user can decide whether to download it. Alternatively, if the target vehicle already has a map, the map server will be requested to confirm whether there is an update for the current map, and the user can decide whether to update the map. After the target vehicle has the latest map, the target vehicle is triggered to start the fully automatic valet battery swap service (i.e. AVBS) function.

[0108] For example, if the map server does not have a map, the user can be prompted through HMI or voice interaction to ask whether to build a map for the current area. If the user confirms to build a map for the car, the Figure 5The process shown completes the map building process, and the map is stored by the map server.

[0109] Step 702: Initially locate the target vehicle to obtain an initial position and posture of the target vehicle. The initial position and posture represents the position and posture of the target vehicle in the map coordinate system. The position and posture may include position and posture.

[0110] For example, if the GPS signal converges and the positioning accuracy is high (e.g., positioning accuracy is about 0.1 meters), the target vehicle can be initially positioned based on the GPS signal to obtain the initial position and posture of the target vehicle. Alternatively, if the GPS signal is poor and the positioning accuracy is low, the target vehicle can be initially positioned based on the initial positioning algorithm to obtain the initial position and posture of the target vehicle. There is no limitation on this initial positioning algorithm.

[0111] Step 703: After the initial positioning is successful, a panoramic image of the target vehicle in the battery swap station scene is obtained.

[0112] For example, while the target vehicle is driving, panoramic images of the target vehicle in the battery swap station scenario can be periodically collected. For example, based on the sensor data collected by multiple surround-view cameras, the sensor data collected by multiple surround-view cameras can be stitched together through methods such as IPM to form a panoramic image. This panoramic image can also be called a bird's-eye view, and the physical scale represented by each pixel in the panoramic image is fixed.

[0113] Step 704: Perform semantic segmentation on each pixel in the panoramic image to obtain a semantic segmentation map corresponding to the panoramic image (also called a semantic bird's-eye view map). The resolution of the semantic segmentation map is the same as that of the panoramic image, and the semantic segmentation map includes the category corresponding to each pixel in the panoramic image.

[0114] For example, the panoramic image can be fed into a semantic segmentation network, which performs semantic segmentation on each pixel in the panoramic image, generating a semantic segmentation map corresponding to the panoramic image. This map includes the category corresponding to each pixel in the panoramic image, and the categories may include, but are not limited to, at least one of the following: parking space, lane marking, arrow, speed bump, zebra crossing, wall column, curb, wheel well, etc. Of course, the above are just a few examples and are not intended to be limiting.

[0115] Step 705: Based on the semantic segmentation map corresponding to the panoramic image, determine whether the panoramic image contains the target pixel of the wheel well category of the battery swap station. If not, return to step 703 and wait for the next cycle to continue acquiring a panoramic image of the target vehicle in the battery swap station scene. If yes, proceed to step 706.

[0116] Exemplarily, if the category corresponding to at least M pixels in the semantic segmentation map is the battery swap station wheel slot category, and M is a positive integer, then the panoramic image has a target pixel of the battery swap station wheel slot category.

[0117] Step 706: If there are target pixel points (such as multiple target pixel points) of the wheel well category of the battery swap station in the panoramic image, the wheel well point cloud of the target pixel point in the vehicle body coordinate system is determined based on the pixel coordinates of the target pixel point in the panoramic image, and the target wheel well length and target wheel well width are determined based on the wheel well point cloud.

[0118] For each target pixel, the pixel coordinates can be converted to reference coordinates in the central coordinate system based on the pixel coordinates of the target pixel in the panoramic image, the length and height of the panoramic image, and the mapping relationship between pixels and physical dimensions. Based on the reference coordinates and the external parameter relationship between the central coordinate system and the vehicle coordinate system, the reference coordinates can be converted to the corresponding three-dimensional coordinates of the target pixel in the vehicle coordinate system.

[0119] For example, the pixel coordinates of the target pixel in the panoramic image are marked as The pixel coordinates can be converted to Convert to reference coordinates in the central coordinate system The reference coordinates can be converted to Convert to the three-dimensional coordinates of the target pixel point in the vehicle coordinate system

[0120] After obtaining the three-dimensional coordinates of each target pixel point in the vehicle body coordinate system, the wheel well point cloud is determined based on the three-dimensional coordinates corresponding to each target pixel point in the vehicle body coordinate system. The wheel well point cloud may include the three-dimensional coordinates corresponding to each target pixel point in the vehicle body coordinate system. Then, the target wheel well length and target wheel well width may be determined based on the wheel well point cloud. For example, the wheel well point cloud may include the three-dimensional coordinates of multiple target pixel points in the vehicle body coordinate system, and the maximum value x in the x direction among the multiple three-dimensional coordinates may be obtained. max , minimum value x in the x direction min , maximum value y in the y direction max and the minimum value y in the y direction min On this basis, we can calculate the maximum value x in the x direction. max and the minimum value x in the x direction min Determine the target wheel groove length L, L = x max -x min The target wheel groove width W can be determined based on the maximum value and minimum value in the y direction, W=y max -y min .

[0121] Step 707: Determine whether the difference between the target wheel groove length and the preset wheel groove length is less than a length threshold, and determine whether the difference between the target wheel groove width and the preset wheel groove width is less than a width threshold.

[0122] If yes, execute step 708; if not, clear the wheel groove point cloud, return to step 703, wait for the next cycle, and continue to obtain a panoramic image of the target vehicle in the battery swap station scene.

[0123] For example, a first difference between the target wheel well length and the preset wheel well length can be calculated, and a second difference between the target wheel well width and the preset wheel well width can be calculated. If the absolute value of the first difference is less than the length threshold, and the absolute value of the second difference is less than the width threshold, then the physical size priors for the wheel well at the battery swap station are met, indicating that the current vehicle has reached the battery swapping surface, and step 708 can be executed. If the absolute value of the first difference is not less than the length threshold, and / or the absolute value of the second difference is not less than the width threshold, then the physical size priors for the wheel well at the battery swap station are not met, indicating that the current vehicle has not reached the battery swapping surface. If the target vehicle is located on a non-planar surface, the ranging error is large, and the wheel well point cloud is cleared, and steps 702-706 are repeated.

[0124] Step 708: Convert the wheel well point cloud of the target sample pixel in the vehicle coordinate system into a target wheel well map point in the map coordinate system. Determine the target pose corresponding to the target vehicle based on the target wheel well map point and the calibrated wheel well map point in the map coordinate system. The target wheel well map point represents the three-dimensional coordinates of the wheel well at the battery swap station in the map coordinate system. These three-dimensional coordinates are labeled as the target wheel well map point.

[0125] For example, the target posture corresponding to the target vehicle can be determined by the following steps:

[0126] Step 7081: Based on the pixel coordinates of the reference pixel point of the ground sign category in the panoramic image, determine the ground sign outline point cloud of the reference pixel point in the vehicle body coordinate system.

[0127] Exemplarily, ground signs may include but are not limited to parking spaces, lane lines, arrows, speed bumps, zebra crossings, etc., taking the ground signs as lane lines as an example. Based on the semantic segmentation map corresponding to the panoramic image, the category corresponding to each pixel in the panoramic image is determined. For reference pixels whose category is a ground sign category (such as a lane line category) (such as pixels of the ground sign category as reference pixels, or contour extraction is performed on pixels of the ground sign category, and contour pixels are used as reference pixels), the pixel coordinates of the reference pixels in the image coordinate system I are marked as The pixel coordinates can be converted to Convert to the reference coordinates under the central coordinate system F The reference coordinates can be converted to Converted to three-dimensional coordinates in vehicle coordinate system B 3D coordinates of multiple reference pixels It is the ground sign outline point cloud, that is, the ground sign outline point cloud includes the three-dimensional coordinates of multiple reference pixel points in the vehicle body coordinate system B.

[0128] Step 7082: Based on the acquired initial pose, the ground marker outline point cloud is converted into a target ground marker map point in the map coordinate system. For example, after successful initial positioning, the initial pose after initial positioning can be obtained. Based on the initial pose, the ground marker outline point cloud can be converted into the map coordinate system to obtain the target ground marker map point in the map coordinate system. There are no restrictions on this conversion process.

[0129] During the driving process of the target vehicle, the position and posture of the target vehicle will change. The changed position and posture can be updated to the initial position and posture. Based on the initial position and posture, the ground marker contour point cloud can be converted to the map coordinate system to obtain the target ground marker map point in the map coordinate system. There is no restriction on this conversion process.

[0130] Step 7083: Perform a nearest neighbor search between the target ground landmark map point and the calibrated ground landmark map points in the map coordinate system to obtain a matching relationship between the point pairs. Based on the matching relationship between the point pairs, a matching algorithm can be used to determine the reference pose corresponding to the target vehicle, also known as the positioning pose.

[0131] For example, referring to step 504, calibrated ground marker map points in a map coordinate system have been obtained and stored. After obtaining the target ground marker map point, a nearest neighbor search is performed on the target ground marker map point and the calibrated ground marker map point to obtain a matching relationship between the point pairs. Based on the matching relationship between the point pairs, a matching algorithm can be used to determine the reference pose corresponding to the target vehicle. The matching algorithm includes but is not limited to ICP (Iterative Closest Point) algorithm, NDT (Normal Distributions Transform) algorithm, etc. There is no restriction on the method for determining this reference pose.

[0132] Illustratively, steps 7081-7083 are executed each time a panoramic image is captured, thereby updating the reference pose corresponding to the target vehicle. For each captured panoramic image, if the difference between the target well length and the preset well length is less than a length threshold, and the difference between the target well width and the preset well width is less than a width threshold, then step 7084 is executed. Otherwise, only the reference pose is updated.

[0133] Step 7084: Based on the reference pose, the wheel well point cloud (i.e., the wheel well point cloud of multiple target pixel points in the vehicle body coordinate system) is converted into target wheel well map points in the map coordinate system.

[0134] For example, after obtaining the reference pose, the wheel groove point cloud can be converted to the map coordinate system based on the reference pose to obtain the target wheel groove map point in the map coordinate system, that is, the wheel groove point cloud is corrected based on the reference pose to obtain a more accurate target wheel groove map point. There is no restriction on this conversion process.

[0135] Step 7085: Perform nearest neighbor search on the target wheel well map point and the calibrated wheel well map point to obtain a matching relationship between the point pairs; based on the matching relationship between the point pairs, a matching algorithm can be used to determine the target posture corresponding to the target vehicle, which can also be called a positioning posture.

[0136] For example, referring to step 508, calibrated wheel well map points in a map coordinate system have been obtained and stored (the calibrated wheel well map points represent the three-dimensional coordinates of the wheel well at the battery swap station in the map coordinate system). A nearest neighbor search can be performed on the target wheel well map points and the calibrated wheel well map points to obtain a matching relationship between the point pairs. Based on the matching relationship between the point pairs, a matching algorithm can be used to determine the target pose corresponding to the target vehicle. The matching algorithm includes but is not limited to ICP, NDT, etc., and the method for determining this target pose is not limited.

[0137] At this point, step 708 is completed, and the target posture corresponding to the target vehicle is obtained.

[0138] Step 709: Determine whether the difference between the target pose and the marked pose is less than the pose threshold. If yes, execute step 710; if not, continue to move the target vehicle until the difference is less than the pose threshold.

[0139] For example, referring to step 509, the marked posture (i.e., the posture corresponding to the sample vehicle when it is in the wheel slot of the battery swap station) has been obtained and stored. After obtaining the target posture, the difference between the target posture and the marked posture can be calculated. If the difference is less than the posture threshold, it is determined that the target vehicle is in the wheel slot of the battery swap station, that is, the target vehicle has accurately arrived at the designated position for battery swapping. If the difference is not less than the posture threshold, it is determined that the target vehicle is not in the wheel slot of the battery swap station and the target vehicle needs to be moved further.

[0140] Exemplarily, the difference between the target posture and the marked posture is less than the posture threshold, which may include: the distance between the target posture (such as position) and the marked posture (such as position) is less than a preset distance threshold (which can be configured based on experience, such as 0.1 meters), and the angle difference between the target posture (such as posture) and the marked posture (such as posture) is less than a preset angle threshold (which can be configured based on experience, such as 1 degree).

[0141] Step 710: Determine that the target vehicle is located in the wheel well of the battery swap station.

[0142] For example, after determining that the target vehicle is located in the wheel slot of the battery swap station, a query message can be sent to the battery swap station to inquire whether the target vehicle is located in the wheel slot of the battery swap station; if a successful response is received from the battery swap station to the query message, the successful response is used to confirm that the target vehicle is located in the wheel slot of the battery swap station, then the automatic driving mode can be exited, and the battery swap station will perform automatic battery replacement on the target vehicle, that is, automatically replace the battery.

[0143] For example, the target vehicle can request the battery swap station through the cloud platform server to confirm whether the target vehicle is located at the wheel well of the battery swap station. If the battery swap station confirms that the target vehicle is already located at the wheel well of the battery swap station (by determining whether the wheel has fallen into the wheel well through the laser transmitter and receiver installed at the front wheel well), it means that the target vehicle has been correctly positioned, and the battery swap station sends a confirmation signal (i.e., a successful response) to the target vehicle through the cloud platform server. After receiving the confirmation signal, the target vehicle exits the automatic driving mode and sets the gear position to neutral. At the same time, the battery swap station determines that the target vehicle is in neutral through near-field communication and then performs automatic battery swapping.

[0144] If the battery swap station does not detect the target vehicle in the correct position, the target vehicle will send a non-confirmation signal to the target vehicle through the cloud platform server. After receiving the non-confirmation signal, the target vehicle will remind the user through the vehicle HMI or voice interaction that the vehicle position is not accurate and request to take over. After the user takes over, the autonomous driving mode is exited. When the user confirms that the target vehicle is in the correct position, he can request a battery swap through the vehicle HMI or mobile device HMI, and the battery swap station will respond to the target vehicle's battery swap action.

[0145] If the target vehicle has not exited automatic driving and the battery swap is completed, the user will be reminded through the vehicle HMI or voice interaction that the battery swap is complete and whether to proceed to the battery swap destination. After the user confirms, he or she will continue to complete the subsequent steps.

[0146] It can be seen from the above technical solutions that in the embodiment of the present application, a vehicle self-positioning method based on the prior size of the wheel slot of the battery swap station is proposed for self-driving electric vehicles. Combined with visual semantic positioning, it can achieve a very high positioning accuracy in the battery swap station, so that the vehicle can be driven to the wheel slot of the battery swap station, and it can detect whether the vehicle is located in the wheel slot of the battery swap station. Based on the actual size of the wheel slot of the battery swap station, by continuously comparing the difference between the measured size and the actual size of the wheel slot of the battery swap station, it is accurately determined whether the vehicle is located on the battery swap platform. If it is already located on the battery swap platform, the accurate vehicle positioning posture is obtained by matching the wheel slots. By comparing the difference between the positioning posture and the marked posture, it can be determined whether the vehicle is accurately in place, with very high positioning accuracy. If it is necessary to solve the problem of accurate positioning at the intersection of the ramp in the future, if the actual size of the elements at the intersection (such as arrows, parking spaces and other special artificial marks) is known, this method can also be used to screen out observations that meet the ground plane assumption for positioning, which can effectively avoid the positioning error caused by not meeting the ground plane assumption.

[0147] Based on the same application concept as the above method, a positioning device for wheel slots in a battery swap station is proposed in the embodiment of the present application, see Figure 8 FIG. 1 is a schematic diagram of the structure of the device, which may include:

[0148] The acquisition module 81 is used to acquire a panoramic image of the target vehicle in the battery swap station scene; the determination module 82 is used to determine the wheel well point cloud of the target pixel point in the vehicle body coordinate system based on the pixel coordinates of the target pixel point in the panoramic image if there is a target pixel point of the wheel well category of the battery swap station, wherein the vehicle body coordinate system takes the rear wheel center of the target vehicle as the coordinate origin; determine the target wheel well length and target wheel well width based on the wheel well point cloud; the processing module 83 is used to determine the target wheel well length and target wheel well width if the target wheel well length is different from the preset wheel well length. If the difference is less than the length threshold, and the difference between the target wheel well width and the preset wheel well width is less than the width threshold, the wheel well point cloud is converted into a target wheel well map point in a map coordinate system, and the target posture corresponding to the target vehicle is determined based on the target wheel well map point and the calibrated wheel well map point in the map coordinate system; the positioning module 84 is used to determine that the target vehicle is located in the wheel well of the battery swap station if the difference between the target posture and the calibrated posture is less than the posture threshold; wherein the calibrated posture is the posture of the calibrated vehicle when it is located in the wheel well of the battery swap station.

[0149] Exemplarily, the determination module 82 is also used to perform semantic segmentation on each pixel point in the panoramic image to obtain a semantic segmentation map corresponding to the panoramic image, the resolution of the semantic segmentation map is the same as the resolution of the panoramic image, and the semantic segmentation map includes the category corresponding to each pixel point in the panoramic image; based on the semantic segmentation map, it is determined whether there are target pixel points of the battery swap station wheel slot category in the panoramic image; wherein, if the category corresponding to at least M pixel points in the semantic segmentation map is the battery swap station wheel slot category, then there are target pixel points of the battery swap station wheel slot category in the panoramic image.

[0150] Exemplarily, when determining the wheel well point cloud of the target pixel point in the vehicle body coordinate system based on the pixel coordinates of the target pixel point in the panoramic image, the determining module 82 is specifically configured to:

[0151] Based on the pixel coordinates of the target pixel point in the panoramic image, the length and height of the panoramic image, and the mapping relationship between the calibrated pixels and physical dimensions, converting the pixel coordinates into reference coordinates in a central coordinate system, where the center of the target vehicle is the coordinate origin;

[0152] Based on the reference coordinates and the extrinsic relationship between the calibrated central coordinate system and the vehicle body coordinate system, converting the reference coordinates into the three-dimensional coordinates corresponding to the target pixel point in the vehicle body coordinate system;

[0153] The wheel well point cloud is determined based on the three-dimensional coordinates corresponding to each target pixel point in the vehicle body coordinate system, wherein the wheel well point cloud includes the three-dimensional coordinates corresponding to each target pixel point in the vehicle body coordinate system.

[0154] Exemplarily, when determining the target wheel groove length and the target wheel groove width based on the wheel groove point cloud, the determining module 82 is specifically configured to: if the wheel groove point cloud includes multiple three-dimensional coordinates, obtain the maximum value in the x direction, the minimum value in the x direction, the maximum value in the y direction, and the minimum value in the y direction from the multiple three-dimensional coordinates;

[0155] Determine the target wheel groove length based on the maximum value in the x-direction and the minimum value in the x-direction;

[0156] The target wheel groove width is determined based on the maximum value and the minimum value in the y direction.

[0157] Exemplarily, the processing module 83 converts the wheel well point cloud into a target wheel well map point in a map coordinate system, and determines the target posture corresponding to the target vehicle based on the target wheel well map point and the calibrated wheel well map point in the map coordinate system, specifically for:

[0158] Determining a ground sign outline point cloud of a reference pixel of a ground sign category in the panoramic image based on pixel coordinates of the reference pixel in the panoramic image in the vehicle body coordinate system;

[0159] Based on the acquired initial pose, the ground marker outline point cloud is converted into a target ground marker map point in a map coordinate system, and a nearest neighbor search is performed on the target ground marker map point and the calibrated ground marker map point in the map coordinate system to obtain a matching relationship between point pairs; based on the matching relationship between the point pairs, a matching algorithm is used to determine a reference pose corresponding to the target vehicle;

[0160] The wheel well point cloud is converted into a target wheel well map point in a map coordinate system based on the reference posture; a nearest neighbor search is performed on the target wheel well map point and the calibrated wheel well map point to obtain a matching relationship between point pairs; and a matching algorithm is used to determine a target posture corresponding to the target vehicle based on the matching relationship between the point pairs.

[0161] Exemplarily, the difference between the target posture and the marked posture is less than the posture threshold, which includes: the distance between the target posture and the marked posture is less than a preset distance threshold, and the angle difference between the target posture and the marked posture is less than a preset angle threshold; after the positioning module 84 determines that the target vehicle is located in the wheel slot of the battery swap station, it is also used to: send a query message to the battery swap station to inquire whether it is located in the wheel slot of the battery swap station; if a successful response returned by the battery swap station is received, the successful response is used to confirm that the target vehicle is located in the wheel slot of the battery swap station, then the automatic driving mode is exited, and the battery swap station performs automatic battery swapping on the target vehicle.

[0162] Exemplarily, the acquisition module 81 is further configured to acquire the calibrated wheel well map points in the map coordinate system in the following manner: acquiring a sample image of a sample vehicle in the battery swap station scenario;

[0163] If the sample image contains a sample pixel point of the wheel well category of the battery swap station, determine the wheel well point cloud of the sample pixel point in the vehicle body coordinate system based on the pixel coordinates of the sample pixel point in the sample image, and determine the sample wheel well length and sample wheel well width based on the wheel well point cloud;

[0164] If the difference between the sample wheel well length and the preset wheel well length is less than a length threshold, and the difference between the sample wheel well width and the preset wheel well width is less than a width threshold, the wheel well point cloud is converted into a calibrated wheel well map point in a map coordinate system and the calibrated wheel well map point is stored. Exemplarily, the acquisition module 81 is further configured to, after the sample vehicle arrives at the wheel well of the battery swap station, acquire the calibrated pose corresponding to the sample vehicle in the wheel well of the battery swap station and store the calibrated pose.

[0165] Based on the same application concept as the above method, an autonomous driving vehicle is proposed in the embodiment of the present application. Figure 9 As shown, the autonomous driving vehicle includes a processor 91 and a machine-readable storage medium 92, and the machine-readable storage medium 92 stores machine-executable instructions that can be executed by the processor 91; the processor 91 is used to execute the machine-executable instructions to implement the positioning method of the wheel slot of the battery swap station disclosed in the above example of this application.

[0166] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by the processor, the positioning method of the wheel slot of the battery swap station disclosed in the above example of the present application can be implemented.

[0167] The machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that may contain or store information, such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as a CD, DVD, etc.), or similar storage media, or a combination thereof.

[0168] The systems, devices, modules, or units described in the above embodiments may be implemented by a computer entity or by a product having certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email transceiver, game console, tablet computer, wearable device, or any combination of these devices.

[0169] For the convenience of description, the above devices are described as being divided into various units according to their functions. Of course, when implementing this application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0170] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] Furthermore, these computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0174] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A method for positioning a wheel well in a battery swap station, characterized in that: The method comprises: Obtain a panoramic image of the target vehicle in the battery swap station scene; If the panoramic image contains a target pixel of the wheel well category of the battery swap station, then based on the pixel coordinates of the target pixel in the panoramic image, determine the wheel well point cloud of the target pixel in the vehicle body coordinate system, where the vehicle body coordinate system takes the center of the rear wheel of the target vehicle as the coordinate origin; Determining a target wheel groove length and a target wheel groove width based on the wheel groove point cloud; If the difference between the target wheel well length and the preset wheel well length is less than a length threshold, and the difference between the target wheel well width and the preset wheel well width is less than a width threshold, converting the wheel well point cloud into a target wheel well map point in a map coordinate system, and determining a target pose corresponding to the target vehicle based on the target wheel well map point and the calibrated wheel well map point in the map coordinate system; If the difference between the target posture and the calibrated posture is less than the posture threshold, it is determined that the target vehicle is located in the wheel well of the battery swap station; wherein the calibrated posture is the posture of the calibrated vehicle when it is located in the wheel well of the battery swap station.

2. The method according to claim 1, characterized in that After obtaining the panoramic image of the target vehicle in the battery swap station scene, the method further includes: Performing semantic segmentation on each pixel in the panoramic image to obtain a semantic segmentation map corresponding to the panoramic image, where the resolution of the semantic segmentation map is the same as that of the panoramic image, and the semantic segmentation map includes a category corresponding to each pixel in the panoramic image; Based on the semantic segmentation map, it is determined whether the panoramic image contains target pixel points of the battery swap station wheel slot category; wherein, if the category corresponding to at least M pixel points in the semantic segmentation map is the battery swap station wheel slot category, and M is a positive integer, then the panoramic image contains target pixel points of the battery swap station wheel slot category.

3. The method according to claim 1, characterized in that Determining, based on the pixel coordinates of the target pixel point in the panoramic image, a wheel well point cloud of the target pixel point in a vehicle body coordinate system includes: Based on the pixel coordinates of the target pixel point in the panoramic image, the length and height of the panoramic image, and the mapping relationship between the calibrated pixels and physical dimensions, converting the pixel coordinates into reference coordinates in a central coordinate system, where the center of the target vehicle is the coordinate origin; Based on the reference coordinates and the extrinsic relationship between the calibrated central coordinate system and the vehicle body coordinate system, converting the reference coordinates into the three-dimensional coordinates corresponding to the target pixel point in the vehicle body coordinate system; The wheel well point cloud is determined based on the three-dimensional coordinates corresponding to each target pixel point in the vehicle body coordinate system, wherein the wheel well point cloud includes the three-dimensional coordinates corresponding to each target pixel point in the vehicle body coordinate system.

4. The method according to claim 1, wherein The determining of a target wheel groove length and a target wheel groove width based on the wheel groove point cloud includes: If the wheel groove point cloud includes multiple three-dimensional coordinates, obtaining the maximum value in the x direction, the minimum value in the x direction, the maximum value in the y direction, and the minimum value in the y direction of the multiple three-dimensional coordinates; Determine the target wheel groove length based on the maximum value in the x-direction and the minimum value in the x-direction; The target wheel groove width is determined based on the maximum value and the minimum value in the y direction.

5. The method according to claim 1, wherein The converting the wheel well point cloud into a target wheel well map point in a map coordinate system, and determining a target posture corresponding to the target vehicle based on the target wheel well map point and the calibrated wheel well map point in the map coordinate system, comprises: Determining a ground sign outline point cloud of a reference pixel of a ground sign category in the panoramic image based on pixel coordinates of the reference pixel in the panoramic image in the vehicle body coordinate system; Based on the acquired initial pose, the ground marker outline point cloud is converted into a target ground marker map point in a map coordinate system, and a nearest neighbor search is performed on the target ground marker map point and the calibrated ground marker map point in the map coordinate system to obtain a matching relationship between point pairs; based on the matching relationship between the point pairs, a matching algorithm is used to determine a reference pose corresponding to the target vehicle; The wheel well point cloud is converted into a target wheel well map point in a map coordinate system based on the reference posture; a nearest neighbor search is performed on the target wheel well map point and the calibrated wheel well map point to obtain a matching relationship between point pairs; and a matching algorithm is used to determine a target posture corresponding to the target vehicle based on the matching relationship between the point pairs.

6. The method according to claim 1, characterized in that The difference between the target posture and the marked posture is less than the posture threshold value, which includes: the distance between the target posture and the marked posture is less than a preset distance threshold value, and the angle difference between the target posture and the marked posture is less than a preset angle threshold value; The method further includes determining that the target vehicle is located behind a wheel well of a battery swap station: A query message is sent to the battery swap station to inquire whether the target vehicle is located in the wheel slot of the battery swap station; if a successful response is received from the battery swap station, the successful response is used to confirm that the target vehicle is located in the wheel slot of the battery swap station, then the automatic driving mode is exited, and the battery swap station performs automatic battery swapping on the target vehicle.

7. The method according to any one of claims 1 to 6, characterized in that The method for obtaining the calibrated wheel groove map points in the map coordinate system includes: Acquire a sample image of a sample vehicle in the battery swap station scenario; If the sample image contains a sample pixel point of the wheel well category of the battery swap station, determine the wheel well point cloud of the sample pixel point in the vehicle body coordinate system based on the pixel coordinates of the sample pixel point in the sample image, and determine the sample wheel well length and sample wheel well width based on the wheel well point cloud; If the difference between the sample wheel groove length and the preset wheel groove length is less than the length threshold, and the difference between the sample wheel groove width and the preset wheel groove width is less than the width threshold, the wheel groove point cloud is converted into a calibrated wheel groove map point in a map coordinate system, and the calibrated wheel groove map point is stored.

8. The method according to claim 7, characterized in that The method further comprises: After the sample vehicle arrives at the wheel well of the battery swap station, the marked posture corresponding to the sample vehicle when it is in the wheel well of the battery swap station is obtained, and the marked posture is stored.

9. A positioning device for a wheel groove in a battery swap station, characterized in that: The device comprises: An acquisition module is used to acquire a panoramic image of the target vehicle in the battery swap station scene; a determination module configured to, if a target pixel point of the wheel well category of the battery swap station exists in the panoramic image, determine a wheel well point cloud of the target pixel point in a vehicle body coordinate system based on the pixel coordinates of the target pixel point in the panoramic image, wherein the vehicle body coordinate system has the center of the rear wheel of the target vehicle as the coordinate origin; and determine a target wheel well length and a target wheel well width based on the wheel well point cloud; a processing module, configured to, if a difference between the target wheel well length and the preset wheel well length is less than a length threshold, and a difference between the target wheel well width and the preset wheel well width is less than a width threshold, convert the wheel well point cloud into a target wheel well map point in a map coordinate system, and determine a target pose corresponding to the target vehicle based on the target wheel well map point and the calibrated wheel well map points in the map coordinate system; A positioning module is used to determine that the target vehicle is located in the wheel well of the battery swap station if the difference between the target posture and the calibrated posture is less than a posture threshold; wherein the calibrated posture is the posture of the calibrated vehicle when it is located in the wheel well of the battery swap station.

10. An autonomous driving vehicle, characterized in that: include: A processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; wherein the processor is configured to execute the machine-executable instructions to implement the method according to any one of claims 1 to 8.

Images