Vehicle positioning method, device, equipment and storage medium

Abstract

Embodiments of the present application provide a vehicle positioning method, apparatus, device, and storage medium. The vehicle positioning method includes: during the movement of the vehicle along a preset trajectory, acquiring an image of the vehicle's surrounding environment and vehicle wheel speedometer data, where there is one or more parking spaces on the preset trajectory, and at least some of the parking spaces are equipped with charging piles; when the vehicle needs to be charged, controlling the vehicle to drive into a parking space with a charging pile and not occupied and aligning the vehicle's charging port with the charging pile according to the image of the vehicle's surrounding environment, the vehicle wheel speedometer data, and a pre-constructed map. The present application can improve the accuracy of the vehicle in finding a charging pile and aligning the charging port with the charging pile.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer vision technology, and in particular, to a vehicle positioning method, device, equipment, and storage medium. Background Art

[0002] With the rapid development of autonomous driving-related technologies in recent years, the application of unmanned logistics vehicles in the logistics field has gradually emerged. The main task of unmanned logistics vehicles is to autonomously load and unload goods, and at the same time, they need to be charged irregularly. When charging is required, the unmanned logistics vehicle drives into a parking space with a charging pile for autonomous charging. However, after the unmanned logistics vehicle drives into a parking space with a charging pile, it cannot well align the vehicle charging port with the charging pile. Moreover, when the unmanned logistics vehicle drives into the parking space, it often relies on a single feature for positioning, resulting in low positioning accuracy. Summary of the Invention

[0003] To solve at least one of the above technical problems, embodiments of the present application provide a vehicle positioning method, device, equipment, and computer-readable storage medium.

[0004] In a first aspect, embodiments of the present application provide a map construction method, including:

[0005] Obtaining a first vehicle surrounding environment image when the vehicle moves along a preset trajectory;

[0006] Performing ORB feature point extraction on the first vehicle surrounding environment image, and constructing a feature point map based on the extracted ORB feature points;

[0007] Determining the parking space semantic feature and the charging pile object feature according to the first vehicle surrounding environment image, where the charging pile object feature includes a charging pile semantic feature;

[0008] Determining the positions of the parking space semantic feature and the charging pile object feature in a preset coordinate system;

[0009] Fusing the parking space semantic feature, the charging pile object feature, the position of the parking space semantic feature in the preset coordinate system, and the position of the charging pile object feature in the preset coordinate system into the feature point map to form a map.

[0010] In a possible implementation, the constructing a feature point map based on the extracted ORB feature points includes:

[0011] Generating an initial map according to two frames of images of the first vehicle surrounding environment image;

[0012] Generating new feature points in the initial map according to each pair of matching ORB feature points in the two frames of images of the first vehicle surrounding environment image to form the feature point map.

[0013] In a possible implementation, constructing the feature point map based on the extracted ORB feature points further includes:

[0014] When the current frame image of the surrounding environment image of the first vehicle moves a certain distance from the last key frame image, add the current frame image to the initialized map to update the feature point map.

[0015] In a possible implementation, determining the parking space semantic feature and the charging pile object feature according to the surrounding environment image of the first vehicle includes:

[0016] Perform semantic segmentation on the surrounding environment image of the first vehicle, so that each pixel in the surrounding environment image of the first vehicle corresponds to a classification label, and the classification label includes the parking space line semantic feature, the corner point semantic feature, the limit plate semantic feature, and the charging pile semantic feature;

[0017] Determine the pixels corresponding to the parking space line semantic feature, the pixels corresponding to the corner point semantic feature, and the pixels corresponding to the limit plate semantic feature as the parking space semantic feature;

[0018] Determine the charging pile object feature according to the pixels corresponding to the charging pile semantic feature.

[0019] In a possible implementation, determining the charging pile object feature according to the pixels corresponding to the charging pile semantic feature includes:

[0020] Set the pixel value of the pixels corresponding to the charging pile semantic feature to 1, and set the pixel values of the remaining pixels except the pixels corresponding to the charging pile semantic feature to 0;

[0021] Take the pixels with a pixel value of 1 and adjacent pixel positions as charging pile pixels to form the charging pile object feature.

[0022] In a possible implementation, determining the position of the parking space semantic feature in a preset coordinate system includes:

[0023] Based on the parameters and calibration parameters of the camera used to collect the surrounding environment image of the first vehicle, generate a top view according to the surrounding environment image of the first vehicle;

[0024] Determine the coordinates of the parking space semantic feature in the pixel coordinate system of the top view;

[0025] Convert the coordinates of the parking space semantic feature into three-dimensional coordinates in the preset coordinate system, and use the three-dimensional coordinates in the preset coordinate system as the position of the parking space semantic feature in the preset coordinate system.

[0026] In a possible implementation, determining the position of the charging pile object feature in the world coordinate system includes:

[0027] In the surrounding environment image of the first vehicle, determining the ORB feature points corresponding to the charging pile object feature;

[0028] Determining the three-dimensional coordinates of the charging pile according to the three-dimensional coordinates of the ORB feature points corresponding to the charging pile object feature;

[0029] Converting the three-dimensional coordinates of the charging pile into the three-dimensional coordinates in a preset coordinate system, and using the three-dimensional coordinates in the preset coordinate system as the position of the charging pile object feature in the preset coordinate system.

[0030] In a possible implementation, after fusing the parking space semantic feature, the charging pile object feature, the position of the parking space semantic feature in the preset coordinate system, and the position of the charging pile object feature in the preset coordinate system into the feature point map to form a map, it further includes:

[0031] If the parking space corresponding to the parking space semantic feature does not have a charging pile, it is marked;

[0032] If the parking space corresponding to the parking space semantic feature has a charging pile, it is marked.

[0033] In a second aspect, an embodiment of the present application provides a vehicle positioning method, including:

[0034] During the movement of the vehicle along a preset trajectory, obtaining the surrounding environment image of the second vehicle and the vehicle wheel speedometer data, where one or more parking spaces are provided on the preset trajectory, and at least some of the parking spaces have charging piles;

[0035] When the vehicle needs to be charged, according to the surrounding environment image of the second vehicle, the vehicle wheel speedometer data, and a pre-constructed map, controlling the vehicle to drive into a parking space with a charging pile and not occupied and aligning the vehicle charging port with the charging pile.

[0036] In a possible implementation, controlling the vehicle to drive into a parking space with a charging pile and not occupied and aligning the vehicle charging port with the charging pile according to the surrounding environment image of the second vehicle, the vehicle wheel speedometer data, and a pre-constructed map includes:

[0037] When the parking space has a charging pile and is not occupied, based on the parking space semantic feature in the pre-constructed map and the position of the parking space semantic feature in the world coordinate system, controlling the vehicle to drive into the parking space with a charging pile and not occupied according to the surrounding environment image of the second vehicle and the vehicle wheel speedometer data;

[0038] Based on the charging pile object features in the pre-constructed map and the positions of the charging pile object features in the world coordinate system, control the vehicle according to the second vehicle surrounding environment image and the vehicle wheel speedometer data so that the vehicle charging port is aligned with the charging pile.

[0039] In a possible implementation manner, the controlling the vehicle to drive into a parking space with a charging pile and not occupied based on the parking space semantic features in the pre-constructed map and the positions of the parking space semantic features in the world coordinate system, according to the second vehicle surrounding environment image and the vehicle wheel speedometer data includes:

[0040] Detect and identify the parking space, the parking space lines of the parking space, the corner points of the parking space, and the limit plates of the parking space according to the second vehicle surrounding environment image;

[0041] Locate the vehicle according to the parking space lines of the parking space, the corner points of the parking space, the limit plates of the parking space, the parking space semantic features, and the positions of the parking space semantic features in the world coordinate system, where the parking space semantic features include parking space line semantic features, corner point semantic features, and limit plate semantic features;

[0042] Based on the vehicle positioning, control the vehicle to drive into a parking space with a charging pile and not occupied according to the vehicle wheel speedometer data.

[0043] In a possible implementation manner, the controlling the vehicle so that the vehicle charging port is aligned with the charging pile based on the charging pile object features in the pre-constructed map and the positions of the charging pile object features in the world coordinate system, according to the second vehicle surrounding environment image and the vehicle wheel speedometer data includes:

[0044] Detect and identify the charging pile according to the second vehicle surrounding environment image;

[0045] Locate the vehicle according to the detected and identified charging pile, the charging pile object features, and the positions of the charging pile object features in the world coordinate system;

[0046] Generate a virtual charging parking space according to the positions of the charging pile object features in the world coordinate system, where the size of the virtual charging parking space is greater than or equal to the vehicle body size;

[0047] Based on the vehicle positioning, control the vehicle to drive into the virtual charging parking space so that the vehicle charging port is aligned with the charging port of the virtual charging parking space according to the vehicle wheel speedometer data.

[0048] In a possible implementation manner, the method for constructing a map includes:

[0049] Obtain the first vehicle surrounding environment image when the vehicle moves along a preset trajectory;

[0050] Extract ORB feature points from the surrounding environment image of the first vehicle, and construct a feature point map based on the extracted ORB feature points;

[0051] Determine the parking space semantic feature and the charging pile object feature according to the surrounding environment image of the first vehicle, and the charging pile object feature includes the charging pile semantic feature;

[0052] Determine the positions of the parking space semantic feature and the charging pile object feature in a preset coordinate system;

[0053] Fuse the parking space semantic feature, the charging pile object feature, the position of the parking space semantic feature in the preset coordinate system, and the position of the charging pile object feature in the preset coordinate system into the feature point map to form a map.

[0054] In a possible implementation manner, the constructing a feature point map based on the extracted ORB feature points includes:

[0055] Generate an initial map according to two frames of the surrounding environment image of the first vehicle;

[0056] Generate new feature points in the initial map according to each pair of matching ORB feature points in the two frames of the surrounding environment image of the first vehicle to form the feature point map.

[0057] In a possible implementation manner, the constructing a feature point map based on the extracted ORB feature points further includes:

[0058] When there is a certain distance of movement between the current frame image and the last key frame image of the surrounding environment image of the first vehicle, add the current frame image to the initial map to update the feature point map.

[0059] In a possible implementation manner, the determining the parking space semantic feature and the charging pile object feature according to the surrounding environment image of the first vehicle includes:

[0060] Perform semantic segmentation on the surrounding environment image of the first vehicle so that each pixel in the surrounding environment image of the first vehicle corresponds to a classification label, and the classification label includes the parking space line semantic feature, the corner point semantic feature, the limit plate semantic feature, and the charging pile semantic feature;

[0061] Determine the pixels corresponding to the parking space line semantic feature, the pixels corresponding to the corner point semantic feature, and the pixels corresponding to the limit plate semantic feature as the parking space semantic feature;

[0062] Determine the charging pile object feature according to the pixels corresponding to the charging pile semantic feature.

[0063] In a possible implementation, determining the charging pile object feature according to the pixels corresponding to the charging pile semantic feature includes:

[0064] Set the pixel value of the pixels corresponding to the charging pile semantic feature to 1, and set the pixel values of the remaining pixels except the pixels corresponding to the charging pile semantic feature to 0;

[0065] Take the pixels with a pixel value of 1 and adjacent pixel positions as charging pile pixels to form the charging pile object feature.

[0066] In a possible implementation, determining the position of the parking space semantic feature in the world coordinate system includes:

[0067] Based on the parameters and calibration parameters of the camera for collecting the surrounding environment image of the first vehicle, generate a top view according to the surrounding environment image of the first vehicle;

[0068] Determine the coordinates of the parking space semantic feature in the pixel coordinate system of the top view;

[0069] Convert the coordinates of the parking space semantic feature into three-dimensional coordinates in the preset coordinate system, and use the three-dimensional coordinates in the preset coordinate system as the position of the parking space semantic feature in the preset coordinate system.

[0070] In a possible implementation, determining the position of the charging pile object feature in the world coordinate system includes:

[0071] In the surrounding environment image of the first vehicle, determine the ORB feature points corresponding to the charging pile object feature;

[0072] Determine the three-dimensional coordinates of the charging pile according to the three-dimensional coordinates of the ORB feature points corresponding to the charging pile object feature;

[0073] Convert the three-dimensional coordinates of the charging pile into three-dimensional coordinates in the preset coordinate system, and use the three-dimensional coordinates in the preset coordinate system as the position of the charging pile object feature in the preset coordinate system.

[0074] In a possible implementation, the step of fusing the parking space semantic feature, the charging pile object feature, the position of the parking space semantic feature in the world coordinate system, and the position of the charging pile object feature in the world coordinate system into the feature point map to form a map further includes:

[0075] If the parking space corresponding to the parking space semantic feature does not have a charging pile, mark it;

[0076] If the parking space corresponding to the parking space semantic feature has a charging pile, mark it.

[0077] In a third aspect, an embodiment of the present application provides a map construction device, including:

[0078] A first acquisition module, configured to acquire a first image of the vehicle's surrounding environment when the vehicle moves along a preset trajectory;

[0079] A map construction module, configured to extract ORB feature points from the first image of the vehicle's surrounding environment, and construct a feature point map based on the extracted ORB feature points;

[0080] A feature determination module, configured to determine the parking space semantic feature and the charging pile object feature according to the first image of the vehicle's surrounding environment, where the charging pile object feature includes a charging pile semantic feature;

[0081] A position determination module, configured to determine the positions of the parking space semantic feature and the charging pile object feature in the world coordinate system;

[0082] A map fusion module, configured to fuse the parking space semantic feature, the charging pile object feature, the position of the parking space semantic feature in the world coordinate system, and the position of the charging pile object feature in the world coordinate system into the feature point map to form a map.

[0083] In a fourth aspect, an embodiment of the present application provides a vehicle positioning device, including:

[0084] A second acquisition module, configured to acquire an image of the vehicle's surrounding environment during the process of the vehicle moving along a preset trajectory, where there is one or more parking spaces on the preset trajectory, and at least some of the parking spaces are equipped with charging piles;

[0085] A third acquisition module, configured to acquire vehicle wheel speed meter data during the process of the vehicle moving along a preset trajectory;

[0086] A vehicle control module, configured to, when the vehicle needs to be charged, control the vehicle to drive into a parking space equipped with a charging pile and not occupied according to the image of the vehicle's surrounding environment, the vehicle wheel speed meter data, and a pre-constructed map, and align the vehicle's charging port with the charging pile.

[0087] In a fifth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, where a computer program is stored on the memory, and when the processor executes the computer program, the method described in any item of the first aspect or the method described in any item of the second aspect is implemented.

[0088] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any item of the first aspect or the method described in any item of the second aspect is implemented.

[0089] In the map construction method, vehicle positioning method, device, equipment, and storage medium provided by the embodiments of the present application, a first surrounding environment image of the vehicle is acquired when the vehicle moves according to a preset trajectory; ORB feature points are extracted from the first surrounding environment image of the vehicle, and a feature point map is constructed based on the extracted ORB feature points; a parking space semantic feature and a charging pile object feature are determined according to the first surrounding environment image of the vehicle; the positions of the parking space semantic feature and the charging pile object feature in the world coordinate system are determined; the parking space semantic feature, the charging pile object feature, the position of the parking space semantic feature in the world coordinate system, and the position of the charging pile object feature in the world coordinate system are fused into the feature point map to form a map, so that a map including the parking space semantic feature and the charging pile object feature can be constructed, without solely relying on a certain feature, and thus accurate positioning in various environments can be achieved.

[0090] Further, after the map construction is completed, during the process of the vehicle moving according to the preset trajectory, a surrounding environment image of the vehicle and vehicle wheel speedometer data are acquired. When the vehicle needs to be charged, according to the surrounding environment image of the vehicle, the vehicle wheel speedometer data, and the pre-constructed map, the vehicle is controlled to drive into a parking space with a charging pile and not occupied, and the charging port of the vehicle is aligned with the charging pile, so that the accuracy of the vehicle searching for a charging pile and aligning the charging port with the charging pile can be improved.

[0091] It should be understood that the content described in the summary of the invention section is not intended to limit the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Combined with the drawings and referring to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present application will become more obvious. In the drawings, the same or similar reference numerals represent the same or similar elements.

[0093] Figure 1 The schematic diagram of the application scenario of the vehicle positioning method according to the embodiments of the present application is shown.

[0094] Figure 2 The schematic diagram of a vehicle positioning method according to an embodiment of the present application is shown.

[0095] Figure 3 The structural diagram of an electronic device according to an embodiment of the present application is shown.

[0096] Figure 4 The flowchart of the map construction method according to an embodiment of the present application is shown.

[0097] Figure 5 The schematic diagram of the vehicle driving route during the map construction according to an embodiment of the present application is shown.

[0098] Figure 6 The flowchart of another vehicle positioning method according to an embodiment of the present application is shown.

[0099] Figure 7 The schematic diagram of vehicle control according to an embodiment of the present application is shown.

[0100] Figure 8 The schematic diagram of a virtual charging parking space according to an embodiment of the present application is shown.

[0101] Figure 9 The schematic diagram of the setting of a virtual charging parking space according to an embodiment of the present application is shown.

[0102] Figure 10 The flowchart of yet another vehicle positioning method according to an embodiment of the present application is shown.

[0103] Figure 11 The block diagram of a map construction device according to an embodiment of the present application is shown.

[0104] Figure 12 The block diagram of a vehicle positioning device according to an embodiment of the present application is shown. Detailed implementation manners

[0105] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

[0106] To facilitate the understanding of the embodiments of the present application, some terms related to the embodiments of the present application will be explained first.

[0107] The vehicle surrounding environment image is collected by a fish-eye camera arranged on the vehicle, including but not limited to the front view image, rear view image, left view image, and right view image of the vehicle, and each image includes but not limited to the image of the front of the vehicle and the ground.

[0108] The vehicle wheel speedometer data is collected by a wheel speedometer arranged on the vehicle, including but not limited to the steering wheel angle and the vehicle forward speed.

[0109] Next, the application scenarios related to the embodiments of the present application will be introduced. It should be noted that the application scenario described in the embodiments of the present application is the scenario of autonomous charging during the loading and unloading of goods by a driverless logistics vehicle, which is only for more clearly explaining the technical solutions in the embodiments of the present application and does not constitute a limitation on the technical solutions provided in the embodiments of the present application. The vehicle positioning method provided in the embodiments of the present application is also applicable to similar or analogous scenarios where other driverless vehicles need to be charged.

[0110] Figure 1 The schematic diagram of the application scenario of the vehicle positioning method according to an embodiment of the present application is shown. InFigure 1 In the scene shown, the vehicle loading and unloading route and the delivery points and parking spaces passed through on this route are schematically shown, and at least some of the parking spaces are equipped with charging piles. It should be noted that in the actual application scenario, the number of delivery points and the number of parking spaces can be any number, and there is no restriction on whether the parking spaces have charging piles. In the actual application scenario, when the vehicle passes through each delivery point along the loading and unloading route to load and unload goods, if the vehicle needs to be charged, it is necessary to control the vehicle to drive into a parking space with a charging pile and align the charging port of the vehicle with the charging pile for charging operation.

[0111] In the embodiments of the present application, the vehicle is an autonomous vehicle capable of realizing autonomous driving. In some optional embodiments, the control device arranged on the vehicle is used to control the vehicle to drive into a parking space with a charging pile and align the charging port of the vehicle with the charging pile. In some other optional embodiments, the remote control device is used to control the vehicle to drive into a parking space with a charging pile and align the charging port of the vehicle with the charging pile.

[0112] Figure 2 The flowchart of a vehicle positioning method according to an embodiment of the present application is shown.

[0113] See Figure 2 , the image acquisition device acquires the vehicle surrounding environment image and transmits the vehicle surrounding environment image to the control device, and the wheel speed meter acquires the vehicle wheel speed meter data and transmits the vehicle wheel speed meter data to the control device. On the one hand, the control device can construct a map based on the vehicle surrounding environment image. Specifically, the control device obtains the first vehicle surrounding environment image when the vehicle moves along the loading and unloading route, and processes the first vehicle surrounding environment image to construct a map including the semantic features of the parking space, the object features of the charging pile, the position of the semantic features of the parking space in the world coordinate system, and the position of the object features of the charging pile in the world coordinate system. On the other hand, the control device can control the vehicle to drive into a parking space with a charging pile and align the charging port of the vehicle with the charging pile based on the constructed map, the vehicle surrounding environment image, and the vehicle wheel speed meter data. It should be noted that the construction process of the map and the control process of the vehicle are described below, and will not be elaborated here.

[0114] Figure 3 The structure diagram of an electronic device according to an embodiment of the present application is shown. In some optional embodiments, Figure 3 the electronic device shown is a control device arranged on the vehicle or a remote control device for controlling the vehicle.

[0115] See Figure 3, the electronic device 300 includes a processor 301 and a memory 303. Among them, the processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 300 may further include a transceiver 304. It should be noted that in practical applications, the transceiver 304 is not limited to one, and the structure of the electronic device 300 does not constitute a limitation to the embodiments of the present application.

[0116] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logic blocks, modules, and circuits described in connection with the disclosure of the present application. The processor 301 may also be a combination that implements computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0117] The bus 302 may include a path for transmitting information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 3 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0118] The memory 303 can be a ROM (Read Only Memory), or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory), or other types of dynamic storage devices that can store information and instructions. It can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0119] The memory 303 is used to store the application program code for executing the solution of this application and is controlled and executed by the processor 301. The processor 301 is used to execute the application program code stored in the memory 303 to implement the construction of the map or control the vehicle to drive into a parking space with a charging pile and align the charging port of the vehicle with the charging pile.

[0120] Among them, the electronic device includes but is not limited to: mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. It should be noted that Figure 3 The illustrated electronic device is only an example and should not impose any limitations on the functions and usage scope of the embodiments of this application.

[0121] Figure 4 The flowchart of the map construction method according to the embodiments of this application is shown. This method is applied to a control device arranged on a vehicle or a remote control device for controlling the vehicle to travel. Refer to Figure 4 and this method includes the following steps:

[0122] Step 401, obtain the first surrounding environment image of the vehicle when the vehicle moves along a preset trajectory.

[0123] In the embodiments of this application, the surrounding environment image of the vehicle can be obtained through an image acquisition device arranged on the vehicle. When the vehicle moves along a preset trajectory, the surrounding environment image of the vehicle when the vehicle moves along the preset trajectory can be obtained.

[0124] In some alternative embodiments, a fisheye camera may be arranged on each of the front and rear bumpers of the vehicle, and a fisheye camera may be arranged at the lower edges of the left and right rearview mirrors of the vehicle. When the vehicle moves along a preset trajectory, the first vehicle surrounding environment image is collected by the four fisheye cameras.

[0125] Exemplarily, referring to Figure 5 , the manned vehicle travels along Route 1, Route 2, and Route 3 respectively. During the driving process, the first vehicle surrounding environment image is collected by the image acquisition device arranged on the vehicle. Among them, Route 1 is the route for the vehicle to enter and exit each parking space and delivery point, Route 2 is the route for the vehicle to pass through each delivery point to determine the loading and unloading of goods, and Route 3 is the route for the vehicle to pass through each parking space to determine parking and charging.

[0126] Step 402: Extract ORB feature points from the first vehicle surrounding environment image, and construct a feature point map based on the extracted ORB feature points.

[0127] In the embodiment of the present application, extracting ORB feature points from the first vehicle surrounding environment image may, for example, detect Oriented FAST key points and calculate BRIEF descriptors based on the front view image in the vehicle surrounding environment image to extract ORB feature points.

[0128] In the embodiment of the present application, when constructing a feature point map based on the extracted ORB feature points, first generate an initial map according to two frames of images of the front view image in the first vehicle surrounding environment image. For each pair of matching ORB feature points in the two frames of images, generate new feature points in the map to form a feature point map. The information of the newly generated feature points includes the key point position and descriptor attributes.

[0129] Further, if the formed feature point map needs to be optimized, when the current frame of the front view image in the first vehicle surrounding environment image moves a certain distance from the last key frame image, add the current frame image to the initial map to update the feature point map.

[0130] It should be noted that constructing a feature point map based on the front view image in the first vehicle surrounding environment image is only exemplary. It is also possible to construct a feature point map using the left view image, right view image, or rear view image in the first vehicle surrounding environment image. The specific construction method is the same as the above method and will not be elaborated here.

[0131] Step 403: Determine the parking space semantic feature and the charging pile object feature according to the first vehicle surrounding environment image.

[0132] In the embodiments of the present application, the parking space semantic features include, but are not limited to, parking space line semantic features, corner point semantic features, and limit plate semantic features, and the charging pile object features include charging pile semantic features. Determining the parking space semantic features and the charging pile object features means labeling the pixels in the surrounding environment image of the first vehicle, and taking the pixels labeled with the same label as the same type of features.

[0133] Specifically, semantic segmentation is performed on the surrounding environment image of the first vehicle, so that each pixel in the surrounding environment image of the first vehicle corresponds to a classification label. In the embodiments of the present application, the classification labels include parking space line semantic features, corner point semantic features, limit plate semantic features, and charging pile semantic features. The pixels corresponding to the parking space line semantic features, the pixels corresponding to the corner point semantic features, and the pixels corresponding to the limit plate semantic features are used as the parking space semantic features.

[0134] In some optional embodiments, when performing semantic segmentation on the surrounding environment image of the first vehicle, methods such as a fully convolutional neural network (FCN) that replaces the fully connected layer of a traditional convolutional neural network with a convolutional layer, a UNet network based on an encoder-decoder structure, and a Deeplab network using pyramid atrous pooling and fully connected conditional random field (CRF) can be used to achieve image semantic segmentation.

[0135] The specific way to determine the charging pile object features is to set the pixel value of the pixels corresponding to the charging pile semantic features to 1, set the pixel values of the pixels except those corresponding to the charging pile semantic features to 0, and take the pixels with a pixel value of 1 and adjacent pixel positions as charging pile pixels to form the charging pile object features.

[0136] In some optional embodiments, the connected component analysis (CCA) method can be used to divide the pixel set in the semantic segmentation image into image regions composed of pixels with the same label and adjacent positions to determine the charging pile objects in the surrounding environment image of the first vehicle.

[0137] In some other optional embodiments, the seed filling method can be used to achieve the division of the image region. Specifically, the image after semantic segmentation is binarized and divided into charging pile pixels and non-charging pile pixels. Let B(charging pile pixels)=1 and B(non-charging pile pixels)=0. Scan the image until the current pixel point B is 1. Take this pixel point as the seed and set label as the charging pile. According to the two conditions of the connected region (same pixel value, adjacent positions), merge all the pixels adjacent to this pixel into the same pixel set. Finally, this pixel set is obtained as a connected region. Repeat the above steps until the scanning ends, and all the charging pile regions in the image can be obtained.

[0138] In some other alternative embodiments, the semantically segmented image is subjected to binary classification into charging pile pixels and non-charging pile pixels, and the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm is used to divide the pixel set in the semantically segmented image into two classes composed of pixels with the same label, thereby determining the charging pile object features in the image.

[0139] It should be noted that the DBSCAN algorithm assumes that the category can be determined by the tightness of the sample distribution. Samples of the same category should be closely connected. By classifying samples that are density-connected into one category, a clustering category is obtained. Then, by classifying all samples into multiple different categories that are density-connected, the final clustering result is obtained. Input the sample set, starting from a selected core point, continuously expand into the density-reachable area, so as to obtain a maximized area containing the core point and boundary points. Any two points in the area are density-connected. When the current clustering cluster is generated, then select a core point from the remaining core point set to generate a clustering cluster until the core point set is empty, and output the cluster division.

[0140] Step 404, determine the positions of the parking space semantic features and the charging pile object features in the world coordinate system.

[0141] In some embodiments, to determine the position of the parking space semantic features in the world coordinate system, it can be achieved by means of coordinate transformation. Based on the parameters and calibration parameters of the camera used to collect the images of the surrounding environment of the first vehicle, a top view is generated according to the images of the surrounding environment of the first vehicle. In the pixel coordinate system of the top view, the coordinates of the parking space semantic features are determined, and the coordinates of the parking space semantic features are converted into three-dimensional coordinates in a preset coordinate system, and the three-dimensional coordinates in the preset coordinate system are used as the positions of the parking space semantic features in the preset coordinate system.

[0142] Specifically, first, determine the position of the parking space semantic features in the fisheye camera coordinate system; then, convert the position of the parking space semantic features in the fisheye camera coordinate system into the position in the top view coordinate system; then, convert the position of the parking space semantic features in the top view coordinate system into the position in the virtual camera coordinate system; then, convert the position of the parking space semantic features in the virtual camera coordinate system into the position in the Base-link coordinate system; finally, convert the position of the parking space semantic features in the Base-link coordinate system into the position in the world coordinate system.

[0143] The following introduces each coordinate system involved in combination with specific examples. It should be noted that the coordinate systems involved in the following text are only for more clearly explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application.

[0144] Top - view coordinate system, the coordinate system is located in the vehicle's underbody plane, with the upper - left corner of the image as the origin, the positive direction of the u - axis is horizontally to the right, and the positive direction of the v - axis is vertically downwards.

[0145] Fisheye camera coordinate system (taking the front - view fisheye camera as an example), with the camera optical center as the origin, the x - axis is parallel to the u - axis of the top - view coordinate system, the y - axis is parallel to the v - axis of the top - view coordinate system, that is, the x - axis points to the right of the camera, the y - axis points downwards of the camera, and the z - axis is the camera optical axis pointing forward of the camera and perpendicular to the image plane.

[0146] Virtual camera coordinate system (the virtual camera is a camera assumed to be looking down from above the vehicle center), with the camera optical center as the origin, the x - axis points to the right of the vehicle, the y - axis points to the rear of the vehicle, and the z - axis points downwards of the vehicle.

[0147] Base - link coordinate system, with the center of the vehicle's rear axle as the origin, the x - axis points to the front of the vehicle, the y - axis points to the left of the vehicle, and the z - axis points upwards of the vehicle.

[0148] World coordinate system, with the center of the rear axle at the initial position of the vehicle as the origin, the x - axis points to the front of the vehicle, the y - axis points to the left of the vehicle, and the z - axis points upwards of the vehicle.

[0149] Converting the position in the fisheye camera coordinate system to the position in the top - view coordinate system:

[0150] Before determining the position of the parking space semantic features in the fisheye camera coordinate system, it is necessary to clarify the parameters of the fisheye camera and the calibration parameters. The parameters of the fisheye camera are: the internal parameters of each fisheye camera after rasterization of the front - view fisheye camera, rear - view fisheye camera, left - view fisheye camera, and right - view fisheye camera respectively. Before introducing the calibration parameters, the concept of the top - view needs to be introduced. The top - view is the image generated by the inverse perspective projection of the observations of multiple fisheye cameras on the vehicle at a certain moment.

[0151] In the embodiment of the present application, the method of calibration parameters is as follows:

[0152] During calibration, the vehicle is stationary on a flat road surface, and a calibration board is used to directly obtain the correspondence from the fisheye camera image to the top - view coordinate system, and calculate the mapping relationship between the pixels of the original image of each fisheye camera and the top - view image.

[0153] The mapping relationship between the pixels of the original image of each fisheye camera and the top - view image can be expressed as follows:

[0154]

[0155] where, (x p , y p ) represents the coordinates of the top - view, (x f , y f) represents the coordinates of the fish-eye camera, and f represents the camera focal length.

[0156] Eight unknown parameters can be calculated from the calibrated 4 pairs of feature points, thereby obtaining the homography matrix

[0157]

[0158] The top-down transformation is completed using the homography matrix.

[0159] It can be seen that the position of the parking space semantic feature in the fish-eye camera coordinate system can be determined by the parameters of the fish-eye camera. Through the above top-down transformation, the position of the parking space semantic feature in the fish-eye camera coordinate system can be converted into the position of the parking space semantic feature in the top view coordinate system.

[0160] It should be noted that to convert the position of the parking space semantic feature from the top view coordinate system to the world coordinate system, existing conversion methods can be used, and the embodiments of the present application do not limit this.

[0161] In some other embodiments, determining the position of the charging pile object feature in the world coordinate can also be achieved through coordinate system conversion. In the first vehicle surrounding environment image, the ORB feature points corresponding to the charging pile object feature are determined, the three-dimensional coordinates of the charging pile are determined according to the three-dimensional coordinates of the ORB feature points corresponding to the charging pile object feature, and the three-dimensional coordinates of the charging pile are converted into the three-dimensional coordinates in the preset coordinate system, and the three-dimensional coordinates in the preset coordinate system are used as the position of the charging pile object feature in the preset coordinate system.

[0162] In the top view coordinate system, the three-dimensional coordinate position of the parking space semantic feature pixel points can be determined, and each point is matched and tracked through the ICP algorithm with semantic constraints to calculate the positions between each image, where the parking space lines, corner points, and limit plates are all on the ground and their depth is 0.

[0163] In some optional implementation manners, the semantic constraint algorithm can, through non-linear optimization, solve the rotation matrix R and the translation vector t for the point clouds in two first vehicle surrounding environment images to minimize the result of the error function containing semantic constraints, optimize the minimum error through the Gauss-Newton method, and finally obtain the rotation matrix R and the translation vector t that minimize the error function.

[0164] It should be noted that the coordinate system conversion method is the same as the above conversion method. In the embodiments of the present application, the preset coordinate system is the world coordinate system.

[0165] Step 405, fuse the parking space semantic feature, the charging pile object feature, the position of the parking space semantic feature in the preset coordinate system, and the position of the charging pile object feature in the preset coordinate system into the feature point map to form a map.

[0166] In some embodiments, after fusing the parking space semantic features, charging pile object features, the positions of the parking space semantic features in the world coordinate system, and the positions of the charging pile object features in the world coordinate system into the feature point map to form a map, if the parking space corresponding to the parking space semantic feature does not have a charging pile, it is marked; if the parking space corresponding to the parking space semantic feature has a charging pile, it is marked. For example, it is marked that the parking space does not have a charging pile, and the parking space has a charging pile.

[0167] It should be noted that the constructed map can be a multi-layer map, and the difference between different layers of the map is the different road signs. In the embodiments of the present application, the road signs are parking space lines, corner points, limit plates, and charging piles, and the road signs include their own description information and position information. For example, in the parking space layer map, the road sign is the parking space feature; in the slam layer map, the road sign is the orb feature point.

[0168] Figure 6 The flowchart of another vehicle positioning method according to the embodiments of the present application is shown. This method is applied to a control device arranged on a vehicle or a remote control device for controlling the driving of the vehicle. Refer to Figure 6 , and this method includes the following steps:

[0169] Step 601, during the process of the vehicle moving along a preset trajectory, obtain the surrounding environment image of the second vehicle and the vehicle wheel speedometer data. There is one or more parking spaces on the preset trajectory, and at least some of the parking spaces have charging piles.

[0170] In the embodiments of the present application, the surrounding environment image of the vehicle is obtained by an image acquisition device arranged on the vehicle, and the vehicle wheel speedometer data is obtained by a wheel speedometer arranged on the vehicle. In some alternative embodiments, a fisheye camera is arranged on each of the front and rear bumpers of the vehicle, and a fisheye camera is arranged on the lower edges of the left and right rearview mirrors of the vehicle. The surrounding environment image of the second vehicle can be collected by the four fisheye cameras. In some alternative embodiments, a wheel speedometer is arranged at each of the four wheels of the vehicle to collect the vehicle wheel speedometer data.

[0171] It should be noted that the preset trajectory is the route for the vehicle to load and unload goods, which can be pre-arranged in the control device, and the vehicle is controlled to drive along the route for loading and unloading goods through the surrounding environment image of the second vehicle and the vehicle wheel speedometer data.

[0172] Continue to refer to Figure 1 , assuming that the vehicle is located in the positive direction of the entrance, estimate the vehicle pose according to the surrounding environment image of the second vehicle between adjacent frames, and perform real-time positioning using the pre-constructed map. During the movement of the vehicle, match the features in the surrounding environment image of the second vehicle extracted with the features in the pre-constructed map to obtain the difference between the two features, so as to update the vehicle pose to achieve vehicle positioning.

[0173] In some alternative embodiments, the extended Kalman filter method can be used to update the pose of the vehicle to achieve vehicle positioning. Specifically, first, the pose of the vehicle is predicted according to the vehicle motion model. Then, the image of the vehicle surrounding environment collected is matched with the pre-constructed map. Finally, the image that the vehicle should observe is calculated based on the predicted vehicle pose and the matched features, and the pose of the vehicle is updated using the difference between the image that should be observed and the actually collected second vehicle surrounding environment image.

[0174] In some alternative embodiments, a vehicle kinematic model is established. For a given loading and unloading route, it is used to describe the motion trajectory of the reference vehicle. PID feedback control is adopted to control the steering wheel angle and the forward speed, so that the vehicle can drive along the loading and unloading route.

[0175] It should be noted that if the vehicle does not need to be charged, it continues to drive along the loading and unloading route. If the vehicle needs to be charged, step 602 is executed.

[0176] Step 602, when the vehicle needs to be charged, according to the second vehicle surrounding environment image, the vehicle wheel speedometer data, and the pre-constructed map, control the vehicle to drive into a parking space with a charging pile and not occupied, and align the vehicle charging port with the charging pile.

[0177] In the embodiments of the present application, if the vehicle needs to be charged, first, it is necessary to query whether there is a charging pile in the parking space located on the vehicle driving trajectory according to the second vehicle surrounding environment image and the pre-constructed map. When there is a charging pile, it is necessary to detect whether the parking space is occupied by other vehicles.

[0178] In some alternative embodiments, the Faster RCNN network is used to extract the features of the vehicle surrounding environment image. Combining semantic, context information, position prior information, and target shape prior information, etc., the energy loss function of the detection frame is calculated, and an accurate target detection frame is extracted to determine the parking space.

[0179] After the parking space is determined, query the map and detect whether the parking space can be charged. In some embodiments, it can be determined whether there is a charging pile in the parking space by querying the mark in the map. If there is a charging pile in the parking space, it is initially determined that the parking space can be used for charging. Further, it is detected whether the parking space is occupied. If it is not occupied, it means that the parking space can be used for charging.

[0180] In some embodiments, controlling the vehicle to drive into a parking space with a charging pile and not occupied and aligning the vehicle charging port with the charging pile according to the second vehicle surrounding environment image, the vehicle wheel speedometer data, and the pre-constructed map includes the following steps:

[0181] Step 6021, when the parking space has a charging pile and is unoccupied, based on the parking space semantic features in the pre-constructed map and the positions of the parking space semantic features in the world coordinate system, control the vehicle to drive into the parking space with a charging pile and unoccupied according to the surrounding environment image of the second vehicle and the vehicle wheel speedometer data.

[0182] In the embodiment of the present application, the pre-constructed map includes parking space semantic features, charging pile object features, the positions of the parking space semantic features in the world coordinate system, and the positions of the charging pile object features in the world coordinate system. The parking space semantic features include parking space line semantic features, corner point semantic features, and limit plate semantic features.

[0183] In the embodiment of the present application, control the vehicle to drive into the parking space with a charging pile and unoccupied. First, detect and identify the parking space, the parking space line of the parking space, the corner points of the parking space, and the limit plate of the parking space according to the surrounding environment image of the vehicle.

[0184] Then, position the vehicle according to the parking space line of the parking space, the corner points of the parking space, the limit plate of the parking space, the parking space semantic features, and the positions of the parking space semantic features in the world coordinate system. In some alternative embodiments, based on the constructed map, the registration of point clouds is realized through the ICP algorithm with semantic constraints. The camera pose is optimized by the change of the paired point cloud coordinate positions after registration to estimate the camera pose and realize vehicle positioning. Then, based on the semantic and position information of the points on the map, the positioning of the vehicle relative to the current parking space is further calculated.

[0185] Finally, based on the positioning of the vehicle, control the vehicle to drive into the parking space with a charging pile and unoccupied according to the vehicle wheel speedometer data. In some alternative embodiments, see Figure 7 , the RRT algorithm can be used to plan the path of the vehicle. There is a starting point in the environment, and a point is randomly scattered. If the point is in the drivable area, connect the starting point and the point, and the line segment between the two points forms the simplest tree. Continue to repeat scattering points in the environment and judge whether to add them to the existing tree until the target point is added to the tree, and a path from the starting point to the target point can be found. Specifically, a controller based on PID feedback control can be used to control the steering wheel angle and the forward speed, realize the lateral control of the vehicle according to the angle difference α and β and the distance dx, and realize the longitudinal control of the vehicle according to the distance difference dx.

[0186] Step 6022, based on the charging pile object features in the pre-constructed map and the positions of the charging pile object features in the world coordinate system, control the vehicle according to the surrounding environment image of the second vehicle and the vehicle wheel speedometer data so that the vehicle charging port is aligned with the charging pile.

[0187] In the embodiments of the present application, the vehicle is controlled to align the vehicle charging port with the charging pile. First, the charging pile is detected and recognized based on the image of the vehicle surrounding environment. In some alternative embodiments, the charging pile is determined by means of object detection.

[0188] Then, the vehicle is positioned according to the detected and recognized charging pile, the charging pile object features, and the position of the charging pile object features in the world coordinate system. In some alternative embodiments, the ORB features of the charging pile images in adjacent frames are detected, the ORB features of the charging pile images in adjacent frames are matched, and the vehicle pose change is calculated according to the corresponding relationship of the feature points between the charging pile images in adjacent frames to achieve the positioning of the vehicle relative to the charging pile.

[0189] Then, a virtual charging parking space is generated according to the position of the charging pile object features in the world coordinate system. In the embodiments of the present application, refer to Figure 8 , the size of the virtual charging parking space can be set according to the body sizes of different vehicle models, which is greater than or equal to the body size (5 cm more than the body on each side at the maximum size) and the position of the charging port is calibrated. When generating the virtual charging parking space, the position of the charging port of the virtual charging parking space is aligned with the charging pile, and the virtual charging parking space is kept parallel to the real parking space line.

[0190] In some alternative embodiments, assuming the vehicle length is l, the width is w, the charging pile coordinates in the virtual camera coordinate system are (x, y, z), the horizontal distance from the vehicle charging port to the rear center of the vehicle is Δx, and the maximum distance that the charging device can protrude from the charging pile is Δy, then the size of the virtual charging parking space is (l + 10) * (w + 10), the center coordinates of the virtual charging parking space are (x + Δx, y - Δy - l / 2, 0), and the coordinates of the lower left corner point of the virtual charging parking space are (x + Δx - w / 2, y - Δy, 0). Exemplarily, the setting method of the generated virtual charging parking space is as shown in Figure 9 shown.

[0191] In some alternative embodiments, if the charging pile is relatively smooth and has fewer feature points thereon, it is necessary to sample the contour line of the charging pile, perform triangulation on each sampling point based on the movement of the camera, and determine the depth of the point by observing the angle of the same point at two locations. Specifically, assume a spatial point Q. During the movement of the camera, an image I1 is transformed to an image I2 through a transformation matrix T. There is a feature point Q1 in I1 and a feature point Q2 in I2. These two feature points are the corresponding points of the three-dimensional spatial point Q. x1 and x2 are the normalized coordinates of the two feature points, and s1 and s2 are the depths of the two feature points. According to the definition in epipolar geometry, we have s1x1 = s2Rx2 + t. Multiply both sides of the above equation on the left by an x1^, we get s1x1^x1 = 0 = s2x1^Rx2 + x1^t. The right side can be regarded as an equation of s2, and s2 can be directly obtained. Subsequently, s1 can be obtained, and the depth of the point in two frames can be obtained, and then the spatial coordinates of point Q can be determined. Calculate the coordinate average value of each sampling point including both edges. Take (X, Y, Z) as the geometric center of the charging pile object, and thus the spatial point position of the charging pile object can be obtained.

[0192] Finally, based on the positioning of the vehicle, control the vehicle to drive into the virtual charging space according to the vehicle wheel speed data so that the charging port of the vehicle is aligned with the charging port of the virtual charging space.

[0193] According to the embodiments of the present application, during the movement of the vehicle along a preset trajectory, obtain the surrounding environment image of the second vehicle and the vehicle wheel speed data. When the vehicle needs to be charged, according to the surrounding environment image of the second vehicle, the vehicle wheel speed data, and the pre-constructed map, control the vehicle to drive into a parking space with a charging pile and not occupied and make the charging port of the vehicle aligned with the charging pile, thereby improving the accuracy of aligning the charging port of the vehicle with the charging pile.

[0194] Figure 10 The flowchart of another vehicle positioning method according to the embodiments of the present application is shown. Refer to Figure 10 This method includes the following steps:

[0195] Step 1001, obtain the surrounding environment image of the first vehicle when the vehicle moves along a preset trajectory.

[0196] Step 1002, extract ORB feature points from the surrounding environment image of the first vehicle, and construct a feature point map based on the extracted ORB feature points.

[0197] Step 1003, determine the parking space semantic feature and the charging pile object feature according to the surrounding environment image of the first vehicle. The charging pile object feature includes the charging pile semantic feature.

[0198] Step 1004, determine the position of the parking space semantic feature in the world coordinate system and the position of the charging pile object feature in the world coordinate system.

[0199] Step 1005: Integrate the parking space semantic features, the charging pile object features, the positions of the parking space semantic features in the world coordinate system, and the positions of the charging pile object features in the world coordinate system into the feature point map to form a map.

[0200] Step 1006: During the process of the vehicle moving along a preset trajectory, obtain the surrounding environment image of the second vehicle and the vehicle wheel speedometer data. There is one or more parking spaces on the preset trajectory, and at least some of the parking spaces are equipped with charging piles.

[0201] Step 1007: When the vehicle needs to be charged, control the vehicle to drive into a parking space equipped with a charging pile and unoccupied, and align the vehicle's charging port with the charging pile according to the surrounding environment image of the second vehicle, the vehicle wheel speedometer data, and the constructed map.

[0202] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0203] The above is the introduction of the method embodiments. The following further illustrates the solution of the present application through device embodiments.

[0204] Figure 11 The block diagram of the map construction device according to the embodiment of the present application is shown. This device is applied to a control device arranged on a vehicle or a remote control device for controlling the vehicle to travel, or can be implemented as a control device arranged on a vehicle or a remote control device for controlling the vehicle to travel. Refer to Figure 11 As shown, this device includes a first acquisition module 1101, a map construction module 1102, a feature determination module 1103, a position determination module 1104, and a map fusion module 1105.

[0205] The first acquisition module 1101 is configured to acquire the surrounding environment image of the first vehicle when the vehicle moves along a preset trajectory.

[0206] The map construction module 1102 is configured to extract ORB feature points from the surrounding environment image of the first vehicle, and construct a feature point map based on the extracted ORB feature points.

[0207] The feature determination module 1103 is configured to determine the parking space semantic features and the charging pile object features according to the surrounding environment image of the first vehicle. The charging pile object features include charging pile semantic features.

[0208] A position determination module 1104 is configured to determine the positions of the parking space semantic features and the charging pile object features in the world coordinate system.

[0209] A map fusion module 1105 is configured to fuse the parking space semantic features, the charging pile object features, the positions of the parking space semantic features in the world coordinate system, and the positions of the charging pile object features in the world coordinate system into the feature point map to form a map.

[0210] Figure 12 The block diagram of a vehicle positioning device according to an embodiment of the present application is shown. The device is applied to a control device arranged on a vehicle or a remote control device for controlling the driving of the vehicle, or can be implemented as a control device arranged on a vehicle or a remote control device for controlling the driving of the vehicle. Refer to Figure 12 This device includes a second acquisition module 1201, a third acquisition module 1202, and a vehicle control module 1203.

[0211] The second acquisition module 1201 is configured to acquire an image of the vehicle surrounding environment during the movement of the vehicle along a preset trajectory. One or more parking spaces are provided on the preset trajectory, and at least some of the parking spaces are equipped with charging piles.

[0212] The third acquisition module 1202 is configured to acquire vehicle wheel speedometer data during the movement of the vehicle along a preset trajectory.

[0213] The vehicle control module 1203 is configured to, when the vehicle needs to be charged, control the vehicle to drive into a parking space with a charging pile and not occupied and align the vehicle charging port with the charging pile according to the vehicle surrounding environment image, the vehicle wheel speedometer data, and a pre-constructed map.

[0214] In some embodiments, the vehicle control module 1102 is specifically configured to:

[0215] When the parking space has a charging pile and is not occupied, based on the parking space semantic features and the positions of the parking space semantic features in the world coordinate system in the pre-constructed map, control the vehicle to drive into a parking space with a charging pile and not occupied according to the vehicle surrounding environment image and the vehicle wheel speedometer data;

[0216] Based on the charging pile object features and the positions of the charging pile object features in the world coordinate system in the pre-constructed map, control the vehicle to align the vehicle charging port with the charging pile according to the vehicle surrounding environment image and the vehicle wheel speedometer data.

[0217] In some embodiments, the vehicle control module 1102 is specifically further configured to:

[0218] Detect and identify parking spaces, parking space lines, corner points of parking spaces, and limit plates of parking spaces according to the vehicle surrounding environment image.

[0219] Position the vehicle according to the parking space line of the parking space, the corner points of the parking space, the limit plates of the parking space, the parking space semantic features, and the positions of the parking space semantic features in the world coordinate system. The parking space semantic features include parking space line semantic features, corner point semantic features, and limit plate semantic features;

[0220] Based on the positioning of the vehicle, control the vehicle to drive into a parking space with a charging pile and not occupied according to the vehicle wheel speedometer data.

[0221] In some embodiments, the vehicle control module 1102 is further specifically configured to:

[0222] Detect and identify a charging pile according to the image of the vehicle surrounding environment;

[0223] Position the vehicle according to the detected and identified charging pile, the charging pile object features, and the positions of the charging pile object features in the world coordinate system;

[0224] Generate a virtual charging parking space according to the position of the charging pile object features in the world coordinate system. The virtual charging parking space has a charging port, and the charging port of the virtual charging parking space is aligned with the charging pile, and the size of the virtual charging parking space is greater than or equal to the vehicle body size;

[0225] Based on the positioning of the vehicle, control the vehicle to drive into the virtual charging parking space according to the vehicle wheel speedometer data so that the vehicle charging port is aligned with the charging port of the virtual charging parking space.

[0226] In some embodiments, the device further includes a map construction module, and the map construction module is specifically configured to:

[0227] Obtain the first vehicle surrounding environment image when the vehicle moves along a preset trajectory;

[0228] Extract ORB feature points from the first vehicle surrounding environment image, and construct a feature point map based on the extracted ORB feature points;

[0229] Determine the parking space semantic features and the charging pile object features according to the first vehicle surrounding environment image. The charging pile object features include charging pile semantic features;

[0230] Determine the positions of the parking space semantic features and the charging pile object features in the world coordinate system;

[0231] Fuse the parking space semantic features, the charging pile object features, the positions of the parking space semantic features in the world coordinate system, and the positions of the charging pile object features in the world coordinate system into the feature point map to form a map.

[0232] In some embodiments, the map construction module is further specifically configured to:

[0233] Generate an initial map based on two frames of the first vehicle's surrounding environment image;

[0234] Generate new feature points in the initial map based on each pair of matching ORB feature points in the two frames of the first vehicle's surrounding environment image to form the feature point map.

[0235] In some embodiments, the map construction module is further specifically configured to:

[0236] When the current frame image of the first vehicle's surrounding environment image moves a certain distance from the last key frame image, add the current frame image to the initial map to update the feature point map.

[0237] In some embodiments, the map construction module is further specifically configured to:

[0238] Perform semantic segmentation on the first vehicle's surrounding environment image so that each pixel in the first vehicle's surrounding environment image corresponds to a classification label, and the classification labels include the parking space line semantic feature, the corner point semantic feature, the limit plate semantic feature, and the charging pile semantic feature;

[0239] Determine the pixels corresponding to the parking space line semantic feature, the corner point semantic feature, and the limit plate semantic feature as the parking space semantic feature;

[0240] Determine the charging pile object feature according to the pixels corresponding to the charging pile semantic feature.

[0241] In some embodiments, the map construction module is further specifically configured to:

[0242] Set the pixel value of the pixels corresponding to the charging pile semantic feature to 1, and set the pixel values of the remaining pixels except the pixels corresponding to the charging pile semantic feature to 0;

[0243] Take the pixels with a pixel value of 1 and adjacent pixel positions as charging pile pixels to form the charging pile object feature.

[0244] In some embodiments, the map construction module is further specifically configured to:

[0245] Based on the parameters and calibration parameters of the camera used to collect the first vehicle's surrounding environment image, generate a top view according to the first vehicle's surrounding environment image;

[0246] Determine the coordinates of the parking space semantic feature in the pixel coordinate system of the top view;

[0247] Convert the coordinates of the parking space semantic feature into three-dimensional coordinates in the world coordinate system, and use the three-dimensional coordinates in the world coordinate system as the position of the parking space semantic feature in the world coordinate system.

[0248] In some embodiments, the map construction module is further specifically configured to:

[0249] In the surrounding environment image of the first vehicle, determine the ORB feature points corresponding to the charging pile object feature;

[0250] Determine the three-dimensional coordinates of the charging pile according to the three-dimensional coordinates of the ORB feature points corresponding to the charging pile object feature;

[0251] Convert the three-dimensional coordinates of the charging pile into three-dimensional coordinates in the world coordinate system, and use the three-dimensional coordinates in the world coordinate system as the position of the charging pile object feature in the world coordinate system.

[0252] In some embodiments, the map construction module is further specifically configured to:

[0253] If the parking space corresponding to the parking space semantic feature does not have a charging pile, mark it;

[0254] If the parking space corresponding to the parking space semantic feature has a charging pile, mark it.

[0255] It should be noted that when determining the road capacity, the road capacity determination device provided in the above embodiments only takes the division of the above functional modules as an example. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the road capacity determination device provided in the above embodiments and the road capacity determination method embodiments belong to the same concept. For the specific implementation process, please refer to the method embodiments, which will not be elaborated here.

[0256] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a digital versatile disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)). It should be noted that the computer-readable storage medium mentioned in the embodiments of the present application can be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0257] It should be understood that the "at least one" mentioned herein refers to one or more, and "a plurality" refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B can mean A or B; the "and / or" herein is merely a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily limit to be different.

[0258] The above are the exemplary embodiments provided by the present application, which are not intended to limit the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the embodiments of the present application shall be included in the protection scope of the present application.

Claims

1. A vehicle positioning method, characterized in that, including: During the movement of the vehicle along a preset trajectory, obtain the surrounding environment image of the second vehicle and the vehicle wheel speedometer data. There are one or more parking spaces on the preset trajectory, and at least some of the parking spaces are equipped with charging piles. When the vehicle needs to be charged, according to the surrounding environment image of the second vehicle, the vehicle wheel speedometer data, and a pre-constructed map, control the vehicle to drive into a parking space equipped with a charging pile and unoccupied, and align the vehicle charging port with the charging pile. The controlling the vehicle to drive into a parking space equipped with a charging pile and unoccupied and align the vehicle charging port with the charging pile according to the surrounding environment image of the second vehicle, the vehicle wheel speedometer data, and a pre-constructed map includes: When the parking space is equipped with a charging pile and unoccupied, based on the parking space semantic features in the pre-constructed map and the position of the parking space semantic features in the world coordinate system, control the vehicle to drive into the parking space equipped with a charging pile and unoccupied according to the surrounding environment image of the second vehicle and the vehicle wheel speedometer data. Based on the charging pile object features in the pre-constructed map and the position of the charging pile object features in the world coordinate system, control the vehicle according to the surrounding environment image of the second vehicle and the vehicle wheel speedometer data to align the vehicle charging port with the charging pile. The controlling the vehicle to drive into the parking space equipped with a charging pile and unoccupied based on the parking space semantic features in the pre-constructed map and the position of the parking space semantic features in the world coordinate system according to the surrounding environment image of the second vehicle and the vehicle wheel speedometer data includes: Detect and identify the parking space, the parking space lines, the corner points of the parking space, and the limit plates of the parking space according to the surrounding environment image of the second vehicle. Locate the vehicle according to the parking space lines, the corner points of the parking space, the limit plates of the parking space, the parking space semantic features, and the position of the parking space semantic features in the world coordinate system. The parking space semantic features include parking space line semantic features, corner point semantic features, and limit plate semantic features. Based on the vehicle positioning, control the vehicle to drive into the parking space equipped with a charging pile and unoccupied according to the vehicle wheel speedometer data. The controlling the vehicle to align the vehicle charging port with the charging pile based on the charging pile object features in the pre-constructed map and the position of the charging pile object features in the world coordinate system according to the surrounding environment image of the second vehicle and the vehicle wheel speedometer data includes: Detect and identify the charging pile according to the surrounding environment image of the second vehicle. Locate the vehicle according to the detected and identified charging pile, the charging pile object features, and the position of the charging pile object features in the world coordinate system. Generate a virtual charging parking space according to the position of the charging pile object features in the world coordinate system. The size of the virtual charging parking space is greater than or equal to the vehicle body size. Based on the vehicle positioning, control the vehicle to drive into the virtual charging parking space so that the vehicle charging port is aligned with the charging port of the virtual charging parking space according to the vehicle wheel speedometer data.

2. The method according to claim 1, wherein The method for constructing a map includes: Obtain the surrounding environment image of the first vehicle when the vehicle moves along a preset trajectory. Extract ORB feature points from the surrounding environment image of the first vehicle, and construct a feature point map based on the extracted ORB feature points. Determine the semantic features of the parking space and the feature of the charging pile object according to the image of the surrounding environment of the first vehicle, where the feature of the charging pile object includes the semantic feature of the charging pile; Determine the positions of the semantic features of the parking space and the feature of the charging pile object in a preset coordinate system; Fuse the semantic features of the parking space, the feature of the charging pile object, the position of the semantic features of the parking space in the preset coordinate system, and the position of the feature of the charging pile object in the preset coordinate system into the feature point map to form a map.

3. The method according to claim 2, wherein Constructing a feature point map based on the extracted ORB feature points includes: Generate an initial map according to two frames of images of the surrounding environment image of the first vehicle; Generate new feature points in the initial map according to each pair of matching ORB feature points in two frames of images of the surrounding environment image of the first vehicle to form the feature point map.

4. The method according to claim 3, characterized in that, The constructing of the feature point map based on the extracted ORB feature points further includes: When there is a certain distance of movement between the current frame image and the last key frame image of the surrounding environment image of the first vehicle, add the current frame image to the initial map to update the feature point map.

5. The method according to claim 2, wherein The determining the semantic features of the parking space and the feature of the charging pile object according to the image of the surrounding environment of the first vehicle includes: Perform semantic segmentation on the image of the surrounding environment of the first vehicle so that each pixel in the image of the surrounding environment of the first vehicle corresponds to a classification label, and the classification label includes the semantic feature of the parking space line, the semantic feature of the corner point, the semantic feature of the limit plate, and the semantic feature of the charging pile; Determine the pixels corresponding to the semantic feature of the parking space line, the pixels corresponding to the semantic feature of the corner point, and the pixels corresponding to the semantic feature of the limit plate as the semantic features of the parking space; Determine the feature of the charging pile object according to the pixels corresponding to the semantic feature of the charging pile.

6. The method according to claim 5, wherein The determining the feature of the charging pile object according to the pixels corresponding to the semantic feature of the charging pile includes: Set the pixel value of the pixels corresponding to the semantic feature of the charging pile to 1, and set the pixel values of the remaining pixels except the pixels corresponding to the semantic feature of the charging pile to 0; Take the pixels with a pixel value of 1 and adjacent pixel positions as charging pile pixels to form the feature of the charging pile object.

7. The method according to claim 2, characterized in that, Determine the position of the semantic features of the parking space in the world coordinate system includes: Based on the parameters of the camera used to collect the image of the surrounding environment of the first vehicle, generate a top view according to the image of the surrounding environment of the first vehicle; Determine the coordinates of the semantic features of the parking space in the pixel coordinate system of the top view; Convert the coordinates of the semantic features of the parking space into three-dimensional coordinates in the preset coordinate system, and use the three-dimensional coordinates in the preset coordinate system as the position of the semantic features of the parking space in the preset coordinate system.

8. The method according to claim 2, wherein Determine the position of the feature of the charging pile object in the world coordinate system includes: In the image of the surrounding environment of the first vehicle, determine the ORB feature points corresponding to the feature of the charging pile object; Determine the three-dimensional coordinates of the charging pile according to the three-dimensional coordinates of the ORB feature points corresponding to the feature of the charging pile object; Convert the three-dimensional coordinates of the charging pile into three-dimensional coordinates in a preset coordinate system, and use the three-dimensional coordinates in the preset coordinate system as the position of the charging pile object feature in the preset coordinate system.

9. The method according to claim 2, wherein The step of fusing the parking space semantic feature, the charging pile object feature, the position of the parking space semantic feature in the world coordinate system, and the position of the charging pile object feature in the world coordinate system into the feature point map to form a map further includes: If the parking space corresponding to the parking space semantic feature does not have a charging pile, mark it as that the parking space does not have a charging pile; if the parking space corresponding to the parking space semantic feature has a charging pile, mark it as that the parking space has a charging pile.

10. A vehicle positioning device, characterized in that, It includes: A second acquisition module, configured to acquire an image of the environment around the second vehicle during the movement of the vehicle along a preset trajectory, where there is one or more parking spaces on the preset trajectory, and at least some of the parking spaces have charging piles; A third acquisition module, configured to acquire vehicle wheel speed meter data during the movement of the vehicle along a preset trajectory; A vehicle control module, configured to, when the vehicle needs to be charged, control the vehicle to drive into a parking space with a charging pile and not occupied and align the vehicle charging port with the charging pile according to the image of the environment around the second vehicle, the vehicle wheel speed meter data, and a pre-constructed map; The step of controlling the vehicle to drive into a parking space with a charging pile and not occupied and align the vehicle charging port with the charging pile according to the image of the environment around the second vehicle, the vehicle wheel speed meter data, and a pre-constructed map includes: When the parking space has a charging pile and is not occupied, based on the parking space semantic feature and the position of the parking space semantic feature in the world coordinate system in the pre-constructed map, control the vehicle to drive into a parking space with a charging pile and not occupied according to the image of the environment around the second vehicle and the vehicle wheel speed meter data; Based on the charging pile object feature and the position of the charging pile object feature in the world coordinate system in the pre-constructed map, control the vehicle to align the vehicle charging port with the charging pile according to the image of the environment around the second vehicle and the vehicle wheel speed meter data; The step of controlling the vehicle to drive into a parking space with a charging pile and not occupied based on the parking space semantic feature and the position of the parking space semantic feature in the world coordinate system in the pre-constructed map according to the image of the environment around the second vehicle and the vehicle wheel speed meter data includes: Detect and identify the parking space, the parking space lines, the corner points of the parking space, and the limit plates of the parking space according to the image of the environment around the second vehicle; Locate the vehicle according to the parking space lines, the corner points of the parking space, the limit plates of the parking space, the parking space semantic feature, and the position of the parking space semantic feature in the world coordinate system, where the parking space semantic feature includes a parking space line semantic feature, a corner point semantic feature, and a limit plate semantic feature; Based on the positioning of the vehicle, control the vehicle to drive into a parking space with a charging pile and not occupied according to the vehicle wheel speed meter data; The step of controlling the vehicle to align the vehicle charging port with the charging pile based on the charging pile object feature and the position of the charging pile object feature in the world coordinate system in the pre-constructed map according to the image of the environment around the second vehicle and the vehicle wheel speed meter data includes: Detect and identify a charging pile based on the surrounding environment image of the second vehicle; Locate the vehicle according to the detected and identified charging pile, the object characteristics of the charging pile, and the position of the object characteristics of the charging pile in the world coordinate system; Generate a virtual charging space based on the position of the object characteristics of the charging pile in the world coordinate system, and the size of the virtual charging space is greater than or equal to the size of the vehicle body; Based on the positioning of the vehicle, control the vehicle to drive into the virtual charging space according to the vehicle wheel speed data so that the charging port of the vehicle is aligned with the charging port of the virtual charging space.

11. An electronic device, comprising a memory and a processor, wherein a computer program is stored on the memory, characterized in that, When the processor executes the computer program, the method described in any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the method described in any one of claims 1 to 9 is implemented.

Images