Parking method and apparatus, and electronic device system and storage medium

Abstract

A parking method and apparatus, and an electronic device system and a storage medium. The parking method comprises: converting image features, which are collected by means of a plurality of image recording devices around a vehicle, into bird's-eye-view (BEV) features of a panoramic BEV; on the basis of the BEV features and in view of a relative position relationship between an obstacle around the vehicle and the vehicle, determining an angular point of the obstacle as a reference point of an obstacle detection frame; on the basis of the reference point, predicting position information of the reference point to two adjacent-side midpoints which are adjacent to the reference point, so as to obtain the size, position and orientation of the obstacle detection frame, wherein the size, position and orientation of the obstacle detection frame are used for reflecting an obstacle area where the obstacle is located; and on the basis of the size, position and orientation of the obstacle detection frame, determining whether to avoid the obstacle area for parking.

Description

Method, device, electronic equipment system and storage medium for parking CROSS-REFERENCE TO RELATED APPLICATIONS

[0000] This application claims priority to the Chinese patent application filed with the China Patent Office on October 20, 2023, with application number 202311367819.7 and invention name “Parking method, device, electronic equipment system and storage medium”, the entire contents of which are incorporated by reference in this application. Technical Field The present application relates to the technical field of parking control, and in particular to a parking method, device, electronic equipment system and storage medium. Background Art In the related art, during the parking process, it is necessary to combine the 3D target detection method to realize the detection of obstacle targets. Generally, the goal of the 3D target detection method is to classify and locate the target of interest. Among them, the one-stage solution in 3D target detection usually directly predicts the position, length, width, height and orientation information of the 3D box. Specifically, the center key point detection head (CenterPoint Head) is usually used as the detection head to output the heat map (heatmap), the length, width and height [I,J,K] (3D size) of the three-dimensional voxel image, the yaw angle (yaw) rotated around the Y axis, and the center height (center height) information to form the 3D box of the target. And the 3D detection task defines the object orientation angle by predicting the sine (sin) and cosine (cos) values ​​of the orientation angle. Summary of the invention The following is a summary of the subject matter described in detail in this application. This summary is not intended to limit the scope of the claims. The present application provides a parking method, device, electronic equipment system and storage medium. The present application provides a parking method, comprising: converting image features collected by multiple image recording devices around a vehicle into bird's-eye view BEV features of a panoramic bird's-eye view; determining, based on the BEV features, the corner points of the obstacles around the vehicle and the relative positional relationship between the vehicle and the obstacles around the vehicle as reference points of an obstacle detection frame; predicting, based on the reference points of the obstacle detection frame, the position information of the reference points to the midpoints of two adjacent sides adjacent to the reference points, and obtaining the size, position and orientation of the obstacle detection frame; and obtaining the size, position and orientation of the obstacle detection frame. The size, position and orientation of the obstacle detection frame are used to reflect the obstacle area where the obstacle is located; and whether to avoid the obstacle area for parking is determined according to the size, position and orientation of the obstacle detection frame. In some embodiments, the position information from the reference point to the midpoints of the two adjacent edges includes an offset vector from the reference point to the midpoints of the two adjacent edges and a specific position of the midpoints of the adjacent edges; predicting the position information from the reference point to the midpoints of the two adjacent edges adjacent to the reference point based on the reference point to obtain the size, position and orientation of the obstacle detection frame includes: inputting the reference point into a detection frame generation model to output the offset vector from the reference point to the midpoints of the two adjacent edges and the positions of the two midpoints of the adjacent edges of the obstacle detection frame; the detection frame generation model is obtained by training a training sample set containing the reference point; and obtaining the size, position and orientation of the obstacle detection frame based on the offset vector from the reference point to the midpoints of the two adjacent edges and the positions of the two midpoints of the adjacent edges. In some embodiments, according to the bev feature, by combining the relative position relationship between the obstacles around the vehicle and the vehicle, determining the corner point of the obstacle as a reference point of the obstacle detection frame, including: according to the bev feature, by combining the relative position relationship between the obstacle and the vehicle, determining the corner point of the obstacle closest to the vehicle as a reference point of the obstacle detection frame. In some embodiments, the corner points of the obstacles around the vehicle are determined as reference points of the obstacle detection frame based on the bev features through the relative position relationship between the obstacles around the vehicle and the vehicle, including: performing bev sampling on image features according to multiple preset heights to obtain bev features at the multiple preset heights; compressing and fusing the bev features at the multiple preset heights according to the height dimension to obtain bev features with height information; and determining any corner point of the obstacles around the vehicle according to the bev features with height information as a reference point of the obstacle detection frame. In some embodiments, predicting the position information from the reference point to the midpoints of two adjacent sides adjacent to the reference point based on the reference point to obtain the size, position and orientation of the obstacle detection frame includes: predicting the position information from the reference point to the midpoints of two adjacent sides adjacent to the reference point based on the reference point of the obstacle detection frame to obtain the size, position, orientation of the obstacle detection frame and the obstacle height corresponding to the bev feature with height information; the obstacle height is used to reflect the importance of the impact on parking; the higher the obstacle height, the higher the importance of the impact on parking; determining whether to avoid the obstacle area for parking based on the size, position and orientation of the obstacle detection frame includes: determining whether to avoid the obstacle area for parking based on the size, position, orientation of the obstacle detection frame and the obstacle height. In some embodiments, determining whether to avoid the obstacle area for parking according to the size, position, orientation and height of the obstacle detection frame includes: determining the size, position, orientation and height of the obstacle detection frame according to the size, position, orientation and height of the obstacle ... The method comprises the following steps: determining the position, direction, and height of the obstacle to estimate whether the obstacle collides with the vehicle; if the obstacle does not collide with the vehicle, controlling the vehicle to park without avoiding the obstacle area; and if the obstacle collides with the vehicle, controlling the vehicle to park without avoiding the obstacle area. In some embodiments, the image features collected by multiple image recording devices around the vehicle are converted into a bird's-eye view bev features of a panoramic bird's-eye view, including: acquiring images collected by multiple image recording devices; based on the image features in the images, and according to the position mapping relationship between the bev features and the image features, determining the coordinates of each 3D point in the panoramic bird's-eye view in each image coordinate system as the bev features. The present application provides a parking device, comprising: a feature conversion module, configured to convert image features collected by multiple image recording devices around a vehicle into bird's-eye view (BEV) features of a panoramic bird's-eye view; a reference point determination module, configured to determine, based on the BEV features and the relative positional relationship between the obstacles around the vehicle and the vehicle, the corner points of the obstacles as reference points of an obstacle detection frame; a detection frame determination module, configured to predict, based on the reference point, position information from the reference point to the midpoints of two adjacent edges adjacent to the reference point, to obtain the size, position and orientation of the obstacle detection frame; the size, position and orientation of the obstacle detection frame are used to reflect the obstacle area where the obstacle is located; and a parking determination module, configured to determine, based on the size, position and orientation of the obstacle detection frame, whether to avoid the obstacle area for parking. The present application provides an electronic device, including one or more processors, configured to implement any of the methods described above. The present application provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the method described in any one of the above items is implemented. In some embodiments, the parking method of the present application converts the image features collected by multiple image recording devices around the vehicle into bird's-eye view bev features of a panoramic bird's-eye view; according to the bev features, by combining the relative position relationship between the obstacles around the vehicle and the vehicle, the corner points of the obstacles are determined as reference points of the obstacle detection frame; according to the reference point of the obstacle detection frame, the position information from the reference point to the midpoints of the two adjacent edges adjacent to the reference point is predicted to obtain the size, position and orientation of the obstacle detection frame; the size, position and orientation of the obstacle detection frame are used to reflect the obstacle area where the obstacle is located; according to the size, position and orientation of the obstacle detection frame, it is determined whether to avoid the obstacle area for parking. This method of predicting the position information of the midpoints of the two adjacent edges from the reference point to obtain the size, position and orientation of the obstacle detection frame is similar to reverse thinking. Even if the relative position of the object and the vehicle changes or the object is blocked, the obstacle area where the obstacle is located can be inferred, reducing the impact of being blocked and the change of relative position, thereby improving the accuracy of the obstacle detection frame during parking, making it easier to avoid obstacles during parking, thereby reducing the possibility of collision. Other aspects will be apparent upon reading and understanding the drawings and detailed description. BRIEF DESCRIPTION OF THE DRAWINGS FIG1 is a flow chart of a parking method provided in an embodiment of the present application. FIG. 2 is a schematic diagram showing the parking method shown in FIG. 1 including a corner point and midpoints of two adjacent sides. FIG. 3 is a schematic diagram showing a final result of an obstacle detection frame in the parking method shown in FIG. 1 . FIG. 4 is a schematic diagram showing the structure of a parking device according to an embodiment of the present application. FIG5 is a module block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of the present specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of the present specification as detailed in the appended claims. It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments. During parking, when the relative position of an obstacle target (such as a vehicle) and the vehicle changes, or when the obstacle target is blocked, the accuracy of the orientation angle prediction is low. As a result, the 3D frame of the obstacle is prone to orientation deviation, making parking more prone to collision. In order to solve the technical problem of low accuracy in heading angle prediction when the relative position of an object and the vehicle changes or the object is blocked, an embodiment of the present application provides the following parking method. As shown in FIG. 1 , the parking method may include, but is not limited to, the following steps 110 to 140 . Step 110 , converting image features collected by a plurality of image recording devices around the vehicle into bird's eye view (BEV) features of a panoramic bird's eye view. The image recording device may be used to obtain an image of the vehicle's surroundings. The image recording device may include, but is not limited to, one or more of a camera, a radar, and a camera. Accordingly, the bird's-eye view may be an image of the vehicle's surroundings obtained by using a plurality of image recording devices. The image features may include image coordinates. The bird's-eye view bev feature includes the bev coordinates of each 3D point in the bird's-eye view. Step 120, based on the bev feature, by combining the relative position relationship between the obstacles around the vehicle and the vehicle, determine the corner points of the obstacles as reference points of the obstacle detection frame. The above-mentioned obstacles refer to targets that affect the parking of the vehicle. The above-mentioned obstacles may include, but are not limited to, one or more of people, other vehicles other than the vehicle, and support pillars of the underground parking lot. The corner points of the obstacles around the vehicle in the above step 120 are used to reflect the corner points of the obstacles framed by the obstacle detection frame. The size of these obstacle detection frames is related to the size of the obstacles and can be learned through a neural network. The above obstacle detection box is represented by the category of the obstacle and the bounding box in the 3D world space to reflect each obstacle. In general, the 3D bounding box is parameterized by its position [x, y, z], size [h, w, l] and orientation [θ, φ, ψ] relative to a predefined reference coordinate system (such as the coordinate system of the vehicle recording the data). In most autonomous driving situations, the heading angle θ around the Y axis, also known as the yaw angle, is considered. Among them, the orientation is an absolute orientation. 0 degrees is toward the front. Step 130 , predicting the position information of the reference point to the midpoints of two adjacent edges adjacent to the reference point based on the reference point, and obtaining the size, position and orientation of the obstacle detection frame. The size, position and orientation of the obstacle detection frame are used to reflect the obstacle area where the obstacle is located. Step 140: Determine whether to park in an obstacle-avoiding area based on the size, position, and orientation of the obstacle detection frame. In the embodiment of the present application, the corner points of obstacles around the vehicle are determined as reference points of the obstacle detection frame through the bird's-eye view bev features; based on the reference point, the position information of the reference point to the midpoints of the two adjacent edges adjacent to the reference point is predicted to obtain the size, position and orientation of the obstacle detection frame. This method of predicting the position information of the midpoints of two adjacent edges from the reference point to obtain the size, position and orientation of the obstacle detection frame is similar to reverse thinking. Even if the relative position of the object and the vehicle changes or the object is blocked, the obstacle area where the obstacle is located can be inferred, reducing the impact of being blocked and the change of relative position, thereby improving the accuracy of the obstacle detection frame, making it easier to park without obstacles, and thus reducing the possibility of collision. In the embodiment of the above step 110, images collected by multiple image recording devices around the vehicle are obtained; and based on the image features in the images, the position mapping relationship between the bev features and the image features is determined in the panoramic bird's eye view. The coordinates of each 3D point in the panoramic view in each image coordinate system are used as bev features. In this way, it is convenient to detect obstacles in the 360° panoramic image, and then obtain the obstacle detection area to prevent the car from scratching. Furthermore, the image recording device may include a fisheye camera. Correspondingly, the image coordinate system may include a fisheye image coordinate system. For example, the multiple image recording devices are four fisheye cameras, which may be placed in the front, rear, left, and right directions of the driving car, respectively, to ensure that the camera field of view is sufficient to cover the surroundings of the car, and the fisheye camera is calibrated with a chessboard calibration board to obtain its internal parameters and perform distortion correction. The above position mapping relationship may refer to a mapping table of pixel coordinates (pix coords) corresponding to the center point of the pre-divided grid of the BEV space and the image feature positions of the four fisheye cameras. In the subsequent use process, the images of the four fisheye cameras are obtained, and based on the image feature positions, the center of the target grid pre-divided in the bird's-eye view BEV space corresponding to these image feature positions can be determined through this position mapping table. In this way, the center of the target grid can be used to determine the bird's-eye view BEV features. Furthermore, according to the pix coords mapping table, for each point of the bev feature, the value of the corresponding position is sampled from the four camera image features to obtain the required bev feature. This bev feature is subsequently used as the input of the downstream network task for 3D object detection. Continue to convert the 2D (Two Dimensional) image features into bird's-eye view Bev features: define the correspondence between 3D space and image coordinates through the camera's internal and external parameters, and pre-divide the BEV space into W times H grids. The position mapping table of each grid center point and the four camera images can be calculated by the camera's internal and external parameters. The coordinates of each 3D point in the panoramic bird's-eye view in each fisheye image coordinate system are marked as pixel coordinates (pix coords). The calculation method of pixel coordinates varies with the camera calibration method, and this application does not describe this in detail. Pixel coordinates can be pre-calculated and saved offline in the real-time inference stage. The calculation process of pixel coordinates Pix_coords is as follows: First, the angle theta between the 3D point incident ray and the optical center: Among them, theta is the angle between the incident ray of the 3D point and the optical center, and (x, y, z) is the xyz three-dimensional coordinate position of the 3D point. Second, the fisheye distortion thetad is approximated by an 11th-order polynomial, with distortion parameters a0~a 10 is given by the calibration: thetad = a 10 theta 10 +a9theta 9 +…+a1theta+a0 Third, the coordinates (u, v) of the camera coordinate system are approximated by isometric projection: Fourth, convert the camera coordinate system to the pixel coordinate system through the camera internal and external parameters: P=A.[u,v] T +C Among them, P is the pixel coordinate matrix, A is the scaling matrix from the camera coordinate system to the pixel coordinate system, which is calculated by the camera internal and external parameters, and C is the optical center position matrix. Through the above process, the coordinates of each 3D point in the panoramic bird's-eye view (BEV Plane) in each fisheye image coordinate system can be obtained as pix_coords pixel coordinates, which are used as bev features. FIG. 2 is a schematic diagram showing the parking method shown in FIG. 1 including a corner point and midpoints of two adjacent sides. As shown in FIG. 1 , there are multiple ways to determine the corner points in step 120 as follows. In a determination method as shown in FIG2, based on the bev feature, by combining the relative position relationship between the obstacle and the vehicle, the corner point of the obstacle closest to the vehicle is determined as a reference point. In this way, the corner point of the obstacle closest to the vehicle is determined as a reference point by the relative position relationship between the corner point in the bev feature of the obstacle and the vehicle. In this way, the corner points of the obstacle detection frame can be determined more quickly, and the obstacle detection frame can be quickly generated, so as to facilitate the subsequent framing of the obstacle in the obstacle detection frame. Among them, the above-mentioned obstacle detection box can be a rectangular box. Among the four corner points of the rectangular box of the surrounding vehicles, the point closest to the center of the vehicle has the strongest semantics and the highest precision requirement. This application uses the corner point closest to the vehicle as the reference point, selects the corner point closest to the vehicle among the four corner points projected from the target obstacle to the BEV's perspective as the reference corner point, and predicts the offset vector from the reference point to the midpoints of the two adjacent edges and the specific positions of the midpoints of the two adjacent edges. As shown in Figure 2, the corner point closest to the vehicle among the four corner points of 0bj1 is marked with a circular dot, and the midpoints of the two adjacent edges adjacent to the corner point are marked with a triangle and a square, respectively. In another determination method, based on the bev feature, by combining the relative position relationship between the obstacle and the vehicle, any corner point of the obstacle is determined as the reference point of the obstacle detection frame. Of course, the obstacle height reflected by the above bev feature can include the actual obstacle height, and the obstacle height reflected by the above bev feature can also be the height of continued compression and fusion according to multiple preset heights. Please see below for detailed description. Further, in the embodiment of the above step 120, the image features are bev sampled according to multiple preset heights to obtain bev features at multiple preset heights; the bev features at multiple preset heights are sampled according to the height dimension. The bev features with high information are compressed and fused to obtain the bev features with high information; the corner points of obstacles around the vehicle are determined according to the bev features with high information. In this way, this bev feature is used as the input of the downstream tasks of the network, providing a richer and more accurate feature representation for subsequent 3D perception, which is conducive to improving the perception performance of the model. The embodiment of the present application queries the value of the position corresponding to each camera image feature for each grid of bev according to the pixel coordinate (pix coords) mapping table, samples the corresponding position of the image feature into the bev grid, and after predicting the pixel coordinates (pix coords) at multiple preset heights according to the height distribution, the bev features at multiple heights can be finally obtained. Furthermore, multiple preset heights are used to reflect the height distribution of different obstacle types. Due to the different heights of different obstacle types, the impact on vehicle parking is different. Therefore, by predicting the height distribution based on image features, image features at multiple heights can be provided. This sampling process performs bev sampling for each preset height, and finally bev features at multiple heights can be obtained. The bev features are compressed and fused according to the height dimension, and a bev feature representation that integrates multiple height information can be obtained. The embodiment of the present application can fuse the height dimension through a convolution submodule, and the network autonomously learns to compress the bev features with multiple height information. I will not go into details here. FIG. 3 is a schematic diagram showing a final result of an obstacle detection frame in the parking method shown in FIG. 1 . Combined with FIG1 , as shown in FIG3 , the position information from the reference point to the midpoints of the two adjacent edges includes the offset vector from the reference point to the midpoints of the two adjacent edges, and the specific position of the midpoints of the adjacent edges; the above step 130 may further include inputting the reference point into the detection frame generation model to output the offset vector from the reference point to the midpoints of the two adjacent edges of the obstacle detection frame, and the specific position of the midpoints of the two adjacent edges; the detection frame generation model is obtained by machine learning training of a training sample set containing the reference point; according to the offset vector from the reference point to the midpoints of the two adjacent edges, and the specific position of the midpoints of the two adjacent edges, the size, position and orientation of the obstacle detection frame are obtained. In this way, by predicting the reference point using the detection frame generation model to output the offset vector from the reference point to the midpoints of the two adjacent edges of the obstacle detection frame, and the specific position of the midpoints of the two adjacent edges, the size, position and orientation of the obstacle detection frame are obtained, which can improve the accuracy of the offset vector from the reference point to the midpoints of the two adjacent edges of the obstacle detection frame, and the specific position of the midpoints of the two adjacent edges. When the detection frame generation model is post-processed, the approximate positions of the midpoints of the two adjacent edges can be regressed through the nearest point and the vector from the nearest point to the midpoints of the two adjacent edges. The predicted midpoint information is matched with the nearest reference point through distance matching, and the 2D mapping of the 3D frame under the BEV perspective is adjusted to obtain the final result as shown in Figure 3. Compared with the detection head based on the center key point (CenterPoint), the reference point-based 3D detection head proposed in this application can provide more accurate 3D position information and a more stable orientation angle with less deviation. In conjunction with FIG. 1 to FIG. 3, the above step 130 may further include (1) predicting the position information of the reference point to the midpoints of two adjacent edges adjacent to the reference point according to the reference point, and obtaining the size, position and direction of the obstacle detection frame. direction, and the obstacle height corresponding to the bev feature with height information; the obstacle height is used to reflect the importance of the impact on parking; the higher the obstacle height, the greater the importance of the impact on parking. Accordingly, the above step 140 may further include (2) determining whether to park in a way that avoids an obstacle area according to the obstacle area, position and orientation, and obstacle height. In this way, obstacles that do not affect the parking of the vehicle, such as the unlocked state of the parking lock, have a relatively small impact on parking, and the obstacle height can be set to a relatively small importance to reduce its impact on parking. Obstacles that affect the parking of the vehicle, such as people or pillars, have a relatively large impact on parking, and the obstacle height can be set to a relatively large importance to highlight its impact on parking. The above step (2) may further include ①, based on the obstacle area, position and direction, as well as the obstacle height, estimating whether the obstacle collides with the vehicle; if not, that is, when the obstacle and the vehicle do not collide, executing step ②, controlling the vehicle to park without avoiding the obstacle area; if yes, that is, when the obstacle and the vehicle collide, executing step ③, controlling the vehicle to park without avoiding the obstacle area. In this way, the obstacle area can be automatically avoided or the vehicle can be parked directly according to the obstacle height, thereby improving parking efficiency. The 3D obstacle detection head based on the reference point of this application can have a direction deviation index of 2 to 3 degrees when the obstacle is a vehicle type. FIG. 4 is a schematic diagram showing the structure of a parking device according to an embodiment of the present application. As shown in FIG4 , the parking device may further include the following modules: A feature conversion module 41 is configured to convert image features collected by a plurality of image recording devices around the vehicle into bird's-eye view bev features of a panoramic bird's-eye view; The reference point determination module 42 is configured to determine the corner points of the obstacles as reference points of the obstacle detection frame according to the bev feature by combining the relative position relationship between the obstacles around the vehicle and the vehicle; The detection frame determination module 43 is configured to predict the position information from the reference point to the midpoints of the two adjacent edges adjacent to the reference point according to the reference point, and obtain the size, position and orientation of the obstacle detection frame; the size, position and orientation of the obstacle detection frame are used to reflect the obstacle area where the obstacle is located; The parking determination module 44 is configured to determine whether to park in a way that avoids an obstacle area according to the size, position and orientation of the obstacle detection frame. The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and the same technical effect can be achieved, which will not be repeated here. A person skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware (such as a processor) through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, Disk or CD, etc. Optionally, all or part of the steps of the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware, such as implementing its corresponding functions through an integrated circuit, or can be implemented in the form of a software function module, such as implementing its corresponding functions through a processor executing a program / instruction stored in a memory. The present application is not limited to any specific form of combination of hardware and software. An embodiment of the present application provides an electronic device, including the parking device as described above. FIG5 is a module block diagram of an electronic device 50 provided in an embodiment of the present application. As shown in FIG. 5 , the electronic device 50 includes one or more processors 51 for implementing the above parking method. In some embodiments, the electronic device 50 may include a computer-readable storage medium 59, which may store a program that can be called by the processor 51 and may include a non-volatile storage medium. In some embodiments, the electronic device 50 may include a memory 58 and an interface 57. In some embodiments, the electronic device 50 may also include other hardware according to actual applications. The above-mentioned electronic device may be a processor, a controller, a mobile terminal, a computer, etc., which will not be listed one by one here. The computer-readable storage medium 59 of the embodiment of the present application stores a program thereon, and when the program is executed by the processor 51, it is used to implement the parking method described above. The present application may take the form of a computer program product implemented on one or more computer-readable storage media 59 (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media 59 include permanent and non-permanent, removable and non-removable media, and information storage may be implemented by any method or technology. The information may be computer-readable instructions, data structures, modules of a program, or other data. Examples of computer-readable storage media 59 include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically-erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. In the description of this application, it should be understood that the terms "middle", "longitudinal", "lateral", "length", The orientations or positional relationships indicated by “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc. are based on the orientations or positional relationships shown in the accompanying drawings and are only for the convenience of describing the present application and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application. In the description of the present application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include at least one of the features. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined. The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification. It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the phrase "includes a ..." defines an element, and does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

Claims

1. A parking method, comprising: Convert image features collected by multiple image recording devices around the vehicle into bird's-eye view bev features of a panoramic bird's-eye view; According to the bev feature, by combining the relative position relationship between the obstacles around the vehicle and the vehicle, the corner points of the obstacles are determined as reference points of the obstacle detection frame; According to the reference point of the obstacle detection frame, predict the position information from the reference point to the midpoints of two adjacent edges adjacent to the reference point to obtain the size, position and orientation of the obstacle detection frame; The size, position and orientation of the obstacle detection frame are used to reflect the obstacle area where the obstacle is located; Whether to park in a manner avoiding the obstacle area is determined according to the size, position and orientation of the obstacle detection frame.

2. The parking method according to claim 1, wherein: The position information from the reference point to the midpoints of the two adjacent edges includes an offset vector from the reference point to the midpoints of the two adjacent edges and the position of the midpoints of the adjacent edges; and the predicting, based on the reference point, the position information from the reference point to the midpoints of the two adjacent edges adjacent to the reference point to obtain the size, position and orientation of the obstacle detection frame includes: Inputting the reference point into a detection frame generation model to output an offset vector from the reference point of the obstacle detection frame to the midpoints of the two adjacent edges and the positions of the midpoints of the two adjacent edges; the detection frame generation model is obtained by training a training sample set containing the reference point; The size, position and orientation of the obstacle detection frame are obtained according to the offset vector from the reference point to the midpoints of the two adjacent edges and the positions of the midpoints of the two adjacent edges.

3. The parking method according to claim 1 or 2, wherein: The determining, based on the bev feature, of the corner points of the obstacles as reference points of the obstacle detection frame by combining the relative position relationship between the obstacles around the vehicle and the vehicle, includes: According to the bev feature, by combining the relative position relationship between the obstacle and the vehicle, the corner point of the obstacle closest to the vehicle is determined as the reference point of the obstacle detection frame.

4. The parking method according to claim 1 or 2, wherein: The method of determining the corner points of the obstacles as reference points of the obstacle detection frame according to the bev feature and the relative position relationship between the obstacles around the vehicle and the vehicle includes: According to a plurality of preset heights, bev sampling is performed on the image features to obtain bev features at the plurality of preset heights; The bev features at the multiple preset heights are compressed and fused according to the height dimension to obtain a high confidence The bev characteristics of the information; According to the bev feature with height information, any corner point of the obstacle around the vehicle is determined as a reference point of the obstacle detection frame.

5. The parking method according to claim 4, wherein: The step of predicting, based on the reference point, the position information of the reference point to the midpoints of two adjacent edges adjacent to the reference point, and obtaining the size, position and orientation of the obstacle detection frame includes: According to the reference point of the obstacle detection frame, predict the position information from the reference point to the midpoints of two adjacent sides adjacent to the reference point, and obtain the size, position, orientation of the obstacle detection frame and the obstacle height corresponding to the bev feature with height information; the obstacle height is used to reflect the importance of the impact on parking; the higher the obstacle height, the higher the importance of the impact on parking; The determining whether to avoid the obstacle area for parking according to the size, position and orientation of the obstacle detection frame includes: Whether to park in a way that avoids the obstacle area is determined according to the size, position, orientation of the obstacle detection frame and the height of the obstacle.

6. The parking method according to claim 5, wherein: The determining whether to avoid the obstacle area for parking according to the size, position, orientation of the obstacle detection frame and the height of the obstacle includes: estimating whether the obstacle collides with the vehicle according to the size, position, orientation of the obstacle detection frame, and the height of the obstacle; When the obstacle does not collide with the vehicle, controlling the vehicle to park without avoiding the obstacle area; When the obstacle collides with the vehicle, the vehicle is controlled to avoid the obstacle area and park.

7. The parking method according to any one of claims 1 to 6, wherein: The step of converting the image features collected by the multiple image recording devices around the vehicle into bird's-eye view bev features of the panoramic bird's-eye view includes: Acquire images captured by the multiple image recording devices; Based on the image features in the image and in accordance with the position mapping relationship between the bev features and the image features, the coordinates of each 3D point in the panoramic bird's-eye view in each image coordinate system are determined as the bev features.

8. A parking device, comprising: A feature conversion module is configured to convert image features collected by a plurality of image recording devices around the vehicle into bird's-eye view bev features of a panoramic bird's-eye view; A reference point determination module is configured to determine the reference point of the vehicle through the obstacles around the vehicle and the vehicle according to the bev feature. The relative position relationship of the vehicle is determined to determine the corner point of the obstacle as the reference point of the obstacle detection frame; a detection frame determination module, configured to predict, based on the reference point, position information from the reference point to the midpoints of two adjacent edges adjacent to the reference point, to obtain the size, position and orientation of the obstacle detection frame; the size, position and orientation of the obstacle detection frame are used to reflect the obstacle area where the obstacle is located; The parking determination module is configured to determine whether to park in a manner avoiding the obstacle area according to the size, position and orientation of the obstacle detection frame.

9. An electronic device, comprising one or more processors, configured to implement the parking method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a program stored thereon, wherein when the program is executed by a processor, the parking method according to any one of claims 1 to 7 is implemented.

Images