A parking method, system, readable storage medium, and vehicle

Abstract

This invention provides a parking method, system, readable storage medium, and vehicle. The method includes real-time acquisition of surrounding images of the vehicle and stitching these images together to form a bird's-eye view. Based on a parking space model in a deep learning model, parking spaces and obstacles in the bird's-eye view are identified, and image coordinates of each parking space and obstacle coordinates are obtained. Vehicle coordinates are acquired in real-time and converted into vehicle world coordinates. Based on the vehicle world coordinates, the image coordinates of each parking space are converted into world coordinates for each parking space. Obstacle coordinates are also converted into obstacle world coordinates based on the vehicle world coordinates. Based on the world coordinates of each parking space and the obstacle world coordinates, the optimal parking space world coordinates are obtained. Finally, a parking route is planned based on the optimal parking space world coordinates and the obstacle world coordinates. This invention can more accurately determine the location of parking spaces and search for more effective parking spaces, thereby making vehicle parking more precise.

Description

Technical Field

[0001] This invention relates to the field of automatic parking technology, and in particular to a parking method, system, readable storage medium, and vehicle. Background Technology

[0002] With the rapid development of cities, the number of cars is increasing year by year. Coupled with the limited parking space in cities, parking cars in narrow spaces has become an essential skill. However, for some drivers, parallel parking is not an easy task, and sometimes they have to keep looking for an empty parking space.

[0003] In the early days, reversing cameras and reversing radars were commonly used to remind drivers to park. With the development of autonomous driving technology, intelligent parking assistance systems and autonomous valet parking systems have emerged. After arriving at their destination, drivers can free their hands and even get out of the car first, allowing the vehicle to autonomously search for a suitable parking space and intelligently park itself.

[0004] In existing technologies, the accuracy of the search for parking spaces, the effectiveness of the search, and the precision of the confirmation of obstacles around the parking spaces are not high during the process of vehicles autonomously searching for and intelligently parking in parking spaces, thus failing to guarantee safe parking of vehicles. Summary of the Invention

[0005] Therefore, the purpose of this invention is to provide a parking method, system, readable storage medium, and vehicle to at least address the shortcomings of the prior art.

[0006] This invention proposes a parking method, the parking method comprising:

[0007] Real-time acquisition of surround view images of the vehicle and stitching of the surround view images into a bird's-eye view;

[0008] Based on the storage location model in the deep learning model, the storage locations and obstacles in the bird's-eye view are identified, and the image coordinates of each storage location and the coordinates of the obstacles are obtained;

[0009] The system acquires vehicle coordinates in real time and converts them into vehicle world coordinates. Based on the vehicle world coordinates, it converts the image coordinates of each storage location into the world coordinates of each storage location. It also converts the obstacle coordinates into obstacle world coordinates based on the vehicle world coordinates.

[0010] Based on the world coordinates of each storage location and the world coordinates of the obstacles, the optimal world coordinates of the storage location are obtained, and parking routes are planned according to the optimal world coordinates of the storage location and the world coordinates of the obstacles.

[0011] Compared with existing technologies, the beneficial effects of this invention are as follows: By stitching together images acquired around the vehicle to obtain a bird's-eye view, and using a parking space model in a deep learning model to identify parking spaces and obstacles in the bird's-eye view, the image coordinates of the parking spaces and the coordinates of the obstacles are obtained. Then, the vehicle coordinates are acquired in real time and converted into vehicle world coordinates. Based on the vehicle world coordinates, the image coordinates of the parking spaces are then converted into the world coordinates of the parking spaces. This makes the coordinates of the parking spaces more accurate. Furthermore, by converting the coordinates of the obstacles into obstacle world coordinates, and then obtaining the optimal world coordinates of the parking spaces using both world coordinates, the searched parking spaces become more effective. Finally, parking routes are planned based on the optimal world coordinates of the parking spaces, resulting in more accurate parking of the vehicle.

[0012] Furthermore, the step of stitching the surrounding panoramic images into a bird's-eye view includes:

[0013] The four-sided panoramic images are sequentially corrected and orthodontized, and the four-sided panoramic images include four individual images;

[0014] The four individual images are converted into four individual bird's-eye view images, and the four individual bird's-eye view images are stitched together to obtain a bird's-eye view image.

[0015] Furthermore, the step of identifying storage locations and obstacles in the bird's-eye view based on the storage location model in the deep learning model, and obtaining the image coordinates of each storage location and the coordinates of the obstacles, includes:

[0016] Based on the storage location model in the deep learning model, the bird's-eye view is inferred and identified to obtain the image coordinates and storage location type of each storage location;

[0017] The deep learning model is used to infer and identify the bird's-eye view, and a high-level abstract feature map is generated by performing a nonlinear transformation on the bird's-eye view. The coordinates and types of the obstacles are then output based on the high-level abstract feature map.

[0018] Furthermore, the step of converting the obstacle coordinates into obstacle world coordinates based on the vehicle world coordinates includes:

[0019] The relative distance between the obstacle and the vehicle is obtained in real time based on the vehicle coordinates, and the relative position of the obstacle is obtained based on the relative distance.

[0020] The world coordinates of the obstacle are obtained by performing coordinate transformation based on its relative position.

[0021] Furthermore, following the step of basing the steps on the world coordinates of each storage location and the world coordinates of the obstacle, the method further includes:

[0022] Scan the obstacles and perform redundant detection on them;

[0023] Based on the world coordinates of the obstacles, the algorithm fuses the obstacles after the redundancy detection is completed, so as to remove invalid storage locations with obstacles.

[0024] In another aspect, the present invention also proposes a parking system, the parking system comprising:

[0025] The first acquisition module is used to acquire images of the vehicle's surroundings in real time and stitch the images together to form a bird's-eye view.

[0026] The identification module is used to identify the storage locations and obstacles in the bird's-eye view based on the storage location model in the deep learning model, and to obtain the image coordinates of each storage location and the coordinates of the obstacles;

[0027] The second acquisition module is used to acquire vehicle coordinates in real time, convert the vehicle coordinates into vehicle world coordinates, convert the image coordinates of each storage location into the world coordinates of each storage location based on the vehicle world coordinates, and convert the obstacle coordinates into obstacle world coordinates based on the vehicle world coordinates.

[0028] The planning module is used to obtain the optimal world coordinates of each storage location based on the world coordinates of the storage locations and the world coordinates of the obstacles, and to plan parking routes based on the optimal world coordinates of the storage locations and the world coordinates of the obstacles.

[0029] Furthermore, the first acquisition module includes:

[0030] A correction unit is used to sequentially correct and orthodontize the four-sided panoramic images, which include four individual images.

[0031] The stitching unit is used to convert the four individual images into four individual bird's-eye view images, and stitch the four individual bird's-eye view images together to obtain a bird's-eye view image.

[0032] The second acquisition module includes:

[0033] The acquisition unit is used to acquire the relative distance between the obstacle and the vehicle in real time based on the vehicle coordinates, and to obtain the relative position of the obstacle based on the relative distance;

[0034] The transformation unit is used to perform coordinate transformation based on the relative position of the obstacle to obtain the world coordinates of the obstacle.

[0035] Furthermore, the identification module includes:

[0036] The first reasoning unit is used to perform reasoning and recognition on the bird's-eye view based on the storage location model in the deep learning model, and to obtain the image coordinates and storage location type of each storage location.

[0037] The second reasoning unit is used to perform reasoning and recognition on the bird's-eye view based on the deep learning model, perform nonlinear transformation on the bird's-eye view to generate a high-level abstract feature map, and output the obstacle coordinates and obstacle type based on the high-level abstract feature map.

[0038] The planning module includes:

[0039] A scanning unit is used to scan the obstacles and perform redundant detection on the obstacles;

[0040] The filtering unit is used to perform algorithmic fusion on the obstacles after the redundancy detection is completed based on the world coordinates of the obstacles, so as to remove invalid locations with obstacles.

[0041] In another aspect, the present invention provides a readable storage medium on which a computer program is stored, which, when executed by a processor, implements the above-described parking method.

[0042] The present invention also proposes a vehicle including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described parking method. Attached Figure Description

[0043] Figure 1 This is a flowchart of the parking method in the first embodiment of the present invention;

[0044] Figure 2 This is a flowchart of the parking method in the second embodiment of the present invention;

[0045] Figure 3 This is a structural diagram of the parking system in the third embodiment of the present invention;

[0046] Figure 4 This is a structural diagram of the vehicle according to the fourth embodiment of the present invention.

[0047] Explanation of key component symbols:

[0048] First Acquisition Module 11 Recognition Module 12 Second acquisition module 13 Planning module 14 memory 10 processor 20 Computer program 30

[0049] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0050] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0051] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0053] Example 1

[0054] Please see Figure 1 The image shows a parking method according to the first embodiment of the present invention, which specifically includes steps S101 to S104:

[0055] S101, acquire the surrounding view images of the vehicle in real time, and stitch the surrounding view images into a bird's-eye view;

[0056] In practice, after the vehicle receives the autonomous parking task, the wide-angle fisheye cameras installed on the front, rear, left and right sides of the vehicle acquire real-time images of the vehicle's surroundings, which are called surround view images. Then, the surround view images are stitched together to form a bird's-eye view.

[0057] It is worth noting that in this embodiment, the surrounding view images are four sets of images: the front image of the vehicle captured by the front wide-angle fisheye camera, the front image of the vehicle captured by the rear wide-angle fisheye camera, the left image of the vehicle captured by the left wide-angle fisheye camera, and the left image of the vehicle captured by the right wide-angle fisheye camera. The front image, rear image, left image, and right image of the vehicle are converted into a bird's-eye view of the front image, a bird's-eye view of the rear image, a bird's-eye view of the left image, and a bird's-eye view of the right image, respectively. Then, the bird's-eye view of the front image, the rear image, the left image, and the right image are stitched together to form a total bird's-eye view.

[0058] S102, Based on the storage location model in the deep learning model, identify the storage locations and obstacles in the bird's-eye view, and obtain the image coordinates of each storage location and the coordinates of the obstacles;

[0059] It needs to be explained that deep learning models have powerful representation and modeling capabilities. Through supervised learning, they automatically learn the feature abstraction map of the target layer by layer, thereby identifying parking spaces and obstacles in the bird's-eye view and obtaining the coordinates of the parking spaces in the bird's-eye view, which is the image coordinate of the parking spaces. At the same time, they also obtain the coordinates of the obstacles in the bird's-eye view, which is the coordinate of the obstacles. This makes the search for parking spaces and obstacles around the vehicle more effective and avoids searching for invalid parking spaces.

[0060] It is worth noting that when identifying obstacles around a vehicle, a millimeter-wave radar is installed on the vehicle. The millimeter-wave radar emits millimeter waves of a corresponding segment through a transmitting antenna. When the millimeter waves encounter an obstacle, they are reflected back. The reflected millimeter waves are received by a receiving antenna. Based on the waveband and flight time of the millimeter waves, combined with the vehicle's speed, the relative distance between the millimeter-wave radar and the obstacle is calculated. This relative distance between the vehicle and the obstacle reveals the relative position of the obstacle, i.e., the obstacle's coordinates.

[0061] S103, acquire vehicle coordinates in real time, convert the vehicle coordinates into vehicle world coordinates, convert the image coordinates of each storage location into the world coordinates of each storage location based on the vehicle world coordinates, and convert the obstacle coordinates into obstacle world coordinates based on the vehicle world coordinates.

[0062] In practice, the bird's-eye view is first converted into vehicle coordinates based on the extrinsic calibration parameters of the wide-angle fisheye cameras on the four sides of the vehicle, thereby obtaining the vehicle coordinates. At the same time, the world coordinates of the vehicle's IMU are obtained, and the vehicle coordinates are then converted into world coordinates using an algorithm. This allows the bird's-eye view to be converted into image world coordinates. Since the bird's-eye view is an image of the vehicle's surroundings, the world coordinates of each storage location and the world coordinates of obstacles can be obtained based on the vehicle's world coordinates, which are the image world coordinates of the bird's-eye view. This makes the determination of the storage location coordinates more accurate and effective.

[0063] S104. Based on the world coordinates of each storage location and the world coordinates of the obstacle, the optimal storage location world coordinates are obtained, and a parking route is planned according to the optimal storage location world coordinates and the obstacle world coordinates.

[0064] In practical implementation, during vehicle parking, there may be multiple parking spaces, and the obstacles around each parking space are different. Therefore, based on the world coordinates of each parking space and the world coordinates of the obstacles, the world coordinates of the obstacles around each parking space and the world coordinates of the obstacles around each parking space can be known. Thus, the optimal world coordinates of the parking space can be obtained. In this embodiment, the optimal world coordinates of the parking space are the parking space with the fewest obstacles around it and the farthest obstacles around it. The world coordinates of the parking space are extracted, and a parking route is planned based on the world coordinates of the parking space to enable vehicles to park more accurately and safely.

[0065] In summary, the parking method in the above embodiments of the present invention stitches together images acquired around the vehicle to obtain a bird's-eye view of the vehicle's surroundings. A parking space model in a deep learning model identifies parking spaces and obstacles in the bird's-eye view, obtaining the image coordinates of the parking spaces and the coordinates of the obstacles. Then, the vehicle coordinates are acquired in real time and converted into vehicle world coordinates. Based on the vehicle world coordinates, the image coordinates of the parking spaces are then converted into the world coordinates of the parking spaces. This makes the coordinates of the parking spaces more accurate. Furthermore, the obstacle coordinates are converted into obstacle world coordinates. The optimal world coordinates of the parking spaces are then obtained using both the parking space world coordinates and the obstacle world coordinates, making the searched parking spaces more effective. Finally, a parking route is planned based on the optimal parking space world coordinates, resulting in more accurate parking of the vehicle.

[0066] Example 2

[0067] Please see Figure 2 The image shows a parking method according to a second embodiment of the present invention, which specifically includes steps S201 to S209:

[0068] S201, acquire the vehicle's four-sided surround view image in real time, and correct and distort the four-sided surround view image in sequence. The four-sided surround view image includes four individual images.

[0069] S202, convert the four individual images into four individual bird's-eye view images, and stitch the four individual bird's-eye view images together to obtain a bird's-eye view image;

[0070] In practice, after the vehicle receives the autonomous parking task, the wide-angle fisheye cameras installed on the front, rear, left and right sides of the vehicle acquire real-time images of the vehicle's surroundings, which are called surround view images. Then, the surround view images are stitched together to form a bird's-eye view.

[0071] It is worth noting that in this embodiment, the surrounding view images are four sets of images: the front image of the vehicle captured by the front wide-angle fisheye camera, the front image of the vehicle captured by the rear wide-angle fisheye camera, the left image of the vehicle captured by the left wide-angle fisheye camera, and the left image of the vehicle captured by the right wide-angle fisheye camera. The front image, rear image, left image, and right image of the vehicle are converted into a bird's-eye view of the front image, a bird's-eye view of the rear image, a bird's-eye view of the left image, and a bird's-eye view of the right image, respectively. Then, the bird's-eye view of the front image, the rear image, the left image, and the right image are stitched together to form a total bird's-eye view.

[0072] S203, based on the storage location model in the deep learning model, perform reasoning and recognition on the bird's-eye view to obtain the image coordinates and storage location type of each storage location;

[0073] It needs to be explained that the storage location is identified based on the storage location model in the deep learning model to obtain the storage location type, which includes horizontal storage location, vertical storage location and diagonal storage location, and the image coordinates of the storage location in the bird's-eye view are obtained, that is, the image coordinates of the storage location.

[0074] S204, based on the deep learning model, perform reasoning and recognition on the bird's-eye view, perform nonlinear transformation on the bird's-eye view to generate a high-level abstract feature map, and output the obstacle coordinates and obstacle type based on the high-level abstract feature map;

[0075] It should be explained that deep learning models have powerful representation and modeling capabilities. Through supervised learning, they automatically learn the feature abstraction map of the target layer by layer, and then can infer and identify the coordinates and types of obstacles in the bird's-eye view. Obstacle types include vehicles, pedestrians, traffic cones, trash cans, and parking locks, etc.

[0076] S205, acquire vehicle coordinates in real time, convert the vehicle coordinates into vehicle world coordinates, convert the image coordinates of each storage location into the world coordinates of each storage location based on the vehicle world coordinates, acquire the relative distance between the obstacle and the vehicle in real time based on the vehicle coordinates, and obtain the relative position of the obstacle based on the relative distance;

[0077] It is worth noting that when identifying obstacles around a vehicle, a millimeter-wave radar is installed on the vehicle. The millimeter-wave radar emits millimeter waves of a corresponding segment through a transmitting antenna. When the millimeter waves encounter an obstacle, they are reflected back. The reflected millimeter waves are received by a receiving antenna. Based on the waveband and flight time of the millimeter waves, combined with the vehicle's speed, the relative distance between the millimeter-wave radar and the obstacle is calculated. This relative distance between the vehicle and the obstacle reveals the relative position of the obstacle, i.e., the obstacle's coordinates.

[0078] S206, Perform coordinate transformation based on the relative position of the obstacle to obtain the world coordinates of the obstacle;

[0079] In practice, the bird's-eye view is first converted into vehicle coordinates based on the extrinsic calibration parameters of the wide-angle fisheye cameras on the four sides of the vehicle, thereby obtaining the vehicle coordinates. At the same time, the world coordinates of the vehicle's IMU are obtained, and the vehicle coordinates are then converted into world coordinates using an algorithm. This allows the bird's-eye view to be converted into image world coordinates. Since the bird's-eye view is an image of the vehicle's surroundings, the world coordinates of each storage location and the world coordinates of obstacles can be obtained based on the vehicle's world coordinates, which are the image world coordinates of the bird's-eye view. This makes the determination of the storage location coordinates more accurate and effective.

[0080] S207, Based on the world coordinates of each storage location and the world coordinates of the obstacle, scan the obstacle and perform redundant detection on the obstacle;

[0081] S208, Based on the world coordinates of the obstacles, the obstacles after the redundancy detection is completed are fused by algorithm to remove invalid storage locations with obstacles;

[0082] In practical implementation, there will be multiple parking spaces and obstacles during the parking process. Therefore, it is necessary to perform fine and redundant detection on multiple obstacles. In this embodiment, redundant detection refers to dual detection: first, obstacle detection is performed through a deep learning model, and second, obstacle detection is performed through millimeter-wave radar. Even if one side misses an obstacle, the other side can be used for secondary detection to reduce the obstacle miss rate. Then, the redundant detection is combined with the obstacle information. In this embodiment, algorithm fusion refers to the fusion of obstacle information and parking space information detected by millimeter-wave radar and deep learning model to determine whether the vehicle can park in the parking space. The obstacle information includes the world coordinates and obstacle type of the obstacle, and the parking space information includes the world coordinates and parking space type of the parking space. Finally, a judgment strategy is formulated based on the obstacle information and parking space information to eliminate parking spaces with obstacles, that is, to eliminate invalid parking spaces, and finally output valid parking spaces.

[0083] S209, obtain the optimal storage location world coordinates, and plan the parking route based on the optimal storage location world coordinates and the obstacle world coordinates;

[0084] In practical implementation, during vehicle parking, there may be multiple parking spaces, and the obstacles around each parking space are different. Therefore, based on the world coordinates of each parking space and the world coordinates of the obstacles, the world coordinates of the obstacles around each parking space and the world coordinates of the obstacles around each parking space can be known. Thus, the optimal world coordinates of the parking space can be obtained. In this embodiment, the optimal world coordinates of the parking space are the parking space with the fewest obstacles around it and the farthest obstacles around it. The world coordinates of the parking space are extracted, and a parking route is planned based on the world coordinates of the parking space to enable the vehicle to park more accurately and safely.

[0085] In summary, compared with Embodiment 1, the parking method in the above embodiments of the present invention confirms the obstacle type and parking space type based on a deep learning model, and detects obstacles around the parking space by performing redundant detection on the world coordinates of each obstacle. Therefore, parking spaces with obstacles around them can be eliminated, avoiding vehicles from choosing parking spaces that are difficult to park in. This allows the optimal parking space to be selected, and the parking route of the vehicle can be planned using the world coordinates of the optimal parking space. After the parking route is planned, the vehicle mobilizes various modules to cooperate and complete the autonomous parking task so that the vehicle can park in the optimal parking space.

[0086] Example 3

[0087] In another aspect, the present invention also proposes a parking system, please refer to [link / reference]. Figure 3 The image shows a parking system according to a third embodiment of the present invention, the parking system comprising:

[0088] The first acquisition module 11 is used to acquire the surrounding view images of the vehicle in real time and stitch the surrounding view images into a bird's-eye view;

[0089] The identification module 12 is used to identify the storage locations and obstacles in the bird's-eye view based on the storage location model in the deep learning model, and to obtain the image coordinates of each storage location and the coordinates of the obstacle.

[0090] The second acquisition module 13 is used to acquire vehicle coordinates in real time, convert the vehicle coordinates into vehicle world coordinates, convert the image coordinates of each storage location into the world coordinates of each storage location based on the vehicle world coordinates, and convert the obstacle coordinates into obstacle world coordinates based on the vehicle world coordinates.

[0091] The planning module 14 is used to obtain the optimal world coordinates of each storage location based on the world coordinates of each storage location and the world coordinates of the obstacles, and to plan a parking route based on the optimal world coordinates of the storage location and the world coordinates of the obstacles.

[0092] In some optional embodiments, the first acquisition module 11 includes:

[0093] A correction unit is used to sequentially correct and orthodontize the four-sided panoramic images, which include four individual images.

[0094] The stitching unit is used to convert the four individual images into four individual bird's-eye view images, and stitch the four individual bird's-eye view images together to obtain a bird's-eye view image.

[0095] The second acquisition module 13 includes:

[0096] The acquisition unit is used to acquire the relative distance between the obstacle and the vehicle in real time based on the vehicle coordinates, and to obtain the relative position of the obstacle based on the relative distance;

[0097] The transformation unit is used to perform coordinate transformation based on the relative position of the obstacle to obtain the world coordinates of the obstacle.

[0098] In some alternative embodiments, the identification module includes:

[0099] The first reasoning unit is used to perform reasoning and recognition on the bird's-eye view based on the storage location model in the deep learning model, and to obtain the image coordinates and storage location type of each storage location.

[0100] The second reasoning unit is used to perform reasoning and recognition on the bird's-eye view based on the deep learning model, perform nonlinear transformation on the bird's-eye view to generate a high-level abstract feature map, and output the obstacle coordinates and obstacle type based on the high-level abstract feature map.

[0101] The planning module includes:

[0102] A scanning unit is used to scan the obstacles and perform redundant detection on the obstacles;

[0103] The filtering unit is used to perform algorithmic fusion on the obstacles after the redundancy detection is completed based on the world coordinates of the obstacles, so as to remove invalid locations with obstacles.

[0104] The parking system provided in the third embodiment of the present invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0105] Example 4

[0106] The present invention also proposes a vehicle, please refer to [link / reference]. Figure 4 The vehicle shown is a fourth embodiment of the present invention, including a memory 10, a processor 20, and a computer program 30 stored on the memory 10 and executable on the processor 20. When the processor 20 executes the program 30, it implements the parking method as described above.

[0107] In a specific implementation, the processor 20 acquires the surrounding view images of the vehicle in real time and stitches the surrounding view images into a bird's-eye view;

[0108] The processor 20 identifies the storage locations and obstacles in the bird's-eye view based on the storage location model in the deep learning model, and obtains the image coordinates of each storage location and the coordinates of the obstacles;

[0109] The processor 20 acquires vehicle coordinates in real time, converts the vehicle coordinates into vehicle world coordinates, converts the image coordinates of each storage location into the world coordinates of each storage location based on the vehicle world coordinates, and converts the obstacle coordinates into obstacle world coordinates based on the vehicle world coordinates.

[0110] The processor 20 obtains the optimal world coordinates of each storage location based on the world coordinates of the storage locations and the world coordinates of the obstacles, and plans a parking route based on the optimal world coordinates of the storage locations and the world coordinates of the obstacles.

[0111] In some embodiments, the processor 20 may be an electronic control unit (ECU, also known as a vehicle computer), a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing access restriction programs.

[0112] The memory 10 includes at least one type of readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10 may be an internal storage unit of the vehicle, such as the vehicle's hard disk. In other embodiments, the memory 10 may be an external storage device of the vehicle, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 10 may include both internal and external storage devices. The memory 10 can be used not only to store application software and various types of data installed in the vehicle, but also to temporarily store data that has been output or will be output.

[0113] It should be pointed out that, Figure 4The structure shown does not constitute a limitation on the vehicle. In other embodiments, the vehicle may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0114] This invention also proposes a readable storage medium storing a computer program that, when executed by a processor, implements the parking method described above.

[0115] Those skilled in the art will understand that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0116] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0117] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0118] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0119] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A parking method, characterized in that, The parking method includes: Real-time acquisition of surround view images of the vehicle and stitching of the surround view images into a bird's-eye view; Based on the storage location model in the deep learning model, the storage locations and obstacles in the bird's-eye view are identified, and the image coordinates of each storage location and the coordinates of the obstacles are obtained. The system acquires vehicle coordinates in real time and converts them into vehicle world coordinates. Based on the vehicle world coordinates, it converts the image coordinates of each storage location into the world coordinates of each storage location. It also converts the obstacle coordinates into obstacle world coordinates based on the vehicle world coordinates. Based on the world coordinates of each storage location and the world coordinates of the obstacles, the optimal world coordinates of the storage location are obtained, and parking routes are planned according to the optimal world coordinates of the storage location and the world coordinates of the obstacles.

2. The parking method according to claim 1, characterized in that, The step of stitching the four surrounding view images into a bird's-eye view includes: The four-sided panoramic images are sequentially corrected and orthodontized, and the four-sided panoramic images include four individual images; The four individual images are converted into four individual bird's-eye view images, and the four individual bird's-eye view images are stitched together to obtain a bird's-eye view image.

3. The parking method according to claim 1, characterized in that, The step of identifying storage locations and obstacles in the bird's-eye view based on the storage location model in the deep learning model, and obtaining the image coordinates of each storage location and the coordinates of the obstacles, includes: Based on the storage location model in the deep learning model, the bird's-eye view is inferred and identified to obtain the image coordinates and storage location type of each storage location; The deep learning model is used to infer and identify the bird's-eye view, and a high-level abstract feature map is generated by performing a nonlinear transformation on the bird's-eye view. The coordinates and types of the obstacles are then output based on the high-level abstract feature map.

4. The parking method according to claim 1, characterized in that, The step of converting the obstacle coordinates into obstacle world coordinates based on the vehicle world coordinates includes: The relative distance between the obstacle and the vehicle is obtained in real time based on the vehicle coordinates, and the relative position of the obstacle is obtained based on the relative distance. The world coordinates of the obstacle are obtained by performing coordinate transformation based on its relative position.

5. The parking method according to claim 1, characterized in that, Following the step of using the world coordinates of each storage location and the world coordinates of the obstacle, the method further includes: Scan the obstacles and perform redundant detection on them; Based on the world coordinates of the obstacles, the algorithm fuses the obstacles after the redundancy detection is completed, so as to remove invalid storage locations with obstacles.

6. A parking system, characterized in that, The parking system includes: The first acquisition module is used to acquire images of the vehicle's surroundings in real time and stitch the images together to form a bird's-eye view. The identification module is used to identify the storage locations and obstacles in the bird's-eye view based on the storage location model in the deep learning model, and to obtain the image coordinates of each storage location and the coordinates of the obstacles. The second acquisition module is used to acquire vehicle coordinates in real time, convert the vehicle coordinates into vehicle world coordinates, convert the image coordinates of each storage location into the world coordinates of each storage location based on the vehicle world coordinates, and convert the obstacle coordinates into obstacle world coordinates based on the vehicle world coordinates. The planning module is used to obtain the optimal world coordinates of each storage location based on the world coordinates of the storage locations and the world coordinates of the obstacles, and to plan parking routes based on the optimal world coordinates of the storage locations and the world coordinates of the obstacles.

7. The parking system according to claim 6, characterized in that, The first acquisition module includes: A correction unit is used to sequentially correct and orthodontize the four-sided panoramic images, which include four individual images. The stitching unit is used to convert the four individual images into four individual bird's-eye view images, and stitch the four individual bird's-eye view images together to obtain a bird's-eye view image. The second acquisition module includes: The acquisition unit is used to acquire the relative distance between the obstacle and the vehicle in real time based on the vehicle coordinates, and to obtain the relative position of the obstacle based on the relative distance; The transformation unit is used to perform coordinate transformation based on the relative position of the obstacle to obtain the world coordinates of the obstacle.

8. The parking system according to claim 6, characterized in that, The identification module includes: The first reasoning unit is used to perform reasoning and recognition on the bird's-eye view based on the storage location model in the deep learning model, and to obtain the image coordinates and storage location type of each storage location. The second reasoning unit is used to perform reasoning and recognition on the bird's-eye view based on the deep learning model, perform nonlinear transformation on the bird's-eye view to generate a high-level abstract feature map, and output the obstacle coordinates and obstacle type based on the high-level abstract feature map. The planning module includes: A scanning unit is used to scan the obstacles and perform redundant detection on the obstacles; The filtering unit is used to perform algorithmic fusion on the obstacles after the redundancy detection is completed based on the world coordinates of the obstacles, so as to remove invalid locations with obstacles.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the parking method as described in any one of claims 1 to 5.

10. A vehicle comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the parking method as described in any one of claims 1-5.

Images