Memory parking method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a memory parking method, electronic equipment and a storage medium, and relates to the technical field of automatic driving. Parking environment information is obtained through a sensor based on the fusion parking system, perception fusion is conducted on the parking environment information, first perception fusion information is obtained, a memory path diagram matched with the first perception fusion information is retrieved from a stored memory path diagram set according to the first perception fusion information, and the memory path diagram is stored in the memory path diagram set. And according to the memory path diagram and the first perception fusion information, parking path planning is carried out, a parking path is obtained, parking is carried out according to the parking path, second perception fusion information is obtained in real time in the parking process, and the parking path is adjusted according to the second perception fusion information. Therefore, according to the parking environment information, the specific position of the parking space and the obstacle are judged, and parking is performed according to the memory path diagram.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a memory parking method, an electronic device and a storage medium. Background Art

[0002] At present, when automatic parking perceives parking spaces, the obstacle information collected by the ultrasonic radar is easily interfered by the external environment. For example, high temperature, heavy rain, and other sound wave signals will affect the accuracy and stability of ultrasonic signal ranging, resulting in the inability to accurately identify obstacle information and determine the specific location of the parking space, thus failing to successfully park. Summary of the invention

[0003] In view of this, an object of embodiments of the present invention is to provide a memory parking method based on perception fusion, an electronic device and a storage medium to at least partially improve the above-mentioned problem.

[0004] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0005] In a first aspect, an embodiment of the present invention provides a memory parking method, which is applied to an autonomous driving processor of a fusion parking system, wherein the fusion parking system further includes a sensor, wherein the sensor is communicatively connected to the autonomous driving processor, and the method includes:

[0006] acquiring parking environment information based on the sensor;

[0007] Performing perception fusion on the parking environment information to obtain first perception fusion information;

[0008] According to the first perception fusion information, retrieving a memory path map matching the first perception fusion information from a stored memory path map set;

[0009] Performing parking path planning according to the memory path map and the first perception fusion information to obtain a parking path;

[0010] Parking is performed according to the parking path, and second perception fusion information is acquired in real time during the parking process, and the parking path is adjusted according to the second perception fusion information.

[0011] Optionally, the fusion parking system further includes a micro control unit, the sensors include an AK2 ultrasonic radar, an AVM surround view camera, and a front view camera, the autonomous driving processor is respectively connected to the AVM surround view camera, the front view camera, and the micro control unit, the micro control unit is connected to the AK2 ultrasonic radar, and the acquisition of parking environment information based on the sensors includes:

[0012] Acquire ultrasonic environment information through the AK2 ultrasonic radar, and process it through the micro control unit to obtain ultrasonic parking space information and ultrasonic obstacle information;

[0013] Acquire surround image information through the AVM surround camera, and obtain linear parking space information, spatial parking space information, surround obstacles and ground markings through surround stitching and surround point cloud processing;

[0014] The forward-looking camera is used to obtain forward obstacles, lane lines, road surface markings, weather information, and light visibility.

[0015] Optionally, the perception fusion information includes a parking space semantic feature map, a parking space occupancy grid map, a parking space semantic point cloud map, and an obstacle point cloud map. The step of perceptually fusing the parking environment information to obtain the first perception fusion information includes:

[0016] Performing parking space fusion on the ultrasonic parking space information, the line parking space information and the spatial parking space information to generate the parking space semantic feature map, the parking space occupancy grid map and the parking space semantic point cloud map;

[0017] Obstacle fusion is performed on the ultrasonic obstacle information, the surround obstacle and the forward obstacle to obtain the obstacle point cloud map.

[0018] Optionally, the step of retrieving a memory path map matching the first perception fusion information from a stored memory path map set according to the first perception fusion information comprises:

[0019] Extracting feature information from the first perceptual fusion information;

[0020] Calculating the similarity between the feature information and the features in the memory path graph set;

[0021] According to the similarity calculation result, the memory path diagram that best matches the feature information is retrieved.

[0022] Optionally, the step of performing parking path planning according to the memory path map and the first perception fusion information to obtain a parking path includes:

[0023] Determine the current position of the vehicle and the position of the target parking space;

[0024] constructing a local map around the vehicle according to the first perception fusion information;

[0025] Identifying and marking obstacles and ground markings from the local map;

[0026] A path planning algorithm is used to generate a path from the current position of the vehicle to the target parking space position.

[0027] Optionally, parking according to the parking path, acquiring second perception fusion information in real time during the parking process, and adjusting the parking path according to the second perception fusion information includes:

[0028] Calculate the vehicle position during the current parking process in real time and obtain the second perception fusion information;

[0029] detecting whether there is an obstacle in the parking path according to the second perception fusion information;

[0030] If yes, re-plan the parking path and determine whether a new parking path can be planned;

[0031] If a new parking path can be planned, parking will continue; if a new parking path cannot be planned, parking will be stopped and an alarm will be sounded.

[0032] Optionally, the method further comprises the step of performing memory mapping of the parking path according to the memory mapping instruction, the step comprising:

[0033] Acquire parking space information and ground marking information through the AVM surround view camera and the front view camera;

[0034] The vehicle's moving distance and direction are obtained through the vehicle odometer, and the vehicle's acceleration and angular velocity are obtained through the inertial measurement unit;

[0035] Performing dead reckoning according to the moving distance, the moving direction, the acceleration, and the angular velocity to obtain a vehicle position and a vehicle attitude;

[0036] According to the parking space information, the ground sign information, the vehicle position and the vehicle posture, a vehicle driving path and a vehicle surrounding environment map are constructed to obtain a memory path map; the vehicle surrounding environment map includes roads, traffic signs and parking spaces;

[0037] The memory path map is stored in the fusion parking system of the vehicle for use in memory parking.

[0038] Optionally, the method further comprises:

[0039] When memory parking is completed, determining whether the actual parking path is consistent with the matched parking path;

[0040] If not, issuing an instruction on whether to enter the actual parking path;

[0041] If an input instruction is received, the actual parking path is input into the fusion parking system.

[0042] In a second aspect, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described methods when executing the program.

[0043] In a third aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned methods when executed by a processor.

[0044] The embodiment of the present invention provides a memory parking method, electronic device and storage medium, which obtains parking environment information through sensors, performs perception fusion, plans parking paths according to the perception fusion information and the memory path map, and performs parking, and obtains new perception fusion information in real time, and adjusts the parking path in real time. Thus, the specific location of the parking space and obstacles are determined according to the parking environment information, and parking is performed according to the memory path map.

[0045] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0047] Figure 1 A schematic structural block diagram of an electronic device provided by an embodiment of the present invention;

[0048] Figure 2 A schematic structural block diagram of a fusion parking system provided in an embodiment of the present invention;

[0049] Figure 3 A schematic diagram of a flow chart of a memory parking method provided by an embodiment of the present invention;

[0050] Figure 4 Another schematic structural block diagram of a fusion parking system provided by an embodiment of the present invention;

[0051] Figure 5 Another schematic diagram of a flow chart of a memory parking method provided by an embodiment of the present invention;

[0052] Figure 6 A schematic diagram of sensor installation provided by an embodiment of the present invention;

[0053] Figure 7 An exploded view and an assembly view of an AK2 ultrasonic radar provided by an embodiment of the present invention;

[0054] Figure 8 An exploded view and an assembly view of an AVM surround view camera provided by an embodiment of the present invention;

[0055] Fig. 9 An exploded view and an assembly view of a front-view camera provided by an embodiment of the present invention;

[0056] Fig.10 Another schematic diagram of a flow chart of a memory parking method provided by an embodiment of the present invention;

[0057] Fig.11 A schematic diagram of memory mapping and memory cruising provided by an embodiment of the present invention.

[0058] Icon: 100-electronic device; 101-memory; 102-communication interface; 103-processor; 104-bus; 300-fusion parking system; 310-autonomous driving processor; 320-sensor; 321-AK2 ultrasonic radar; 322-AVM surround view camera; 323-front view camera; 330-micro control unit. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0062] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0063] The commonly used parking system is an assisted driving system that uses on-board sensors and vehicle control chips to automatically park the vehicle into a parking space. Similar to other assisted driving systems, the automatic parking system is mainly composed of several parts such as environmental perception, recognition decision-making, and motion execution. The main function of this system is to realize the recognition of parking spaces and the planning and tracking of parking paths.

[0064] The specific processing steps are as follows:

[0065] 1. The client initiates a parking command;

[0066] 2. The automatic parking system begins to issue perception instructions;

[0067] 3. The sensor senses the environment and sends the sensed information to the vehicle parking system control chip;

[0068] 4. The recognition and decision module makes recognition decisions on the perceived information;

[0069] 5. The motion execution module receives and executes the motion control signal to realize a series of actions such as steering, shifting, deceleration and braking;

[0070] 6. The driver can select a parking space and choose parallel or perpendicular parking modes through the human-computer interaction module;

[0071] 7. Parking is completed.

[0072] However, at present, when automatic parking is sensing parking spaces, the obstacle information collected by the ultrasonic radar is easily interfered by the external environment. For example, high temperature, heavy rain, and other sound wave signals will affect the accuracy and stability of ultrasonic signal ranging, resulting in the inability to accurately identify obstacle information and determine the specific location of the parking space, thus failing to park successfully.

[0073] Based on the above situation, the embodiments of the present invention provide a memory parking method, an electronic device and a storage medium, which obtain accurate obstacle information, parking space information and other information by fusing environmental information, thereby realizing automatic parking.

[0074] To implement the process steps and functions of each example of the present invention, please refer to Figure 1 , Figure 1 A schematic structural block diagram of an electronic device provided by an embodiment of the present invention. The electronic device 100 includes a memory 101 and a processor 103, and the memory 101 and the processor 103 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses 104 or signal lines. The memory 101 can be used to store software programs and modules, and the processor 103 executes the software programs and modules stored in the memory 101 to perform various functional applications and data processing.

[0075] The electronic device 100 may be, but is not limited to, a personal computer (PC), a server, a distributed computer, etc. It is understandable that the electronic device 100 is not limited to a physical server, but may also be a virtual machine on a physical server, a virtual machine built on a cloud platform, etc., which can provide the same functions as the server or virtual machine. The operating system of the electronic device 100 may be, but is not limited to, a Windows system, a Linux system, etc.

[0076] Among them, the memory 101 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0077] The communication connection between the electronic device 100 and an external device is achieved through at least one communication interface 102 (which can be wired or wireless).

[0078] The processor 103 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the embodiment of the present invention can be completed by the hardware integrated logic circuit in the processor 103 or the instructions in the form of software. The processor 103 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0079] Understandably, Figure 1 The structure shown is for illustration only. The electronic device 100 may also include Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0080] The following is an exemplary description of the memory parking method provided by the present invention. Specifically, Figure 2 A schematic structural block diagram of a fusion parking system provided by an embodiment of the present invention, Figure 3 A flowchart of a memory parking method provided by an embodiment of the present invention is shown in FIG. Figure 2 , Figure 3 The memory parking method is applied to the autonomous driving processor 310 of the fusion parking system 300. The fusion parking system 300 further includes a sensor 320. The sensor 320 is in communication with the autonomous driving processor 310. The method is as follows Figure 3 The following steps are shown:

[0081] S210: Acquire parking environment information based on sensors.

[0082] When receiving a parking instruction, the sensor 320 can obtain parking environment information such as sound waves and vision. The parking instruction can be the vehicle using reverse gear or other specified forms to send a reverse instruction to the vehicle body to wake up the memory parking function. The fusion parking system 300 receives the reverse instruction and sends a control instruction to start environmental perception.

[0083] S220: Perform perception fusion on the parking environment information to obtain first perception fusion information.

[0084] The sensor 320 obtains various parking environment information and performs perception fusion to obtain information that is richer and easier for the processor to understand, which is convenient for subsequent memory path map retrieval and parking path planning.

[0085] S230: According to the first perception fusion information, a memory path map matching the first perception fusion information is retrieved from a stored memory path map set.

[0086] After obtaining the first perception fusion information, the memory path map set is searched to match the memory path map closest to the first perception fusion information. The memory path map in the memory path map set may be actively created by the user when parking, or may be created during the use of the memory parking method, or may be a memory path map shared by other vehicles.

[0087] S240: Perform parking path planning according to the memory path map and the first perception fusion information to obtain a parking path.

[0088] The historical memory path map is matched, and the parking path planning is performed according to the first perception fusion information obtained by the perception fusion. According to the historical memory path map and the first perception fusion information, the same parking path may be obtained, or different parking paths may be obtained.

[0089] S250: Parking according to the parking path, acquiring the second perception fusion information in real time during the parking process, and adjusting the parking path according to the second perception fusion information.

[0090] After obtaining the parking path, the vehicle will automatically park according to the parking path. However, there may be unexpected situations during the parking process, such as unrecognized obstacles or newly appeared moving obstacles. Therefore, during the automatic parking process, it is necessary to obtain parking environment information in real time, perceive and fuse the parking environment information, obtain the second perception fusion information, and adjust the parking path in real time according to the second perception fusion information.

[0091] The parking environment information is acquired through sensors, and perception fusion is performed. The parking path is planned based on the perception fusion information and the matching memory path map, and automatic parking is performed. New perception fusion information is acquired in real time, and the parking path is adjusted in real time. Thus, the specific location of the parking space and obstacles are determined based on the parking environment information, new parking environment information is acquired in real time, and parking is performed according to the memory path map.

[0092] In order to obtain all-round parking environment information, different types of sensors 320 may be provided to obtain more parking environment information. Figure 4 Another schematic structural block diagram of a fusion parking system provided by an embodiment of the present invention is shown in FIG. Figure 5Another flowchart of a memory parking method provided by an embodiment of the present invention is shown in FIG. Figure 4 , Figure 5 The fusion parking system 300 may further include a micro control unit 330, the sensor 320 includes an AK2 ultrasonic radar 321, an AVM surround camera 322, and a front camera 323, the automatic driving processor 310 is respectively connected to the AVM surround camera 322, the front camera 323, and the micro control unit 330, the micro control unit 330 is connected to the AK2 ultrasonic radar 321, and the above step S210 may include the following: Figure 5 The following steps are shown:

[0093] S211: Acquire ultrasonic environment information through the AK2 ultrasonic radar and process it through the micro control unit to obtain ultrasonic parking space information and ultrasonic obstacle information.

[0094] S212: Acquire surround image information through the AVM surround camera, and obtain linear parking space information, spatial parking space information, surround obstacles and ground signs through surround stitching and surround point cloud processing.

[0095] S213: Obtain forward obstacles, weather information, and light visibility through a forward-looking camera.

[0096] Optionally, the sensor 320 may include 12 AK2 ultrasonic radars 321, 4 AVM surround view cameras 322 and 1 front view camera 323. The installation location can be found in Figure 6 .

[0097] Compared with ordinary radar sensors, AK2 ultrasonic radar 321 has the advantages of better obstacle detection performance, more accurate ranging information, and longer detection distance. Figure 7 , Figure 7 The exploded view and assembly view of an AK2 ultrasonic radar provided by an embodiment of the present invention. The AVM surround view camera 322 uses a fisheye lens, and its fisheye protruding structure can increase the viewing angle to more than 180 degrees, and can detect the surrounding information of the vehicle body in all directions. Figure 8 , Figure 8 The exploded view and assembly view of an AVM surround view camera provided by an embodiment of the present invention. The camera includes a lens, a camera upper cover, a shielding cover, a printed circuit board assembly (PCBA), a camera lower cover, and a FAKRA cable. The front-view camera 323 uses a camera and a processor combined in one (front-view integrated machine). The camera sensor obtains environmental information and then sends the obtained information to the processor for preprocessing. It has forward-view perception functions such as obstacle recognition, lane line recognition, and road sign recognition. See Fig. 9 , Fig. 9 An exploded view and an assembly view of a front-view camera provided in an embodiment of the present invention.

[0098] The ultrasonic environment information obtained by the AK2 ultrasonic radar can be processed by the micro control unit first to obtain ultrasonic parking space information and ultrasonic obstacle information. The processed ultrasonic environment information can be directly used by the automatic driving processor 310.

[0099] AVM surround view cameras are generally installed around the vehicle. AVM surround view cameras capture surround view image information, stitch the images of each camera together to form a panoramic image around the vehicle, and pre-process the stitched image, including denoising, edge enhancement, etc., to facilitate feature extraction. Useful features such as parking space lines, traffic signs, obstacle contours, etc. are extracted from the image. The surround view image information is converted into point cloud data, and depth information is assigned to each pixel point to create a three-dimensional environment model to obtain line parking space information, spatial parking space information, surround view obstacles, and ground markings. Among them, line parking space information is to identify line parking space information, and the location and size of the parking space are determined by analyzing the markings and shapes on the ground; spatial parking space information includes the situation of obstacles above the parking space, and the height of the parking space is known.

[0100] The forward-looking camera is generally installed on the front windshield of the vehicle and is mainly used to obtain information such as forward obstacles, lane lines, road signs, weather information, and light visibility.

[0101] In order to reduce the errors in obtaining parking space information and obstacle information, the parking environment information can also be sensed and fused to reduce the errors in the obtained information. The sensed fusion information can include parking space semantic feature map, parking space occupancy grid map, parking space semantic point cloud map and obstacle point cloud map, see Fig.10 , Fig.10 Another flowchart of a memory parking method provided by an embodiment of the present invention is shown in FIG. 1 . The above step S220 may include: Fig.10 The following steps are shown:

[0102] S221: Fusing the ultrasonic parking space information, the line parking space information, and the spatial parking space information to generate a parking space semantic feature map, a parking space occupancy grid map, and a parking space semantic point cloud map.

[0103] The parking space semantic feature map, parking space occupancy grid map and parking space semantic point cloud map obtained after parking space fusion can accurately identify the specific information of the parking space and facilitate the subsequent parking route planning.

[0104] Among them, the parking space semantic feature map is used to represent the semantic information in the environment, such as the boundaries of parking spaces and traffic signs. The parking space occupancy grid map is a two-dimensional grid in which each cell represents a certain spatial area in the environment, indicating which cells in the environment are occupied by obstacles and which are free. The semantic point cloud map is a three-dimensional data structure composed of point cloud data, and each point has a semantic label to indicate the spatial position of the parking space.

[0105] The parking space fusion method can be to extract features from ultrasonic parking space information, line parking space information and spatial parking space information, extract the shape, size and position features of the parking space from the line parking space information and spatial parking space information, extract the relative distance and angle features of the parking space from the ultrasonic parking space information, and synchronize the parking space information in time for correct association. Convert different parking space information into the same coordinate system for easy fusion. Finally, use the fusion algorithm to fuse the features to obtain the parking space semantic feature map, parking space occupancy grid map and parking space semantic point cloud map.

[0106] S222: Obstacle fusion is performed on the ultrasonic obstacle information, the surround obstacle and the forward obstacle to obtain an obstacle point cloud map.

[0107] The obstacle point cloud map that performs obstacle fusion can accurately identify the location, category, size and other information of the obstacles. Among them, the obstacle point cloud map is specifically used to represent obstacles in the environment, indicating the location and shape of obstacles in the environment. These obstacles may include other vehicles, pedestrians, trees or any objects that may block the path.

[0108] The main content of obstacle fusion is the same as the above-mentioned parking space fusion. Feature extraction can be to extract obstacle features from the surrounding obstacles and forward obstacles, such as shape, size, color, etc. The distance feature of the obstacle is extracted from the ultrasonic obstacle information.

[0109] After perceiving and fusing the environmental information, the processed information can be fed back to the car screen, which will display the perception fusion results in the parking environment, such as obstacles, parking spaces, weather, light visibility and other information.

[0110] After obtaining the current first perception fusion information, a memory path map matching the first perception fusion information may be retrieved from the stored memory path map set, and the steps may include:

[0111] Extract feature information from the first perceptual fusion information.

[0112] Calculate the similarity between the feature information and the features in the memory path graph collection.

[0113] According to the similarity calculation results, the memory path diagram that best matches the feature information is retrieved.

[0114] Extract key features from the first perception fusion information. These features may include road boundaries, traffic signs, lane lines, parking space lines, etc. Convert the extracted features into numerical form, create feature vectors, and calculate the similarity between the current feature vector and the feature vectors in the memory path map set. The similarity can be calculated using Euclidean distance, cosine similarity, and Mahalanobis distance. The memory path map corresponding to the feature vector in the memory path map set with the highest similarity can be used as the memory path map used for this parking.

[0115] The memory path map in the stored memory path map set may be actively created by the user when parking. The memory parking method may further include memory mapping the parking path according to the memory mapping instruction, which may specifically include:

[0116] The parking space information and ground marking information are obtained through the AVM surround view camera and the front view camera.

[0117] The vehicle's moving distance and direction are obtained through the vehicle's body odometer, and the vehicle's acceleration and angular velocity are obtained through the inertial measurement unit.

[0118] According to the moving distance, moving direction, acceleration and angular velocity, dead reckoning is performed to obtain the vehicle position and vehicle attitude.

[0119] According to the parking space information, ground sign information, vehicle position and vehicle posture, the vehicle driving path and the vehicle surrounding environment map are constructed to obtain a memory path map. Among them, the vehicle surrounding environment map includes roads, traffic signs, and parking spaces;

[0120] The memory route map is stored in the vehicle's fusion parking system for use in memory parking.

[0121] See also Fig.11 , memory mapping is to obtain environmental information for a parking path and construct a memory path map, which is stored in the fusion parking system 300. Memory mapping requires the support of surround view cameras, front view cameras, vehicle body odometers and inertial measurement units. The surround view camera and front view camera provide parking space, ground signs, and obstacle perception information, the vehicle body odometer provides visual inertial odometer information, and the vehicle body odometer performs dead reckoning together with the inertial measurement unit. Finally, the construction of the memory path map is completed, and the memory path map can include point cloud maps, vector maps, and trajectory data.

[0122] It is understandable that after retrieving the memory path map, it is not necessarily possible to directly use the memory path map for automatic parking. In the current situation, there may be new obstacles on the parking path of the memory path map. Therefore, it is necessary to plan the parking path according to the memory path map and the first perception fusion information. Step S240 may include:

[0123] Determine the vehicle's current position and the target parking space position.

[0124] Based on the first perception fusion information, a local map around the vehicle is constructed.

[0125] Identify and mark obstacles and ground landmarks from the local map.

[0126] Use a path planning algorithm to generate a path from the vehicle's current position to the target parking space.

[0127] First, based on the parking space information, the starting point and end point of the vehicle parking path are determined. Secondly, based on the first perception fusion information, which may include a parking space semantic feature map, a parking space occupancy grid map, a parking space semantic point cloud map, an obstacle point cloud map, ground signs, weather information, and light visibility, a local map around the vehicle is constructed through automatic driving processing 300. In the temporary local map, obstacles and ground signs are identified and marked, and combined with the above-mentioned matched memory path map, a path planning algorithm is used to generate a path from the vehicle's current position to the target parking space position.

[0128] The path planning algorithm can use A* algorithm, Dijkstra algorithm, RRT (rapid exploration random tree). Extract the road network structure that can be used for path planning in the temporary local map, and design a cost function to evaluate the cost of the path, which can include path length, estimated travel time, and safety. The expression can be:

[0129] Cost = w 1 *D+w 2 *T+w 3 *S

[0130] Among them, w 1 、w 2 、w 3 is the weight coefficient, D is the path length, T is the estimated travel time, and S is safety.

[0131] Use a path planning algorithm to search for a feasible path from the starting point to the end point on the road network.

[0132] It is understandable that after the parking path is selected, unexpected situations may still occur during the parking process. Therefore, during the parking process, the parking path may be adjusted in real time. Step S250 may include:

[0133] Calculate the vehicle position during the current parking process in real time and obtain the second perception fusion information.

[0134] According to the second perception fusion information, detect whether there is an obstacle in the parking path.

[0135] If so, the parking path is replanned and it is determined whether a new parking path can be planned.

[0136] If a new parking path can be planned, parking will continue; if a new parking path cannot be planned, parking will be stopped and an alarm will be sounded.

[0137] During parking, the fusion parking system will perform parking path following control. This process is to continuously detect the environment through the sensor 320, and fuse it to obtain the second perception fusion information. According to the second perception fusion information, the vehicle position in the current parking process will be estimated in real time. If there is an obstacle in the parking route, the parking path will be re-planned, the vehicle's turning angle and speed will be automatically adjusted, and the obstacle will be bypassed. If the obstacle cannot be bypassed and parking is not possible, the vehicle will stop and automatically avoid it, and an alarm will be issued. It can realize functions such as automatic speed adjustment, automatic turning, automatic bypassing of obstacles, and automatic parking avoidance.

[0138] After parking is successful, the actual parking path may be inconsistent with the planned parking path, and the user may also choose to enter the actual parking path into the fusion parking system 300. Specifically, the memory parking method may also include:

[0139] When memory parking is completed, it is determined whether the actual parking path is consistent with the planned parking path.

[0140] If not, an instruction is issued whether to enter the actual parking path.

[0141] If an input command is received, the actual parking path is entered into the fusion parking system.

[0142] During the actual parking process, the vehicle screen displays the results of the parking path planning and the parking path following control. The planned parking path and the actual parking path will be displayed in different colors on the vehicle screen. If the actual parking path is completely in accordance with the planned parking path, the two routes on the vehicle screen will almost overlap; if the actual parking path is different from the planned parking path due to environmental information, the two routes will not overlap, and the driver will need to choose whether to re-enter the new parking path.

[0143] In summary, the memory parking method, electronic device and storage medium provided by the embodiments of the present invention obtain parking environment information through sensors, perform perception fusion, plan parking paths according to the perception fusion information and the memory path map, perform parking, obtain new second perception fusion information in real time, adjust the parking path in real time according to the second perception fusion information, and store the actual parking path selection in the fusion parking system. Therefore, the specific location and obstacles of the parking space can be judged according to the parking environment information, and an adaptive parking control strategy can be automatically made according to the perceived environment information, thereby improving the user's parking success rate and improving the convenience of parking.

[0144] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a module, a program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0146] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a computer-readable storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0147] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0148] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A memory parking method, characterized in that: An autonomous driving processor applied to a fusion parking system, the fusion parking system further comprising a sensor, the sensor being communicatively connected to the autonomous driving processor, the method comprising: acquiring parking environment information based on the sensor; Performing perception fusion on the parking environment information to obtain first perception fusion information; According to the first perception fusion information, retrieving a memory path map matching the first perception fusion information from a stored memory path map set; Performing parking path planning according to the memory path map and the first perception fusion information to obtain a parking path; Parking is performed according to the parking path, and second perception fusion information is acquired in real time during the parking process, and the parking path is adjusted according to the second perception fusion information.

2. The method according to claim 1, characterized in that The fusion parking system further includes a micro control unit, the sensors include an AK2 ultrasonic radar, an AVM surround view camera, and a front view camera, the autonomous driving processor is respectively connected to the AVM surround view camera, the front view camera, and the micro control unit in communication, the micro control unit is connected to the AK2 ultrasonic radar in communication, and the parking environment information is obtained based on the sensors, including: Acquire ultrasonic environment information through the AK2 ultrasonic radar, and process it through the micro control unit to obtain ultrasonic parking space information and ultrasonic obstacle information; Acquire surround image information through the AVM surround camera, and obtain linear parking space information, spatial parking space information, surround obstacles and ground markings through surround stitching and surround point cloud processing; The forward-looking camera is used to obtain forward obstacles, lane lines, road surface markings, weather information, and light visibility.

3. The method according to claim 2, characterized in that The perception fusion information includes a parking space semantic feature map, a parking space occupancy grid map, a parking space semantic point cloud map, and an obstacle point cloud map. The step of perceiving and fusing the parking environment information to obtain the first perception fusion information includes: Performing parking space fusion on the ultrasonic parking space information, the line parking space information and the spatial parking space information to generate the parking space semantic feature map, the parking space occupancy grid map and the parking space semantic point cloud map; Obstacle fusion is performed on the ultrasonic obstacle information, the surround obstacle and the forward obstacle to obtain the obstacle point cloud map.

4. The method according to claim 1, characterized in that: The step of retrieving a memory path map matching the first perception fusion information from a stored memory path map set according to the first perception fusion information comprises: Extracting feature information from the first perceptual fusion information; Calculating the similarity between the feature information and the features in the memory path graph set; According to the similarity calculation result, the memory path diagram that best matches the feature information is retrieved.

5. The method according to claim 1, characterized in that The step of performing parking path planning according to the memory path map and the first perception fusion information to obtain a parking path includes: Determine the current position of the vehicle and the position of the target parking space; constructing a local map around the vehicle according to the first perception fusion information; Identifying and marking obstacles and ground markings from the local map; A path planning algorithm is used to generate a path from the current position of the vehicle to the target parking space position.

6. The method according to claim 1, characterized in that Parking according to the parking path, acquiring second perception fusion information in real time during the parking process, and adjusting the parking path according to the second perception fusion information includes: Calculate the vehicle position during the current parking process in real time and obtain the second perception fusion information; detecting whether there is an obstacle in the parking path according to the second perception fusion information; If yes, re-plan the parking path and determine whether a new parking path can be planned; If a new parking path can be planned, parking will continue; if a new parking path cannot be planned, parking will be stopped and an alarm will be sounded.

7. The method according to claim 2, characterized in that The method further includes the step of performing memory mapping of the parking path according to the memory mapping instruction, the step comprising: Acquire parking space information and ground marking information through the AVM surround view camera and the front view camera; The vehicle's moving distance and direction are obtained through the vehicle odometer, and the vehicle's acceleration and angular velocity are obtained through the inertial measurement unit; Performing dead reckoning according to the moving distance, the moving direction, the acceleration, and the angular velocity to obtain a vehicle position and a vehicle attitude; According to the parking space information, the ground sign information, the vehicle position and the vehicle posture, a vehicle driving path and a vehicle surrounding environment map are constructed to obtain a memory path map; the vehicle surrounding environment map includes roads, traffic signs and parking spaces; The memory path map is stored in the fusion parking system of the vehicle for use in memory parking.

8. The method according to claim 1, characterized in that The method further comprises: When memory parking is completed, determining whether the actual parking path is consistent with the planned parking path; If not, issuing an instruction on whether to enter the actual parking path; If an input instruction is received, the actual parking path is input into the fusion parking system.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images