Automatic auxiliary parking method and system based on cloud assistance, terminal and medium

Abstract

The invention provides an automatic auxiliary parking method and system based on cloud assistance, and the method comprises the steps: recording the data of a look-around camera, an ultrasonic radar, a wheel speed meter and inertial navigation during the automatic parking of a vehicle, uploading the corresponding data when the parking fails, and requesting the cloud parking calculation; the cloud server performs dead reckoning according to the look-around camera, the wheel speedometer and the inertial navigation data to obtain pose information; target detection is carried out according to the surround-view camera data, the size and position of an obstacle are obtained, and the obstacle is detected according to the ultrasonic radar data; the obstacles detected by the two are combined to obtain obstacle information; acquiring a panoramic look-around image based on the look-around camera data, acquiring parking space coordinate information in combination with the obstacle information, and sending the obstacle information and the parking space coordinate information to a vehicle end; a parking track is solved under the parking local coordinate system, and parking track information is sent to the vehicle end; and after the vehicle side fuses and verifies the information, a parking track is executed, and cloud-side-assisted automatic parking is completed.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving, and in particular to an automatic assisted parking method, system, terminal and medium based on cloud assistance. Background Art

[0002] The automatic parking assistance (APA) system can sense the surrounding environment, control the acceleration and deceleration of the vehicle and the steering angle, and automatically complete the parking task, thereby reducing the burden on human drivers and the occurrence of accidents such as scratches. Some commercial applications have been put into use. Most of the current APA systems are limited by the hardware computing power of the vehicle. In order to ensure real-time performance, complex perception and planning algorithms cannot be used. Therefore, parking space detection failure or path planning failure may occur in difficult scenarios such as small parking spaces and narrow traversable areas. Cloud servers can provide cheap and convenient computing power, perform real-time perception and planning in the cloud, and send commands to vehicles for execution. However, uploading sensor data and sending commands through cloud servers are easily affected by network instability and delay, which can easily cause safety hazards. Therefore, how to combine cloud computing power with vehicle-side computing power is an effective solution to solve automatic parking assistance.

[0003] Through searching, it is found that the Chinese invention patent application "Autonomous parking system and method based on cloud sharing and map fusion" with publication number CN112802346A collects camera images, uses simultaneous localization and mapping (SLAM) to establish semantic maps of parking routes and parking spaces, and classifies trajectory information to obtain trajectory maps, and generates target trajectory maps based on the selected trajectory maps, wherein the target trajectory map is a new trajectory map generated by fusion of the selected trajectory maps; the trajectory map is stored locally and / or in the cloud; autonomous parking parameters are set according to the target trajectory map; and repositioning is performed from coarse to fine according to the autonomous parking parameters. Although this method realizes the combination of cloud computing power and vehicle-side computing power, it requires the pre-establishment of semantic maps of parking routes and parking spaces. The autonomous parking function cannot be realized in parking lots where semantic maps have not been established, and the generalization is poor. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides an automatic assisted parking method, system, terminal and medium based on cloud assistance.

[0005] According to one aspect of the present invention, there is provided an automatic assisted parking method based on cloud assistance, comprising:

[0006] During the operation of the vehicle's APA, the surround camera data, ultrasonic radar data, wheel speed meter data, and inertial navigation data are recorded during the vehicle's automatic parking. When the automatic parking fails, the recorded data is uploaded and the cloud parking calculation is requested;

[0007] The cloud server performs dead reckoning based on the surround view camera data, wheel speed meter data and inertial navigation data to obtain the position information of the vehicle during the movement;

[0008] The cloud server performs 3D target detection based on the surround view camera data to obtain the size and position of surrounding obstacles, and detects surrounding obstacles based on the ultrasonic radar data; combines the obstacles detected by both to obtain obstacle information; obtains a panoramic surround view image based on the surround view camera data, and obtains parking space coordinate information in combination with the obstacle information, and sends the obstacle information and the parking space coordinate information to the vehicle end;

[0009] The cloud server uses the center of the rear axle of the vehicle as the starting point and the center of the rear axle of the vehicle corresponding to the target parking space as the end point, solves the parking trajectory in the parking local coordinate system, and sends the parking trajectory information to the vehicle end;

[0010] After the vehicle obtains the obstacle information, vehicle coordinate information and parking trajectory information sent by the cloud server and integrates and verifies them, the vehicle APA executes the parking trajectory to complete cloud-assisted automatic parking.

[0011] Preferably, performing dead reckoning based on the surround view camera data, wheel speed meter data and inertial navigation data to obtain the position information of the vehicle during movement includes:

[0012] The initial position of the vehicle is taken as the origin of the map coordinate system; the surround camera image is preprocessed and feature extracted to obtain feature points, and the relative position and posture changes of the camera at different time points are estimated by matching the feature points between consecutive frames of the image. The wheel speed meter data and inertial navigation data are combined to apply Kalman filtering or factor graph optimization to solve the current camera position and posture, and the vehicle's position and posture information (x, y, yaw) is solved through prior calibration data, and the posture information P = {(x1, y1, yaw1), ..., (xN, yN, yawN)} at each moment is stored.

[0013] Preferably, performing 3D target detection according to the surround camera data to obtain the size and position of surrounding obstacles, and detecting surrounding obstacles according to the ultrasonic radar data; merging the obstacles detected by both to obtain obstacle information, includes:

[0014] Use multi-view 3D target detection method to detect surrounding objects from the surround camera image and obtain the bounding box information of each obstacle;

[0015] Calculate the location information of the detected obstacle based on the measurement distance of the ultrasonic radar and the installation position and orientation of the ultrasonic radar;

[0016] Both obstacles are transformed into the map coordinate system through the current vehicle posture (x, y, yaw), the detected obstacles are merged, and the obtained bounding box information and position information are used as obstacle information.

[0017] Preferably, the acquiring of the panoramic surround image based on the surround view camera data and obtaining the parking space coordinate information in combination with the obstacle information includes:

[0018] The surround view camera data with pre-calibrated camera internal and external parameters is transformed into a bird's-eye view and stitched to obtain a panoramic surround view image, including:

[0019] For each camera, the fisheye camera image image is dedistorted by using the pre-calibrated internal parameters to obtain image image1; the fisheye camera external parameters are used to obtain the bird's-eye view image image2 through the perspective transformation algorithm for image1; all the obtained images image2 are superimposed to obtain the panoramic view image from the bird's-eye view;

[0020] The parking space is detected by deep learning, and combined with the obstacle information, appropriate parking space coordinate information is obtained, including:

[0021] The obtained panoramic surround image is used for parking space detection using a deep learning algorithm to obtain the four vertices point1 to point4 of the parking space in the parking space coordinate system. The vehicle posture (x, y, yaw) is transformed into the map coordinate system containing obstacle information to obtain vertices point1' to point4', and the center slot_center and direction slot_yaw of the parking space are calculated to obtain the parking space coordinate information.

[0022] Preferably, the method of solving the parking trajectory in the parking local coordinate system with the center of the rear axle of the vehicle as the starting point and the center position of the rear axle of the vehicle corresponding to the target parking space as the end point includes:

[0023] A grid map M of appropriate size is established in the map coordinate system, and the positions overlapping with the detected obstacle information are set as occupied. The current vehicle position and orientation are used as the starting state S, and the detected parking space center and orientation are used as the end state G. A collision-free path T from the starting point S to the end point G is planned in the map M using the hybrid A* algorithm.

[0024] Preferably, obtaining the obstacle information, the vehicle coordinate information and the parking trajectory information and fusing and verifying them includes:

[0025] Calculate whether each path point of path T collides with any obstacle in the obstacle information. If no collision occurs, the test passes and is handed over to the underlying controller for execution.

[0026] Preferably, the automatic parking failure includes:

[0027] If parking space detection fails or path planning fails, the vehicle stops moving.

[0028] According to another aspect of the present invention, an automatic assisted parking system based on cloud assistance is provided, characterized in that it includes: an APA module and a vehicle-cloud fusion module arranged on the vehicle side, and a positioning module, a perception module and a planning module arranged on the cloud server; wherein:

[0029] The APA module is used to record the surround camera data, ultrasonic radar data, wheel speed meter data and inertial navigation data when the vehicle is automatically parking, and upload the recorded data to the cloud server and request cloud parking calculation when the automatic parking fails; execute the parking trajectory verified by the vehicle-cloud fusion module;

[0030] The vehicle-cloud fusion module is used to obtain the obstacle information, vehicle coordinate information and parking trajectory information sent by the cloud server, fuse and verify them, and then send the verified parking trajectory information to the APA module;

[0031] The positioning module is used to perform dead reckoning based on the surround view camera data, wheel speed meter data and inertial navigation data to obtain the position information of the vehicle during movement;

[0032] The perception module is used to perform 3D target detection based on the surround view camera data, obtain the size and position of surrounding obstacles, and detect surrounding obstacles based on the ultrasonic radar data; merge the obstacles detected by both to obtain obstacle information; obtain a panoramic surround view image based on the surround view camera data, and obtain parking space coordinate information in combination with the obstacle information, and send the obstacle information and the parking space coordinate information to the vehicle-cloud fusion module;

[0033] The planning module is used to solve the parking trajectory in the parking local coordinate system with the center of the vehicle's rear axle as the starting point and the center position of the vehicle's rear axle corresponding to the target parking space as the end point, and send the parking trajectory information to the vehicle-cloud fusion module.

[0034] According to a third aspect of the present invention, there is provided a computer terminal, comprising a vehicle-mounted terminal and a cloud computing platform terminal; wherein the vehicle-mounted terminal and the cloud computing platform terminal both comprise: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it can be used to execute any one of the methods described above in the present invention, or to run the system described above in the present invention.

[0035] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute any of the methods described above in the present invention, or to run the system described above in the present invention.

[0036] Due to the adoption of the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0037] The cloud-assisted automatic assisted parking method, system, terminal and medium provided by the present invention utilize cloud computing power to assist in solving the vehicle automatic parking task. By uploading vehicle-side information and solving it in the cloud, a better-performing automatic assisted parking algorithm is implemented, thereby improving the success rate of the automatic assisted parking task.

[0038] The cloud-assisted automatic assisted parking method, system, terminal and medium provided by the present invention integrate the information of the vehicle end and the cloud end to improve the accuracy of the information and ensure the safety of the parking task.

[0039] The cloud-assisted automatic assisted parking method, system, terminal and medium provided by the present invention request cloud-assisted parking calculation when APA automatic parking fails, without the need to continuously communicate with the cloud computing platform, thereby reducing communication costs and delay effects and improving the robustness of the system.

[0040] The cloud-assisted automatic assisted parking method, system, terminal and medium provided by the present invention do not require complicated modifications to the vehicle side, can adapt to a variety of different vehicle models, and have low additional costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0042] Figure 1 4 is a flowchart of an automatic assisted parking method based on cloud assistance in one embodiment of the present invention.

[0043] Figure 2 The figure is a flowchart of an automatic assisted parking method based on cloud assistance in a preferred embodiment of the present invention.

[0044] Figure 3 Schematic diagram of component modules of an automatic assisted parking system based on cloud assistance in one embodiment of the present invention.

[0045] Figure 4 It is a working schematic diagram of an automatic assisted parking method and system based on cloud assistance in a specific application example of the present invention. DETAILED DESCRIPTION

[0046] The following is a detailed description of the embodiments of the present invention: This embodiment is implemented on the premise of the technical solution of the present invention, and a detailed implementation method and a specific operation process are given. It should be pointed out that for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0047] The automatic assisted parking (APA) system is an important application of the active anti-collision system of automobiles in low-speed and complex urban environments, and is also a specific embodiment of the intelligence of the active anti-collision system of automobiles. Most of the current APA systems are limited by the hardware computing power of the vehicle, and when using cloud computing power, they are easily affected by network instability and delays. In response to the above problems, an embodiment of the present invention provides an automatic assisted parking method based on cloud assistance. This method combines the computing power of the vehicle with the computing power of the cloud, and improves the success rate of automatic assisted parking through cloud assistance to address the problems that insufficient computing power on the vehicle side affects the parking success rate, and that the cloud server is easily affected by network instability and delays when uploading sensor data and issuing instructions.

[0048] Specifically, Figure 1 As shown, the automatic assisted parking method based on cloud assistance provided by this embodiment may include the following operations:

[0049] S1, during the operation of vehicle APA, the surround camera data, ultrasonic radar data, wheel speed meter data and inertial navigation data of the vehicle during automatic parking are recorded, and when automatic parking fails, the recorded data is uploaded and cloud parking calculation is requested;

[0050] S2, the cloud server performs dead reckoning based on the surround camera data, wheel speed meter data, and inertial navigation data to obtain the vehicle's position information during motion;

[0051] S3, the cloud server performs 3D target detection based on the surround view camera data, obtains the size and position of surrounding obstacles, and detects surrounding obstacles based on the ultrasonic radar data; merges the obstacles detected by the two to obtain obstacle information; obtains a panoramic surround image based on the surround view camera data, and obtains parking space coordinate information in combination with the obstacle information, and sends the obstacle information and parking space coordinate information to the vehicle end;

[0052] S4, the cloud server uses the center of the rear axle of the vehicle as the starting point and the center of the rear axle of the vehicle corresponding to the target parking space as the end point, solves the parking trajectory in the parking local coordinate system, and sends the parking trajectory information to the vehicle end;

[0053] S5, the vehicle side obtains the obstacle information, vehicle coordinate information and parking trajectory information sent by the cloud server, integrates and verifies them, and then executes the parking trajectory through the vehicle APA to complete cloud-assisted automatic parking.

[0054] In some preferred implementations, the above S1, automatic parking failure, includes:

[0055] If parking space detection fails or path planning fails, the vehicle stops moving.

[0056] In some preferred embodiments, the above S2, performing dead reckoning based on the surround camera data, the wheel speed meter data and the inertial navigation data to obtain the position information of the vehicle during the movement, may further include the following operations:

[0057] The visual localization and mapping technology (VSLAM) is used, which includes the following steps:

[0058] The initial position of the vehicle is taken as the origin of the map coordinate system; the surround camera image is preprocessed and feature extracted to obtain feature points, and the relative position and posture changes of the camera at different time points are estimated by matching the feature points between consecutive frames of the image. The wheel speed meter data and inertial navigation data are combined to apply Kalman filtering or factor graph optimization to solve the current camera position and posture, and the vehicle's position and posture information (x, y, yaw) is solved through prior calibration data, and the posture information P = {(x1, y1, yaw1), ..., (xN, yN, yawN)} at each moment is stored.

[0059] In some preferred embodiments, the above S3, performing 3D target detection based on the surround camera data to obtain the size and position of surrounding obstacles, and detecting surrounding obstacles based on the ultrasonic radar data; merging the obstacles detected by both to obtain obstacle information, may further include the following operations:

[0060] Use multi-view 3D target detection methods to detect surrounding objects from the surround camera image and obtain the bounding box information of each obstacle, including: center position, orientation and size information;

[0061] The position information of the detected obstacle is calculated according to the measurement distance of the ultrasonic radar and the installation position and orientation of the ultrasonic radar; wherein the obstacle can be regarded as a cylinder, the center position of the cylinder is the obstacle position, and the radius can be selected as a smaller value;

[0062] Both obstacles are transformed into the map coordinate system through the current vehicle posture (x, y, yaw), the detected obstacles are merged, and the obtained bounding box information and position information are used as obstacle information, denoted as O.

[0063] In some preferred implementations, the above S3, acquiring a panoramic surround image based on surround view camera data and obtaining parking space coordinate information in combination with obstacle information, may further include:

[0064] The surround view camera data with pre-calibrated camera internal and external parameters is transformed into a bird's-eye view and stitched to obtain a panoramic surround view image; the specific steps include:

[0065] For each camera, the fisheye camera image image is dedistorted by using the pre-calibrated internal parameters to obtain image image1; the fisheye camera external parameters are used to obtain the bird's-eye view image image2 through the perspective transformation algorithm for image1; all the obtained images image2 are superimposed to obtain the panoramic view image from the bird's-eye view;

[0066] The parking space is detected by deep learning, and the appropriate parking space coordinate information is obtained by combining the obstacle information. The specific steps include:

[0067] The panoramic surround image obtained in the previous step is used to detect the parking space using a deep learning algorithm (such as the YOLO series, GCN Parking Slot, etc.), and the four vertices point1 to point4 of the parking space in the parking space coordinate system are obtained. The vertices point1' to point4' are obtained by transforming the vehicle posture (x, y, yaw) to the map coordinate system containing obstacle information, and the center slot_center and the direction slot_yaw of the parking space are calculated to obtain the parking space coordinate information.

[0068] In some preferred embodiments, the above S4, taking the center of the rear axle of the vehicle as the starting point and the center of the rear axle of the vehicle corresponding to the target parking space as the end point, solving the parking trajectory in the parking local coordinate system may further include the following operations:

[0069] The hybrid A* path optimization algorithm is used to solve the smooth parking trajectory in the parking local coordinate system, which specifically includes the following steps:

[0070] The hybrid A* path optimization algorithm is used to solve the smooth parking trajectory in the parking local coordinate system, which specifically includes the following steps:

[0071] A grid map M of appropriate size is established in the map coordinate system, and the position overlapping with the obstacle information O detected in S3 is set as occupied; the current vehicle position and orientation are used as the starting state S, and the detected parking space center and orientation are used as the end state G, and a collision-free path T from the starting point S to the end point G is planned in the map M using the hybrid A* algorithm.

[0072] In some preferred implementations, the above S5, obtaining obstacle information, parking space coordinate information and parking trajectory information and fusing and verifying them, may further include the following operations:

[0073] Calculate whether each path point of path T collides with any obstacle in obstacle information O. If no collision occurs, the test passes and is handed over to the underlying controller for execution.

[0074] An embodiment of the present invention provides an automatic assisted parking system based on cloud assistance.

[0075] Specifically, Figure 2 As shown, the automatic parking assistance system based on cloud assistance provided by this embodiment may include: an APA module and a vehicle-cloud fusion module arranged on the vehicle side, and a positioning module, a perception module and a planning module arranged on the cloud server; wherein:

[0076] The APA module is used to record the surround camera data, ultrasonic radar data, wheel speed meter data and inertial navigation data during the vehicle's automatic parking. When the automatic parking fails, it uploads the recorded data to the cloud server and requests cloud parking calculation; executes the parking trajectory after verification by the vehicle-cloud fusion module;

[0077] The vehicle-cloud fusion module is used to obtain the obstacle information, vehicle coordinate information and parking trajectory information sent by the cloud server, fuse and verify them, and then send the verified parking trajectory information to the APA module;

[0078] The positioning module is used to perform dead reckoning based on the surround camera data, wheel speed meter data and inertial navigation data to obtain the vehicle's position information during movement;

[0079] The perception module is used to detect 3D targets based on the surround-view camera data, obtain the size and position of surrounding obstacles, and detect surrounding obstacles based on the ultrasonic radar data; merge the obstacles detected by the two to obtain obstacle information; obtain a panoramic surround image based on the surround-view camera data, and obtain parking space coordinate information in combination with the obstacle information, and send the obstacle information and parking space coordinate information to the vehicle-cloud fusion module;

[0080] The planning module is used to solve the parking trajectory in the local parking coordinate system with the center of the vehicle's rear axle as the starting point and the center of the vehicle's rear axle corresponding to the target parking space as the end point, and send the parking trajectory information to the vehicle-cloud fusion module.

[0081] It should be noted that the steps in the method provided by the present invention can be implemented by using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system, that is, the embodiments in the method can be understood as preferred examples for constructing the system, which will not be elaborated here.

[0082] The technical solution provided by the above embodiment of the present invention is further described in detail below in conjunction with a specific application example.

[0083] like Figure 3 As shown, combined with Figure 2 As shown, the cloud-assisted automatic assisted parking system adopted in this specific application example, based on the technical solution provided by the above-mentioned embodiment of the present invention, includes:

[0084] -Vehicle-side module

[0085] 1. APA module:

[0086] The vehicle uses surround-view cameras and ultrasonic radars to perceive the environment, determine the location of obstacles and parking spaces, and use wheel speed meters and inertial navigation for local positioning to plan a collision-free path, control the movement of the vehicle into the parking space, and complete APA. If the parking space detection fails or the path planning fails, the vehicle stops moving and uploads the sensor data during the parking process to the cloud computing platform.

[0087] 2. Car-cloud fusion module:

[0088] After the vehicle fails to park automatically, it obtains the parking perception and planning results from the cloud. It integrates the obstacle perception information to obtain accurate obstacle information to prevent scratches and collisions; integrates the parking space perception information to obtain the accurate parking space location to improve the accuracy of parking; integrates the path planning information to obtain a safe, collision-free, smooth and comfortable trajectory, and the APA module executes the trajectory.

[0089] -Cloud Module

[0090] 1. Positioning module:

[0091] The starting point of automatic parking is used as the coordinate origin of the local parking coordinate system. The wheel speed meter, inertial navigation and surround camera images are used to obtain the vehicle's posture information during the parking process using visual simultaneous localization and mapping (VSLAM) technology. A local parking coordinate system is established with the origin when the vehicle starts the APA function.

[0092] 2. Perception module:

[0093] The vehicle's surround view image is obtained through four surround view cameras, and 3D target detection is performed in the surround view image to obtain the size and position of surrounding obstacles. At the same time, ultrasonic waves are used to detect obstacles around the vehicle, and the obstacles detected by the two are merged. The camera image is converted into a bird's-eye view through the pre-calibrated camera internal and external parameters and stitched to obtain a panoramic surround view image. A deep learning algorithm is used in the image to detect parking spaces, and the coordinates of suitable parking spaces are obtained in combination with obstacle information. Combined with the vehicle's own posture, the obstacle information and parking space information are transformed into the local parking coordinate system.

[0094] 3. Planning module

[0095] The center of the rear axle of the vehicle is taken as the starting point, and the center of the rear axle corresponding to the parking space selected by the user is taken as the end point. In the local parking coordinate system, hybrid A*, path optimization and other algorithms are used to solve a safe and smooth parking trajectory, and the trajectory is sent to the vehicle. With the support of the cloud computing platform, a method with higher computational complexity can be used to calculate the parking trajectory in difficult scenarios.

[0096] Through the above-mentioned cloud-assisted automatic assisted parking system, a corresponding automatic assisted parking method is implemented, such as Figure 4 As shown, combined with Figure 3 As shown, the following steps are included:

[0097] S1, Figure 3 There is a vehicle that fails to perform the automatic assisted parking task. After the APA failure, the vehicle uploads the surround camera, ultrasonic radar, wheel speed meter, and inertial navigation data recorded during the parking process to the cloud computing platform and requests the cloud parking computing service.

[0098] S2, the cloud uses VSLAM technology to calculate the vehicle's position information during movement based on the uploaded data from the wheel speed meter, inertial navigation and surround view camera, and establishes a local parking map.

[0099] S3, 3D target detection is performed based on the image of the surround view camera, and combined with the detection results of the ultrasonic radar, the two cars in the parking space are detected as obstacles. Through the pre-calibrated camera internal and external parameters, the camera image is processed into a bird's-eye view and stitched to obtain a panoramic surround image. Using the deep learning method, it can be detected that there are 3 parking spaces around. Combined with the obstacle information, parking space 2 is selected as the available parking space, and the obstacle information is sent to the vehicle.

[0100] S4, taking the center of the rear axle of the vehicle as the starting point and the center of the rear axle of the vehicle corresponding to the target parking space as the end point, use the hybrid A\* algorithm to solve a safe and smooth parking trajectory and send the trajectory to the vehicle.

[0101] S5, the vehicle obtains the parking perception and planning results from the cloud, and integrates the obstacle perception information, parking space perception information, and path planning information. After the information is integrated and verified, the APA module executes the parking trajectory.

[0102] The automatic assisted parking method, system, terminal and medium based on cloud assistance provided by the above-mentioned embodiments of the present invention use cloud computing power to assist in solving the vehicle automatic parking task. By uploading the vehicle-side information and solving it in the cloud, a better-performing automatic assisted parking algorithm is implemented, thereby improving the success rate of the automatic assisted parking task. By integrating the vehicle-side information with the cloud, the accuracy of the information is improved, and the safety of the parking task is guaranteed. When APA automatic parking fails, cloud-assisted parking calculation is requested without continuous communication with the cloud computing platform, thereby reducing communication costs and delay effects and improving the robustness of the system. There is no need to make complex modifications to the vehicle-side, and it can adapt to a variety of different vehicle models with low additional costs.

[0103] An embodiment of the present invention provides a computer terminal, including a vehicle-mounted terminal and a cloud computing platform terminal; the vehicle-mounted terminal and the cloud computing platform terminal both include: a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, it can be used to execute any method of the above-mentioned embodiments of the present invention, or, to run any system of the above-mentioned embodiments of the present invention.

[0104] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0105] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0106] The processor is used to execute the computer program stored in the memory to implement the various steps of the method or various modules of the system involved in the above embodiments. For details, please refer to the relevant descriptions in the above method and system embodiments.

[0107] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0108] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it can be used to execute any method in the above embodiments of the present invention, or to run any system in the above embodiments of the present invention.

[0109] All matters not covered in the above embodiments of the present invention are well known in the art.

[0110] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. An automatic assisted parking method based on cloud assistance, characterized in that: include: During the operation of the vehicle's APA, the surround camera data, ultrasonic radar data, wheel speed meter data, and inertial navigation data are recorded during the vehicle's automatic parking. When the automatic parking fails, the recorded data is uploaded and the cloud parking calculation is requested; The cloud server performs dead reckoning based on the surround view camera data, wheel speed meter data and inertial navigation data to obtain the position information of the vehicle during the movement; The cloud server performs 3D target detection based on the surround view camera data to obtain the size and position of surrounding obstacles, and detects surrounding obstacles based on the ultrasonic radar data; combines the obstacles detected by both to obtain obstacle information; obtains a panoramic surround view image based on the surround view camera data, and obtains parking space coordinate information in combination with the obstacle information, and sends the obstacle information and the parking space coordinate information to the vehicle end; The cloud server uses the center of the rear axle of the vehicle as the starting point and the center of the rear axle of the vehicle corresponding to the target parking space as the end point, solves the parking trajectory in the parking local coordinate system, and sends the parking trajectory information to the vehicle end; After the vehicle obtains the obstacle information, vehicle coordinate information and parking trajectory information sent by the cloud server and integrates and verifies them, the vehicle APA executes the parking trajectory to complete cloud-assisted automatic parking.

2. The cloud-assisted automatic parking method according to claim 1, characterized in that: The dead reckoning is performed according to the surround view camera data, the wheel speed meter data and the inertial navigation data to obtain the position information of the vehicle during the movement, including: The initial position of the vehicle is taken as the origin of the map coordinate system; the surround camera image is preprocessed and feature extracted to obtain feature points, and the relative position and posture changes of the camera at different time points are estimated by matching the feature points between consecutive frames of the image. The wheel speed meter data and inertial navigation data are combined to apply Kalman filtering or factor graph optimization to solve the current camera position and posture, and the vehicle's position and posture information (x, y, yaw) is solved through prior calibration data, and the posture information P = {(x1, y1, yaw1), ..., (xN, yN, yawN)} at each moment is stored.

3. The cloud-assisted automatic parking method according to claim 1, characterized in that: The 3D target detection is performed based on the surround camera data to obtain the size and position of surrounding obstacles, and the surrounding obstacles are detected based on the ultrasonic radar data; The obstacles detected by both are combined to obtain obstacle information, including: Use multi-view 3D target detection method to detect surrounding objects from the surround camera image and obtain the bounding box information of each obstacle; Calculate the location information of the detected obstacle based on the measurement distance of the ultrasonic radar and the installation position and orientation of the ultrasonic radar; Both obstacles are transformed into the map coordinate system through the current vehicle posture (x, y, yaw), the detected obstacles are merged, and the obtained bounding box information and position information are used as obstacle information.

4. The cloud-assisted automatic parking method according to claim 1, characterized in that: The method of acquiring a panoramic view image based on the surround view camera data and obtaining parking space coordinate information in combination with the obstacle information includes: The surround view camera data with pre-calibrated camera internal and external parameters is transformed into a bird's-eye view and stitched to obtain a panoramic surround view image, including: For each camera, the fisheye camera image image is dedistorted by using the pre-calibrated internal parameters to obtain image image1; the fisheye camera external parameters are used to obtain the bird's-eye view image image2 through the perspective transformation algorithm for image1; all the obtained images image2 are superimposed to obtain the panoramic view image from the bird's-eye view; The parking space is detected by deep learning, and combined with the obstacle information, appropriate parking space coordinate information is obtained, including: The obtained panoramic surround image is used for parking space detection using a deep learning algorithm to obtain the four vertices point1 to point4 of the parking space in the parking space coordinate system. The vehicle posture (x, y, yaw) is transformed into the map coordinate system containing obstacle information to obtain vertices point1' to point4', and the center slot_center and direction slot_yaw of the parking space are calculated to obtain the parking space coordinate information.

5. The cloud-assisted automatic parking method according to claim 1, characterized in that: The method of solving the parking trajectory in the parking local coordinate system with the center of the rear axle of the vehicle as the starting point and the center of the rear axle of the vehicle corresponding to the target parking space as the end point includes: A grid map M of appropriate size is established in the map coordinate system, and the positions overlapping with the detected obstacle information are set as occupied. The current vehicle position and orientation are used as the starting state S, and the detected parking space center and orientation are used as the end state G. A collision-free path T from the starting point S to the end point G is planned in the map M using the hybrid A* algorithm.

6. The cloud-assisted automatic parking method according to claim 1, characterized in that: Obtaining the obstacle information, vehicle coordinate information, and parking trajectory information and fusing and verifying them includes: Calculate whether each path point of path T collides with any obstacle in the obstacle information. If no collision occurs, the test passes and is handed over to the underlying controller for execution.

7. The cloud-assisted automatic assisted parking method according to any one of claims 1 to 6, characterized in that: The automatic parking failure includes: If parking space detection fails or path planning fails, the vehicle stops moving.

8. An automatic assisted parking system based on cloud assistance, characterized in that: include: The APA module and vehicle-cloud fusion module are arranged on the vehicle side, and the positioning module, perception module and planning module are arranged on the cloud server; among which: The APA module is used to record the surround camera data, ultrasonic radar data, wheel speed meter data and inertial navigation data when the vehicle is automatically parking, and upload the recorded data to the cloud server and request cloud parking calculation when the automatic parking fails; execute the parking trajectory verified by the vehicle-cloud fusion module; The vehicle-cloud fusion module is used to obtain the obstacle information, vehicle coordinate information and parking trajectory information sent by the cloud server, fuse and verify them, and then send the verified parking trajectory information to the APA module; The positioning module is used to perform dead reckoning based on the surround view camera data, wheel speed meter data and inertial navigation data to obtain the position information of the vehicle during movement; The perception module is used to perform 3D target detection based on the surround view camera data, obtain the size and position of surrounding obstacles, and detect surrounding obstacles based on the ultrasonic radar data; merge the obstacles detected by both to obtain obstacle information; obtain a panoramic surround view image based on the surround view camera data, and obtain parking space coordinate information in combination with the obstacle information, and send the obstacle information and the parking space coordinate information to the vehicle-cloud fusion module; The planning module is used to solve the parking trajectory in the parking local coordinate system with the center of the vehicle's rear axle as the starting point and the center position of the vehicle's rear axle corresponding to the target parking space as the end point, and send the parking trajectory information to the vehicle-cloud fusion module.

9. A computer terminal, comprising a vehicle-mounted terminal and a cloud computing platform terminal; wherein: The vehicle-mounted terminal and the cloud computing platform terminal both include: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor can be used to execute the method described in any one of claims 1 to 7, or to run the system described in claim 8 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it can be used to perform the method described in any one of claims 1 to 7, or to run the system described in claim 8.

Images