Parking control method and device, automatic parking auxiliary controller, terminal and system

Abstract

The application discloses a parking control method, device, automatic parking auxiliary controller, terminal and system. The parking control method is applied to an automatic parking auxiliary (APA) controller and includes the following steps: determining parking space information according to images collected by multiple cameras arranged on a vehicle; sending the parking space information, radar data and an electronic map constructed according to the images to a terminal, so that the terminal generates parking control information according to the parking space information, the radar data and the electronic map; and controlling the vehicle to park in a target parking space according to the parking control information. The technical scheme saves the hardware resources of the vehicle and improves the user experience by sending the parking space information, the radar data and the electronic map to the terminal through the APA controller, generating the parking control information in the terminal and returning the parking control information to the APA controller, so that the APA controller controls the vehicle to park in the target parking space according to the parking control information.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, and in particular to a parking control method, device, automatic parking assist controller, terminal and system. Background Technology

[0002] With the development of vehicle control technology, automatic parking technology is gradually being integrated into more and more vehicle models. Automatic parking technology uses radar and cameras installed on the vehicle to identify parking spaces, and plans the parking trajectory through a control algorithm. It then controls the vehicle to complete the automatic parking process by communicating with the vehicle's sensors. Specifically, the control algorithm can plan the parking trajectory using parking space information, radar data, and electronic maps to control the vehicle and complete the automatic parking.

[0003] In existing technologies, the control algorithms for automatic parking are mostly deployed on the vehicle's Automatic Park Assist (APA) controller, making the vehicle's hardware relatively complex. When the control algorithm is improved and needs to be updated to the vehicle, it is necessary to consider whether the vehicle's hardware resources can support the updated control algorithm. Furthermore, updating the control algorithm requires multiple remote upgrades via Over-The-Air (OTA) technology, making the upgrade process cumbersome. Summary of the Invention

[0004] This invention provides a parking control method, device, automatic parking assist controller, terminal, and system. By setting the control algorithm on the terminal, it saves vehicle hardware resources, solves the problem of the cumbersome upgrade of the control algorithm in the vehicle's APA controller, and improves the user experience.

[0005] In a first aspect, embodiments of the present invention provide a parking control method applied to an automatic parking assist (APA) controller, comprising:

[0006] Parking space information is determined based on images captured by multiple cameras installed on the vehicle;

[0007] The parking space information, radar data, and electronic map constructed based on the image are sent to the terminal, so that the terminal can generate parking control information based on the parking space information, radar data, and electronic map;

[0008] The vehicle is controlled to park in the target parking space based on the parking control information.

[0009] Secondly, embodiments of the present invention provide a parking control method, applied to a terminal, comprising:

[0010] It receives parking space information, radar data, and an electronic map constructed from images collected by multiple cameras from the Automatic Parking Assist (APA) controller.

[0011] Parking control information is generated based on the parking space information, the radar data, and the electronic map;

[0012] The parking control information is returned to the APA controller so that the APA controller can control the vehicle to park in the target parking space according to the parking control information.

[0013] Thirdly, embodiments of the present invention provide a parking control device, comprising:

[0014] The parking space information determination module is used to determine parking space information based on images captured by multiple cameras installed on the vehicle;

[0015] The information sending module is used to send the parking space information, radar data, and electronic map constructed based on the image to the terminal, so that the terminal can generate parking control information based on the parking space information, radar data, and electronic map;

[0016] The control module is used to control the vehicle to park in the target parking space based on the parking control information.

[0017] Fourthly, embodiments of the present invention provide a parking control device, comprising:

[0018] The information receiving module is used to receive parking space information, radar data, and electronic maps constructed based on images collected by multiple cameras sent by the Automatic Parking Assist (APA) controller.

[0019] A parking control information generation module is used to generate parking control information based on the parking space information, the radar data, and the electronic map;

[0020] The information return module is used to return parking control information to the APA controller, so that the APA controller can control the vehicle to park in the target parking space according to the parking control information.

[0021] Fifthly, embodiments of the present invention provide an automatic parking assist controller, comprising:

[0022] At least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0023] The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the parking control method as described in the first aspect.

[0024] Sixthly, embodiments of the present invention provide a terminal, including:

[0025] At least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0026] The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the parking control method as described in the second aspect.

[0027] In a seventh aspect, embodiments of the present invention provide a parking control system, including an APA controller as described in the fifth aspect and a terminal as described in the sixth aspect.

[0028] The technical solution of this invention sends parking space information, radar data, and electronic maps to the terminal through the APA controller. The terminal generates parking control information and returns it to the APA controller, so that the APA controller can control the vehicle to park in the target parking space according to the parking control information, which saves vehicle hardware resources and improves user experience.

[0029] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart of a parking control method provided in Embodiment 1 of the present invention;

[0032] Figure 2 This is a schematic diagram of determining parking space information based on images captured by a camera installed on a vehicle, according to Embodiment 1 of the present invention.

[0033] Figure 3 This is a flowchart of a parking control method provided according to Embodiment 2 of the present invention;

[0034] Figure 4 This is a schematic diagram of a vehicle parking into a target parking space according to parking control information, provided in Embodiment 2 of the present invention;

[0035] Figure 5This is a schematic diagram of another vehicle parking into a target parking space according to parking control information, provided by Embodiment 2 of the present invention;

[0036] Figure 6 This is a schematic diagram of the connection between the APA controller and the mobile phone according to Embodiment 2 of the present invention;

[0037] Figure 7 This is a flowchart of another parking control method provided according to Embodiment 2 of the present invention;

[0038] Figure 8 This is a schematic diagram of a parking control device according to Embodiment 3 of the present invention;

[0039] Figure 9 This is a schematic diagram of a parking control device according to Embodiment 4 of the present invention;

[0040] Figure 10 This is a schematic diagram of the structure of an automatic parking assist controller that implements the parking control method of this invention.

[0041] Figure 11 This is a schematic diagram of the structure of a terminal that implements the parking control method of this invention.

[0042] Figure 12 This is a schematic diagram of the structure of a parking control system that implements the parking control method of the present invention. Detailed Implementation

[0043] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0044] It should be noted that the terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0045] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0046] Example 1

[0047] Figure 1 This is a flowchart of a parking control method according to Embodiment 1 of the present invention. This embodiment is applicable to situations where an Automatic Parking Assist (APA) controller controls a vehicle to perform automatic parking. The method can be executed by a parking control device, which can be implemented in software and / or hardware and integrated into the APA controller. The APA controller can perceive the parking environment through cameras and radar, and automatically or manually set a target parking space according to the user's selection, enabling the vehicle to automatically park along a parking trajectory until it reaches the final target parking space.

[0048] like Figure 1 As shown, the method includes:

[0049] S110. Determine parking space information based on images captured by multiple cameras installed on the vehicle.

[0050] Here, "camera" can refer to a video input device. "Image" can refer to all graphics and images captured by the camera; images can be photographs or videos. "Parking space information" can refer to information about where vehicles can park.

[0051] There is no limit to the number of cameras installed on the vehicle, as long as the images captured by the cameras can be used to determine parking space information. For example, the number of cameras on the vehicle can be determined based on actual application requirements.

[0052] There are no restrictions on the type of camera, as long as it can capture images, such as a wide-angle camera, fisheye camera, monocular camera, binocular camera, or ordinary viewing angle camera.

[0053] The location of cameras installed on vehicles is not limited, as long as the images captured by the cameras can determine parking space information. The specific location of the cameras on the vehicle can be determined based on the number of cameras or actual application needs. Cameras in different locations can have different functions. For example, a camera installed on the windshield can automatically detect pedestrians, vehicles, road signs, or traffic lights; another example is that cameras installed on the sides of the vehicle can be used for blind spot monitoring. Since the range of a vehicle's rearview mirror is limited, blind spots can easily form in areas not visible in the rearview mirror. By installing cameras on the sides of the vehicle, blind spots can be largely covered. When a vehicle enters the blind spot, the user can observe the vehicle entering the blind spot through the side camera, which can reduce the occurrence of traffic accidents to a certain extent.

[0054] In one embodiment, the type of camera can be determined based on its location on the vehicle. For example, a camera mounted on the windshield can be a monocular or binocular camera. A monocular or binocular camera can acquire distance information from the captured images, allowing the user to determine the distance between the vehicle and pedestrians, other vehicles, or obstacles. Alternatively, a camera mounted on the side of the vehicle can be a wide-angle camera. By installing multiple wide-angle cameras on the side of the vehicle, the user can observe the situation on the side of the vehicle through the camera. Another example is a camera mounted at the rear of the vehicle, which can be a wide-angle or fisheye camera. This camera can be used for parking assistance, allowing the user to observe the situation behind the vehicle while parking.

[0055] In one embodiment, at least four cameras are installed on the vehicle. The cameras are either wide-angle cameras or fisheye cameras. The four cameras are respectively positioned in the front, rear, left, and right directions of the vehicle. By stitching together the images captured by the four cameras, a panoramic view of the vehicle's surroundings can be obtained. Combined with algorithms, road line perception or parking control can be achieved.

[0056] Parking space information may include, but is not limited to, the coordinates of the four corner points of the parking space or the parking space lines. The coordinates of the four corner points can be determined by a coordinate system established with the vehicle's center position as the origin. The parking space lines can refer to lines used to distinguish the parking space from other areas. Parking space information can be used to obtain one or more parking spaces available for the vehicle. When the parking space information indicates that multiple parking spaces are available, the user can select a target parking space from among the multiple spaces to park the vehicle, or the vehicle's APA controller can randomly select a target parking space from among the multiple spaces to park the vehicle. The target parking space can refer to the parking space where the vehicle will ultimately be parked.

[0057] The method for determining parking space information based on images captured by multiple cameras mounted on a vehicle is not limited, as long as it allows for the determination of parking space information from these images. For example, images from multiple cameras can be stitched together, and the resulting image can be transformed to form a bird's-eye view. This bird's-eye view can then be binarized by setting the grayscale value of each pixel to 0 or 255, effectively creating a black and white effect. This significantly reduces the amount of data in the bird's-eye view, highlighting the outline of the parking space lines and thus allowing for the determination of the parking space information. The bird's-eye view can be a three-dimensional drawing created from a high vantage point using perspective principles, preserving the geometric features of the ground lines, which is beneficial for parking space line detection.

[0058] In one embodiment, Figure 2 This is a schematic diagram illustrating the determination of parking space information based on images captured by a camera installed on a vehicle, according to Embodiment 1 of the present invention. Figure 2 As shown, the camera is positioned in front of the vehicle. By transforming the viewpoint of the image captured by this camera to form a bird's-eye view, and then binarizing the bird's-eye view, features that conform to the parking lines (such as...) can be extracted. Figure 2 (See the black border shown). By processing the bird's-eye view, the coordinates of the two corner points of the parking space in front of the vehicle or the parking space line can be obtained. By combining this with images captured by cameras placed at other locations on the vehicle, complete parking space information can be obtained.

[0059] S120: The parking space information, radar data, and electronic map constructed based on the image are sent to the terminal so that the terminal can generate parking control information based on the parking space information, radar data, and electronic map.

[0060] Radar data can refer to data collected by radar installed on a vehicle. Radar can be an electronic device that uses electromagnetic waves to detect targets. Radar emits electromagnetic waves to illuminate a target and receives its echo, thereby obtaining information such as the distance from the target to the electromagnetic wave emission point, the rate of change of distance (radial velocity), azimuth, or altitude. By installing radar on a vehicle, the relative distance, relative speed, angle, or direction of motion between the vehicle and other objects can be detected. Radar data can include, but is not limited to, the relative distance, relative speed, angle, or direction of motion between the vehicle and other objects.

[0061] There are no restrictions on the type of radar installed on the vehicle, as long as it can acquire radar data. This could include millimeter-wave radar, infrared radar, ultrasonic radar, or lidar. Different types of radar can have different effects when collecting radar data. For example, millimeter-wave radar has a longer detection range, but its obstacle detection capability is relatively weaker than lidar; while lidar has a longer detection range and can accurately locate obstacles.

[0062] There is no limit to the number of radars installed on the vehicle, nor is there a limit to the location of the radars on the vehicle, as long as radar data can be obtained through the radars. The specific number and location of the radars can be determined according to the actual application needs.

[0063] In one embodiment, 12 radars are installed on the vehicle and evenly distributed around the vehicle, and radar data can be acquired through the 12 radars.

[0064] An electronic map can refer to a map stored and viewed digitally using computer technology; it is also known as a digital map. The electronic map in this invention may include, but is not limited to, information about obstacles around the vehicle, information about other vehicles, and parking space information. The information about obstacles around the vehicle may include, but is not limited to, the types of obstacles and their locations, while the information about other vehicles may include, but is not limited to, the locations of other vehicles.

[0065] There are no restrictions on the method of constructing electronic maps from images, as long as it is possible to construct an electronic map from images. For example, image processing software can be used to identify images captured by multiple cameras on a vehicle, obtain information about obstacles around the vehicle and other vehicles in the images, and then combine this information with parking space information to form an electronic map. The electronic map can display information about obstacles around the vehicle, other vehicles, and parking spaces.

[0066] The display method of obstacle information and other vehicle information around the vehicle on the electronic map is not limited. For example, it can be displayed as text on the electronic map; or it can be displayed as icons on the electronic map, with different icons used to distinguish the obstacle information and other vehicle information around the vehicle; or it can extract the outlines of obstacles and other vehicles around the vehicle through image processing, represent these outlines as points, establish a coordinate system with the vehicle center as the origin, and then display the obstacle information and other vehicle information around the vehicle in the form of points on the electronic map using coordinates.

[0067] There are no restrictions on how parking space information is displayed on the electronic map. For example, it can be displayed as a box on the electronic map; or a coordinate system can be established with the center of the vehicle as the origin, and the parking space information can be displayed in the form of coordinates.

[0068] A terminal can refer to an input / output device. The type of terminal is not limited in this invention. For example, it can be a laptop, tablet computer, laptop computer, personal digital assistant and other suitable computer. It can also be various forms of mobile terminals, such as personal digital processors, smartphones, wearable devices and other similar devices.

[0069] The method of sending parking space information, radar data, and the electronic map constructed from the images to the terminal is not limited, as long as it enables the transmission of these data. For example, the method involves establishing a wireless connection with the terminal after verifying its permissions, and then sending the parking space information, radar data, and the electronic map to the terminal. The method of verifying terminal permissions is not limited; it could be through password verification. The method of establishing a wireless connection with the terminal is also not limited; it could be through wireless communication technologies such as Bluetooth, Wi-Fi, or 4G.

[0070] This step involves sending parking space information, radar data, and an electronic map constructed from the images to the terminal, enabling the terminal to generate parking control information based on the parking space information, radar data, and electronic map. This saves vehicle hardware resources and improves the user experience.

[0071] S130. Control the vehicle to park in the target parking space based on parking control information.

[0072] Parking control information can refer to information that controls a vehicle to park in a target parking space. The target parking space can refer to the parking space where the vehicle will ultimately be parked.

[0073] There are no restrictions on parking control information, which may include, but is not limited to, steering wheel information, vehicle speed information, gear information, brake information, and vehicle light information.

[0074] Among them, steering wheel information can characterize the state of the steering wheel when the vehicle is parked, such as the steering wheel rotation angle or the steering wheel rotation direction.

[0075] Vehicle speed information can represent the speed that a vehicle should maintain when parking. For example, a preset speed can be set to keep the vehicle at the preset speed when parking. The preset speed can be set according to actual needs. Alternatively, the vehicle speed can be controlled according to the actual situation to make the vehicle park. The vehicle speed can be reduced when the vehicle needs to turn, or the preset speed can be maintained when the vehicle is going straight.

[0076] Gear information can indicate the gear that the vehicle should be in when parking. For example, it can enable the vehicle to park in a set gear, which can be set according to actual needs. Alternatively, it can change the gear to park the vehicle according to the actual situation, such as using reverse gear when the vehicle needs to reverse, or selecting a lower gear when the vehicle is traveling at a low speed.

[0077] Braking information can characterize the braking status of a vehicle when it is parked, such as the frequency or force of braking.

[0078] Vehicle light information can indicate the status of the vehicle's lights when the vehicle is parked, such as turning on the hazard lights when the vehicle is parked to alert surrounding vehicles to take evasive action.

[0079] The method of controlling a vehicle to park in a target parking space based on parking control information is not limited, as long as it can be done. For example, the system can determine the steering wheel position when parking based on steering wheel information, the appropriate speed for parking based on vehicle speed information, the appropriate gear for parking based on gear information, the brake status for parking based on brake information, or the status of vehicle lights based on headlight information. A connection is established with the vehicle's hardware associated with the parking control information, and the relevant vehicle hardware is controlled to perform corresponding operations based on the parking control information, so that the vehicle can park in the target parking space.

[0080] The technical solution of this invention sends parking space information, radar data, and electronic maps to the terminal through the APA controller. The terminal generates parking control information and returns it to the APA controller, so that the APA controller can control the vehicle to park in the target parking space according to the parking control information, which saves vehicle hardware resources and improves user experience.

[0081] Furthermore, before controlling the vehicle to park in the target parking space based on parking control information, the parking control method also includes:

[0082] The system performs a handshake with the vehicle's associated components, which include at least one of the following: the body control module, the electronic parking brake system, the electronic power steering system, the electronic stability control unit, and the automatic transmission control unit.

[0083] The associated component can refer to the vehicle's hardware related to controlling vehicle parking. A handshake with the associated component can be understood as establishing a connection between the APA controller and the vehicle's associated component. The APA controller can then send parking control information to the corresponding associated component and control the associated component to perform corresponding operations so that the vehicle can park in the target parking space according to the parking control information.

[0084] The associated components include at least one of the following: body control module, electronic parking brake system, electronic power steering system, electronic stability control unit, and automatic transmission control unit.

[0085] The Body Control Module (BCM) refers to a module used to control commonly used vehicle functions, such as headlight control, window control, or central locking. By establishing a handshake between the APA controller and the Body Control Module, hazard lights can be activated during parking to alert surrounding vehicles to take evasive action.

[0086] Electronic Parking Brake (EPB) refers to a system that controls the vehicle's parking brake through electronic circuitry. By connecting the APA controller with the electronic parking brake system, the vehicle can be controlled to apply the parking brake when parking is required during the parking process.

[0087] Electronic Power Steering (EPS) is a power steering system that directly relies on an electric motor to provide auxiliary torque. EPS uses signals from vehicle speed and torque sensors to determine the motor's rotation direction and the magnitude of the assist current, thereby controlling the steering wheel. EPS can provide different levels of assistance from the motor at different vehicle speeds, ensuring the vehicle is light and agile when turning at low speeds and stable and reliable when turning at high speeds. By integrating the APA controller with the electronic power steering system, the steering wheel can be controlled via the electronic power steering system during parking.

[0088] Electronic Stability Control (ESC) refers to a unit that controls vehicle stability. ESC combines multiple sensors, electronic control units, and actuators to control the longitudinal and lateral stability of the vehicle. By integrating the APA controller with the ESC, the vehicle's acceleration can be controlled by the ESC during parking to maintain stability.

[0089] An automatic transmission control unit (TCU) can be a unit that enables automatic gear shifting in a vehicle through computer and power electronic drive technology. By connecting the APA controller with the automatic transmission control unit, the vehicle's gear status can be automatically changed during parking.

[0090] By handshaking with the vehicle's associated components before controlling the vehicle to park in the target parking space based on the parking control information, the APA controller can control the corresponding associated components to perform corresponding operations based on the parking control information, thereby controlling the vehicle to park in the target parking space.

[0091] Furthermore, before sending parking space information, radar data, and the electronic map constructed from the images to the terminal, the parking control method also includes:

[0092] The target parking space is determined based on the user's selection of parking space information.

[0093] The parking space information can include one or more parking spaces, and users can select the target parking space for their vehicle.

[0094] There are no restrictions on how users select parking spaces, as long as it allows them to choose and specify a target parking space. For example, users can select a parking space through the central control screen on the vehicle's APA controller. This central control screen can display information such as vehicle audio, navigation, or vehicle information. It enables user interaction with the vehicle and can show different parking spaces. Users can select any one of these spaces as their target parking space through touch controls. The APA controller then determines the target parking space based on the user's selection.

[0095] Before sending parking space information, radar data, and electronic maps constructed from images to the terminal, the target parking space can be determined based on the user's selection of parking space information. This allows the user to select the target parking space even when the parking space information indicates that there are multiple parking spaces, making the selection of the target parking space more in line with the user's needs.

[0096] Furthermore, parking control methods also include:

[0097] The images are stitched together based on the camera calibration data, and the resulting stitched image is displayed on the central control screen.

[0098] Camera calibration data refers to data that reflects the relative position of a camera on a vehicle. Camera calibration data can characterize the positions of multiple cameras installed on the vehicle. This can be achieved by establishing a coordinate system with the vehicle's center as the origin, and then calibrating the positions of the multiple cameras relative to the vehicle's center within that coordinate system.

[0099] Multiple cameras positioned at different locations on a vehicle can capture different images. For example, a camera positioned at the front of the vehicle can capture images of the front of the vehicle, a camera positioned on the side of the vehicle can capture images of the side of the vehicle, and a camera positioned at the rear of the vehicle can capture images of the rear of the vehicle. By stitching together the different images captured by multiple cameras, a complete field of view around the vehicle can be obtained.

[0100] The images are stitched together based on the camera calibration data, and the resulting stitched image is displayed on the central control screen. This can be understood as stitching together images collected by different cameras into a complete 360-degree surround view and displaying it on the central control screen, allowing users to observe the situation around the vehicle through the central control screen, making the parking process safer.

[0101] Example 2

[0102] Figure 3 This is a flowchart of a parking control method according to Embodiment 2 of the present invention. This embodiment is applicable to situations where the terminal performs parking control planning by receiving information sent by the APA controller. This method can be executed by a parking control device, which can be implemented in software and / or hardware and integrated into the terminal. The type of terminal is not limited; it can be a laptop, tablet, computer, personal digital assistant, or other suitable computer. Alternatively, it can be various forms of mobile terminals, such as personal digital processors, smartphones, wearable devices, and other similar devices.

[0103] like Figure 3 As shown, the method includes:

[0104] S210 receives parking space information, radar data, and an electronic map constructed from images collected by multiple cameras from the Automatic Parking Assist (APA) controller.

[0105] The parking space information sent by the APA controller may include, but is not limited to, the coordinates of the four corner points of the parking space or the parking space line. The parking space information sent by the APA controller can be used to obtain a target parking space that can be used for vehicle parking. The target parking space may be a target parking space selected by the user from multiple parking spaces in the APA controller, or it may be a target parking space randomly selected by the vehicle's APA controller from multiple parking spaces.

[0106] The radar data sent by the APA controller may include, but is not limited to, the relative distance, relative speed, angle, or direction of motion between the vehicle and other objects.

[0107] The electronic map sent by the APA controller, constructed from images captured by multiple cameras, may include, but is not limited to, information about obstacles around the vehicle, information about other vehicles, and parking space information. The electronic map allows the system to determine the position of the target parking space relative to the vehicle, as well as the location of obstacles or other vehicles along the path from the vehicle to the target parking space.

[0108] The method of receiving parking space information, radar data, and electronic maps constructed from images captured by multiple cameras sent by the Automatic Parking Assist (APA) controller is not limited, as long as it can receive these data. For example, the terminal can establish a wireless connection with the APA controller to receive the parking space information, radar data, and electronic maps from the APA controller.

[0109] By receiving parking space information, radar data, and an electronic map constructed from images collected by multiple cameras sent by the APA controller, the terminal can plan the path for a vehicle to park in the target parking space based on the parking space information, radar data, and electronic map.

[0110] S220 generates parking control information based on parking space information, radar data, and electronic maps.

[0111] Parking control information refers to information used to control a vehicle's parking into a target parking space. This information may include, but is not limited to, steering wheel information, vehicle speed information, gear information, braking information, and headlight information.

[0112] The method of generating parking control information based on parking space information, radar data, and electronic maps is not limited, as long as it can be done. For example, a target parking space can be determined based on parking space information; the relative distance, relative speed, angle, or direction of movement between the vehicle and other objects can be determined based on radar data; the position of the target parking space relative to the vehicle, as well as obstacles or other vehicles around the path from the vehicle to the target parking space, can be determined through an electronic map; and the path for the vehicle to enter the target parking space can be planned by combining parking space information, radar data, and electronic maps.

[0113] S230: Return parking control information to the APA controller so that the APA controller can control the vehicle to park in the target parking space according to the parking control information.

[0114] There are no restrictions on the method of returning parking control information to the APA controller. For example, parking control information can be returned to the APA controller by establishing a wireless connection between the terminal and the APA controller.

[0115] Returning parking control information to the APA controller so that the APA controller can control the vehicle to park in the target parking space according to the parking control information can be understood as follows: after receiving the parking control information, the APA controller establishes a connection with the vehicle's associated components, and controls the relevant associated components to perform corresponding operations according to the parking control information, so that the vehicle parks in the target parking space according to the parking control information.

[0116] Figure 4 This is a schematic diagram of a vehicle parking into a target parking space according to parking control information, provided in Embodiment 2 of the present invention. Figure 4 As shown, the vehicle is parked perpendicularly into the target parking space. Figure 4 From top to bottom, the steps are as follows: First, plan ahead of the target parking space to position the vehicle in front of the target parking space; second, plan ahead of the inner entrance area of ​​the target parking space, and if an obstacle is detected, replan ahead of the target parking space; third, upon reaching the parking point planned in the second step or encountering an obstacle, replan and adjust the vehicle's posture forward so that the vehicle can park into the target parking space at a small angle.

[0117] Figure 5 This is a schematic diagram of another vehicle parking into a target parking space according to parking control information, provided by Embodiment 2 of the present invention. Figure 5 As shown, the vehicle is parked horizontally in the target parking space. Figure 5 From left to right, the steps are as follows: First, position the vehicle to the left front of the target parking space; second, position the vehicle to the right rear, and determine the turning angle into the target parking space based on the rear position of the vehicle in the first step; third, park the vehicle in the target parking space; fourth, adjust the vehicle's posture.

[0118] The technical solution of this invention involves receiving parking space information, radar data, and electronic maps sent by the APA controller via a terminal, generating parking control information on the terminal, and returning it to the APA controller. This enables the APA controller to control the vehicle to park in the target parking space based on the parking control information, saving vehicle hardware resources and improving user experience.

[0119] The present invention will be described by way of example below:

[0120] Taking a mobile phone as an example, this paper proposes a parking control algorithm that uses an APA controller to process images to identify parking spaces and then sends the parking space information to the mobile application (APP). The algorithm performs trajectory calculation (i.e., the terminal generates parking control information based on parking space information, radar data, and electronic maps). The APA controller performs a handshake with related components, and the APA controller and the mobile APP connect via WIFI or Bluetooth to complete the parking process.

[0121] Figure 6 This is a schematic diagram of the connection between the APA controller and the mobile phone according to Embodiment 2 of the present invention, as shown below. Figure 6 As shown, the cameras and radar installed on the vehicle can connect to the APA controller, sending images captured by the cameras and radar data to the APA controller. The cameras can be four cameras positioned at the front, rear, left, and right of the vehicle, and the radar can be twelve. The APA controller can include a vision algorithm (i.e., determining parking space information based on images captured by multiple cameras on the vehicle, and constructing an electronic map based on the parking space information, radar data, and images), a state machine for user interaction, and a related component control module. The mobile app can contain a traffic control algorithm. The APA controller and the mobile app connect via Wi-Fi or Bluetooth. The APA controller can communicate with the vehicle's CAN bus via the Controller Area Network (CAN).

[0122] The system includes four cameras (front, rear, left, and right) that acquire image information in real time. Twelve radars detect distances in real time. The APA controller communicates with a mobile app; it acquires image information to identify parking spaces (i.e., obtains parking space information and builds an electronic map); it transmits the image information and radar data to the mobile app via Wi-Fi (i.e., the APA controller sends parking space information, radar data, and the electronic map to the terminal); it establishes handshakes with related components; and it acquires the real-time output of the parking control algorithm from the mobile app (steering wheel angle, acceleration, braking force, and gear position, i.e., parking control information) and transmits it to relevant components. The vehicle's central control display (i.e., the central screen) shows the video content transmitted from the 360-degree surround view controller to the user.

[0123] Figure 7 This is a flowchart of another parking control method provided according to Embodiment 2 of the present invention, as shown below. Figure 7As shown, the APA controller connects to a mobile app. The APA controller uses visual algorithms to search for and identify parking spaces. When the user confirms parking, the APA controller hands with the vehicle's associated components and sends the parking space information and radar information (i.e., radar data) to the mobile app via Wi-Fi. This allows the mobile app to calculate the parking trajectory (i.e., generate parking control information) and transmit it to the APA controller via Wi-Fi. The mobile app outputs steering wheel angle, acceleration, and gear information (i.e., parking control information) to the APA controller in real time, and the APA controller controls the vehicle to park.

[0124] By running the control algorithm on a mobile app, hardware resources for embedded devices in the vehicle are saved, reducing costs. Meanwhile, the short update cycle of mobile phones means users have a greater chance of accessing more powerful algorithm chips as they become available. Mobile phones have richer resources and better performance, allowing them to run better control algorithms and run different levels of control algorithms depending on the phone's hardware resources. Mobile app upgrades are stable, convenient, and fast, reducing the risk of controller failure after OTA upgrades of embedded device control algorithms. Control parameters can be flexibly adjusted to adapt to various vehicles.

[0125] Example 3

[0126] Figure 8 This is a schematic diagram of a parking control device according to Embodiment 3 of the present invention. This embodiment is applicable to situations where an Automatic Parking Assist (APA) controller controls a vehicle to perform automatic parking. Figure 8 As shown, the specific structure of the device includes:

[0127] The parking space information determination module 31 is used to determine parking space information based on images captured by multiple cameras installed on the vehicle.

[0128] The information sending module 32 is used to send parking space information, radar data, and an electronic map constructed based on the image to the terminal, so that the terminal can generate parking control information based on the parking space information, radar data, and electronic map.

[0129] The control module 33 is used to control the vehicle to park in the target parking space based on the parking control information.

[0130] The parking control device provided in this embodiment first determines the parking space information based on images collected by multiple cameras installed on the vehicle through the parking space information determination module 31; then, it sends the parking space information, radar data, and an electronic map constructed based on the images to the terminal through the information sending module 32, so that the terminal can generate parking control information based on the parking space information, radar data, and electronic map; finally, it controls the vehicle to park in the target parking space through the control module 33 based on the parking control information.

[0131] Furthermore, before controlling the vehicle to park in the target parking space based on parking control information, the device also includes:

[0132] The component handshake module is used to perform component handshakes with the vehicle. The components include at least one of the following: body control module, electronic parking brake system, electronic power steering system, electronic stability control unit, and automatic transmission control unit.

[0133] Furthermore, before sending parking space information, radar data, and the electronic map constructed from the images to the terminal, the device also includes:

[0134] The target parking space determination module is used to determine the target parking space based on the user's selection of parking space information.

[0135] Furthermore, the device also includes:

[0136] The image stitching display module is used to stitch images according to camera calibration data and display the resulting stitched image on the central control screen.

[0137] The parking control device provided in this embodiment of the invention can execute the parking control method provided in Embodiment 1 of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0138] Example 4

[0139] Figure 9 This is a schematic diagram of a parking control device according to Embodiment 4 of the present invention. This embodiment is applicable to situations where the terminal performs parking control planning by receiving information sent by the APA controller. Figure 9 As shown, the specific structure of the device includes:

[0140] The information receiving module 41 is used to receive parking space information, radar data, and an electronic map constructed based on images collected by multiple cameras sent by the automatic parking assist (APA) controller.

[0141] The parking control information generation module 42 is used to generate parking control information based on parking space information, radar data, and electronic maps.

[0142] The information return module 43 is used to return parking control information to the APA controller so that the APA controller can control the vehicle to park in the target parking space according to the parking control information.

[0143] The parking control device provided in this embodiment first receives parking space information, radar data, and an electronic map constructed from images collected by multiple cameras from the Automatic Parking Assist (APA) controller via the information receiving module 41; then, the parking control information generation module generates parking control information based on the parking space information, radar data, and electronic map; finally, the information return module 43 returns the parking control information to the APA controller so that the APA controller can control the vehicle to park in the target parking space based on the parking control information.

[0144] The parking control device provided in this embodiment of the invention can execute the parking control method provided in Embodiment 2 of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0145] Example 5

[0146] Figure 10 This is a schematic diagram of the structure of an automatic parking assist controller that implements the parking control method of this invention. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0147] like Figure 10 As shown, the automatic parking assist controller 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the automatic parking assist controller 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0148] Multiple components in the automatic parking assist controller 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, optical disk, etc.; and a communication unit 19, such as a network card, modem, wireless transceiver, etc. The communication unit 19 allows the automatic parking assist controller 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0149] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as parking control methods.

[0150] In some embodiments, the parking control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the automatic parking assist controller 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the parking control method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the parking control method by any other suitable means (e.g., by means of firmware).

[0151] Figure 11 This is a schematic diagram of the structure of a terminal implementing the parking control method of this invention. The terminal is intended to represent a laptop, tablet, computer, personal digital assistant, and other suitable computer, or various forms of mobile terminal, such as a personal digital processor, smartphone, wearable device, and other similar device.

[0152] like Figure 11 As shown, terminal 20 includes at least one processor 21 and a memory, such as read-only memory (ROM) 22 and random access memory (RAM) 23, communicatively connected to at least one processor 21. The memory stores computer programs executable by at least one processor. Processor 21 can perform various appropriate actions and processes based on the computer program stored in ROM 22 or loaded from storage unit 28 into RAM 23. RAM 23 can also store various programs and data required for the operation of terminal 20. Processor 21, ROM 22, and RAM 23 are interconnected via bus 24. Input / output (I / O) interface 25 is also connected to bus 24.

[0153] Multiple components in terminal 20 are connected to I / O interface 25, including: input unit 26, such as keyboard, mouse, etc.; output unit 27, such as various types of displays, speakers, etc.; storage unit 28, such as disk, optical disk, etc.; and communication unit 29, such as network card, modem, wireless transceiver, etc. Communication unit 29 allows terminal 20 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0154] Processor 21 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 21 performs the various methods and processes described above, such as parking control methods.

[0155] In some embodiments, the parking control method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on terminal 20 via ROM 22 and / or communication unit 29. When the computer program is loaded into RAM 23 and executed by processor 21, one or more steps of the parking control method described above may be performed. Alternatively, in other embodiments, processor 21 may be configured to perform the parking control method by any other suitable means (e.g., by means of firmware).

[0156] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0157] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0158] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0159] To provide interaction with the user, the systems and techniques described herein can be implemented on an automatic parking assist controller or terminal, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the automatic parking assist controller or terminal. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0160] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0161] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0162] Figure 12 This is a schematic diagram of the structure of a parking control system that implements the parking control method of this invention. Figure 12 As shown, the parking control system 50 includes an APA controller 51 and a terminal 52. The APA controller 51 and the terminal 52 can communicate by establishing a wireless connection. The method of establishing the wireless connection is not limited, and can be established through wireless communication technologies such as Bluetooth, Wi-Fi, or fourth-generation mobile communication technology.

[0163] A wireless connection is established between the APA controller 51 and the terminal 52. The APA controller 51 can send parking space information, radar data, and an electronic map constructed based on the image to the terminal 52. The terminal 52 receives the parking space information, radar data, and electronic map constructed based on the image sent by the APA controller 51 and generates parking control information. Then, it returns the parking control information to the APA controller 51 so that the APA controller 51 can control the vehicle to park in the target parking space according to the parking control information.

[0164] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0165] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A parking control method, characterized in that, The method, applied to an automatic parking assist (APA) controller, includes: Parking space information is determined based on images captured by multiple cameras installed on the vehicle; The parking space information, radar data, and electronic map constructed based on the image are sent to the terminal, so that the terminal can generate parking control information based on the parking space information, radar data, and electronic map; The vehicle is controlled to park in the target parking space based on the parking control information; The process of generating parking control information based on the parking space information, the radar data, and the electronic map includes: Determine the target parking space based on the parking space information; The relative distance, relative speed, angle, or direction of motion between the vehicle and other objects is determined based on the radar data. The electronic map is used to determine the position of the target parking space relative to the vehicle, as well as the situation of obstacles or other vehicles around the path from the vehicle to the target parking space; Based on the parking space information, radar data, and information determined by the electronic map, a path is planned for the vehicle to enter the target parking space, and parking control information is generated. The parking control information includes at least steering wheel information, vehicle speed information, gear information, brake information, and headlight information.

2. The method according to claim 1, characterized in that, Before controlling the vehicle to park in the target parking space based on the parking control information, the method further includes: The system performs a handshake with the vehicle, the associated components including at least one of the following: body control module, electronic parking brake system, electronic power steering system, electronic stability control unit, and automatic transmission control unit.

3. The method according to claim 1, characterized in that, Before sending the parking space information, radar data, and electronic map constructed based on the image to the terminal, the method further includes: The target parking space is determined based on the user's selection of the parking space information.

4. The method according to claim 1, characterized in that, Also includes: The images are stitched together based on the camera calibration data, and the resulting stitched image is displayed on the central control screen.

5. A parking control method, characterized in that, Applied to a terminal, the method includes: It receives parking space information, radar data, and an electronic map constructed from images collected by multiple cameras from the Automatic Parking Assist (APA) controller. Parking control information is generated based on the parking space information, the radar data, and the electronic map; The parking control information is returned to the APA controller so that the APA controller can control the vehicle to park in the target parking space according to the parking control information; The process of generating parking control information based on the parking space information, the radar data, and the electronic map includes: Determine the target parking space based on the parking space information; The relative distance, relative speed, angle, or direction of motion between the vehicle and other objects is determined based on the radar data. The electronic map is used to determine the position of the target parking space relative to the vehicle, as well as the situation of obstacles or other vehicles around the path from the vehicle to the target parking space; Based on the parking space information, radar data, and information determined by the electronic map, a path is planned for the vehicle to enter the target parking space, and parking control information is generated. The parking control information includes at least steering wheel information, vehicle speed information, gear information, brake information, and headlight information.

6. A parking control device, characterized in that, include: The parking space information determination module is used to determine parking space information based on images captured by multiple cameras installed on the vehicle. The information sending module is used to send the parking space information, radar data, and electronic map constructed based on the image to the terminal, so that the terminal can generate parking control information based on the parking space information, radar data, and electronic map; The control module is used to control the vehicle to park in the target parking space according to the parking control information; The process of generating parking control information based on the parking space information, the radar data, and the electronic map includes: Determine the target parking space based on the parking space information; The relative distance, relative speed, angle, or direction of motion between the vehicle and other objects is determined based on the radar data. The electronic map is used to determine the position of the target parking space relative to the vehicle, as well as the situation of obstacles or other vehicles around the path from the vehicle to the target parking space; Based on the parking space information, radar data, and information determined by the electronic map, a path is planned for the vehicle to enter the target parking space, and parking control information is generated. The parking control information includes at least steering wheel information, vehicle speed information, gear information, brake information, and headlight information.

7. A parking control device, characterized in that, include: The information receiving module is used to receive parking space information, radar data, and electronic maps constructed based on images collected by multiple cameras sent by the Automatic Parking Assist (APA) controller. A parking control information generation module is used to generate parking control information based on the parking space information, the radar data, and the electronic map; The information return module is used to return parking control information to the APA controller so that the APA controller can control the vehicle to park in the target parking space according to the parking control information; The parking control information generation module is specifically used for: Determine the target parking space based on the parking space information; The relative distance, relative speed, angle, or direction of motion between the vehicle and other objects is determined based on the radar data. The electronic map is used to determine the position of the target parking space relative to the vehicle, as well as the situation of obstacles or other vehicles around the path from the vehicle to the target parking space; Based on the parking space information, radar data, and information determined by the electronic map, a path is planned for the vehicle to enter the target parking space, and parking control information is generated. The parking control information includes at least steering wheel information, vehicle speed information, gear information, brake information, and headlight information.

8. An automatic parking assist controller, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the parking control method as described in any one of claims 1-4.

9. A terminal, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the parking control method as described in claim 5.

10. A parking control system, characterized in that, It includes the APA controller as described in claim 8 and the terminal as described in claim 9.

Images