Automatic parking methods and devices, electronic devices, vehicles and storage media

Abstract

This invention discloses an automatic parking method and apparatus, electronic device, vehicle, and storage medium. The automatic parking method includes: acquiring a target image of the vehicle's environment; identifying a valid parking space and a drivable area based on the target image; and parking the vehicle in the valid parking space according to the drivable area. This invention's automatic parking method achieves automatic parking functionality, improves the accuracy of automatic parking recognition, and reduces costs by eliminating the need for expensive, specific sensor equipment.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to an automatic parking method and device, electronic device, vehicle, and computer-readable storage medium. Background Technology

[0002] In related technologies, automatic parking systems typically employ multiple sensors, such as ultrasonic radar, millimeter-wave radar, lidar, and inertial measurement units, to perceive the vehicle's own status and the surrounding environment. However, the high cost of lidar and millimeter-wave radar, requiring the integration of high-performance computing chips, increases system costs. Furthermore, some sensors have limited lifespans, further restricting the application scenarios of this method. As for ultrasonic radar, the propagation of sound waves may be interfered with or attenuated by grid-like or flexible obstacles, causing the ultrasonic radar to fail to accurately detect the shape and position of the obstacle, resulting in poor recognition performance. Summary of the Invention

[0003] The present invention aims to at least solve one of the technical problems existing in the prior art. To this end, the first objective of the present invention is to propose an automatic parking method that realizes the function of automatic vehicle parking, improves the accuracy of automatic parking recognition, and reduces costs by eliminating the need for expensive specific sensor equipment.

[0004] The second objective of this invention is to provide an automatic parking device.

[0005] The third objective of this invention is to provide an electronic device.

[0006] The fourth objective of this invention is to provide a vehicle.

[0007] The fifth objective of this invention is to provide a computer-readable storage medium.

[0008] To achieve the above objectives, an automatic parking method according to a first aspect of the present invention includes: acquiring a target image of a vehicle environment; identifying a valid parking space and a drivable area based on the target image of the vehicle environment; and parking the vehicle in the valid parking space according to the drivable area.

[0009] The automatic parking method according to embodiments of the present invention acquires a target image of the vehicle's environment, detects and identifies suitable parking spaces and drivable areas based on the image information, thereby realizing the function of automatic parking. It identifies suitable parking spaces and drivable areas based solely on visual information of the vehicle's surrounding environment, with minimal interference, thus improving the accuracy of automatic parking identification. Furthermore, it does not rely on expensive specific sensor equipment such as lidar and millimeter-wave radar, thereby reducing costs.

[0010] In some embodiments, the vehicle environment target image is a bird's-eye view of the vehicle environment with brightness values ​​within a first preset brightness threshold range.

[0011] In some embodiments, the vehicle environment target image is a bird's-eye view of the vehicle environment where the brightness value of the corresponding left or right area of ​​the vehicle is within the first preset brightness threshold range.

[0012] In some embodiments, the vehicle environment bird's-eye view is obtained by stitching together multiple vehicle environment images from different angles.

[0013] In some embodiments, the effective parking spaces include a first type of effective parking spaces, which are marked parking spaces in the vehicle environment target image whose corner coordinates satisfy the standard parking space corner position.

[0014] In some embodiments, the first type of valid parking space includes one of vertical parking space, horizontal parking space and angled parking space, and the first type of valid parking space is determined by the included angle between adjacent sides and the length of adjacent sides of the marked parking space.

[0015] In some embodiments, the automatic parking method further includes: when there is a deviation between the corner position corresponding to the corner coordinates of the parking space and the corner position of the standard parking space, adjusting the abnormal corner coordinates of the parking space according to the characteristic parameters of the standard parking space to obtain the first type of effective parking space.

[0016] In some embodiments, the effective parking space further includes a second type of effective parking space, which is a connected region between the circumscribed rectangles of two adjacent obstacles that satisfy the double-boundary parking space condition.

[0017] In some embodiments, the obstacles do not include objects located in untrusted areas, objects with an area smaller than a preset area, or objects of a preset type; wherein, the untrusted area is the area in front of or behind the vehicle in the vehicle environment target image where the brightness value exceeds a second preset brightness threshold range.

[0018] In some embodiments, the final valid parking space is the parking space between the first type of valid parking spaces and the second type of valid parking spaces after removing overlapping parking spaces.

[0019] In some embodiments, the drivable area is determined by the drivability probability of each image grid in the rasterized vehicle environment target image.

[0020] To achieve the above objectives, an automatic parking device according to a second aspect of the present invention includes: an acquisition module for acquiring a target image of a vehicle environment; a processing module for identifying a valid parking space and a drivable area based on the target image of the vehicle environment; and a control module for parking the vehicle in the valid parking space according to the drivable area.

[0021] According to an embodiment of the present invention, the automatic parking device acquires environmental information around the vehicle through an acquisition module, identifies suitable parking spaces and drivable areas, and automatically parks the vehicle in the suitable parking space based on the drivable area, thereby realizing the function of automatic parking, improving the accuracy of automatic parking recognition, and reducing costs by eliminating the need for expensive specific sensor equipment.

[0022] To achieve the above objectives, an electronic device according to a third aspect of the present invention includes: at least one processor; a memory connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the at least one processor executes the computer program to implement the automatic parking method described in the above embodiment.

[0023] According to the electronic device of the present invention, at least one processor employs the automatic parking method described in the above embodiments. By acquiring a target image of the vehicle environment, it detects and identifies suitable parking spaces and drivable areas based on the image information, thereby realizing the function of automatic parking. It identifies suitable parking spaces and drivable areas based solely on visual information of the vehicle's surrounding environment, with minimal interference, thus improving the accuracy of automatic parking identification. Furthermore, it does not rely on expensive specific sensor equipment such as lidar and millimeter-wave radar, thereby reducing costs.

[0024] To achieve the above objectives, the vehicle of the fourth aspect of the present invention includes the electronic device described in the above embodiments.

[0025] According to embodiments of the present invention, a vehicle equipped with the electronic device described in the above embodiments acquires target images of the vehicle environment, detects and identifies suitable parking spaces and drivable areas based on the image information, thereby realizing the function of automatic parking. It identifies suitable parking spaces and drivable areas based solely on visual information of the vehicle's surrounding environment, with minimal interference, improving the accuracy of automatic parking recognition. Furthermore, it does not rely on expensive specific sensor equipment such as lidar and millimeter-wave radar, thereby reducing costs.

[0026] In some embodiments, the vehicle further includes an image acquisition device, and at least one processor of the electronic device is electrically connected to the image acquisition device.

[0027] To achieve the above objectives, a computer-readable storage medium according to a fifth aspect of the present invention stores a computer program thereon, which, when executed, implements the automatic parking method described in the above embodiments.

[0028] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0029] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0030] Figure 1 This is a flowchart of an automatic parking method according to an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of a preset image area according to an embodiment of the present invention;

[0032] Figure 3 This is a schematic diagram of the coordinates of a parking space corner point according to an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of the detection results of a drivable area according to an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram illustrating the calculation of the maximum bounding rectangle according to an embodiment of the present invention;

[0035] Figure 6 This is a schematic diagram of a gridded drivable area according to an embodiment of the present invention;

[0036] Figure 7 This is a detailed flowchart of an automatic parking system according to an embodiment of the present invention;

[0037] Figure 8 This is a block diagram of an automatic parking device according to an embodiment of the present invention;

[0038] Figure 9 This is a block diagram of an electronic device according to an embodiment of the present invention;

[0039] Figure 10 This is a block diagram of a vehicle according to an embodiment of the present invention.

[0040] Figure label:

[0041] Automatic parking device 1;

[0042] Acquisition module 10; Processing module 20; Control module 30;

[0043] 100 vehicles;

[0044] Electronic device 110;

[0045] Processor 111; memory 112; image acquisition device 113. Detailed Implementation

[0046] The embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. The embodiments of the present invention are described in detail below.

[0047] The following is for reference. Figures 1-7 An automatic parking method according to an embodiment of the first aspect of the present invention is described.

[0048] Figure 1 This is a flowchart of an automatic parking method according to an embodiment of the present invention, such as... Figure 1 As shown, the automatic parking method of this invention includes at least the following steps S1-S3.

[0049] S1, acquire the target image of the vehicle's environment.

[0050] In some embodiments, images of the vehicle's surrounding environment can be acquired using an onboard camera, i.e., visual information about the environment around the vehicle. This can be achieved, for example, through a vehicle's four fisheye cameras or other image acquisition devices.

[0051] The four-way fisheye camera is a camera system with four fisheye lenses. A fisheye lens is a wide-angle lens with a very wide field of view, providing a near-panoramic view. The four-way fisheye camera combines four independent fisheye lenses, each responsible for capturing images from different directions, thus forming a panoramic image. This allows users to simultaneously observe the scene in front, behind, to the left, and to the right, providing a more comprehensive and wider field of view.

[0052] In some embodiments, the acquired vehicle environment target image may include environmental information such as roads, driving areas, parking spaces, and obstacles (e.g., other vehicles, pedestrians, walls, or pillars) around the vehicle. This environmental information can help the driver or vehicle system adjust parking operations to ensure parking accuracy and safety.

[0053] S2 identifies valid parking spaces and drivable areas based on the vehicle environment target image.

[0054] In this context, a valid parking space refers to a suitable parking location or space where no vehicle is parked. These spaces meet specific conditions, such as being of appropriate size, free of obstructions, and complying with parking rules. A drivable area refers to an area where a vehicle can safely drive and complete a parking operation. To ensure the safety and smooth execution of the parking process, it is essential to ensure that vehicles can move freely within this area without colliding or interfering with other vehicles, pedestrians, or obstacles.

[0055] In some embodiments, identifying valid parking spaces and drivable areas can be achieved through image processing and computer vision techniques (such as machine learning backbone networks). Specifically, machine learning backbone networks, such as ResNet or MobelNet, can be used to extract features from vehicle environment target images, and these features can be analyzed and processed to identify valid parking spaces and drivable areas. For example, vehicle environment target images can be input into a pre-trained backbone network, utilizing the network's feature extraction capabilities to capture useful information from the image. Then, further analysis and processing, such as object detection and semantic segmentation, can be performed based on the extracted features to determine valid parking spaces and drivable areas.

[0056] S3, park in a valid parking space according to the drivable area.

[0057] Specifically, based on the vehicle's location and the location of available parking spaces, path planning is performed within the drivable area to determine the optimal route for the vehicle to reach the available parking space. Path planning takes into account obstacles, traffic rules, and other constraints within the drivable area to ensure the vehicle can safely and efficiently reach the target location. Based on the path planning results, parameters such as steering, acceleration, and braking are controlled to guide the vehicle along the determined path. During the journey, an onboard camera can monitor surrounding obstacles in real time. If any obstacle is detected that conflicts with the vehicle's path, obstacle avoidance measures must be taken promptly, such as stopping, slowing down, or adjusting the driving direction to ensure the safety of the parking process. Then, precise parking operations are performed based on the size of the parking space and the vehicle's dimensions to ensure the vehicle is accurately parked in the available parking space, thus completing the automatic parking process.

[0058] The automatic parking method according to embodiments of the present invention acquires a target image of the vehicle environment, that is, based on visual information around the vehicle, detects and identifies a suitable parking space, and determines a drivable area, that is, an area where the vehicle can safely drive and complete the parking operation, and automatically parks the vehicle to a suitable parking space, thereby realizing the function of automatic parking, improving the accuracy of automatic parking recognition, and reducing costs by not relying on expensive specific sensor equipment.

[0059] In some embodiments, the target image of the vehicle environment is a bird's-eye view of the vehicle environment with brightness values ​​within a first preset brightness threshold range. Brightness values ​​can be used to filter out images that are insufficiently bright or blurry, thus improving accuracy.

[0060] Specifically, acquiring a vehicle environment target image may include: acquiring a bird's-eye view of the environment centered on the vehicle; calculating the brightness values ​​of preset image regions in the bird's-eye view; and determining that if the brightness values ​​are within a preset brightness threshold range, then the bird's-eye view is a vehicle environment target image. Otherwise, acquiring the next frame's vehicle environment bird's-eye view, and then further determining whether the next frame's bird's-eye view can be used as a vehicle environment target image based on the brightness values ​​of each preset image region, in order to filter out images with insufficient brightness or blurriness and improve accuracy.

[0061] In this embodiment, multiple vehicle environment images from different angles are acquired; the multiple vehicle environment images are stitched together according to the camera calibration parameters of the camera that acquired the vehicle environment images to obtain a bird's-eye view centered on the vehicle.

[0062] Specifically, firstly, acquiring multiple images of the vehicle's environment from different angles can be achieved using four fisheye cameras mounted on the vehicle. Each fisheye camera is positioned at a different angle and is responsible for capturing images from a specific angle. These four fisheye cameras can capture images simultaneously or sequentially to cover different perspectives around the vehicle.

[0063] For example, after acquiring multiple vehicle environment images from different angles, these images can be stitched together. During the stitching process, camera calibration parameters are required. Camera calibration refers to a specific calibration process for the camera, and the calibration parameters can include the camera's intrinsic parameter matrix, distortion coefficients, and extrinsic parameter matrix. The intrinsic parameter matrix describes parameters such as the camera's focal length, principal point position, and pixel size, used to correct geometric distortions in the image. Distortion coefficients are used to further correct distortion effects in the image, such as radial and tangential distortion. The extrinsic parameter matrix describes the camera's position and orientation in three-dimensional space, used to align images from multiple viewpoints to the same coordinate system.

[0064] Furthermore, by applying the camera's calibration parameters, each vehicle environment image undergoes distortion correction and alignment processing, and then they are stitched together to generate a vehicle-centric bird's-eye view. A bird's-eye view is an image that looks down on the vehicle and its surrounding environment, similar to observing a scene from a bird's-eye view. It provides a panoramic view of the vehicle's surroundings, allowing the driver or vehicle systems to observe the surrounding environment more clearly and more accurately determine available parking spaces and drivable areas.

[0065] In some embodiments, after obtaining a bird's-eye view centered on the vehicle, the bird's-eye view can be divided into multiple preset image regions. These preset image regions can be of equal size and shape (e.g., rectangles, squares, trapezoids, etc.), and the specific shape can be determined according to specific application requirements and algorithm design.

[0066] Figure 2 This is a schematic diagram of a preset image area according to an embodiment of the present invention, such as... Figure 2 As shown, the bird's-eye view can be divided into four preset image regions, Area1, Area2, Area3, and Area4, based on the correspondence between the fisheye camera and the bird's-eye view. For each divided image region, the RGB values ​​of all pixels within that region are extracted. The average brightness of the corresponding region is then calculated based on the RGB values ​​of each region. Assuming the average brightness is represented by B, the calculation formula is as follows:

[0067]

[0068] Furthermore, by calculating the average brightness of each image region, it can be determined whether the region meets the preset brightness threshold range. By judging whether the average brightness values ​​of different image regions meet the requirements, image regions that meet the preset brightness threshold range can be selected, thereby obtaining a target image of the vehicle environment. This target image contains key information about the environment around the vehicle and can be used for subsequent identification and judgment of parking spaces and drivable areas.

[0069] like Figure 2 As shown, Area4 and Area2 can be represented as the left and right sides of the vehicle, respectively. The brightness value of either the left or right side of the vehicle is compared with a preset brightness threshold. If the brightness value of either the left or right side falls within the first preset brightness threshold range, the requirement is met, and the bird's-eye view is used as the target image of the vehicle's environment for subsequent identification and judgment of valid parking spaces and drivable areas. It is understandable that since parking spaces are located to the left or right of the vehicle during parking, only the environmental conditions on the left and right sides of the vehicle need to be considered. This improves image processing efficiency while still satisfying the requirement of parking space identification.

[0070] In addition, the preset image area of ​​the vehicle environment target image also includes the left and right regions of the vehicle, for example... Figure 2The diagram shows Area1 and Area3. Area1 can represent the area in front of the vehicle, and Area3 can represent the area behind the vehicle. The brightness of the areas in front and behind the vehicle is processed as follows: The brightness value of the area in front of the vehicle is compared with a second preset brightness threshold. The second preset brightness threshold can be equal to or different from the first preset brightness threshold; no specific restriction is placed here. If the brightness of the area in front of the vehicle exceeds the range of the second preset brightness threshold, then that area will be marked as an untrusted area. Similarly, the brightness value of the area behind the vehicle is compared with the second preset brightness threshold. If the brightness of the area behind the vehicle exceeds the range of the second preset brightness threshold, then that area will also be marked as an untrusted area.

[0071] The purpose of marking the area in front of or behind the vehicle as an unreliable area is to remind the system to pay special attention to these areas during parking. This is because unreliable areas may contain interference or erroneous information, which could affect the accuracy and safety of the parking operation.

[0072] In some embodiments, after acquiring a vehicle environment target image, the image can be input into a backbone network. By performing a series of convolution and pooling operations on the image, marked parking spaces in the vehicle environment target image can be identified. For example, a feature map corresponding to the vehicle environment target image is obtained. The feature map contains information at different levels and semantic levels in the image. Based on the feature map, marked parking spaces are identified in the vehicle environment image. Marked parking spaces refer to parking spaces marked or drawn with specific lines in the vehicle environment image. These lines can be used to identify and define the location of parked vehicles.

[0073] In an embodiment, a valid parking space may include a first type of valid parking space, which is a marked parking space in the vehicle environment target image whose corner coordinates meet the standard parking space corner position.

[0074] Specifically, after identifying the marked parking spaces, the coordinates of the corner points of the parking spaces can be further extracted. Parking space corner points are the boundary points of the parking spaces; determining these corner points accurately describes the location and shape of the parking spaces. By analyzing and processing the corner point coordinates, the system can determine the type of the marked parking space, thus obtaining the first type of valid parking space. The type of the first type of valid parking space can include, but is not limited to, perpendicular parking spaces, horizontal parking spaces, and angled parking spaces.

[0075] Furthermore, the first type of valid parking space may include one of vertical parking space, horizontal parking space and angled parking space, and the first type of valid parking space is determined by the included angle between adjacent sides and the length of adjacent sides of the marked parking space.

[0076] Specifically, obtaining the first type of valid parking space based on the corner coordinates of the parking space includes: obtaining the ratio of the included angle between the adjacent sides of the marked parking space to the length of the adjacent side based on the vector ratio formed by the corner coordinates of the parking space; and determining the parking space type of the marked parking space as one of vertical parking space, horizontal parking space, and inclined parking space based on the ratio of the included angle to the length of the adjacent side.

[0077] In some embodiments, after obtaining the coordinates of the corner points of a marked parking space, vectors of adjacent sides can be constructed using these coordinates. Adjacent sides refer to two adjacent sides connected by their corner points. Then, by calculating the included angle between the adjacent sides of the marked parking space, the angle information of the marked parking space can be obtained. The included angle can be determined using the formula for calculating the angle between vectors (i.e., the ratio of the vectors formed by the corner point coordinates of the parking space). Simultaneously, based on the ratio of the vectors formed by the corner point coordinates of the parking space, the ratio of the lengths of the adjacent sides can also be calculated, thus obtaining the ratio information of the lengths of the adjacent sides of the marked parking space.

[0078] Furthermore, based on the range of different included angles and the ratio of adjacent side lengths, marked parking spaces are classified into different types, such as perpendicular parking spaces, horizontal parking spaces, or angled parking spaces.

[0079] The included angle can be used to distinguish between perpendicular and angled parking spaces. Specifically, for a perpendicular parking space, the included angle between the two adjacent sides should be close to 90 degrees. Therefore, if the calculated included angle is close to 90 degrees, the parking space can be determined to be perpendicular. For angled parking spaces, the included angle will deviate more significantly and will not be close to 90 degrees.

[0080] The ratio of adjacent side lengths can be used to distinguish between parallel and perpendicular parking spaces. These two types of parking spaces differ in terms of area, length-to-width ratio, etc. For example, parallel parking spaces have a relatively longer length and a shorter width, while perpendicular parking spaces are the opposite, having a relatively longer width and a shorter length. Therefore, by calculating the ratio of adjacent side lengths, it is possible to determine whether a parking space is parallel or perpendicular.

[0081] In summary, by calculating and comparing the included angle and the ratio of side lengths, the type of marked parking spaces can be determined. This distinction provides important information for automatic parking systems, enabling them to accurately identify and select suitable parking spaces and perform corresponding parking operations and route planning.

[0082] In some embodiments, the process of identifying valid parking spaces and drivable areas also includes handling the coordinates of abnormal parking space corner points.

[0083] For example, if the corner point coordinates of a parking space deviate from the corner point position of a standard parking space, the corner point coordinates of the abnormal parking space are adjusted according to the characteristic parameters of the standard parking space; the first type of valid parking space is obtained based on the adjusted corner point coordinates. In other words, when the corner point coordinates of a parking space deviate from the corner point position of a standard parking space, the first type of valid parking space can be obtained by adjusting the corner point coordinates of the abnormal parking space according to the characteristic parameters of the standard parking space.

[0084] The standard parking space characteristic parameters can be a predefined or stored set of standardized parameters used to describe and define the characteristics of the parking space. Depending on the parking space type, the system can access the pre-stored standard parking space characteristic parameters to obtain relevant parking space size, location, orientation, and spacing information. Then, the system can adjust the corner coordinates of the marked parking spaces according to these parameters to meet the requirements, thus obtaining the first type of valid parking spaces. The first type of valid parking space is the adjusted parking space that matches the standard parking space characteristic parameters, possessing accurate location and shape information. This parking space can be considered a suitable parking location or space.

[0085] Specifically, when an anomaly is detected in the corner position corresponding to the parking space corner coordinates, it may mean that the position detection of some corner points in the vehicle environment target image may be inaccurate or contain errors. Anomalies can include corner position offset, missing, or incorrect. There are various reasons that may cause anomalies in the corner position corresponding to the parking space corner coordinates, including but not limited to occlusions, changes in viewing angle, and low image quality.

[0086] Furthermore, for cases where the corner coordinates of a parking space are abnormal, adjustments can be made based on the standard parking space characteristic parameters corresponding to the parking space type. These standard parking space characteristic parameters describe the characteristics of a parking space under ideal conditions. By accessing pre-stored standard parking space characteristic parameters according to the parking space type, the system can understand information such as the desired location, shape, and size of the parking space. Based on these standard parking space characteristic parameters, the system adjusts the abnormal corner coordinates of the parking space to conform to the desired shape and location. Adjustments may include translating, rotating, or correcting the corner positions.

[0087] Furthermore, by adjusting the corner coordinates of the parking spaces, the system can identify the first type of valid parking spaces. This means that the parking spaces have been processed, their corner coordinates meet the requirements of standard parking space characteristic parameters, and they possess accurate location and shape information, making them suitable for parking operations.

[0088] Figure 3 This is a schematic diagram of the coordinates of a parking space corner point according to an embodiment of the present invention, as shown below. Figure 3As shown, after the bird's-eye view is processed by the backbone network and the parking space corner detection head, the detectable parking space corners in the bird's-eye view can be output. This bird's-eye view yields two parking spaces, PKS1 and PKS2, with their corner coordinates arranged in reverse order as P0, P1, P2, and P3. In PKS1, the angle A between P0P1 and P1P2 and the ratio LR of the lengths of P0P1 and P1P2 are obtained first through the ratio of the vectors formed by the corner coordinates. The specific calculation formula is as follows:

[0089]

[0090]

[0091] Based on the results, PKS1 can be determined to be a perpendicular parking space. Then, the lengths of |P0P1|, |P1P2|, |P2P3|, and |P3P0| are adjusted to meet the output standard for perpendicular parking spaces. In PKS2, the same formula is used to calculate the angle A between P0P1 and P1P2, and the ratio LR of the lengths of P0P1 and P1P2. Similarly, PKS2 can be determined to be a perpendicular parking space. However, due to occlusion in the PKS2 parking space, the direct results of corner point position detection may have significant errors, such as… Figure 3 As shown in the solid line box. Therefore, the positions of corner points P2 and P3 in PKS2 are adjusted according to the standard parking space characteristic parameters corresponding to the perpendicular parking space to meet the perpendicular parking space output standard, such as... Figure 3 As shown in the dashed box. The adjusted P0, P1, P'2, and P'3 are output as the corner points of PKS2.

[0092] Furthermore, in the embodiments, the effective parking space also includes a second type of effective parking space, which is a connected region between the outer rectangles of two adjacent obstacles that satisfy the double-boundary parking space condition.

[0093] Specifically, image recognition algorithms can be used to identify obstacles in the target image of the vehicle environment; and the bounding rectangle of each obstacle can be obtained; if the connected region between the bounding rectangles of two adjacent obstacles satisfies the double-boundary parking space condition, then the connected region is a second type of valid parking space. The second type of valid parking space is an area without lines or markings, that is, an area where a car can park between two obstacles.

[0094] In this embodiment, obstacles can be obtained through semantic segmentation of the feature map of the vehicle environment target image. Specifically, some features can be predefined to distinguish between road markings and obstacles. A semantic segmentation algorithm processes the feature map of the vehicle environment target image to determine the road markings and obstacles within the image. Specifically, the predefined features can be used to identify road labels, such as driving lanes, sidewalks, and parking spaces. By considering features associated with each road label, such as texture, color, and shape, the semantic segmentation algorithm can assign different regions in the image to the corresponding road label category. The predefined features can also be used to identify obstacles, such as other vehicles, pedestrians, and obstacle objects. By considering features associated with obstacles, such as size, shape, and texture, the semantic segmentation algorithm can distinguish obstacle regions from other regions in the image.

[0095] Furthermore, in the image obtained from semantic segmentation, the location of obstacles can be identified using connected component detection algorithms. Connected component detection is an image analysis method that identifies regions composed of consecutive pixels in an image and treats them as objects or targets. In this case, connected component detection can be applied to the semantically segmented image to locate the obstacles.

[0096] In this process, road markings are removed from the vehicle environment target image. This means that the vehicle environment target image needs to be preprocessed before connected component detection to remove road markings. This is because various road markings, such as stop signs and traffic signs, may be present in the vehicle environment target image. These road markings may be mistaken for obstacles during connected component detection, thus affecting the identification and selection of parking spaces. To avoid this interference, the system removes road markings before processing the vehicle environment target image. This can be achieved through image processing algorithms, such as image segmentation or image filtering techniques, to separate the road markings from the image or set their pixel values ​​to background or invalid values.

[0097] Furthermore, for each detected obstacle region, its circumscribed rectangle can be calculated, which is the smallest rectangle that can completely enclose the obstacle. The position and size of this circumscribed rectangle can provide information about the obstacle, such as its location, boundaries, and dimensions.

[0098] Furthermore, if a connected region exists between the bounding rectangles of two adjacent obstacles, and this connected region satisfies the conditions for a double-boundary parking space, then this connected region is considered a second-type valid parking space. A double-boundary parking space refers to the area between two obstacles that can serve as a suitable parking space. This connected region typically has an appropriate size and shape, capable of accommodating a vehicle and providing sufficient space for parking operations.

[0099] Figure 4 This is a schematic diagram of the drivable area detection results according to an embodiment of the present invention, as shown below. Figure 4 As shown, semantic segmentation was performed on road signs, walls, pillars, vehicles, and the drivable area itself. Then, the drivable area detection results, after removing road signs, were input into connected component detection. The results are as follows: Figure 4 As shown in the marked box, Figure 4 The CCP contains six obstacles.

[0100] like Figure 5 As shown, the largest bounding rectangle that obstacles 5 and 6 can form satisfies the double-boundary parking space output standard, meaning that its aspect ratio, length, width, and the positions of adjacent obstacles all meet the output conditions. Therefore, the corner points P0, P1, P2, and P3 corresponding to output PKS0 are calculated.

[0101] In some embodiments, after obtaining the first type of valid parking spaces and the second type of valid parking spaces, for each first type of valid parking space and each second type of valid parking space, the ratio of their intersection area to their union area is calculated. If the intersection-union ratio is IOU, the specific calculation formula is as follows:

[0102]

[0103] The area of ​​the intersection of two parking spaces represents the overlapping part between the two parking spaces, and the area of ​​the union of two parking spaces represents the total area of ​​the two parking spaces. By calculating the intersection-union ratio, the degree of overlap between the first type of effective parking spaces and the second type of effective parking spaces can be evaluated.

[0104] Furthermore, based on the intersection-merge ratio, it is determined whether there are overlapping parking spaces between the first type of valid parking spaces and the second type of valid parking spaces. If it is determined that there are overlapping parking spaces between the first type of valid parking spaces and the second type of valid parking spaces, the overlapping parking spaces in the second type of valid parking spaces are removed, and the parking spaces in the second type of valid parking spaces after removing the overlapping parking spaces and the first type of valid parking spaces are taken as the final valid parking spaces.

[0105] Therefore, the automatic parking method of the present invention can simultaneously meet the parking space output requirements under both conditions with and without parking lines. Under conditions with parking lines, the system can identify marked parking spaces based on markers or specific lines drawn in the vehicle environment target image, extract the corner coordinates of the parking spaces, and then adjust the corner coordinates according to predefined standard parking space feature parameters to obtain a first-class valid parking space that meets the standard. Under conditions without parking lines, the system uses semantic segmentation and connected component detection methods to identify drivable areas and obstacles, and further determines whether there are second-class valid parking spaces that meet the double-boundary parking space condition. By calculating the intersection-union ratio and removing overlapping parking spaces, the final valid parking spaces can be obtained, which meet the requirements under both conditions with and without parking lines.

[0106] Furthermore, the automatic parking method of this invention introduces several conditions when identifying obstacles, including untrusted areas, preset obstacle areas, and preset obstacle types. These conditions are used to filter out reliable obstacles and mark them. That is, if an object in the target image of the vehicle environment is located in an untrusted area, or its area is smaller than a preset area, or it belongs to a preset type, it is not marked as an obstacle. In other words, obstacles do not include objects located in untrusted areas, objects with areas smaller than a preset area, or objects of the preset type. By filtering out obstacles, the accuracy of obstacle identification can be improved.

[0107] Specifically, obstacles in the vehicle's environmental target image are not marked if they fall within unreliable areas. Unreliable areas can be regions with brightness values ​​exceeding a preset brightness threshold or unclear areas. This is done to avoid including unreliable obstacle information in the automatic parking system's considerations, thereby reducing misjudgments and unnecessary interference.

[0108] Alternatively, if the area of ​​an obstacle is smaller than a preset area, the system will not mark it as an obstacle. The preset area is a threshold set in advance based on the actual situation, used to filter out smaller objects, because these small objects may be noise or less important interference objects, and have little impact on parking operations.

[0109] Alternatively, if an obstacle is identified as an object of a preset type, it will not be marked as an obstacle. The preset types are pre-defined based on the specific application scenario and system requirements, and are used to determine which objects should be identified and treated as obstacles.

[0110] By introducing these conditions, automatic parking systems can more accurately and reliably identify and mark obstacles. The use of untrusted areas, preset obstacle areas, and preset obstacle types helps eliminate interfering objects and noise, thereby improving the performance and accuracy of the automatic parking system.

[0111] In some embodiments, the automatic parking method of this invention further includes marking the region boundaries of the obstacle after determining the obstacle. Specifically, edge detection algorithms (such as Canny, Sobel, etc.) can be used to extract the region boundaries where the obstacle has a 100% probability of being an obstacle. Edge detection algorithms can identify edges in an image based on changes in pixel values. They determine the boundaries between pixels by detecting changes in grayscale values ​​or colors in the image. In the automatic parking method of this invention, these boundaries can be used to represent the contours of the obstacle.

[0112] By marking the boundaries of obstacle areas, the outlines of obstacles can be clearly displayed on the vehicle's environmental target image, enabling drivers to accurately judge the relative position of obstacles to the surrounding environment, thereby better planning and executing parking operations.

[0113] In some embodiments, drivable regions in a vehicle environment target image can be identified based on semantic recognition. That is, semantic segmentation is performed on the feature map of the vehicle environment target image to determine the drivable regions in the vehicle environment target image.

[0114] In semantic segmentation, predefined features can be used to distinguish drivable areas, which can then be directly identified using a semantic segmentation algorithm. Specifically, a drivable area refers to a region where a vehicle can safely travel. When predefining features, characteristics related to drivable areas can be considered, such as a smooth road surface and no obstructions. The semantic segmentation algorithm can then identify regions possessing these features as initially confirmed drivable areas.

[0115] Furthermore, in some embodiments, the semantically segmented vehicle environment target image is rasterized according to a preset pixel matrix. The preset pixel matrix can be an n×n matrix of any size, such as 3×3, 5×5, 7×7, etc. This preset pixel matrix is ​​used to rasterize the semantically segmented vehicle environment target image, dividing the image into a series of grids. The size of each grid is determined by the size of the preset pixel matrix. By dividing the image into grids of equal size, the spatial information in the image can be transformed into a discrete data representation. Rasterization simplifies and improves the efficiency of analyzing and processing the vehicle environment.

[0116] Furthermore, based on the rasterized vehicle environment image, the drivability probability of each image grid is calculated. The drivability probability represents the degree to which each grid is considered a drivable area. The final drivable area can be determined by calculating the drivability probability of each image grid in the rasterized vehicle environment target image, and parking operations can then be performed based on the drivability probability of each grid. Typically, automatic parking systems select grids with higher drivability probabilities as parking targets because these grids are considered safer and more reliable areas. Through rasterization and drivability probability calculation, the system can find suitable and effective parking spaces based on the actual situation in the vehicle environment image and perform the parking operation.

[0117] Figure 6 This is a gridded schematic diagram of a drivable area according to an embodiment of the present invention, such as... Figure 6 As shown, a 3×3 drivable area block can be used as a grid, containing a total of 6 drivable area pixels. Therefore, the drivability probability of the corresponding grid is 6 / 9.

[0118] Referring to the above explanation, Figure 7 This is a detailed flowchart of an automatic parking system according to an embodiment of the present invention, such as... Figure 7 As shown, specifically, the real-time video data acquired by the four fisheye cameras is input into the image stitching module. The stitching module stitches these images into a top-down view image centered on the vehicle. Then, the stitched image is input into the image brightness detection module to detect the brightness of the area obtained by the four cameras, and the brightness of the image is determined based on the detection results to determine whether the brightness meets the detection requirements.

[0119] The qualified stitched image is then input into the backbone network to generate a feature map corresponding to the image. This feature map is then input into the parking space corner detection module and the drivable area detection module. The parking space corner detection module obtains the coordinates of the parking space corners in the image and determines the parking space type based on the coordinates of the four corners. Based on the typical size of the parking space type, the corresponding corner coordinates are output. Simultaneously, the feature map is also output by the drivable area detection module, showing the drivable area under the current bird's-eye view. By performing connected component detection on the drivable area, the maximum bounding rectangle that can be formed by two connected components can be calculated. Using this bounding rectangle, the corner coordinates corresponding to the wireless parking space can be calculated.

[0120] After acquiring parking space corner points and drivable areas, overlapping parking spaces are removed using an overlapping parking space detection module, and the parking spaces in the final bird's-eye view are output. Furthermore, the obtained connected components can be used to denoise the drivable area detection results based on information such as the type, location, and area of ​​the connected components. The denoised drivable area can be downsampled to meet the resolution output requirements of the raster map, and the obstacle boundaries in the bird's-eye view can be further improved in confidence during the downsampling process, ultimately forming a reliable drivable area for output. Parking in a valid parking space based on the drivable area enables automatic parking, improving the accuracy of automatic parking recognition, and reducing costs by eliminating the need for expensive specific sensor equipment.

[0121] The following is for reference. Figure 8 An automatic parking device according to an embodiment of the second aspect of the present invention is described.

[0122] Figure 8 This is a block diagram of an automatic parking device according to an embodiment of the present invention, such as Figure 8 As shown, the automatic parking device 1 includes: an acquisition module 10, a processing module 20, and a control module 30.

[0123] The acquisition module 10 acquires vehicle environment target images through an onboard camera (e.g., a four-channel fisheye camera) or other image acquisition devices. The processing module 20 analyzes the vehicle environment target images using image processing and computer vision algorithms (e.g., object detection, semantic segmentation, edge detection, etc.) to identify suitable parking spaces and areas where the vehicle can safely drive. The control module 30 is used to park the vehicle in a suitable parking space based on the drivable area.

[0124] According to an embodiment of the present invention, the automatic parking device 1, through the combined use of the acquisition module 10, the processing module 20 and the control module 30, can effectively perceive the environmental information around the vehicle and identify the effective parking space and drivable area suitable for parking. Based on the drivable area, the control module 30 can automatically park the vehicle in the effective parking space, thereby realizing the function of automatic parking, improving the accuracy of automatic parking recognition, and reducing costs by not relying on expensive specific sensor equipment.

[0125] The following is for reference. Figure 9 An electronic device according to an embodiment of a third aspect of the present invention is described.

[0126] Figure 9 This is a block diagram of an electronic device according to an embodiment of the present invention, such as Figure 9 As shown, the electronic device 110 includes at least one processor 111 and a memory 112.

[0127] The memory 102 is communicatively connected to at least one processor 111. The memory 112 stores a computer program that can be executed by at least one processor 111. In the embodiment, the memory 112 may be a solid-state memory (e.g., flash memory) or random access memory (RAM) for storing programs and intermediate calculation results.

[0128] The processor 111 is the core component of the electronic device, responsible for processing and executing various instructions, as well as calculating and controlling data. When the processor 111 executes a computer program, it implements the automatic parking method described in the above embodiment.

[0129] According to the electronic device 110 of the present invention, at least one processor 111 employs the automatic parking method described in the above embodiment. By acquiring a target image of the vehicle environment, it detects and identifies suitable parking spaces and drivable areas based on the image information, thereby realizing the function of automatic parking. It identifies suitable parking spaces and drivable areas based solely on visual information of the vehicle's surrounding environment, with minimal interference information, thus improving the accuracy of automatic parking identification. Furthermore, it does not rely on expensive specific sensor equipment such as lidar and millimeter-wave radar, thereby reducing costs.

[0130] The following is for reference. Figure 10 A vehicle according to an embodiment of the fourth aspect of the present invention is described.

[0131] Figure 10 This is a block diagram of a vehicle according to an embodiment of the present invention, such as... Figure 10 As shown, vehicle 100 includes the electronic device 110 of the above embodiment.

[0132] In some embodiments, vehicle 100 may be various types of transportation, including but not limited to electric vehicles, hybrid vehicles, trucks, and buses.

[0133] According to an embodiment of the present invention, the vehicle 100, equipped with the electronic device 110 described in the above embodiment, acquires a target image of the vehicle environment, detects and identifies suitable parking spaces and drivable areas based on the image information, thereby realizing the automatic parking function of the vehicle 100. It identifies suitable parking spaces and drivable areas based solely on visual information of the vehicle's surrounding environment, with minimal interference information, improving the accuracy of automatic parking recognition. Furthermore, it does not rely on expensive specific sensor equipment such as lidar and millimeter-wave radar, thereby reducing costs.

[0134] In some embodiments, the vehicle 100 further includes an image acquisition device 112, and at least one processor 111 of the electronic device 110 is electrically connected to the image acquisition device 112. The image acquisition device 113 can be used to acquire images of the vehicle's environment, and it can be a camera device, such as a fisheye camera. The image acquisition device 113 can capture visual information about the surroundings of the vehicle 100, including the location of the vehicle 100, surrounding obstacles, drivable areas, etc.

[0135] This invention also proposes a computer-readable storage medium storing a computer program thereon. When the computer program is executed by the processor 111, it implements the automatic parking method described in the above embodiments. The specific implementation process of the automatic parking method can be referred to the description in the above embodiments.

[0136] The computer-readable storage medium in embodiments of the present invention may include, but is not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which will not be described in detail here.

[0137] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0138] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. An automatic parking method, characterized in that, include: Acquire images of the vehicle's surrounding environment; Based on the vehicle environment target image, valid parking spaces and drivable areas are identified. The valid parking spaces identified based on the vehicle environment target image include a first type of valid parking spaces and a second type of valid parking spaces. The first type of valid parking spaces are the marked parking spaces in the vehicle environment target image whose corner coordinates meet the standard parking space corner position. The second type of valid parking spaces are the connected areas between the bounding rectangles of two adjacent obstacles that meet the double-boundary parking space condition. The final valid parking spaces are the parking spaces after removing overlapping parking spaces from the first type of valid parking spaces and the second type of valid parking spaces. Park the vehicle in the final available parking space according to the drivable area.

2. The automatic parking method according to claim 1, characterized in that, The vehicle environment target image is a bird's-eye view of the vehicle environment where the brightness value of the area surrounding the corresponding vehicle is within a first preset brightness threshold range.

3. The automatic parking method according to claim 2, characterized in that, The vehicle environment bird's-eye view is obtained by stitching together multiple vehicle environment images from different angles.

4. The automatic parking method according to claim 1, characterized in that, The first type of valid parking space includes one of vertical parking spaces, horizontal parking spaces, and angled parking spaces. The first type of valid parking space is determined by the included angle between adjacent sides and the length of adjacent sides of the marked parking space.

5. The automatic parking method according to claim 1, characterized in that, When there is a deviation between the corner position corresponding to the corner coordinates of the parking space and the corner position of the standard parking space, the first type of valid parking space is obtained by adjusting the abnormal corner coordinates of the parking space according to the characteristic parameters of the standard parking space.

6. The automatic parking method according to claim 1, characterized in that, The obstacles do not include objects located in untrusted areas, objects with an area smaller than the preset area, or objects of the preset type; The untrusted region refers to the area in front of or behind the vehicle in the target image of the vehicle environment where the brightness value exceeds the second preset brightness threshold range.

7. The automatic parking method according to any one of claims 1-6, characterized in that, The drivable area is determined by the drivability probability of each image grid in the rasterized vehicle environment target image.

8. An automatic parking device, characterized in that, include: The acquisition module is used to acquire images of the vehicle's environmental targets. The processing module is used to identify valid parking spaces and drivable areas based on the vehicle environment target image. The valid parking spaces identified based on the vehicle environment target image include a first type of valid parking spaces and a second type of valid parking spaces. The first type of valid parking spaces are marked parking spaces in the vehicle environment target image whose corner coordinates meet the standard parking space corner position. The second type of valid parking spaces are the connected areas between the bounding rectangles of two adjacent obstacles that meet the double-boundary parking space condition. The final valid parking spaces are the parking spaces after removing overlapping parking spaces from the first type of valid parking spaces and the second type of valid parking spaces. A control module is used to park the vehicle in the final available parking space based on the drivable area.

9. An electronic device, characterized in that, include: At least one processor; A memory, the memory being connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, and when the at least one processor executes the computer program, it implements the automatic parking method according to any one of claims 1-7.

10. A vehicle, characterized in that, Includes the electronic device as described in claim 9.

11. The vehicle according to claim 10, characterized in that, The vehicle also includes an image acquisition device, and at least one processor of the electronic device is electrically connected to the image acquisition device.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the automatic parking method according to any one of claims 1-7.

Images