Parking space identification method and related device, electronic device, and storage medium

Abstract

The present application discloses a parking space recognition method and related devices, electronic devices, and storage media. The parking space recognition method includes: obtaining a vehicle surround view image and obtaining a vehicle status, wherein the vehicle status includes: searching for a parking space, parking with the front of the vehicle, or parking with the rear of the vehicle; then, based on the vehicle status, determining a first perspective that needs to be cropped and a second perspective that needs to be retained in the vehicle surround view image; cropping the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and cropping the image area of the second perspective close to the first perspective in the vehicle surround view image; and filling the cropped image area in the vehicle surround view image to obtain an image to be recognized; on this basis, performing recognition based on the image to be recognized to obtain a parking space recognition result. The above scheme can improve the accuracy of parking space recognition.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a parking space recognition method and related devices, electronic equipment, and storage media. Background Art

[0002] In recent years, with the continuous development of intelligent driving technology, the demand for parking functions has also increased.

[0003] Currently, parking space recognition primarily relies on surround-view images generated by visual cameras. A deep learning algorithm, combined with extensive real-world vehicle surround-view images, develops a visual parking space line perception model. However, the surround-view images are prone to distortion and image misalignment at the intersection of images from different perspectives. If the corners of the parking space line are near the seams of the image stitching, deep learning-based visual perception can easily lead to significant errors in recognition, resulting in inaccurate parking space line recognition and positioning, which in turn leads to poor parking posture. Therefore, improving the accuracy of parking space recognition has become an urgent issue. Summary of the Invention

[0004] The main technical problem solved by this application is to provide a parking space identification method and related devices, electronic equipment, and storage media, which can improve the accuracy of parking space identification.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a parking space recognition method, including: obtaining a vehicle surround view image and obtaining a vehicle status; wherein the vehicle status includes: searching for a parking space, front parking, and rear parking; then, based on the vehicle status, determining a first perspective that needs to be cropped and a second perspective that needs to be retained in the vehicle surround view image; cropping the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and cropping the image area of the second perspective close to the first perspective in the vehicle surround view image; filling the cropped image area in the vehicle surround view image to obtain an image to be recognized; and performing recognition based on the image to be recognized to obtain a parking space recognition result.

[0006] In order to solve the above technical problems, the second aspect of the present application provides a parking space recognition device, including: an acquisition module, a determination module, a cropping module, a filling module and an identification module, the acquisition module is used to acquire a vehicle surround view image and obtain a vehicle status; wherein the vehicle status includes: searching for a parking space, front parking, and rear parking; the determination module is used to determine, based on the vehicle status, a first perspective that needs to be cropped and a second perspective that needs to be retained in the vehicle surround view image; the cropping module is used to crop the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and crop the image area of the second perspective close to the first perspective in the vehicle surround view image; the filling module is used to fill the cropped image area in the vehicle surround view image to obtain an image to be recognized; the recognition module is used to perform recognition based on the image to be recognized to obtain a parking space recognition result.

[0007] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, including a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the parking space recognition method of the first aspect above.

[0008] In order to solve the above technical problems, the fourth aspect of the present application provides a computer-readable storage medium storing program instructions that can be executed by a processor, and the program instructions are used to implement the parking space recognition method of the first aspect.

[0009] The above scheme obtains the vehicle surround view image and the vehicle status; wherein the vehicle status includes: searching for a parking space, parking in front of the vehicle, and parking in the rear of the vehicle; then, based on the vehicle status, determines the first perspective that needs to be cropped and the second perspective that needs to be retained in the vehicle surround view image; crops the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and crops the image area of the second perspective close to the first perspective in the vehicle surround view image; fills the cropped image area in the vehicle surround view image to obtain the image to be recognized; recognizes based on the image to be recognized to obtain the parking space recognition result, on the one hand, by obtaining the vehicle surround view image and the vehicle status, determines the vehicle model itself and the first perspective. The first perspective that needs to be cropped and the second perspective that needs to be retained in the vehicle surround view image are further cropped, and the vehicle model itself, the image area of the first perspective, and the image area of the second perspective close to the first perspective in the vehicle surround view image are further cropped. This avoids as much as possible the deformation or dislocation of the vehicle surround view image caused by the deviation of the split angle between adjacent perspectives under different vehicle states, which helps to improve the accuracy of parking space recognition. On the other hand, by filling the cropped image area in the vehicle surround view image, the visual deviation caused by the texture of the vehicle model itself during the parking space recognition process is avoided, the clarity of the image to be recognized is improved, and the accuracy of parking space recognition is further improved.

[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0012] Figure 1 This is a flow chart of an embodiment of the parking space identification method of the present application;

[0013] Figure 2 is a schematic diagram of an embodiment of a vehicle surround view image;

[0014] Figure 3 is a schematic diagram of an embodiment of a process for cropping a vehicle surround view image;

[0015] Figure 4 is a schematic diagram of an embodiment of filling a cropped image area in a vehicle surround view image;

[0016] Figure 5 is a schematic diagram of an embodiment of an image to be recognized;

[0017] Figure 6 is a schematic diagram of another embodiment of an image to be recognized;

[0018] Figure 7 is a schematic diagram of another embodiment of an image to be recognized;

[0019] Figure 8 This is a flow chart of another embodiment of the parking space identification method of the present application;

[0020] Figure 9 This is a schematic diagram of the framework of an embodiment of the parking space recognition device of the present application;

[0021] Figure 10 This is a schematic diagram of the framework of an embodiment of the electronic device of the present application;

[0022] Figure 11 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present application in detail with reference to the accompanying drawings.

[0024] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0025] The term "and / or" herein is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may indicate three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " herein generally indicates that the objects associated with each other are in an "or" relationship. In addition, "many" herein means two or more than two. In addition, the term "at least one" herein means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C, may mean including any one or more elements selected from the set consisting of A, B, and C. "Several" means at least one. The terms "first", "second", etc. in the specification and claims herein and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0026] See also Figure 1 , Figure 1This is a flow chart of an embodiment of the parking space identification method of the present application. Specifically, it may include the following steps:

[0027] Step S11: Acquire the vehicle surround view image and vehicle status.

[0028] In the disclosed embodiment, the vehicle status includes any of: searching for a parking space, front-end parking, or rear-end parking. It is understood that searching for a parking space is performed based on the left and right ends of the vehicle. Furthermore, the vehicle status can be determined based on vehicle information stored in a controller (i.e., master control).

[0029] In one implementation scenario, the vehicle surround view image is obtained by stitching the perspective images taken around the vehicle and rendering the position of the vehicle model. Figure 2 , Figure 2 This is a schematic diagram of an embodiment of a vehicle surround view image. Specifically, the perspective images captured from the front, rear, left, and right sides of the vehicle can be stitched together, and the position of the vehicle model can be rendered based on the stitched images to obtain a vehicle surround view image. That is, the vehicle surround view image can include the front perspective (front view), rear perspective (rear view), left perspective (left view), right perspective (right view) and vehicle model of the vehicle.

[0030] Step S12: Based on the vehicle state, determine a first perspective that needs to be cropped and a second perspective that needs to be retained in the vehicle surround view image.

[0031] In one implementation scenario, in response to the vehicle status being a parking search, the first perspective is determined to include the front and rear perspectives, and the second perspective is determined to include the left and right perspectives. Specifically, after acquiring the vehicle surround view image, the vehicle status is further acquired. When the vehicle status is a parking search, the front and rear perspectives are determined to be the first perspectives to be cropped, and the left and right perspectives are determined to be the second perspectives to be retained.

[0032] In another implementation scenario, in response to the vehicle status being rear-end parked, the first perspective is determined to include the front, left, and right perspectives, and the second perspective is determined to include the rear perspective. Specifically, after acquiring the surround view image, the vehicle status is further acquired. When the vehicle status is rear-end parked, the front, left, and right perspectives are determined to be the first perspectives to be cropped, and the rear perspective is determined to be the second perspective to be retained.

[0033] In another implementation scenario, in response to the vehicle status being front-end parked, the first perspective is determined to include the rear, left, and right perspectives, and the second perspective is determined to include the front perspective. Specifically, after acquiring the surround view image, the vehicle status is further acquired. When the vehicle status is front-end parked, the rear, left, and right perspectives are determined to be the first perspectives to be cropped, and the front perspective is determined to be the second perspective to be retained.

[0034] Step S13: cropping the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and cropping the image area of the vehicle surround view image at the second perspective close to the first perspective.

[0035] It should be noted that in the process of cropping the vehicle surround view image, there is no limitation on the cropping order, that is, the image area of the vehicle model itself and the first perspective in the vehicle surround view image can be cropped first, or the image area of the second perspective close to the first perspective in the vehicle surround view image can be cropped first. In addition, since the vehicle surround view image contains images of different perspectives, there is a seam angle deviation between adjacent perspectives in the vehicle surround view image. For example, there is a seam angle deviation between the front view and the right view in the vehicle surround view image. If the seam angle deviation can be 3 degrees, 2 degrees, 1 degree, etc., it can be that the front view is biased toward the right view. Figure 3 Degrees, or the right view can be biased towards the front view Figure 3 The split angle deviation can be determined based on actual conditions and is not specifically limited here. It is understood that split angle deviation occurs during the splicing process of different perspectives in the vehicle surround view image. This split angle deviation can cause visual errors during parking space recognition, leading to inaccurate parking space recognition. Therefore, the split angle deviation caused by image splicing should be removed to minimize the visual errors in parking space recognition.

[0036] In one implementation scenario, the visual error in parking space recognition is minimized by cropping the image area of the vehicle surround view image where the second perspective is close to the first perspective. Specifically, the intersection of the dividing lines between the first perspective, the second perspective, and the vehicle model itself is used as the target point. The image area close to the first perspective is cropped along a target line that starts at the target point and deviates from a reference line by a preset angle; the reference line is the dividing line between the first and second perspectives and passes through the target point. The preset angle can be set to 4 degrees, 5 degrees, 6 degrees, etc. The preset angle can be set according to actual conditions and is not specifically limited here. It should be noted that the preset angle is not less than the gap angle deviation between adjacent perspectives in the vehicle surround view image. The above method, by cropping the image area close to the first perspective, thereby minimizing the gap angle deviation caused by image stitching, reduces the visual error in parking space recognition and further improves the accuracy of parking space recognition.

[0037] For example, see Figure 3 , Figure 3 This is a schematic diagram of an embodiment of the process of cropping a vehicle surround view image. The first perspectives to be cropped include the front and rear perspectives, the second perspectives to be retained include the left and right perspectives, the image areas of the vehicle model itself and the first perspective in the vehicle surround view image need to be cropped, and the image area of the second perspective close to the first perspective in the vehicle surround view image needs to be cropped. Figure 3As shown, the intersection of the dividing lines of the front view, right view and car model image area is the target point. If the seam angle deviation is 3 degrees, in order to ensure that the retained right view image area has no splicing seams, it is necessary to ensure that the preset angle is not less than the seam angle deviation. The preset angle can be calculated by α-β. For example, if the preset angle is 5 degrees, then β = α-5°. Furthermore, the target line is determined based on the line segment forming the β angle, and the image area close to the first view is cropped to the angle area between α and β. The method for determining the cropping area between the front view and the left view, the rear view and the left view, and the rear view and the right view can be deduced in the same way and will not be repeated here.

[0038] Step S14: Filling the cropped image area in the vehicle surround view image to obtain an image to be recognized.

[0039] Please refer to Figure 3 and Figure 4 , Figure 4 This is a schematic diagram of an embodiment of filling the cropped image area in a vehicle surround view image. When the vehicle status is parking search, the first perspective is determined to include the front and rear perspectives, and the second perspective includes the left and right perspectives. The image area of the vehicle model itself and the first perspective in the vehicle surround view image is cropped. The image area of the second perspective close to the first perspective in the vehicle surround view image is cropped. Specifically, the front and rear perspectives and the vehicle model itself are cropped. The image area of the right perspective close to the front perspective (the angle between α and β) is further cropped. The cropped image area is then filled with a preset color to obtain an image to be recognized. The preset color can be black, gray, etc., and is not specifically limited here.

[0040] Please refer to Figure 5 , Figure 5 This is a schematic diagram of an embodiment of an image to be recognized. When the vehicle status is "parking search," the first perspective is determined to include both the front and rear perspectives, and the second perspective is determined to include both the left and right perspectives. The image areas of the vehicle model and the first perspective in the surround view image are cropped. Furthermore, the image area of the surround view image from the second perspective close to the first perspective is cropped. The cropped image areas are then filled to obtain the image to be recognized when the vehicle status is "parking search."

[0041] Please refer to Figure 6 , Figure 6 This is a schematic diagram of another embodiment of an image to be recognized. When the vehicle is in the rear-end parked state, the first perspective is determined to include the front, left, and right perspectives, and the second perspective includes the rear perspective. The image area of the vehicle model itself and the first perspective in the surround view image is cropped. Furthermore, the image area of the surround view image from the second perspective close to the first perspective is cropped. The cropped image area is then filled to obtain the image to be recognized when the vehicle is in the rear-end parked state.

[0042] Please refer to Figure 7 , Figure 7 This is a schematic diagram of another embodiment of an image to be recognized. When the vehicle status is front-end parked, the first perspective is determined to include rear, left, and right perspectives, and the second perspective includes the front perspective. The image area of the vehicle model itself and the first perspective in the surround view image is cropped. Furthermore, the image area of the second perspective close to the first perspective in the surround view image is cropped. The cropped image area is then filled to obtain the image to be recognized when the vehicle status is front-end parked.

[0043] Step S15: performing recognition based on the image to be recognized to obtain a parking space recognition result.

[0044] In one implementation scenario, a to-be-recognized image is recognized based on a deep learning model, thereby obtaining a parking space recognition result. The parking space recognition result may include recognition confidence and the pixel locations of parking space corner points in the to-be-recognized image, and may further include parking space type, etc. Deep learning models may include, but are not limited to, CNN (convolution neural network), RNN (recurrent neural network), etc. In addition, the deep learning model can be trained based on sample data. Specifically, sample vehicle surround view images captured by a surround view camera in an actual vehicle are obtained, and the visible parking spaces in the vehicle surround view images are manually annotated, that is, the locations of the four corner points of the parking spaces in the sample vehicle surround view images and the parking space type are annotated. The parking space type may include horizontal, vertical, diagonal, etc. The annotated sample vehicle surround view images are used as sample data, and the deep learning model is trained based on the sample data, so that the deep learning model has the ability to automatically identify parking space corner points and parking space types, thereby obtaining a parking space recognition result.

[0045] In one implementation scenario, after performing recognition based on the image to be recognized and obtaining a parking space recognition result, it is further possible to detect whether the recognition confidence is greater than a confidence threshold that matches the vehicle state. The confidence threshold can be set to 0.5, 0.6, 0.7, etc. Of course, the confidence threshold can also be determined based on the vehicle state. The confidence threshold can be determined based on actual conditions and is not specifically limited here. When the recognition confidence is greater than the confidence threshold that matches the vehicle state, the pixel position of the parking space corner point is mapped to a spatial coordinate system to obtain the spatial position of the parking space corner point. For example, when the vehicle state is "parking space search", the spatial position of the second vehicle part is used as the coordinate origin of the spatial coordinate system. The second vehicle part can be the center of the vehicle's rear axle or the center point of the vehicle. The second vehicle part can be determined based on actual conditions and is not specifically limited here. In addition, after determining the coordinate origin of the spatial coordinate system, the steps of acquiring the vehicle surround view image and acquiring the vehicle state and subsequent steps can be re-executed based on the current spatial coordinate origin, thereby minimizing errors as much as possible. Specifically, the pixel positions of the parking space corner points in the parking space recognition result can be first obtained. Then, referring to the mutual conversion method between the image coordinate system, the camera coordinate system, and the world coordinate system, the pixel positions of the parking space corner points can be mapped to the spatial coordinate system. After obtaining information such as camera intrinsic parameters and extrinsic parameters, the coordinate conversion between the coordinate systems can be achieved. The spatial position of a reference object that matches the vehicle status is also obtained, and the spatial position of the parking space corner points is verified based on the spatial position of the reference object to determine whether to update the parking space recognition result to the automatic parking system. The reference object can be a vehicle or a billboard. The type of reference object can be determined based on actual conditions and is not specifically limited here. It should be noted that the spatial position of the reference object is also located in the spatial coordinate system. When the detection and recognition confidence is not greater than the confidence threshold matching the vehicle status, the steps of obtaining the vehicle surround view image and obtaining the vehicle status and subsequent steps are re-executed until parking is completed or the vehicle leaves the parking space. The above method can determine whether the current recognition result is credible by detecting whether the recognition confidence is greater than the confidence threshold that matches the vehicle status. If the current recognition result is not credible, the steps of obtaining the vehicle surround view image and obtaining the vehicle status and their subsequent steps are re-executed. If the current recognition result is credible, the parking space recognition result is further judged to determine whether the parking space recognition result is updated to the automatic parking system, which helps to improve the accuracy of the parking space recognition result and further improve the user experience.

[0046] In a specific implementation scenario, when the vehicle status is to search for a parking space, the spatial position of the reference object is the spatial position of the first vehicle part. The first vehicle part can be the vehicle's rearview mirror, door handle, etc. The first vehicle part can be determined based on actual conditions and is not specifically limited here. Specifically, the spatial position of the parking space center axis can be determined based on the spatial position of the parking space corner point, and then the spatial position of the first vehicle part can be detected to determine whether it is within the first preset range of the spatial position of the parking space center axis to determine whether to update the latest parking space recognition result to the automatic parking system. The first preset range can be determined based on the first vehicle part and is not specifically limited here. For example, the first vehicle part is the vehicle's rearview mirror, and the first preset range is set to 0.5 meters. The spatial position of the parking space center axis is calculated through the spatial position of the parking space corner point. The system then checks whether the linear distance between the spatial position of the vehicle's rearview mirror and the spatial position of the parking space's central axis is within 0.5 meters. If so, the system updates the latest parking space recognition result to the automatic parking system. Otherwise, the system re-executes the steps of acquiring the vehicle's surround view image and obtaining the vehicle's status, as well as subsequent steps, until parking is completed or the vehicle leaves the parking space. In the above method, when the vehicle status is "searching for a parking space," the system determines the position of the parking space's central axis based on the spatial position of the parking space's corner points, and then determines whether the current parking space recognition result is usable based on the distance between the central axis and the spatial position of the first vehicle part, further improving the accuracy of the parking space recognition result.

[0047] In a specific implementation scenario, when the vehicle status is front-end parking or rear-end parking, the spatial position of the reference object is the spatial position of the corner point of the parking space when the vehicle status is parking search. That is, the spatial position of the reference object can be the spatial position of any corner point of the parking space when the vehicle status is parking search. Specifically, it can be detected whether the positional deviation between the spatial position of the reference object and the spatial position of the corner point of the parking space is within a second preset range to determine whether to update the latest parking space recognition result to the automatic parking system. Exemplarily, the spatial position of the reference object is the spatial position of any corner point of the parking space when the vehicle status is parking search. The second preset range is set to 15 centimeters. The positional deviation between the spatial position of the reference object and the spatial position of the corner point of the parking space is detected to determine whether it is within 15 centimeters. If the distance between the two is within 15 centimeters, the latest parking space recognition result is updated to the automatic parking system. Otherwise, the steps of acquiring the vehicle surround view image and acquiring the vehicle status and subsequent steps are re-executed until parking is completed or the vehicle leaves the parking space. It should be noted that the confidence thresholds for matching front-end parking and rear-end parking are both greater than the confidence threshold for matching parking spaces. This ensures that the automatic parking system uses higher-quality recognition results during the parking phase than during the parking space search phase, further improving the accuracy of parking space recognition. In this approach, when the vehicle's status is front-end parking or rear-end parking, the spatial position of the parking space corners when the vehicle's status is parking space search is detected and compared with the spatial position of the parking space corners in the most recent parking space recognition result to determine whether the current parking space recognition result is valid, further improving the accuracy of parking space recognition results.

[0048] In one implementation scenario, after verifying the spatial positions of the parking space corners based on the spatial positions of the reference objects and determining whether to update the parking space recognition results to the automatic parking system, the steps of acquiring the vehicle's surround view image and obtaining the vehicle's status, along with subsequent steps, can be re-executed until parking is completed or the vehicle leaves the parking space. This approach, by continuously acquiring the vehicle's status and subsequent steps to obtain the parking space recognition results, thereby continuously updating the most recent parking space recognition results until parking is completed or the vehicle leaves the space, helps improve the real-time and accuracy of parking space recognition and minimizes the possibility of poor parking posture due to parking space recognition errors.

[0049] The above scheme obtains the vehicle surround view image and the vehicle status; wherein the vehicle status includes: searching for a parking space, parking in front of the vehicle, and parking in the rear of the vehicle; then, based on the vehicle status, determines the first perspective that needs to be cropped and the second perspective that needs to be retained in the vehicle surround view image; crops the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and crops the image area of the second perspective close to the first perspective in the vehicle surround view image; fills the cropped image area in the vehicle surround view image to obtain the image to be recognized; recognizes based on the image to be recognized to obtain the parking space recognition result, on the one hand, by obtaining the vehicle surround view image and the vehicle status, determines the vehicle model itself and the first perspective. The first perspective that needs to be cropped and the second perspective that needs to be retained in the vehicle surround view image are further cropped, and the vehicle model itself, the image area of the first perspective, and the image area of the second perspective close to the first perspective in the vehicle surround view image are further cropped. This avoids as much as possible the deformation or dislocation of the vehicle surround view image caused by the deviation of the split angle between adjacent perspectives under different vehicle states, which helps to improve the accuracy of parking space recognition. On the other hand, by filling the cropped image area in the vehicle surround view image, the visual deviation caused by the texture of the vehicle model itself during the parking space recognition process is avoided, the clarity of the image to be recognized is improved, and the accuracy of parking space recognition is further improved.

[0050] See also Figure 8 , Figure 8 This is a flow chart of another embodiment of the parking space identification method of the present application. Specifically, it may include the following steps:

[0051] Step S801: Acquire the vehicle surround view image and vehicle status.

[0052] Specifically, the methods described in the aforementioned disclosed embodiments may be referred to, and will not be described in detail here.

[0053] In one implementation scenario, after obtaining the vehicle surround view image and the vehicle status, it is possible to further determine whether the vehicle has the automatic parking function turned on. If so, the vehicle status is determined; otherwise, the steps of obtaining the vehicle surround view image and obtaining the vehicle status and subsequent steps are re-executed.

[0054] Step S802: Determine whether the vehicle status is parking search; if so, execute step S802; otherwise, execute step S807.

[0055] Step S802: cropping the vehicle model and the front / rear perspectives in the vehicle surround view image.

[0056] In one implementation scenario, when the vehicle status is parking search, it is determined that the first perspective that needs to be cropped in the vehicle surround view image includes the front perspective and the rear perspective, and the second perspective that needs to be retained includes the left perspective and the right perspective. Then, the vehicle model and the front / rear perspectives in the vehicle surround view image are cropped. This can be further performed on the image area of the vehicle surround view image where the second perspective is close to the first perspective, so as to avoid the occurrence of seam angle deviation between adjacent perspectives in the vehicle surround view image as much as possible.

[0057] Step S803: Filling the cropped image area to obtain an image to be recognized.

[0058] In one implementation scenario, after cropping the vehicle surround view image, the cropped image area may be filled to obtain an image to be recognized, thereby minimizing the impact of the texture of the vehicle model itself on the parking space recognition process.

[0059] Step S804: performing recognition based on the image to be recognized to obtain a parking space recognition result.

[0060] Specifically, the method for identifying the image to be identified can refer to the method in the aforementioned disclosed embodiment, and will not be repeated here.

[0061] Step S805: Determine whether the recognition confidence is greater than 0.5; if so, execute step S806; otherwise, execute step S801.

[0062] In one implementation scenario, after obtaining a parking space recognition result, the recognition confidence level of the parking space recognition result can be determined. Specifically, the recognition confidence level is determined to be greater than a confidence threshold for matching the vehicle status. For example, if the vehicle status matching confidence threshold is set to 0.5, the recognition confidence level is determined to be greater than 0.5. Furthermore, if the recognition confidence level is less than 0.5, the current parking space recognition result is unreliable, and the steps of acquiring the vehicle surround view image and the vehicle status, as well as subsequent steps, need to be repeated.

[0063] Step S806: Determine whether the spatial position of the first vehicle part is within a first preset range of the spatial position of the central axis of the parking space; if so, execute step S814; otherwise, execute step S801.

[0064] In one implementation scenario, after determining that the recognition confidence level is greater than 0.5, the accuracy of the parking space recognition result may also be verified. The verification method can be based on the method described in the aforementioned disclosed embodiment and will not be further described here. Furthermore, when the spatial position of the first vehicle part is not within the first preset range of the spatial position of the parking space's central axis, the current parking space recognition result is unusable, and the steps of acquiring the vehicle surround view image and obtaining the vehicle status, as well as subsequent steps, must be re-executed.

[0065] Step S814: Update the latest parking space recognition result to the automatic parking system.

[0066] Step S807: Determine whether the vehicle status is rear-end parking; if so, execute step S808; otherwise, execute step S809.

[0067] Step S808: cropping the vehicle model and the front / left / right perspectives in the vehicle surround view image.

[0068] In one implementation scenario, when the vehicle is in rear-end parking, it is determined that the first perspectives that need to be cropped in the vehicle surround view image include the front perspective, left perspective, and right perspective, and the second perspective that needs to be retained includes the rear perspective. Therefore, the vehicle model and the front / left / right perspectives in the vehicle surround view image are cropped. This can be further performed on the image area of the vehicle surround view image where the second perspective is close to the first perspective, thereby minimizing the occurrence of seam angle deviation between adjacent perspectives in the vehicle surround view image.

[0069] Step S809: cropping the vehicle model and left / right / rear perspectives in the vehicle surround view image.

[0070] In one implementation scenario, when the vehicle state is front-end entry, it is determined that the first perspectives that need to be cropped in the vehicle surround view image include the left perspective, right perspective, and rear perspective, and the second perspective that needs to be retained includes the front perspective. Then, the vehicle model and the left / right / rear perspectives in the vehicle surround view image are cropped. This can be further performed on the image area of the vehicle surround view image where the second perspective is close to the first perspective, so as to minimize the occurrence of seam angle deviation between adjacent perspectives in the vehicle surround view image.

[0071] S810: Filling the cropped image area to obtain an image to be recognized.

[0072] In one implementation scenario, after cropping the vehicle surround view image, the cropped image area may be filled to obtain an image to be recognized, thereby minimizing the impact of the texture of the vehicle model itself on the parking space recognition process.

[0073] S811: Perform recognition based on the image to be recognized to obtain a parking space recognition result.

[0074] Specifically, the method for identifying the image to be identified can refer to the method in the aforementioned disclosed embodiment, and will not be repeated here.

[0075] Step S812: Determine whether the recognition confidence is greater than 0.7. If so, execute step S813; otherwise, execute step S801.

[0076] In one implementation scenario, after obtaining a parking space recognition result, the recognition confidence level of the parking space recognition result can be determined. Specifically, the recognition confidence level is determined to be greater than a confidence threshold for matching the vehicle status. For example, if the vehicle status matching confidence threshold is set to 0.7, the recognition confidence level is determined to be greater than 0.7. Furthermore, if the recognition confidence level is less than 0.7, the current parking space recognition result is unreliable, and the steps of acquiring the vehicle surround view image and the vehicle status, as well as subsequent steps, need to be repeated.

[0077] It should be noted that the confidence threshold for matching the front parking and the confidence threshold for matching the rear parking are both greater than the confidence threshold for matching the parking space search.

[0078] Step S813: Determine whether the positional deviation between the spatial position of the reference object and the spatial position of the parking space corner point is within a second preset range; if so, execute step S814; otherwise, execute step S801.

[0079] In one implementation scenario, after determining that the recognition confidence level is greater than 0.7, the accuracy of the parking space recognition result may also be verified. The verification method may be similar to that described in the aforementioned disclosed embodiments and will not be further described here. Furthermore, if the positional deviation between the spatial position of the reference object and the spatial position of the parking space corner point is not within a second preset range, the current parking space recognition result is unusable, and the steps of acquiring the vehicle surround view image and obtaining the vehicle status, as well as subsequent steps, must be repeated.

[0080] Step S814: Update the latest parking space recognition result to the automatic parking system.

[0081] The above scheme obtains the vehicle surround view image and the vehicle status; wherein the vehicle status includes: searching for a parking space, parking in front of the vehicle, and parking in the rear of the vehicle; then, based on the vehicle status, determines the first perspective that needs to be cropped and the second perspective that needs to be retained in the vehicle surround view image; crops the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and crops the image area of the second perspective close to the first perspective in the vehicle surround view image; fills the cropped image area in the vehicle surround view image to obtain the image to be recognized; recognizes based on the image to be recognized to obtain the parking space recognition result, on the one hand, by obtaining the vehicle surround view image and the vehicle status, determines the vehicle model itself and the first perspective. The first perspective that needs to be cropped and the second perspective that needs to be retained in the vehicle surround view image are further cropped, and the vehicle model itself, the image area of the first perspective, and the image area of the second perspective close to the first perspective in the vehicle surround view image are further cropped. This avoids as much as possible the deformation or dislocation of the vehicle surround view image caused by the deviation of the split angle between adjacent perspectives under different vehicle states, which helps to improve the accuracy of parking space recognition. On the other hand, by filling the cropped image area in the vehicle surround view image, the visual deviation caused by the texture of the vehicle model itself during the parking space recognition process is avoided, the clarity of the image to be recognized is improved, and the accuracy of parking space recognition is further improved.

[0082] See also Figure 9 , Figure 9 The figure is a schematic diagram of a framework of an embodiment of a parking space recognition device of the present application. The parking space recognition device 90 includes: an acquisition module 91, a determination module 92, a cropping module 93, a filling module 94, and an identification module 95. The acquisition module 91 is used to acquire the vehicle surround view image and the vehicle status; wherein the vehicle status includes any one of searching for a parking space, front parking, and rear parking; the determination module 92 is used to determine the first perspective to be cropped and the second perspective to be retained in the vehicle surround view image based on the vehicle status; the cropping module 93 is used to crop the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and crop the image area of the second perspective close to the first perspective in the vehicle surround view image; the filling module 94 is used to fill the cropped image area in the vehicle surround view image to obtain the image to be recognized; and the identification module 95 is used to perform recognition based on the image to be recognized to obtain a parking space recognition result.

[0083] The above scheme obtains the vehicle surround view image and the vehicle status; wherein the vehicle status includes: searching for a parking space, parking in front of the vehicle, and parking in the rear of the vehicle; then, based on the vehicle status, determines the first perspective that needs to be cropped and the second perspective that needs to be retained in the vehicle surround view image; crops the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and crops the image area of the second perspective close to the first perspective in the vehicle surround view image; fills the cropped image area in the vehicle surround view image to obtain the image to be recognized; recognizes based on the image to be recognized to obtain the parking space recognition result, on the one hand, by obtaining the vehicle surround view image and the vehicle status, determines the vehicle model itself and the first perspective. The first perspective to be cropped and the second perspective to be retained in the vehicle surround view image are further cropped. The vehicle model itself, the image area of the first perspective, and the image area of the second perspective in the vehicle surround view image close to the first perspective are further cropped. This minimizes distortion or misalignment of the vehicle surround view image caused by deviations in the split angles between adjacent perspectives under different vehicle conditions, thereby improving the accuracy of parking space recognition. Furthermore, by filling in the cropped image areas in the vehicle surround view image, visual deviations caused by the vehicle model's own texture during parking space recognition are avoided, thereby improving the clarity of the image to be recognized and further enhancing the accuracy of parking space recognition. In some disclosed embodiments, the cropping module 93 includes a selection submodule and a cropping submodule. The selection submodule is configured to determine the intersection of the boundary lines between the first perspective, the second perspective, and the vehicle model itself as a target point. The cropping submodule is configured to crop the image area of the first perspective along a target line starting from the target point and offset from a reference line by a preset angle. The reference line is the boundary between the first perspective and the second perspective and passes through the target point.

[0084] Therefore, by cropping the image area close to the first perspective, the seam angle deviation caused by image stitching is removed as much as possible, the visual error generated in parking space recognition is reduced, and the accuracy of parking space recognition is further improved.

[0085] In some disclosed embodiments, the preset angle is not less than a split angle deviation between adjacent viewing angles in the vehicle surround view image.

[0086] In some disclosed embodiments, the determination module 92 includes a first response submodule, a second response submodule, and a third response submodule. The first response submodule is configured to, in response to the vehicle status being a parking search, determine that the first perspective includes a front perspective and a rear perspective, and the second perspective includes a left perspective and a right perspective; the second response submodule is configured to, in response to the vehicle status being a rear-end parking, determine that the first perspective includes a front perspective, a left perspective, and a right perspective, and the second perspective includes a rear perspective; and the third response submodule is configured to, in response to the vehicle status being a front-end parking, determine that the first perspective includes a rear perspective, a left perspective, and a right perspective, and the second perspective includes a front perspective.

[0087] Therefore, the first and second perspectives are determined based on the vehicle state, so as to avoid the occurrence of split angle deviations between adjacent perspectives in the vehicle surround view image as much as possible, thereby improving the accuracy of parking space recognition.

[0088] In some disclosed embodiments, the parking space recognition device 90 further includes a detection module, a verification module, and an execution module. The detection module is configured to detect whether the recognition confidence level is greater than a confidence threshold that matches the vehicle state; the verification module is configured to map the pixel positions of the parking space corner points to a spatial coordinate system to obtain the spatial positions of the parking space corner points, obtain the spatial positions of reference objects that match the vehicle state, and verify the spatial positions of the parking space corner points based on the spatial positions of the reference objects to determine whether to update the parking space recognition results to the automatic parking system; and the execution module is configured to re-execute the steps of obtaining the vehicle surround view image and obtaining the vehicle state, as well as subsequent steps, until parking is completed or the vehicle leaves the parking space.

[0089] Therefore, by detecting whether the recognition confidence is greater than the confidence threshold that matches the vehicle status, it is possible to determine whether the current recognition result is credible. If the current recognition result is not credible, the steps of obtaining the vehicle surround view image and obtaining the vehicle status and their subsequent steps are re-executed. If the current recognition result is credible, the parking space recognition result is further judged to determine whether the parking space recognition result is updated to the automatic parking system, which helps to improve the accuracy of the parking space recognition result and further improve the user experience.

[0090] In some disclosed embodiments, when the vehicle status is searching for a parking space, the spatial position of the reference object is the spatial position of the first vehicle part, and the verification module includes a determination submodule and a first detection submodule. The determination submodule is used to determine the spatial position of the central axis of the parking space based on the spatial position of the corner point of the parking space; the first detection submodule is used to detect whether the spatial position of the first vehicle part is within a first preset range of the spatial position of the central axis of the parking space, and determine whether to update the latest parking space recognition result to the automatic parking system.

[0091] Therefore, when the vehicle status is searching for a parking space, the position of the central axis of the parking space is determined based on the spatial position of the corner points of the parking space, and then the distance between the position of the central axis and the spatial position of the first vehicle part is used to determine whether the current parking space recognition result is available, thereby further improving the accuracy of the parking space recognition result.

[0092] In some disclosed embodiments, when the vehicle status is front-end parking or rear-end parking, the spatial position of the reference object is the spatial position of the rear corner point of the vehicle when the vehicle status is searching for a parking space. The verification module includes a second detection submodule, and the second detection submodule is used to detect whether the position deviation between the spatial position of the reference object and the spatial position of the parking space corner point is within a second preset range, and determine whether to update the latest parking space recognition result to the automatic parking system.

[0093] Therefore, when the vehicle status is front-end parking or rear-end parking, the spatial position of the parking space corner point when the vehicle status is parking search is detected and compared with the spatial position of the parking space corner point in the latest parking space recognition result to determine whether the current parking space recognition result is available, thereby further improving the accuracy of the parking space recognition result.

[0094] In some disclosed embodiments, the parking space recognition device 90 further includes a loop module, which is configured to re-execute the steps of acquiring the vehicle surround view image and acquiring the vehicle status and subsequent steps until parking is completed or the vehicle leaves the parking space.

[0095] Therefore, by continuously obtaining the vehicle status step and its subsequent steps, the parking space recognition result is obtained, that is, the latest parking space recognition result is continuously updated until parking is completed or the vehicle leaves the parking space. This helps to improve the real-time and accuracy of parking space recognition, and avoid as much as possible the situation where the vehicle has a poor parking posture due to parking space recognition deviation.

[0096] In some disclosed embodiments, the confidence threshold for matching the front parking and the confidence threshold for matching the rear parking are both greater than the confidence threshold for matching the parking space search; and / or, the coordinate origin of the spatial coordinate system is the spatial position of the second vehicle part when the vehicle state is initially the parking space search.

[0097] See also Figure 10 , Figure 10 This is a schematic diagram of the framework of an embodiment of an electronic device of the present application. Electronic device 100 includes a memory 101 and a processor 102 coupled to each other. Memory 101 stores program instructions, and processor 102 is configured to execute the program instructions to implement the steps of any of the aforementioned parking space recognition method embodiments. Specifically, electronic device 100 may include, but is not limited to, desktop computers, laptop computers, servers, mobile phones, tablet computers, and the like, without limitation herein.

[0098] Specifically, the processor 102 is used to control itself and the memory 101 to implement the steps in any of the above-mentioned parking space identification method embodiments. The processor 102 can also be called a CPU (Central Processing Unit). The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 102 can be implemented by an integrated circuit chip.

[0099] The above scheme, on the one hand, determines the first perspective that needs to be cropped and the second perspective that needs to be retained in the vehicle surround view image by obtaining the vehicle surround view image and the vehicle status, and further crops the vehicle model itself, the image area of the first perspective, and the image area of the second perspective close to the first perspective in the vehicle surround view image, thereby avoiding as much as possible the deformation or dislocation of the vehicle surround view image caused by the deviation of the split angle between adjacent perspectives under different vehicle states, which helps to improve the accuracy of parking space recognition. On the other hand, by filling the cropped image area in the vehicle surround view image, the visual deviation caused by the texture of the vehicle model itself during the parking space recognition process is avoided, the clarity of the image to be recognized is improved, and the accuracy of parking space recognition is further improved.

[0100] See also Figure 11 , Figure 11 The computer-readable storage medium 110 stores program instructions 111 that can be executed by a processor, and the program instructions 111 are used to implement the steps of any of the above-mentioned parking space recognition method embodiments.

[0101] The above scheme, on the one hand, determines the first perspective that needs to be cropped and the second perspective that needs to be retained in the vehicle surround view image by obtaining the vehicle surround view image and the vehicle status, and further crops the vehicle model itself, the image area of the first perspective, and the image area of the second perspective close to the first perspective in the vehicle surround view image, thereby avoiding as much as possible the deformation or dislocation of the vehicle surround view image caused by the deviation of the split angle between adjacent perspectives under different vehicle states, which helps to improve the accuracy of parking space recognition. On the other hand, by filling the cropped image area in the vehicle surround view image, the visual deviation caused by the texture of the vehicle model itself during the parking space recognition process is avoided, the clarity of the image to be recognized is improved, and the accuracy of parking space recognition is further improved.

[0102] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0103] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced with each other and will not be repeated herein for the sake of brevity.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0105] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0106] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0108] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

Claims

1. A parking space recognition method, characterized in that: include: Acquire a surround view image of the vehicle and acquire a vehicle status; wherein the vehicle status includes any one of searching for a parking space, front parking, and rear parking; Based on the vehicle state, determining a first perspective that needs to be cropped and a second perspective that needs to be retained in the vehicle surround view image; cropping the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and cropping the image area of the vehicle surround view image of the second perspective close to the first perspective; Filling the cropped image area in the vehicle surround view image to obtain an image to be recognized; Recognition is performed based on the image to be recognized to obtain a parking space recognition result.

2. The method according to claim 1, characterized in that The step of cropping the image area of the vehicle surround view image where the second perspective is close to the first perspective includes: The intersection of the first viewing angle, the second viewing angle, and the image area dividing line of the car model itself is used as the target point; The image area close to the first viewing angle is cropped along a target line starting from the target point and deviating from a reference line by a preset angle; wherein the reference line is a boundary line between the first viewing angle and the second viewing angle and passes through the target point.

3. The method according to claim 2, characterized in that The preset angle is not less than the split angle deviation between adjacent viewing angles in the vehicle surround view image.

4. The method according to claim 1, wherein The determining, based on the vehicle state, a first perspective to be cropped and a second perspective to be retained in the vehicle surround view image includes at least one of the following: In response to the vehicle state being the parking search position, determining that the first perspective includes a front perspective and a rear perspective, and the second perspective includes a left perspective and a right perspective; In response to the vehicle state being rear-end parked, determining that the first perspective includes a front perspective, a left perspective, and a right perspective, and the second perspective includes a rear perspective; In response to the vehicle state being the front-end parking state, it is determined that the first viewing angle includes a rear viewing angle, a left viewing angle, and a right viewing angle, and the second viewing angle includes a front viewing angle.

5. The method according to claim 1, wherein The parking space recognition result includes a recognition confidence and a pixel position of a parking space corner point in the image to be recognized. After performing recognition based on the image to be recognized to obtain the parking space recognition result, the method further includes: detecting whether the recognition confidence is greater than a confidence threshold matching the vehicle state; If so, mapping the pixel position of the parking space corner point to a spatial coordinate system to obtain the spatial position of the parking space corner point, obtaining the spatial position of a reference object that matches the vehicle state, and verifying the spatial position of the parking space corner point based on the spatial position of the reference object to determine whether to update the parking space recognition result to the automatic parking system; If not, the steps of acquiring the vehicle surround view image and acquiring the vehicle status and subsequent steps are executed again until parking is completed or the vehicle leaves the parking space.

6. The method according to claim 5, characterized in that When the vehicle state is the parking space search, the spatial position of the reference object is the spatial position of the first vehicle part, and the spatial position of the parking space corner point is verified based on the spatial position of the reference object to determine whether to update the latest parking space recognition result to the automatic parking system, including: Determining the spatial position of the central axis of the parking space based on the spatial positions of the parking space corner points; It is detected whether the spatial position of the first vehicle part is located within a first preset range of the spatial position of the central axis of the parking space, and it is determined whether to update the latest parking space recognition result to the automatic parking system.

7. The method according to claim 5, characterized in that When the vehicle status is front-end parked or rear-end parked, the spatial position of the reference object is the spatial position of the parking space corner point when the vehicle status is parking space search, and verifying the spatial position of the parking space corner point based on the spatial position of the reference object to determine whether to update the latest parking space recognition result to the automatic parking system includes: Detecting whether a positional deviation between a spatial position of the reference object and a spatial position of the parking space corner point is within a second preset range, and determining whether to update the latest parking space recognition result to the automatic parking system.

8. The method according to claim 5, characterized in that After verifying the spatial position of the parking space corner point based on the spatial position of the reference object and determining whether to update the parking space recognition result to the automatic parking system, the method further includes: The steps of acquiring the vehicle surround view image and acquiring the vehicle status and subsequent steps are repeated until parking is completed or the vehicle leaves the parking space.

9. The method according to claim 5, characterized in that The confidence threshold for matching the front parking condition and the confidence threshold for matching the rear parking condition are both greater than the confidence threshold for matching the parking space search condition; And / or, the coordinate origin of the spatial coordinate system is the spatial position of the second vehicle part when the vehicle state is initially the parking search space.

10. A parking space recognition device, characterized in that: include: An acquisition module is used to acquire a surround view image of the vehicle and obtain a vehicle status; wherein the vehicle status includes: searching for a parking space, front parking, and rear parking; a determination module, configured to determine, based on the vehicle state, a first perspective to be cropped and a second perspective to be retained in the vehicle surround view image; a cropping module, configured to crop the image area of the vehicle model itself and the first perspective in the vehicle surround view image, and crop the image area of the second perspective close to the first perspective in the vehicle surround view image; A filling module, configured to fill the cropped image area in the vehicle surround view image to obtain an image to be recognized; The recognition module is used to perform recognition based on the image to be recognized to obtain a parking space recognition result.

11. An electronic device, characterized in that: The method comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the parking space recognition method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the parking space recognition method according to any one of claims 1 to 9.

Images