Parking space corner detection method, device and equipment and storage medium

Abstract

The invention discloses a parking space corner detection method, device and equipment and a storage medium, and relates to the technical field of network transmission, and the method comprises the steps: obtaining a parking space aerial view image of a target area, inputting the parking space aerial view image into a pre-trained parking space detection model, and obtaining a parking space rectangular contour in the parking space aerial view image; based on the rectangular contour of the parking space, cutting out a local parking space image from the aerial view image of the parking space; inputting the local parking space image into a pre-trained angular point detection model to obtain an initial parking space angular point in the local parking space image; based on the parking space deflection angle of the local parking space image, adjusting the initial parking space angular point in the local parking space image, and determining the parking space angular point in the local parking space image; and restoring the parking space angular points in the local parking space image to the parking space aerial view image of the target area to obtain parking space information of the target area. Through the above mode, the parking space contour is extracted first, then the parking space corner detection is performed, the interference of noise on corner detection is effectively removed, and the detection accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of network transmission technology, and in particular to a parking space corner point detection method, device, equipment and storage medium. Background Art

[0002] At present, the parking space corners are usually detected by relying on entry lines and dividing lines to determine the parking space. However, misjudgment is prone to occur in complex environments, and the accuracy is low, making it difficult to accurately detect the parking space corners.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a parking space corner point detection method, device, equipment and storage medium, aiming to solve the technical problem in the prior art that it is difficult to accurately detect parking space corner points in complex scenarios.

[0005] To achieve the above object, the present application provides a parking space corner point detection method, the method comprising:

[0006] Acquire a parking space bird's-eye view image of a target area, input the parking space bird's-eye view image into a pre-trained parking space detection model, and obtain a rectangular outline of the parking space in the parking space bird's-eye view image;

[0007] Based on the rectangular outline of the parking space, a partial parking space image is cut out from the parking space bird's-eye view image;

[0008] Inputting the local parking space image into a pre-trained corner point detection model to obtain initial parking space corner points in the local parking space image;

[0009] Based on the parking space deflection angle of the partial parking space image, adjusting the initial parking space corner point in the partial parking space image to determine the parking space corner point in the partial parking space image;

[0010] The parking space corner points in the local parking space image are restored to the parking space bird's-eye view image of the target area to obtain the parking space information of the target area.

[0011] In one embodiment, the step of adjusting the initial parking space corner point in the partial parking space image based on the parking space deflection angle of the partial parking space image to determine the parking space corner point in the partial parking space image comprises:

[0012] Acquire a parking space deflection angle of the local parking space image in a preset direction;

[0013] Determining a parking space deflection posture of the partial parking space image based on a parking space deflection angle of the partial parking space image in a preset direction;

[0014] Based on the parking space deflection posture of the partial parking space image, the initial parking space corner point in the partial parking space image is adjusted to obtain the parking space corner point in the partial parking space image.

[0015] In one embodiment, the step of obtaining a parking space deflection angle of the local parking space image in a preset direction includes:

[0016] Acquire a standard parking space image, and determine a standard parking space corner point in the standard parking space image;

[0017] Determine a rotation matrix based on the position coordinates of the corner points of the standard parking space in the standard parking space image and the position coordinates of the corner points of the parking space in the partial parking space image;

[0018] Based on the rotation matrix corresponding to the partial parking space image, a parking space deflection angle of the partial parking space image in a preset direction is determined.

[0019] In one embodiment, before the step of inputting the partial parking space image into a pre-trained corner point detection model to obtain the initial parking space corner points in the partial parking space image, the step further includes:

[0020] Acquire a local training image, and generate a multi-scale feature image based on the local training image and a preset multi-scale convolution step size;

[0021] The corner detection model is trained based on the multi-scale feature image.

[0022] In one embodiment, the step of obtaining a local training image includes:

[0023] Obtain an initial local training image;

[0024] The initial local training image is subjected to data enhancement processing to obtain the local training image, wherein the data enhancement processing at least includes scaling, rotation, inversion and affine transformation.

[0025] In one embodiment, the parking space detection model is a yolov5 network model, and the yolov5 network model includes an input unit, a reference unit, a neck unit, and an output unit connected in sequence. The step of inputting the parking space bird's-eye view image into the pre-trained parking space detection model to obtain the rectangular outline of the parking space in the parking space bird's-eye view image includes:

[0026] Inputting the parking space bird's-eye view image into the input unit for grid segmentation to obtain a grid image, performing Mosaic data enhancement processing and adaptive scaling processing on the grid image to obtain an initial feature image, and determining anchor frame parameters of the grid image;

[0027] Inputting the initial feature image after slicing into the reference unit and the neck unit in sequence to extract features, thereby obtaining a feature image;

[0028] The feature image and anchor frame parameters corresponding to the feature image are input into the output unit for target detection to obtain a predicted bounding box, and the rectangular outline of the parking space is screened out in the predicted bounding box according to the confidence and distance indicators corresponding to the predicted bounding box.

[0029] In one embodiment, the parking space detection model and the corner point detection model are obtained through overall training.

[0030] In addition, to achieve the above purpose, the present application also proposes a parking space corner point detection device, the parking space corner point detection device comprising:

[0031] A contour detection module is used to obtain a parking space bird's-eye view image of a target area, input the parking space bird's-eye view image into a pre-trained parking space detection model, and obtain a rectangular contour of the parking space in the parking space bird's-eye view image;

[0032] A corner point detection module, used for cutting out a partial parking space image from the parking space bird's-eye view image based on the rectangular outline of the parking space;

[0033] The corner point detection module is further used to input the local parking space image into a pre-trained corner point detection model to obtain initial parking space corner points in the local parking space image;

[0034] A deflection adjustment module, configured to adjust an initial parking space corner point in the partial parking space image based on a parking space deflection angle of the partial parking space image, and determine a parking space corner point in the partial parking space image;

[0035] The corner point detection module is further used to restore the parking space corner points in the local parking space image to the parking space bird's-eye view image of the target area to obtain the parking space information of the target area.

[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a parking corner point detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the parking corner point detection method described above.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the parking space corner point detection method described above are implemented.

[0038] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the parking space corner point detection method as described above are implemented.

[0039] The present application provides a parking space corner point detection method, which comprises the steps of acquiring a parking space bird's-eye view image of a target area, inputting the parking space bird's-eye view image into a pre-trained parking space detection model, and obtaining a rectangular outline of the parking space in the parking space bird's-eye view image; based on the rectangular outline of the parking space, cropping a partial parking space image from the parking space bird's-eye view image; inputting the partial parking space image into a pre-trained corner point detection model, and obtaining initial parking space corner points in the partial parking space image; adjusting the initial parking space corner points in the partial parking space image based on a parking space deflection angle of the partial parking space image, and determining the parking space corner points in the partial parking space image; restoring the parking space corner points in the partial parking space image to the parking space bird's-eye view image of the target area, and obtaining parking space information of the target area. This application first extracts the parking space outline in the bird's-eye view, and then performs parking space corner detection according to the parking space outline, which can effectively remove the interference of noise on corner point detection, improve the accuracy of parking space corner point detection in complex scenes, and obtain accurate parking space corner point coordinates. Even if there are no obvious parking space corner points drawn in the parking space, the complete parking space can be accurately detected in the bird's-eye view, solving the technical problem of difficulty in accurately detecting parking space corner points in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 This is a flow chart of Embodiment 1 of the parking space corner point detection method of the present application;

[0043] Figure 2 A schematic diagram of a parking space rectangular outline of a parking space corner point detection method provided in Example 1 of the present application;

[0044] Figure 3 A schematic diagram of parking space corner points of the parking space corner point detection method provided in Example 1 of the present application;

[0045] Figure 4 A schematic diagram of parking space corner point restoration of the parking space corner point detection method provided in Example 1 of the present application;

[0046] Figure 5 A schematic diagram of a brief process of a parking space corner point detection method provided in Example 1 of the present application;

[0047] Figure 6 This is a schematic diagram of the module structure of the parking corner point detection device according to an embodiment of the present application;

[0048] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the parking space corner point detection method in the embodiment of the present application.

[0049] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0052] The main solution of the embodiment of the present application is: obtain a bird's-eye view image of the parking space in the target area, input the parking space bird's-eye view image into a pre-trained parking space detection model, and obtain the rectangular outline of the parking space in the parking space bird's-eye view image; based on the rectangular outline of the parking space, crop a partial parking space image from the parking space bird's-eye view image; input the partial parking space image into a pre-trained corner point detection model to obtain the initial parking space corner points in the partial parking space image; based on the parking space deflection angle of the partial parking space image, adjust the initial parking space corner points in the partial parking space image to determine the parking space corner points in the partial parking space image; restore the parking space corner points in the partial parking space image to the parking space bird's-eye view image of the target area to obtain the parking space information of the target area.

[0053] At present, the parking space corners are usually detected by relying on entry lines and dividing lines to determine the parking space. However, misjudgment is prone to occur in complex environments, and the accuracy is low, making it difficult to accurately detect the parking space corners.

[0054] The present application provides a solution, which first extracts the parking space outline in the bird's-eye view, and then performs parking space corner detection according to the parking space outline, which can effectively remove the interference of noise on corner point detection, improve the accuracy of parking space corner point detection in complex scenes, and obtain accurate parking space corner point coordinates. Even if the parking space does not have obvious parking space corner points drawn in the bird's-eye view, the complete parking space can be accurately detected in the bird's-eye view, solving the technical problem that it is difficult to accurately detect parking space corner points in complex scenes.

[0055] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a parking space corner point detection device, etc., and this embodiment does not specifically limit this. The following takes the parking space corner point detection device as an example to illustrate this embodiment and the following embodiments.

[0056] The present application embodiment provides a parking space corner point detection method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the parking space corner point detection method of the present application.

[0057] In this embodiment, the parking space corner point detection method includes steps S10 to S50:

[0058] Step S10, obtaining a parking space bird's-eye view image of the target area, inputting the parking space bird's-eye view image into a pre-trained parking space detection model, and obtaining a rectangular outline of the parking space in the parking space bird's-eye view image;

[0059] It should be noted that the target area refers to the area where the parking space needs to be detected, which can be selected according to actual needs. The parking space bird's-eye view image is a bird's-eye view of the parking space and its surroundings. The parking space detection model is the model used in this embodiment to detect the parking space contour, which can be a commonly used target detection model, such as: Yolov series model, which is not specifically limited and usually requires pre-training. The parking space rectangular contour is the contour of the detected parking space, usually in the form of a rectangular frame. For example, refer to Figure 2 , Figure 2 The red rectangular frame in the figure is the obtained rectangular outline of the parking space.

[0060] In a feasible implementation, the parking space detection model is a yolov5 network model, which includes an input unit, a reference unit, a neck unit, and an output unit connected in sequence. Step S10 includes steps S101 to S103:

[0061] Step S101: inputting the parking space bird's-eye view image into the input unit for grid segmentation to obtain a grid image, performing Mosaic data enhancement processing and adaptive scaling processing on the grid image to obtain an initial feature image, and determining anchor frame parameters of the grid image;

[0062] It should be noted that after the parking space bird's-eye view image is input into the input unit, the input unit usually divides the parking space bird's-eye view image into a series of grids. The grid image is the image in each grid. Each grid needs to predict a series of bounding boxes to find the final rectangular outline of the parking space. These grid images are respectively subjected to Mosaic data enhancement processing, adaptive scaling processing, and adaptive anchor frame calculation. Mosaic data enhancement processing is to combine multiple images into one image at a certain ratio, so that the model can recognize the target in a smaller range. Through adaptive anchor frame calculation, appropriate anchor frame parameters are obtained for subsequent detection of the rectangular outline of the parking space.

[0063] Step S102, inputting the sliced ​​initial feature image into the reference unit and the neck unit in sequence to extract features, thereby obtaining a feature image;

[0064] It should be noted that the reference unit is the Backbone network unit, and the neck unit is the Neck network unit. The initial feature image is the image obtained after being processed by the input unit, and needs to be input into the Backbone network unit and the Neck network unit for further feature extraction, so as to obtain the corresponding feature image. Among them, before the initial feature image enters the Backbone network unit, the initial feature image needs to be sliced. For example, a value is taken out for every other pixel in the initial feature image to form four images. The four images are complementary and there is no information loss. The relevant information can be concentrated in the channel space. The input channel is expanded by 4 times, and the spliced ​​image becomes 12 channels relative to the original RGB three-channel mode. Finally, the new image is subjected to a convolution operation to obtain a double down-sampled feature map without information loss.

[0065] Step S103, inputting the feature image and the anchor frame parameters corresponding to the feature image into the output unit for target detection to obtain a predicted bounding box, and filtering out the rectangular outline of the parking space in the predicted bounding box according to the confidence and distance indicators corresponding to the predicted bounding box.

[0066] It should be noted that the essence of non-maximum suppression (NMS) is to search for local maxima and suppress non-maximum elements. This embodiment uses NMS to suppress redundant boxes during detection. In the process of detecting parking spaces, a large number of predicted bounding boxes will be generated at the position of the same target, and these bounding boxes may overlap with each other, so non-maximum suppression is needed to find the best predicted bounding box and eliminate redundant bounding boxes.

[0067] It can be understood that first, all the predicted bounding boxes are sorted in descending order according to the confidence, and the predicted bounding box with the highest confidence is selected to confirm that it is a correct prediction, and the distance index between the predicted bounding box and other predicted bounding boxes is calculated. The predicted bounding boxes with high overlap are removed using the distance index, that is, the predicted bounding boxes with distance index values ​​greater than the set threshold are deleted, and the bounding box left at the end is the best predicted bounding box, that is, the rectangular outline of the parking space. Among them, the distance index can be IOU (Intersection of Union) or other suitable indicators, which can be flexibly adjusted according to actual needs and are not specifically limited to this.

[0068] Step S20, based on the rectangular outline of the parking space, cutting out a partial parking space image from the parking space bird's-eye view image;

[0069] It can be understood that, according to the coordinates of the rectangular outline of the parking space, an image within the rectangular outline of the parking space, ie, a partial parking space image, is clipped from the parking space bird's-eye view image.

[0070] Step S30, inputting the local parking space image into a pre-trained corner point detection model to obtain initial parking space corner points in the local parking space image;

[0071] It should be noted that the corner point detection model, i.e., the model used in this embodiment to detect corner points of parking spaces in an image, may be a commonly used key point detection model, which is not specifically limited and usually requires pre-training.

[0072] It can be understood that, by using the trained corner point detection model to detect corner points in the local parking space image, the initial parking space corner points and the coordinates of the initial parking space corner points are obtained, and the initial parking space corner points are the detected corner points.

[0073] In a feasible implementation, a local training image is obtained, and a multi-scale feature image is generated based on the local training image and a preset multi-scale convolution step size; and the corner detection model is trained based on the multi-scale feature image.

[0074] It should be noted that the local training image is the image used to train the corner detection model. Parking spaces usually have strong structural information, such as symmetry and corner distribution. This information helps to locate key points (parking space corners) more accurately. Therefore, when training the corner detection model, a multi-scale feature map is used instead of a single-scale feature map to provide more complete information. Among them, the multi-scale of the multi-scale feature map is achieved by adjusting the convolution step size, which can improve the receptive field. The preset multi-scale convolution step size is the convolution step size required to achieve multi-scale set in this embodiment.

[0075] It is understandable that the corresponding multi-scale feature images are generated by using the local training images and the preset multi-scale convolution step size for corner detection model training. In addition, when entering the prediction stage during the training process, all multi-scale feature maps are connected. The trained corner detection model can predict the complete parking space corner coordinates based on the correlation between the blocked corner and other corners when the parking space corner is partially blocked.

[0076] In a feasible implementation, the step of obtaining a local training image includes: obtaining an initial local training image; performing data enhancement processing on the initial local training image to obtain the local training image, and the data enhancement processing at least includes scaling, rotation, inversion and affine transformation.

[0077] It should be noted that the initial local training image, that is, the initial training image of the corner detection model, in order to improve the generalization ability of the corner detection model for various special parking spaces and ensure stability and accuracy in different scenarios, this embodiment performs data enhancement processing on the initial local training image. The data enhancement processing includes at least scaling, rotation, inversion and affine transformation, and other suitable processing operations may also be set, which are not specifically limited.

[0078] In a feasible implementation manner, the parking space detection model and the corner point detection model are obtained through overall training.

[0079] It is understandable that in the specific implementation, the detection of the parking space rectangular outline and the detection of the parking space corner points can be integrated into the same model for training. At this time, the parking space detection model and the corner point detection model are trained as a whole, which can simplify the overall process and facilitate subsequent maintenance and optimization.

[0080] It should be understood that the outputs of the parking space detection model and the corner point detection model can be directly extracted and used without doing too much calculation, which can save the computing resources of the development board.

[0081] Step S40, adjusting the initial parking space corner point in the partial parking space image based on the parking space deflection angle of the partial parking space image, and determining the parking space corner point in the partial parking space image;

[0082] It should be noted that the initial parking space corner points obtained by the corner point detection model still have certain deviations, so further adjustments are needed to improve the accuracy of the detection results.

[0083] In a feasible implementation, step S40 may include steps S401 to S402:

[0084] Step S401, obtaining a parking space deflection angle of the partial parking space image in a preset direction, and determining a parking space deflection posture of the partial parking space image based on the parking space deflection angle of the partial parking space image in the preset direction;

[0085] It should be noted that the parking space deflection angle is the angle of the parking space deflection, and the parking space deflection posture is the overall deflection posture of the parking space. The preset direction refers to the direction dimension where the deflection angle needs to be calculated, such as: X-axis direction, Y-axis direction, Z-axis direction, and there is no specific limitation on this.

[0086] It can be understood that the deflection posture of the parking space is found by using the deflection angles of the parking space in different directions.

[0087] In a feasible implementation, step S401 may include steps S4011 to S4013:

[0088] Step S4011, acquiring a standard parking space image, and determining a standard parking space corner point in the standard parking space image;

[0089] It should be noted that the standard parking space image is a predefined parking space image used as a reference, and the average value of multiple parking space images may be used. The standard parking space corner points are key points selected as references in this embodiment, which are usually 4 parking space corner points in the standard parking space image.

[0090] Step S4012, determining a rotation matrix based on the position coordinates of the corner points of the standard parking space in the standard parking space image and the position coordinates of the corner points of the parking space in the partial parking space image;

[0091] It can be understood that a rotation matrix is ​​estimated using the four initial parking space corner points in each local parking space image and the four standard parking space corner points in the standard parking space image.

[0092] Step S4013: determining a parking space deflection angle of the partial parking space image in a preset direction based on a rotation matrix corresponding to the partial parking space image.

[0093] It can be understood that the parking space deflection angle in the preset direction is calculated from the rotation matrix.

[0094] Step S402: adjusting the initial parking space corner points in the partial parking space image based on the parking space deflection posture of the partial parking space image to obtain the parking space corner points in the partial parking space image.

[0095] It can be understood that, according to the parking space deflection posture, the initial parking space corner point is adjusted to obtain the final parking space corner point. Figure 3 , Figure 3 The red points marked in the figure are the detected corner points of the parking spaces.

[0096] Step S50: restore the parking space corner points in the partial parking space image to the parking space bird's-eye view image of the target area to obtain the parking space information of the target area.

[0097] It should be noted that the reference Figure 4 After obtaining the parking space corner points in each partial parking space image, these parking space corner points are restored to the parking space bird's-eye view image. The parking space information is the relevant data of the parking space, including at least the coordinates of the parking space corner points of each parking space.

[0098] The present embodiment provides a parking space corner point detection method, which obtains a parking space bird's-eye view image of a target area, inputs the parking space bird's-eye view image into a pre-trained parking space detection model, and obtains a rectangular outline of the parking space in the parking space bird's-eye view image; based on the parking space rectangular outline, a partial parking space image is cropped out from the parking space bird's-eye view image; the partial parking space image is input into a pre-trained corner point detection model to obtain an initial parking space corner point in the partial parking space image; based on the parking space deflection angle of the partial parking space image, the initial parking space corner point in the partial parking space image is adjusted to determine the parking space corner point in the partial parking space image; the parking space corner point in the partial parking space image is restored to the parking space bird's-eye view image of the target area to obtain parking space information in the target area. The present application first extracts the parking space outline from the bird's-eye view image, and then performs parking space corner point detection according to the parking space outline, which can effectively remove the interference of noise on corner point detection, improve the accuracy of parking space corner point detection in complex scenes, and obtain accurate parking space corner point coordinates. Even if the parking space does not have obvious parking space corner points drawn in the bird's-eye view image, the complete parking space can be accurately detected in the bird's-eye view.

[0099] For example, in order to help understand the implementation process of the parking space corner point detection method obtained by combining this embodiment with the above-mentioned embodiment 1, please refer to Figure 5 , Figure 5 A brief flow chart of a parking space corner point detection method is provided, specifically:

[0100] First, input a bird's-eye view image of a parking space; then perform parking space detection on the bird's-eye view image to obtain the rectangular outline of the parking space and the coordinates of the rectangular outline of the parking space; then crop according to the rectangular outline of the parking space to obtain a partial parking space image, perform key point detection on the partial parking space image to obtain the parking space corner points; finally, restore the parking space corner points to the bird's-eye view image of the parking space.

[0101] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the parking corner point detection method of the present application. More simple transformations based on this technical concept are all within the protection scope of the present application.

[0102] This application also provides a parking space corner point detection device, please refer to Figure 6 , the parking space corner point detection device comprises:

[0103] The contour detection module 10 is used to obtain a parking space bird's-eye view image of a target area, input the parking space bird's-eye view image into a pre-trained parking space detection model, and obtain a rectangular contour of the parking space in the parking space bird's-eye view image.

[0104] The corner point detection module 20 is used to cut out a partial parking space image from the parking space bird's-eye view image based on the rectangular outline of the parking space.

[0105] The corner point detection module 20 is further used to input the local parking space image into a pre-trained corner point detection model to obtain initial parking space corner points in the local parking space image.

[0106] The deflection adjustment module 30 is used to adjust the initial parking space corner point in the partial parking space image based on the parking space deflection angle of the partial parking space image, so as to determine the parking space corner point in the partial parking space image.

[0107] The corner point detection module 20 is further used to restore the parking space corner points in the local parking space image to the parking space bird's-eye view image of the target area to obtain the parking space information of the target area.

[0108] In a feasible implementation manner, the deflection adjustment module 30 is further used to obtain a parking space deflection angle of the local parking space image in a preset direction;

[0109] Determining a parking space deflection posture of the partial parking space image based on a parking space deflection angle of the partial parking space image in a preset direction;

[0110] Based on the parking space deflection posture of the partial parking space image, the initial parking space corner point in the partial parking space image is adjusted to obtain the parking space corner point in the partial parking space image.

[0111] In a feasible implementation manner, the deflection adjustment module 30 is further used to obtain a standard parking space image, and determine a standard parking space corner point in the standard parking space image;

[0112] Determine a rotation matrix based on the position coordinates of the corner points of the standard parking space in the standard parking space image and the position coordinates of the corner points of the parking space in the partial parking space image;

[0113] Based on the rotation matrix corresponding to the partial parking space image, a parking space deflection angle of the partial parking space image in a preset direction is determined.

[0114] In a feasible implementation manner, the corner detection module 20 is further used to obtain a local training image, and generate a multi-scale feature image based on the local training image and a preset multi-scale convolution step size;

[0115] The corner detection model is trained based on the multi-scale feature image.

[0116] In a feasible implementation manner, the corner point detection module 20 is further used to obtain an initial local training image;

[0117] The initial local training image is subjected to data enhancement processing to obtain the local training image, wherein the data enhancement processing at least includes scaling, rotation, inversion and affine transformation.

[0118] In a feasible implementation manner, the parking space detection model is a yolov5 network model, and the yolov5 network model includes an input unit, a reference unit, a neck unit, and an output unit connected in sequence. The contour detection module 10 is also used to input the parking space bird's-eye view image into the input unit for grid segmentation to obtain a grid image, perform Mosaic data enhancement processing and adaptive scaling processing on the grid image to obtain an initial feature image, and determine the anchor frame parameters of the grid image;

[0119] Inputting the initial feature image after slicing into the reference unit and the neck unit in sequence to extract features, thereby obtaining a feature image;

[0120] The feature image and anchor frame parameters corresponding to the feature image are input into the output unit for target detection to obtain a predicted bounding box, and the rectangular outline of the parking space is screened out in the predicted bounding box according to the confidence and distance indicators corresponding to the predicted bounding box.

[0121] In a feasible implementation manner, the parking space detection model and the corner point detection model are obtained through overall training.

[0122] The parking space corner point detection device provided by the present application adopts the parking space corner point detection method in the above embodiment, which can solve the technical problem that it is difficult to accurately detect the parking space corner points in complex scenes. Compared with the prior art, the parking space corner point detection device provided by the present application has the same beneficial effects as the parking space corner point detection method provided by the above embodiment, and other technical features in the parking space corner point detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0123] The present application provides a parking corner point detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the parking corner point detection method in the above-mentioned embodiment one.

[0124] Reference below Figure 7, which shows a schematic diagram of the structure of a parking corner point detection device suitable for implementing the embodiment of the present application. The parking corner point detection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The parking space corner point detection device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0125] like Figure 7 As shown, the parking corner detection device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the parking corner detection device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following devices may be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 may allow the parking corner detection device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a parking corner detection device with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead.

[0126] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0127] The parking space corner point detection device provided by the present application adopts the parking space corner point detection method in the above embodiment, which can solve the technical problem that it is difficult to accurately detect the parking space corner points in complex scenes. Compared with the prior art, the beneficial effects of the parking space corner point detection device provided by the present application are the same as the beneficial effects of the parking space corner point detection method provided by the above embodiment, and other technical features in the parking space corner point detection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0128] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0129] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0130] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the parking space corner point detection method in the above-mentioned embodiment.

[0131] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses or components, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution device, apparatus or component. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0132] The computer-readable storage medium may be included in the parking space corner point detection device; or may exist independently without being assembled into the parking space corner point detection device.

[0133] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the parking space corner point detection device, the parking space corner point detection device obtains a bird's-eye view image of the parking space in the target area, inputs the parking space bird's-eye view image into a pre-trained parking space detection model, and obtains the rectangular outline of the parking space in the parking space bird's-eye view image; based on the parking space rectangular outline, a partial parking space image is cropped out from the parking space bird's-eye view image; the partial parking space image is input into the pre-trained corner point detection model to obtain the initial parking space corner points in the partial parking space image; based on the parking space deflection angle of the partial parking space image, the initial parking space corner points in the partial parking space image are adjusted to determine the parking space corner points in the partial parking space image; the parking space corner points in the partial parking space image are restored to the parking space bird's-eye view image of the target area to obtain the parking space information of the target area.

[0134] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the device, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based device that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0137] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned parking space corner point detection method, and can solve the technical problem that it is difficult to accurately detect parking space corner points in complex scenes. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the parking space corner point detection method provided in the above-mentioned embodiment, and will not be repeated here.

[0138] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned parking space corner point detection method when executed by a processor.

[0139] The computer program product provided by the present application can solve the technical problem that it is difficult to accurately detect the corner points of parking spaces in complex scenes. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the parking corner point detection method provided by the above embodiment, which will not be repeated here.

[0140] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A parking space corner point detection method, characterized in that: The method comprises: Acquire a parking space bird's-eye view image of a target area, input the parking space bird's-eye view image into a pre-trained parking space detection model, and obtain a rectangular outline of the parking space in the parking space bird's-eye view image; Based on the rectangular outline of the parking space, a partial parking space image is cut out from the parking space bird's-eye view image; Inputting the local parking space image into a pre-trained corner point detection model to obtain initial parking space corner points in the local parking space image; Based on the parking space deflection angle of the partial parking space image, adjusting the initial parking space corner point in the partial parking space image to determine the parking space corner point in the partial parking space image; The parking space corner points in the local parking space image are restored to the parking space bird's-eye view image of the target area to obtain the parking space information of the target area.

2. The method according to claim 1, characterized in that The step of adjusting the initial parking space corner point in the partial parking space image based on the parking space deflection angle of the partial parking space image to determine the parking space corner point in the partial parking space image comprises: Acquire a parking space deflection angle of the local parking space image in a preset direction; Determining a parking space deflection posture of the partial parking space image based on a parking space deflection angle of the partial parking space image in a preset direction; Based on the parking space deflection posture of the partial parking space image, the initial parking space corner point in the partial parking space image is adjusted to obtain the parking space corner point in the partial parking space image.

3. The method according to claim 2, characterized in that The step of obtaining the parking space deflection angle of the local parking space image in a preset direction comprises: Acquire a standard parking space image, and determine a standard parking space corner point in the standard parking space image; Determine a rotation matrix based on the position coordinates of the corner points of the standard parking space in the standard parking space image and the position coordinates of the corner points of the parking space in the partial parking space image; Based on the rotation matrix corresponding to the partial parking space image, a parking space deflection angle of the partial parking space image in a preset direction is determined.

4. The method according to claim 1, characterized in that Before the step of inputting the local parking space image into a pre-trained corner point detection model to obtain the initial parking space corner points in the local parking space image, the step further includes: Acquire a local training image, and generate a multi-scale feature image based on the local training image and a preset multi-scale convolution step size; The corner detection model is trained based on the multi-scale feature image.

5. The method according to claim 4, characterized in that The step of obtaining a local training image comprises: Obtain an initial local training image; The initial local training image is subjected to data enhancement processing to obtain the local training image, wherein the data enhancement processing at least includes scaling, rotation, inversion and affine transformation.

6. The method according to claim 1, characterized in that The parking space detection model is a yolov5 network model, and the yolov5 network model includes an input unit, a reference unit, a neck unit, and an output unit connected in sequence. The step of inputting the parking space bird's-eye view image into the pre-trained parking space detection model to obtain the rectangular outline of the parking space in the parking space bird's-eye view image includes: Inputting the parking space bird's-eye view image into the input unit for grid segmentation to obtain a grid image, performing Mosaic data enhancement processing and adaptive scaling processing on the grid image to obtain an initial feature image, and determining anchor frame parameters of the grid image; Inputting the initial feature image after slicing into the reference unit and the neck unit in sequence to extract features, thereby obtaining a feature image; The feature image and anchor frame parameters corresponding to the feature image are input into the output unit for target detection to obtain a predicted bounding box, and the rectangular outline of the parking space is screened out in the predicted bounding box according to the confidence and distance indicators corresponding to the predicted bounding box.

7. The method according to any one of claims 1 to 6, characterized in that The parking space detection model and the corner point detection model are obtained through overall training.

8. A parking space corner point detection device, characterized in that: The device comprises: A contour detection module is used to obtain a parking space bird's-eye view image of a target area, input the parking space bird's-eye view image into a pre-trained parking space detection model, and obtain a rectangular contour of the parking space in the parking space bird's-eye view image; A corner point detection module, used for cutting out a partial parking space image from the parking space bird's-eye view image based on the rectangular outline of the parking space; The corner point detection module is further used to input the local parking space image into a pre-trained corner point detection model to obtain initial parking space corner points in the local parking space image; A deflection adjustment module, configured to adjust an initial parking space corner point in the partial parking space image based on a parking space deflection angle of the partial parking space image, and determine a parking space corner point in the partial parking space image; The corner point detection module is further used to restore the parking space corner points in the local parking space image to the parking space bird's-eye view image of the target area to obtain the parking space information of the target area.

9. A parking space corner point detection device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the parking space corner point detection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the parking space corner point detection method according to any one of claims 1 to 7 are implemented.

Images