Parking space area detection method and device and storage medium

Abstract

The invention discloses a parking space area detection method and apparatus, and a storage medium. The method comprises the steps of obtaining a to-be-processed image in a preset area; the preset area comprises a parking space area; inputting the to-be-processed image into a feature extraction model for parking space feature extraction to obtain parking space position features and parking space boundary features; the parking space position features represent position information of parking spaces in the preset area; the parking space boundary features represent boundary information of parking spaces in the preset area; determining a predicted position of the parking space in the preset area according to the parking space position feature; determining a predicted boundary of the parking space in the preset area according to the parking space boundary features; and determining the parking space area in the preset area according to the predicted position of the parking space in the preset area and the predicted boundary of the parking space in the preset area. The accuracy and reliability of the parking space area detection result are improved, and the vehicle can find the parking space conveniently.

Description

Technical Field

[0001] The present application relates to the field of intelligent automatic parking technology, and in particular to a parking area detection method, device and storage medium. Background Art

[0002] With the development of intelligent automatic parking technology, automatic parking of vehicles has gradually been accepted by the public in daily parking. However, due to the environmental conditions of the parking lot, such as lighting changes, weather conditions, etc., the accuracy and stability of the parking space detection system are challenged. The system needs to be able to work reliably in various environments. At the same time, with the increasing busyness of urban traffic, the real-time and accuracy requirements of the parking space detection system are getting higher and higher. The system needs to be able to provide parking space status information quickly and accurately so that the driver can find an available parking space in time.

[0003] However, traditional parking space detection equipment often requires high installation and maintenance costs, and has gradually failed to meet the growing demand for parking space detection. Summary of the invention

[0004] The present application provides a parking area detection method, device and storage medium, which can improve the accuracy and reliability of parking area detection results and facilitate vehicles to find parking spaces.

[0005] In one aspect, the present application provides a parking area detection method, the method comprising:

[0006] Acquire an image to be processed in a preset area; the preset area includes a parking area;

[0007] Input the image to be processed into a feature extraction model to extract parking space features, and obtain parking space position features and parking space boundary features; the parking space position features represent the position information of the parking spaces in the preset area; the parking space boundary features represent the boundary information of the parking spaces in the preset area;

[0008] Determining the predicted position of the parking space within the preset area according to the parking space position characteristics;

[0009] Determining the predicted boundary of the parking space within the preset area according to the parking space boundary characteristics;

[0010] The parking space area within the preset area is determined according to the predicted positions of the parking spaces within the preset area and the predicted boundaries of the parking spaces within the preset area.

[0011] In an exemplary embodiment, the method for training the feature extraction model includes:

[0012] Acquire sample image data within a sample preset area; the sample image data is annotated with sample parking space position labels and sample parking space boundary labels of parking spaces within the sample preset area;

[0013] Based on the feature extraction network of the preset model, the sample image data is subjected to parking space feature extraction processing to obtain the sample parking space position feature and the sample parking space boundary feature corresponding to the parking space in the sample preset area; the feature extraction network includes an initial parking space position detection network and an initial parking space boundary detection network;

[0014] Performing parking space position detection processing on the sample parking space position features based on the initial parking space position detection network to obtain sample parking space position results for parking spaces within the sample preset area;

[0015] Performing parking space boundary detection processing on the sample parking space boundary features based on the initial parking space boundary detection network to obtain sample parking space boundary results for parking spaces in the sample preset area;

[0016] According to the first difference between the sample parking space position result and the sample parking space position label, and the second difference between the sample parking space boundary result and the sample parking space boundary label, the preset model is trained, and the initial parking space position detection network after the training is used as the parking space position detection network, the initial parking space boundary detection network after the training is used as the parking space boundary detection network, and the feature extraction network in the preset model after the training is used as the feature extraction model.

[0017] In an exemplary embodiment, the step of acquiring sample image data within a preset sample area includes:

[0018] Acquire the sample image data within the sample preset area; the sample image data is annotated with the sample parking space position label of the parking space within the sample preset area;

[0019] Determine, according to the sample parking space position label, a sample parking space line of the parking spaces in the sample preset area and a sample parking space entrance direction corresponding to the sample parking space line;

[0020] According to the sample parking space line, determining a sample processed parking space line corresponding to the sample parking space line;

[0021] Determining a sample parking space corner point corresponding to the sample parking space line according to the sample parking space entrance direction corresponding to the sample parking space line;

[0022] Determine the sample parking space corner point corresponding to the sample processed parking space line according to the sample processed parking space line corresponding to the sample parking space line and the sample parking space corner point corresponding to the sample parking space line;

[0023] The sample parking space boundary label is determined according to the sample processed parking space line and the sample parking space corner point corresponding to the sample processed parking space line.

[0024] In an exemplary embodiment, the preset model is trained according to the first difference between the sample parking space position result and the sample parking space position label, and the second difference between the sample parking space boundary result and the sample parking space boundary label, and the initial parking space position detection network after the training is used as the parking space position detection network, the initial parking space boundary detection network after the training is used as the parking space boundary detection network, and the feature extraction network in the preset model after the training is used as the feature extraction model, including:

[0025] determining first loss information according to the first difference between the sample parking space position result and the sample parking space position label;

[0026] determining second loss information according to the second difference between the sample parking space boundary result and the sample parking space boundary label;

[0027] Determining target loss information according to the first loss information and the second loss information;

[0028] According to the target loss information, the parameters of the preset model are adjusted, and the initial parking space position detection network after training is used as the parking space position detection network, the initial parking space boundary detection network after training is used as the parking space boundary detection network, and the feature extraction network in the preset model after training is used as the feature extraction model.

[0029] In an exemplary embodiment, the feature extraction model includes the parking space position detection network and the parking space boundary detection network, and the inputting the image to be processed into the feature extraction model to extract parking space features to obtain parking space position features and parking space boundary features includes:

[0030] Inputting the image to be processed into the parking space line detection subnetwork in the parking space position detection network to perform parking space line detection processing to obtain the parking space lines in the image to be processed;

[0031] Inputting the image to be processed into the parking space entrance direction detection subnetwork in the parking space position detection network to perform parking space entrance direction detection processing, and obtaining the parking space entrance direction corresponding to the parking space line in the image to be processed;

[0032] Determining the parking space position feature according to the parking space line and the parking space entrance direction corresponding to the parking space line;

[0033] Inputting the image to be processed into the processed parking space line detection subnetwork in the parking space boundary detection network to perform processed parking space line detection processing to obtain the processed parking space lines in the image to be processed;

[0034] Inputting the image to be processed into the parking space corner point detection subnetwork in the parking space boundary detection network to perform parking space corner point detection processing, and obtaining parking space corner points corresponding to the processed parking space lines in the image to be processed;

[0035] The parking space boundary feature is determined according to the processed parking space line and the parking space corner point corresponding to the processed parking space line.

[0036] In an exemplary embodiment, determining the predicted position of the parking space in the preset area according to the parking space position feature includes:

[0037] Determine target coordinate information of the parking spaces in the preset area and target parking space images of the parking spaces in the preset area according to the target parking space lines and the target parking space entrance directions of the parking spaces in the preset area;

[0038] The predicted position of the parking space in the preset area is determined according to the target coordinate information of the parking space in the preset area and the target parking space image of the parking space in the preset area.

[0039] In an exemplary embodiment, determining the predicted boundary of the parking space in the preset area according to the parking space boundary feature includes:

[0040] Determine a target boundary image of the parking spaces in the preset area according to the target processed parking space lines and target parking space corner points of the parking spaces in the preset area;

[0041] The predicted boundary of the parking space in the preset area is determined according to the target boundary image of the parking space in the preset area.

[0042] In an exemplary embodiment, determining the parking space area within the preset area according to the predicted positions of the parking spaces within the preset area and the predicted boundaries of the parking spaces within the preset area includes:

[0043] Projecting the target boundary image of the parking spaces in the preset area onto the target parking space image of the parking spaces in the preset area to obtain a target parking space image to be processed;

[0044] Determine the parking space line difference and the parking space position difference according to the target parking space image to be processed; the parking space line difference is determined based on the difference between the target processed parking space line and the target parking space line; the parking space position difference is determined based on the difference between the target parking space corner point and the target parking space entrance direction;

[0045] According to the parking space line difference, the direction of the target parking space line on the to-be-processed parking space image is corrected to the direction of the target processed parking space line to obtain an updated target parking space line direction;

[0046] According to the parking space position difference, the position of the target parking space entrance direction on the parking space image to be processed is corrected to the position of the target parking space corner point to obtain an updated target parking space entrance direction;

[0047] The parking space area within the preset area is determined according to the updated target parking space line direction and the updated target parking space entrance direction.

[0048] In an exemplary embodiment, the method further comprises:

[0049] Extracting parking space status features from the image to be processed in the preset area to obtain parking space status features; the parking space status features represent the status of the parking space in the preset area;

[0050] Determining the parking space status of the parking spaces in the preset area according to the parking space status characteristics; the parking space status includes an idle state and an occupied state;

[0051] The parking space in the preset area whose parking space state is the idle state is set as the target parking space.

[0052] Another aspect provides a parking area detection device, the device comprising:

[0053] The to-be-processed image acquisition module is used to acquire the to-be-processed image in a preset area; the preset area includes a parking area;

[0054] A feature extraction module is used to input the image to be processed into a feature extraction model to extract parking space features, so as to obtain parking space position features and parking space boundary features; the parking space position features represent the position information of the parking spaces in the preset area; the parking space boundary features represent the boundary information of the parking spaces in the preset area;

[0055] A predicted position determination module, used to determine the predicted position of a parking space in the preset area according to the parking space position characteristics;

[0056] A predicted boundary determination module, used to determine the predicted boundary of the parking space in the preset area according to the parking space boundary characteristics;

[0057] The parking space area determination module is used to determine the parking space area in the preset area according to the predicted positions of the parking spaces in the preset area and the predicted boundaries of the parking spaces in the preset area.

[0058] On the other hand, an electronic device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded by the processor and executes the parking area detection method as described above.

[0059] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or at least one program is loaded and executed by a processor to implement the parking area detection method as described above.

[0060] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the parking space area detection method as described above.

[0061] The present application provides a parking area detection method, device and storage medium, which have the following technical effects: the present application obtains an image to be processed within a preset area; the preset area includes a parking area; the image to be processed is input into a feature extraction model to extract parking features, and obtain parking position features and parking boundary features; the parking position features represent the position information of parking spaces within the preset area; the parking boundary features represent the boundary information of parking spaces within the preset area; according to the parking position features, the predicted positions of parking spaces within the preset area are determined; according to the parking boundary features, the predicted boundaries of parking spaces within the preset area are determined; according to the predicted positions of parking spaces within the preset area and the predicted boundaries of parking spaces within the preset area, the parking area within the preset area is determined. The present application extracts parking space features from the image to be processed to extract parking space position features and parking space boundary features; determines the position information of parking spaces in a preset area through the parking space position features, determines the boundary information of parking spaces according to the parking space boundary features, and combines the two to obtain the specific positions and boundaries of parking spaces in the preset area, thereby determining the parking space area in the preset area; obtains the parking space area by combining the parking space position features and the parking space boundary features, and the parking space boundary features constrain the range of parking space detection, thereby improving the accuracy and reliability of the parking space area detection results, and facilitating the vehicle to find a parking space when automatically parking. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions and advantages of the embodiments of this specification or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 It is a flowchart of a parking area detection method provided in an embodiment of this specification;

[0064] Figure 2 It is a flowchart of a parking area detection method provided in an embodiment of this specification;

[0065] Figure 3 is a structural diagram of a feature extraction network provided in an embodiment of this specification;

[0066] Figure 4 It is a flowchart of a parking area detection method provided in an embodiment of this specification;

[0067] Figure 5 This is a schematic diagram of generating a parking space boundary label provided by an embodiment of this specification;

[0068] Figure 6 It is a flowchart of a parking area detection method provided in an embodiment of this specification;

[0069] Figure 7 It is a flowchart of a parking area detection method provided in an embodiment of this specification;

[0070] Figure 8 is a schematic diagram of a parking area correction provided in an embodiment of this specification;

[0071] Fig. 9 is a schematic diagram of a parking area provided in an embodiment of this specification;

[0072] Fig.10 is a schematic diagram of a parking area provided in an embodiment of this specification;

[0073] Fig.11 It is a flowchart of a parking area detection method provided in an embodiment of this specification;

[0074] Fig.12 It is a flowchart of a parking area detection method provided in an embodiment of this specification;

[0075] Fig.13 It is a structural schematic diagram of a parking area detection device provided in an embodiment of this specification.

[0076] Fig.14 It is a structural diagram of a server of a parking area detection method provided in an embodiment of this specification. DETAILED DESCRIPTION

[0077] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0078] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0079] The following describes a parking area detection method of the present application. Figure 1 It is a flowchart of a parking area detection method provided in an embodiment of this specification. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Specifically, Figure 1 As shown, the method can be applied to a control unit in a vehicle, and the method comprises:

[0080] S101: Acquire an image to be processed in a preset area; the preset area includes a parking area.

[0081] In the embodiments of the present specification, the preset area may be a parking lot or other places including parking areas; the image to be processed may be a bird's-eye view image obtained by performing IPM (Inverse Perspective Mapping) processing on the initial image acquired by the image acquisition device on the vehicle. Due to the existence of the perspective effect in the initial image, objects that were originally parallel in the initial image do not appear parallel in the initial image. Therefore, the IPM transformation is used to convert the captured complex perspective image into a simpler and easier-to-understand bird's-eye view, i.e., a bird's-eye view, so that subsequent image analysis becomes more direct. The bird's-eye view data provided to the vehicle control system can help make more accurate decisions, thereby facilitating the provision of parking spaces to the vehicle when the vehicle is automatically parked.

[0082] S103: Input the image to be processed into a feature extraction model to extract parking space features, and obtain parking space position features and parking space boundary features; the parking space position features represent the position information of the parking spaces in the preset area; the parking space boundary features represent the boundary information of the parking spaces in the preset area.

[0083] In the embodiment of the present specification, the parking space segmentation task is added to the feature extraction model, and the parking space feature extraction is performed on the processed image to extract the parking space position feature and the parking space boundary feature. The parking space position feature includes the parking space line and the parking space entrance direction, thereby determining the location information of the parking space; the parking space boundary feature includes the parking space corner point and the processed parking space line, thereby determining the boundary of the parking space, thereby constraining the scope of parking space detection, facilitating the subsequent determination of the parking space area, and improving the accuracy and reliability of the parking space area detection results.

[0084] In the embodiments of this specification, the training method of the feature extraction model is as follows: Figure 2 As shown, Figure 2 The following is a flowchart of a parking area detection method, including:

[0085] S201: Acquire sample image data within a sample preset area; the sample image data is annotated with sample parking space position labels and sample parking space boundary labels of parking spaces within the sample preset area.

[0086] In the embodiments of this specification, Figure 3 As shown, Figure 3 It is a structural diagram of a feature extraction network. Figure 3301 is an input image with a size of 1024*1024*3, 302 is a feature extraction network using Regnet as the backbone network, 303 is a parking space position feature extracted by the feature extraction network, 304 is a parking space boundary feature extracted by the feature extraction network, 305 is a parking space image determined based on the parking space position feature, and 306 is a boundary image determined based on the parking space boundary feature. Specifically, a sample parking space area is set in the sample preset area, the sample image data is an input image with a size of 1024*1024*3, and the sample image data has 3 color channels (RGB), and the sample parking space position label and the sample parking space boundary label are also marked in the sample image data.

[0087] In the embodiment of this specification, the sample image data within the sample preset area is obtained, such as Figure 4 As shown, Figure 4 The following is a flowchart of a parking area detection method, including:

[0088] S401: Acquire the sample image data within the sample preset area; the sample image data is annotated with the sample parking space position labels of the parking spaces within the sample preset area.

[0089] In the embodiment of the present specification, sample image data with sample parking space location labels is obtained, wherein the sample parking space location labels are manually labeled, and the parking space lines and parking spaces of the sample preset area are labeled. The parking spaces are labeled in order, and the entrance directions and parking space corners of the subsequent parking spaces are determined according to the labeled order. For example, Figure 5 As shown, Figure 5 In the figure, 501 represents the original image, 502 represents the sample parking space location label, 503 represents the sample parking space boundary label, A, B, C, and D represent the four marked parking spaces respectively, AB, BC, AD, and CD represent the sample parking space lines respectively, E and F represent the sample parking space corner points respectively, and EF, FG, and EH represent the sample parking space lines after processing respectively. The marking is carried out in the order of A→B→C→D, where A and B are used to determine the parking space corner points in the subsequent sample parking space boundary labels, and the directions of AD and BC are used to determine the parking space entrance direction and the parking space position. The sequential marking facilitates the subsequent distinction and improves the efficiency of parking space area detection.

[0090] S402: Determine, according to the sample parking space position tags, sample parking space lines of parking spaces within the sample preset area and sample parking space entrance directions corresponding to the sample parking space lines.

[0091] In the embodiments of this specification, Figure 5As shown, the sample parking space line is determined according to the parking space line marked in the sample parking space position label, that is, the sample parking space line is determined according to AB, AD, BC, and CD, and the sample parking space entrance direction is determined according to the direction of AD.

[0092] S403: Determine, according to the sample parking space line, a sample processed parking space line corresponding to the sample parking space line.

[0093] In the embodiments of this specification, Figure 5 As shown, the sample parking line AB is thickened from a thin line to a thick line, and the sample processed parking line EF is determined; the sample parking line BC is thickened from a thin line to a thick line, and the sample processed parking line FG is determined; the sample parking line AD is thickened from a thin line to a thick line, and the sample processed parking line EH is determined, and the sample processed parking lines EF, FG, and EH constitute the sample processed parking line label; by thickening the sample parking line, subsequent processing is facilitated, and the efficiency of parking area detection is improved.

[0094] S404: Determine a sample parking space corner point corresponding to the sample parking space line according to the sample parking space entrance direction corresponding to the sample parking space line.

[0095] In the embodiments of this specification, Figure 5 As shown, according to the sample parking space entrance direction AD, the sample parking spots A and B are determined, and the range of the sample parking spot A is expanded from a point to a circle, so as to obtain the sample parking space corner point E; the range of the sample parking spot B is expanded from a point to a circle, so as to obtain the sample parking space corner point F, and the sample parking space corner points E and F constitute the sample parking space corner point label; by expanding the range of the parking space corner point from a single point to a circular area, the visibility and robustness of the parking space corner point are enhanced.

[0096] S405: Determine the sample parking space corner point corresponding to the sample processed parking space line according to the sample processed parking space line corresponding to the sample parking space line and the sample parking space corner point corresponding to the sample parking space line.

[0097] In the embodiment of the present specification, the sample parking space corner point corresponding to the sample processed parking space line is determined based on the sample processed parking space line corresponding to the sample parking space line and the sample parking space corner point corresponding to the sample parking space line, so that the sample processed parking space line corresponds to the sample parking space corner point.

[0098] S406: Determine the sample parking space boundary label according to the sample processed parking space line and the sample parking space corner point corresponding to the sample processed parking space line.

[0099] In the embodiment of the present specification, the sample parking space boundary is constructed by the sample processed parking space line and the sample parking space corner point corresponding to the sample processed parking space line, thereby determining the sample parking space boundary label. By marking the sample parking space position label and automatically generating the sample parking space boundary label according to the sample parking space position label, the workload of manual marking is significantly reduced, the development cost is reduced, and additional information is provided to the feature extraction model through pre-processing, thereby improving the accuracy of the model prediction result.

[0100] S202: Based on a feature extraction network of a preset model, the sample image data is subjected to parking space feature extraction processing to obtain sample parking space position features and sample parking space boundary features corresponding to parking spaces within the sample preset area; the feature extraction network includes an initial parking space position detection network and an initial parking space boundary detection network.

[0101] In the embodiments of this specification, for example, Figure 3 As shown, the feature extraction model uses the Regnet network as the backbone network, i.e., the feature extraction network. The sample image data is input into the preset model, and the parking space features are extracted from the sample image data based on the feature extraction network Regnet. Feature learning is performed from shallow texture, color, and boundary information, thereby extracting the sample parking space position features and sample parking space boundary features corresponding to the parking spaces in the sample preset area. When predicting the model, it is necessary to perform the parking space position detection task and the parking space boundary detection task at the same time, and perform network training on the two feature heads respectively. The Regnet network can well support the ability of multi-task learning, thereby improving the efficiency of the overall system. Therefore, by using the feature extraction network Regnet, an end-to-end network is realized while ensuring that the parking space features of the parking spaces in the sample preset area can be accurately predicted under limited computational complexity.

[0102] S203: Perform parking space position detection processing on the sample parking space position features based on the initial parking space position detection network to obtain sample parking space position results of the parking spaces within the sample preset area.

[0103] In the embodiments of the present specification, a branch network is designed in the feature extraction network for parking space position detection, and the initial parking space position detection network is used to perform parking space position detection on the sample parking space position features, thereby obtaining the sample parking space position result, thereby improving the accuracy of the parking space position detection result.

[0104] S204: Perform parking space boundary detection processing on the sample parking space boundary features based on the initial parking space boundary detection network to obtain sample parking space boundary results of the parking spaces within the sample preset area.

[0105] In the embodiments of the present specification, a branch network is designed in the feature extraction network for parking space boundary detection, and the initial parking space boundary detection network is used to perform parking space boundary detection on the sample parking space boundary features to obtain the sample parking space boundary results, thereby constraining the scope of parking space area detection and improving the accuracy and reliability of the parking space boundary detection results.

[0106] S205: According to the first difference between the sample parking space position result and the sample parking space position label, and the second difference between the sample parking space boundary result and the sample parking space boundary label, the preset model is trained, and the initial parking space position detection network after the training is used as the parking space position detection network, the initial parking space boundary detection network after the training is used as the parking space boundary detection network, and the feature extraction network in the preset model after the training is used as the feature extraction model.

[0107] In the embodiments of the present specification, the model is trained by combining the differences in parking space locations and parking space boundaries, and the feature extraction model and the branch network within the feature extraction model are determined based on the preset model after the training is completed. Through multi-task learning, the model promotes learning between different tasks, which simplifies the process of model deployment and maintenance and improves the accuracy of model detection results.

[0108] In the embodiment of the present specification, the preset model is trained according to the first difference between the sample parking space position result and the sample parking space position label, and the second difference between the sample parking space boundary result and the sample parking space boundary label, and the initial parking space position detection network after the training is used as the parking space position detection network, the initial parking space boundary detection network after the training is used as the parking space boundary detection network, and the feature extraction network in the preset model after the training is used as the feature extraction model, such as Figure 6 As shown, Figure 6 The following is a flowchart of a parking area detection method, including:

[0109] S601: Determine first loss information according to the first difference between the sample parking space location result and the sample parking space location label.

[0110] In the embodiment of the present specification, the sample parking space position result is the predicted position information of the parking space in the sample preset area in the sample image data, and the sample parking space position label is the actual position information of the parking space in the sample preset area in the sample image data. The first loss information is determined according to the difference between the actual position information and the predicted position information. By using the first loss information to update the subsequent model parameters, the error of the parking space position predicted by the model is reduced, thereby improving the accuracy and reliability of the parking space position detection result.

[0111] S602: Determine second loss information according to the second difference between the sample parking space boundary result and the sample parking space boundary label.

[0112] In the embodiment of the present specification, the sample parking space boundary result is the predicted boundary information of the parking space in the sample preset area in the sample image data, and the sample parking space boundary label is the actual boundary information of the parking space in the sample preset area in the sample image data. According to the difference between the actual boundary information and the predicted boundary information, the second loss information is determined. Exemplarily, the second loss information can be determined by a cross entropy loss function. By using the second loss information to update the subsequent model parameters, the gap of the parking space boundary predicted by the model is narrowed, and the accuracy of the parking space boundary detection result is improved.

[0113] S603: Determine target loss information according to the first loss information and the second loss information.

[0114] In the embodiment of the present specification, the target loss information indicating the parking space position difference and the parking space boundary difference is determined based on the first loss information indicating the difference between the actual position information and the predicted position information and the second loss information indicating the difference between the actual boundary information and the predicted boundary information, and the target loss information can be expressed in a weighted summation based on the first loss information and the second loss information. By determining the target loss information, subsequent model updates are facilitated, thereby improving the efficiency of model prediction.

[0115] S604: According to the target loss information, the parameters of the preset model are adjusted, and the initial parking space position detection network after the training is used as the parking space position detection network, the initial parking space boundary detection network after the training is used as the parking space boundary detection network, and the feature extraction network in the preset model after the training is used as the feature extraction model.

[0116] In the embodiments of the present specification, the target loss information is determined to train the model, and the model parameters are adjusted to ensure that the model can learn the best weight configuration to minimize the loss function, wherein these optimized weights represent the model's best representation of the input data features. The model parameters are adjusted to reduce the parking space position differences and parking space boundary differences, and the initial parking space position detection network after training is used as the parking space position detection network, the initial parking space boundary detection network after training is used as the parking space boundary detection network, and the feature extraction network in the preset model after training is used as the feature extraction model, so that two tasks can be completed by one model, and the model can promote learning between different tasks, simplifying the process of model deployment and maintenance, and improving the accuracy of model detection results.

[0117] In the embodiment of this specification, the feature extraction model includes the parking space position detection network and the parking space boundary detection network. The image to be processed is input into the feature extraction model to extract parking space features, and the parking space position features and parking space boundary features are obtained, such as Figure 7 As shown, Figure 7 The following is a flowchart of a parking area detection method, including:

[0118] S701: Inputting the image to be processed into the parking space line detection subnetwork in the parking space position detection network to perform parking space line detection processing to obtain the parking space lines in the image to be processed.

[0119] In the embodiments of the present specification, a branch subnetwork is set in the parking space position detection network, namely, a parking space line detection subnetwork is used for parking space line detection. By performing parking space line detection processing on the image to be processed, the parking space lines in the image to be processed are obtained, thereby achieving comprehensive detection of the parking space lines in the image to be processed and improving the accuracy of the parking space line detection results.

[0120] S702: Input the image to be processed into the parking space entrance direction detection subnetwork in the parking space position detection network to perform parking space entrance direction detection processing to obtain the parking space entrance direction corresponding to the parking space line in the image to be processed.

[0121] In the embodiments of the present specification, a branch subnetwork is set in the parking space position detection network, namely, the parking space entrance direction subnetwork is used for detecting and processing the parking space entrance direction. By performing parking space entrance direction detection processing on the image to be processed, the parking space entrance direction in the image to be processed is obtained, and the parking space entrance direction is matched one-to-one with the parking space line through the position of the parking space entrance direction on the image to be processed, thereby achieving comprehensive detection of the parking space entrance direction in the image to be processed and improving the accuracy of the parking space entrance direction detection result.

[0122] S703: Determine the parking space position feature according to the parking space line and the parking space entrance direction corresponding to the parking space line.

[0123] In the embodiment of this specification, the parking space position features are determined based on the parking space lines in the image to be processed and the parking space entrance directions corresponding to the parking space lines, and the approximate position, shape and parking space entrance direction of the parking space in the image to be processed are determined.

[0124] S704: Inputting the image to be processed into the processed parking space line detection subnetwork in the parking space boundary detection network to perform processed parking space line detection processing to obtain the processed parking space lines in the image to be processed.

[0125] In the embodiments of the present specification, a branch subnetwork is set in the parking space boundary detection network, that is, the processed parking space line detection subnetwork is used for processed parking space line detection, and the processed parking space lines in the image to be processed are obtained by performing processed parking space line detection processing on the image to be processed, thereby achieving comprehensive detection of the processed parking space lines in the image to be processed and improving the accuracy of the processed parking space line detection results.

[0126] S705: Inputting the image to be processed into the parking space corner point detection subnetwork in the parking space boundary detection network to perform parking space corner point detection processing, and obtaining parking space corner points in the image to be processed corresponding to the processed parking space lines.

[0127] In the embodiments of the present specification, a branch sub-network is set in the parking space boundary detection network, that is, the parking space corner point sub-network is used for parking space corner point detection. By performing parking space corner point detection processing on the image to be processed, the parking space corner points in the image to be processed are obtained, and the parking space corner points are matched one-to-one with the processed parking space lines through the positions of the parking space corner points on the image to be processed, thereby achieving comprehensive detection of the parking space corner points in the image to be processed, and improving the accuracy and reliability of the parking space corner point detection results, which is convenient for the subsequent determination of the entrance angle of the parking space.

[0128] S706: Determine the parking space boundary feature according to the processed parking space line and the parking space corner point corresponding to the processed parking space line.

[0129] In the embodiment of this specification, the boundary of the parking space in the preset area is determined based on the processed parking space line and the parking space corner point corresponding to the processed parking space line, thereby determining the parking space boundary feature. By performing multi-task learning, the parking space boundary detection task is added to the model, which constrains the range of parking space area detection and improves the accuracy and reliability of the parking space area detection result.

[0130] S105: Determine the predicted position of the parking space in the preset area according to the parking space position characteristics.

[0131] In the embodiment of the present specification, the predicted position of the parking space in the preset area is determined by the position information of the parking space in the preset area in the image to be processed, the positions of all parking spaces in the preset area are detected, and the comprehensive detection of the parking space position is achieved.

[0132] In the embodiment of the present specification, determining the predicted position of the parking space in the preset area according to the parking space position feature includes:

[0133] Determine target coordinate information of the parking spaces in the preset area and target parking space images of the parking spaces in the preset area according to the target parking space lines and the target parking space entrance directions of the parking spaces in the preset area;

[0134] In the embodiment of the present specification, the target parking space line of the parking space in the preset area is converted into a coordinate value, so as to determine the target coordinate information of the parking space in the preset area, and the target parking space image of the parking space in the preset area is determined according to the area formed by the target parking space line and the target parking space entrance direction. Figure 8 As shown, Figure 8 801 represents the target parking space image, AB, BC, CD, and AD in 801 represent the target parking space lines respectively, the direction of AD represents the entrance direction of the target parking space, the horizontal direction of the image to be processed is set as the x-axis, the vertical direction of the image to be processed is set as the y-axis, the center of the image to be processed is set as the origin, the coordinate values ​​of the parking spots A, B, C, and D corresponding to the target parking space line are obtained, and the target coordinate information is determined according to the coordinate values ​​of A, B, C, and D.

[0135] The predicted position of the parking space in the preset area is determined according to the target coordinate information of the parking space in the preset area and the target parking space image of the parking space in the preset area.

[0136] In the embodiments of this specification, for example, Figure 8 As shown, the horizontal direction of the image to be processed is set as the x-axis, the vertical direction of the image to be processed is set as the y-axis, and the center of the image to be processed is set as the origin. The coordinate points of the parking spaces in the preset area are determined according to the coordinate values ​​of the parking spots A, B, C, and D corresponding to each target parking space line. The position, shape, and parking space entrance angle of the parking spaces in the preset area are determined according to the coordinate points and the target parking space image. By converting the target parking space line and the target parking space entrance direction into coordinate information, and then determining the predicted position of the parking space in the preset area by the coordinate information, the accuracy and reliability of the parking space area detection results are improved.

[0137] S107: Determine the predicted boundary of the parking space in the preset area according to the parking space boundary characteristics.

[0138] In the embodiment of the present specification, the predicted boundaries of the parking spaces in the preset area are determined by the boundary information of the parking spaces in the preset area, the boundaries of all parking spaces in the preset area are detected, all-round detection of parking space boundaries is achieved, and the reliability and accuracy of the parking space boundary detection results are improved.

[0139] In the embodiment of the present specification, determining the predicted boundary of the parking space in the preset area according to the parking space boundary feature includes:

[0140] Determine a target boundary image of the parking spaces in the preset area according to the target processed parking space lines and target parking space corner points of the parking spaces in the preset area;

[0141] In the embodiment of the present specification, the parking spots and parking corner points corresponding to the target processed parking spot lines of the parking spots in the preset area are converted into coordinate values, and the area formed by the target processed parking spot lines and the target parking corner points and the coordinate values ​​are combined to determine the target boundary image of the parking spots in the preset area. For example, Figure 8 As shown, Figure 8 802 represents the target boundary image, E and F in 802 represent the target parking space corner points corresponding to the target processed parking space line, G and H represent the parking spots corresponding to the target processed parking space line, the horizontal direction of the image to be processed is set as the x-axis, the vertical direction of the image to be processed is set as the y-axis, the center of the image to be processed is set as the origin, and the coordinate points of the parking spaces in the preset area are determined according to the coordinate values ​​of the target parking space corner points E and F corresponding to the target processed parking space line and the coordinate values ​​of the parking spots G and H corresponding to the target processed parking space line, and the area formed by the target processed parking space line and the target parking space corner points and the coordinate values ​​are combined to determine the target boundary image.

[0142] The predicted boundary of the parking space in the preset area is determined according to the target boundary image of the parking space in the preset area.

[0143] In the embodiments of this specification, for example, Figure 8 As shown, the horizontal direction of the image to be processed is set as the x-axis, the vertical direction of the image to be processed is set as the y-axis, and the center of the image to be processed is set as the origin. The coordinate points of the parking spaces in the preset area are determined according to the coordinate values ​​of the target parking space corner points E and F corresponding to the parking space lines after target processing and the coordinate values ​​of the parking spots G and H corresponding to the parking space lines after target processing, respectively. According to the coordinate points and the target boundary image, the boundaries of the parking spaces in the preset area and the parking space entrance angle are obtained, thereby determining the predicted boundaries of the parking spaces in the preset area. By determining the predicted boundaries of the parking spaces in the preset area, the range of the parking space area detection and the range of the parking space entrance angle are constrained, thereby improving the accuracy and reliability of the parking space area detection results.

[0144] S109: Determine the parking space area within the preset area according to the predicted positions of the parking spaces within the preset area and the predicted boundaries of the parking spaces within the preset area.

[0145] In the embodiment of this specification, the preset area is an area at a preset distance from the target vehicle, and the preset distance is determined by the acquisition range of the image acquisition device on the target vehicle, such as Figure 9-10 As shown, Fig. 9 Among them, 901 represents the target vehicle, 902, 903, 904 all represent parking spaces within the preset area, and 905 represents the preset area. Fig.101001 represents the target vehicle, 1002 represents the parking area in the preset area, and 1003 represents the preset area. The parking area in the preset area is determined by simultaneously acquiring the predicted position of the parking space in the preset area and the predicted boundary of the parking space in the preset area. The position of the parking space in the preset area is determined, and the parking space detection range and the parking space entrance angle are constrained by detecting the parking space boundary, thereby reducing the detection error and improving the accuracy of the parking space area detection result, so as to provide a parking space for the vehicle when the vehicle is automatically parked.

[0146] In the embodiment of the present specification, the parking space area in the preset area is determined according to the predicted position of the parking space in the preset area and the predicted boundary of the parking space in the preset area, such as Fig.11 As shown, Fig.11 The following is a flowchart of a parking area detection method, including:

[0147] S1101: Projecting the target boundary image of the parking spaces in the preset area onto the target parking space image of the parking spaces in the preset area to obtain a target parking space image to be processed.

[0148] In the embodiments of this specification, Figure 8 As shown, Figure 8 803 represents the target parking space image to be processed obtained by projecting the target boundary image 802 onto the target parking space image 801. By projecting the target boundary image onto the target parking space image, the two are combined into one, which is convenient for subsequent image correction processing.

[0149] S1102: Determine a parking space line difference and a parking space position difference according to the target parking space image to be processed; the parking space line difference is determined based on a difference between the target processed parking space line and the target parking space line; the parking space position difference is determined based on a difference between a target parking space corner point and an entrance direction of the target parking space.

[0150] In the embodiments of the present specification, the detection range of parking spaces within a preset area is limited by target processing of parking space lines and target parking space corner points, thereby facilitating subsequent adjustments of parking space lines and parking space positions by separately determining the differences between parking space lines and parking space positions, thereby improving the efficiency and convenience of parking area detection.

[0151] S1103: According to the parking space line difference, the direction of the target parking space line on the to-be-processed parking space image is corrected to the direction of the target processed parking space line to obtain an updated target parking space line direction.

[0152] In the embodiment of the present specification, according to the difference between the target parking space line and the target processed parking space line, all target parking space lines are adjusted to the position of the target processed parking space line, thereby correcting the error of the target parking space line in the predicted direction, reducing the range of the parking space area, and improving the accuracy of the parking space area detection result.

[0153] S1104: According to the parking space position difference, the position of the target parking space entrance direction on the parking space image to be processed is corrected to the position of the target parking space corner point to obtain an updated target parking space entrance direction.

[0154] In the embodiment of this specification, according to the difference between the target parking space entrance direction and the target parking space corner point, all the target parking space entrance directions are adjusted to the position of the target parking space corner point, so as to correct the deviation of the target parking space entrance direction. Figure 8 As shown, the position of the target parking space entrance direction is adjusted according to the direction of the arrow in 803, thereby improving the accuracy of the parking space entrance angle prediction result, and facilitating providing the vehicle with a parking space and a suitable parking space entrance angle corresponding to the parking space when the vehicle is parking.

[0155] S1105: Determine the parking space area within the preset area according to the updated target parking space line direction and the updated target parking space entrance direction.

[0156] In the embodiments of this specification, Figure 8 As shown, Figure 8 804 represents the preset area and the parking area within the preset area. The target parking space image to be processed is corrected based on the parking space line difference and the parking space position difference, so as to obtain the parking space area within the preset area. The small deviation in the model prediction is corrected through post-processing, and the range of the parking space area detection is constrained. At the same time, the entrance angle of the parking space is accurate, the quality of the final prediction result is improved, and the accuracy and reliability of the parking space area detection result are improved, which is convenient for providing parking spaces for vehicles when parking.

[0157] In the embodiments of this specification, Fig.12 As shown, Fig.12 The following is a flowchart of a parking area detection method, wherein the method further comprises:

[0158] S1201: Extract parking space status features from the image to be processed in the preset area to obtain parking space status features; the parking space status features represent the status of the parking spaces in the preset area.

[0159] In the embodiment of the present specification, by adding the parking space status task to the feature extraction model, the feature extraction model performs multi-task learning, predicts the parking space position features and parking space boundary features within the preset area, and predicts the parking space status features at the same time, thereby clarifying the usage status of the parking spaces within the preset area.

[0160] S1202: Determine the parking space status of the parking spaces in the preset area according to the parking space status characteristics; the parking space status includes an idle state and an occupied state.

[0161] In the embodiment of the present specification, the parking space status characteristics are used to determine whether the parking space in the preset area is idle or occupied, which facilitates the subsequent combination of the vehicle status information with the parking space area information in the preset area, thereby improving the efficiency and convenience of vehicle parking.

[0162] S1203: Setting a parking space in the preset area whose parking space status is the idle state as a target parking space.

[0163] In the embodiments of the present specification, by combining the parking space status of the parking spaces in the preset area with the parking space area in the preset area, the vacant parking spaces in the preset area are obtained, so that when detecting the parking space, the system can quickly and accurately provide parking space status information and parking space position information, so that the vehicle can quickly find an available parking space when automatically parking, thereby improving the efficiency and convenience of parking and realizing the diversification of vehicle functions.

[0164] This specification also provides a parking area detection device, such as Fig.13 As shown, the device comprises:

[0165] The to-be-processed image acquisition module 1301 is used to acquire the to-be-processed image in a preset area; the preset area includes a parking area;

[0166] The feature extraction module 1302 is used to input the image to be processed into a feature extraction model to extract parking space features, thereby obtaining parking space position features and parking space boundary features; the parking space position features represent the position information of the parking spaces in the preset area; the parking space boundary features represent the boundary information of the parking spaces in the preset area;

[0167] The predicted position determination module 1303 is used to determine the predicted position of the parking space in the preset area according to the parking space position characteristics;

[0168] A predicted boundary determination module 1304 is used to determine the predicted boundary of the parking space in the preset area according to the parking space boundary characteristics;

[0169] The parking space area determination module 1305 is used to determine the parking space area in the preset area according to the predicted positions of the parking spaces in the preset area and the predicted boundaries of the parking spaces in the preset area.

[0170] In some embodiments, the apparatus further comprises:

[0171] The sample image data module is used to obtain sample image data within a sample preset area; the sample image data is annotated with sample parking space position labels and sample parking space boundary labels of the parking spaces within the sample preset area.

[0172] The parking space feature extraction module is used to perform parking space feature extraction processing on the sample image data based on the feature extraction network of the preset model to obtain the sample parking space position features and sample parking space boundary features corresponding to the parking spaces in the sample preset area; the feature extraction network includes an initial parking space position detection network and an initial parking space boundary detection network.

[0173] The initial parking space position detection module is used to perform parking space position detection processing on the sample parking space position features based on the initial parking space position detection network to obtain the sample parking space position results of the parking spaces within the sample preset area.

[0174] The initial parking space boundary detection module is used to perform parking space boundary detection processing on the sample parking space boundary features based on the initial parking space boundary detection network to obtain the sample parking space boundary results of the parking spaces within the sample preset area.

[0175] A training module is used to train the preset model according to a first difference between the sample parking space position result and the sample parking space position label, and a second difference between the sample parking space boundary result and the sample parking space boundary label, and use the initial parking space position detection network after the training as the parking space position detection network, use the initial parking space boundary detection network after the training as the parking space boundary detection network, and use the feature extraction network in the preset model after the training as the feature extraction model.

[0176] In some embodiments, the sample image data module further includes:

[0177] The sample parking space position label submodule is used to obtain the sample image data within the sample preset area; the sample image data is marked with the sample parking space position label of the parking space within the sample preset area.

[0178] The first determination submodule is used to determine, according to the sample parking space position label, a sample parking space line of the parking spaces within the sample preset area and a sample parking space entrance direction corresponding to the sample parking space line.

[0179] The sample processed parking space line submodule is used to determine the sample processed parking space line corresponding to the sample parking space line according to the sample parking space line.

[0180] The sample parking space corner point submodule is used to determine the sample parking space corner point corresponding to the sample parking space line according to the sample parking space entrance direction corresponding to the sample parking space line.

[0181] The second determining submodule is used to determine the sample parking space corner point corresponding to the sample processed parking space line according to the sample processed parking space line corresponding to the sample parking space line and the sample parking space corner point corresponding to the sample parking space line.

[0182] The sample parking space boundary label submodule is used to determine the sample parking space boundary label according to the sample processed parking space line and the sample parking space corner point corresponding to the sample processed parking space line.

[0183] In some embodiments, the training module further includes:

[0184] The first loss information submodule is used to determine the first loss information according to the first difference between the sample parking space position result and the sample parking space position label.

[0185] The second loss information submodule is used to determine the second loss information according to the second difference between the sample parking space boundary result and the sample parking space boundary label.

[0186] The target loss information submodule is used to determine target loss information based on the first loss information and the second loss information.

[0187] An updating submodule is used to adjust the parameters of the preset model according to the target loss information, use the initial parking space position detection network after training as the parking space position detection network, use the initial parking space boundary detection network after training as the parking space boundary detection network, and use the feature extraction network in the preset model after training as the feature extraction model.

[0188] In some embodiments, the feature extraction module further includes:

[0189] The parking space line detection sub-network submodule is used to input the image to be processed into the parking space line detection sub-network in the parking space position detection network to perform parking space line detection processing to obtain the parking space lines in the image to be processed.

[0190] The parking space entrance direction detection sub-network sub-module is used to input the image to be processed into the parking space entrance direction detection sub-network in the parking space position detection network to perform parking space entrance direction detection processing to obtain the parking space entrance direction corresponding to the parking space line in the image to be processed.

[0191] The parking space position feature determination submodule is used to determine the parking space position feature according to the parking space line and the parking space entrance direction corresponding to the parking space line.

[0192] The processed parking space line detection sub-network sub-module is used to input the image to be processed into the processed parking space line detection sub-network in the parking space boundary detection network to perform processed parking space line detection processing to obtain the processed parking space lines in the image to be processed.

[0193] The parking space corner point detection sub-network sub-module is used to input the image to be processed into the parking space corner point detection sub-network in the parking space boundary detection network to perform parking space corner point detection processing, and obtain the parking space corner points in the image to be processed corresponding to the processed parking space lines.

[0194] The parking space boundary feature determination submodule is used to determine the parking space boundary feature according to the processed parking space line and the parking space corner point corresponding to the processed parking space line.

[0195] In some embodiments, the predicted position determination module further includes:

[0196] The position intermediate information determination submodule is used to determine the target coordinate information of the parking spaces in the preset area and the target parking space images of the parking spaces in the preset area according to the target parking space lines and the target parking space entrance directions of the parking spaces in the preset area.

[0197] The predicted position determination submodule is used to determine the predicted position of the parking space in the preset area according to the target coordinate information of the parking space in the preset area and the target parking space image of the parking space in the preset area.

[0198] In some embodiments, the prediction boundary determination module further includes:

[0199] The boundary intermediate information determination submodule is used to determine the target boundary image of the parking spaces in the preset area according to the target processed parking space lines and the target parking space corner points of the parking spaces in the preset area.

[0200] The predicted boundary determination submodule is used to determine the predicted boundary of the parking space in the preset area according to the target boundary image of the parking space in the preset area.

[0201] In some embodiments, the parking area determination module further includes:

[0202] The target parking space image to be processed submodule is used to project the target boundary image of the parking space in the preset area onto the target parking space image of the parking space in the preset area to obtain the target parking space image to be processed.

[0203] The difference determination submodule is used to determine the parking space line difference and the parking space position difference according to the target parking space image to be processed; the parking space line difference is determined based on the difference between the target processed parking space line and the target parking space line; the parking space position difference is determined based on the difference between the target parking space corner point and the target parking space entrance direction.

[0204] The target parking space line direction updating submodule is used to correct the direction of the target parking space line on the to-be-processed parking space image to the direction of the target processed parking space line according to the parking space line difference, so as to obtain an updated target parking space line direction.

[0205] The target parking space entrance direction updating submodule is used to correct the position of the target parking space entrance direction on the parking space image to be processed to the position of the target parking space corner point according to the parking space position difference, so as to obtain an updated target parking space entrance direction.

[0206] The parking space area determination submodule is used to determine the parking space area within the preset area according to the updated target parking space line direction and the updated target parking space entrance direction.

[0207] In some embodiments, the apparatus further comprises:

[0208] The parking space status feature module is used to extract the parking space status feature of the image to be processed in the preset area to obtain the parking space status feature; the parking space status feature represents the status of the parking space in the preset area.

[0209] The parking space status determination module is used to determine the parking space status of the parking space in the preset area according to the parking space status characteristics; the parking space status includes an idle state and an occupied state.

[0210] The target parking space determination module is used to set the parking space in the preset area whose parking space state is the idle state as the target parking space.

[0211] The device and method embodiments in the described device embodiments are based on the same inventive concept.

[0212] An embodiment of the present specification provides a vehicle electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement a parking area detection method as provided in the above method embodiment.

[0213] An embodiment of the present application also provides a computer storage medium, which can be set in a terminal to store at least one instruction or at least one program related to a parking area detection method in a method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the parking area detection method provided in the above method embodiment.

[0214] The embodiment of the present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes to implement the parking area detection method provided in the above method embodiment.

[0215] The memory described in the embodiments of this specification can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, application programs required for functions, etc.; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor with access to the memory.

[0216] The parking area detection method embodiment provided in the embodiments of this specification can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Fig.14 is a hardware structure block diagram of a server of a parking area detection method provided in an embodiment of this specification. Fig.14As shown, the server 1400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1410 (the central processing unit 1410 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1430 for storing data, and one or more storage media 1420 (such as one or more mass storage devices) for storing application programs 1423 or data 1422. Among them, the memory 1430 and the storage medium 1420 can be short-term storage or permanent storage. The program stored in the storage medium 1420 may include one or more modules, each of which may include a series of instruction operations on the server. Furthermore, the central processing unit 1410 may be configured to communicate with the storage medium 1420 and execute a series of instruction operations in the storage medium 1420 on the server 1400. The server 1400 may also include one or more power supplies 1460, one or more wired or wireless network interfaces 1450, one or more input and output interfaces 1440, and / or one or more operating systems 1421, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0217] The input / output interface 1440 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the server 1400. In one example, the input / output interface 1440 includes a network adapter (NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 1440 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0218] It can be understood by those skilled in the art that Fig.14 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig.14 More or fewer components as shown, or with Fig.14 Different configurations are shown.

[0219] It can be seen from the embodiments of the parking space area detection method, device and storage medium provided by the present application that the present application obtains an image to be processed within a preset area; the preset area includes a parking space area; the image to be processed is input into a feature extraction model to extract parking space features to obtain parking space position features and parking space boundary features; the parking space position features represent the position information of the parking spaces within the preset area; the parking space boundary features represent the boundary information of the parking spaces within the preset area; based on the parking space position features, the predicted positions of the parking spaces within the preset area are determined; based on the parking space boundary features, the predicted boundaries of the parking spaces within the preset area are determined; based on the predicted positions of the parking spaces within the preset area and the predicted boundaries of the parking spaces within the preset area, the parking space area within the preset area is determined. The present application uses a feature extraction model with Regnet as the backbone network to extract features from the image to be processed after IPM change processing, and through pre-processing before model prediction, the system can automatically generate parking space boundary labels according to parking space position labels, thereby reducing the need for manual labeling and saving cost budget in the actual development process; by adding parking space boundary tasks to the model, the model realizes multi-task learning, and predicts parking space position features and parking space detection features at the same time, thereby enhancing the generalization ability, increasing the constraints on parking space area prediction, and improving the accuracy and reliability of model prediction; by post-processing the extracted parking space features, correcting small deviations in the results obtained by the model prediction, improving the quality of the final prediction results, effectively eliminating the parking space deviations obtained by parking space detection, and realizing accurate prediction of parking space information within the field of view through an end-to-end network while ensuring limited computational complexity, thereby improving the accuracy and reliability of parking space area detection results, improving the efficiency and convenience of vehicle automatic parking, and realizing vehicle functional diversification.

[0220] It should be noted that the above sequence of the embodiments of this specification is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0221] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0222] Those skilled in the art can understand that all or part of the steps of the above embodiments can be completed by hardware, or by a program to instruct the relevant hardware to complete the steps. The program can be stored in a computer storage medium, and the storage medium mentioned above can be a read-only memory, a disk or an optical disk, etc. The above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A parking area detection method, characterized in that: The method comprises: Acquire an image to be processed in a preset area; the preset area includes a parking area; Input the image to be processed into a feature extraction model to extract parking space features, and obtain parking space position features and parking space boundary features; the parking space position features represent the position information of the parking spaces in the preset area; the parking space boundary features represent the boundary information of the parking spaces in the preset area; Determining the predicted position of the parking space within the preset area according to the parking space position characteristics; Determining the predicted boundary of the parking space within the preset area according to the parking space boundary characteristics; The parking space area within the preset area is determined according to the predicted positions of the parking spaces within the preset area and the predicted boundaries of the parking spaces within the preset area.

2. The parking area detection method according to claim 1, characterized in that: The training method of the feature extraction model comprises: Acquire sample image data within a sample preset area; the sample image data is annotated with sample parking space position labels and sample parking space boundary labels of parking spaces within the sample preset area; Based on the feature extraction network of the preset model, the sample image data is subjected to parking space feature extraction processing to obtain the sample parking space position feature and the sample parking space boundary feature corresponding to the parking space in the sample preset area; the feature extraction network includes an initial parking space position detection network and an initial parking space boundary detection network; Performing parking space position detection processing on the sample parking space position features based on the initial parking space position detection network to obtain sample parking space position results for parking spaces within the sample preset area; Performing parking space boundary detection processing on the sample parking space boundary features based on the initial parking space boundary detection network to obtain sample parking space boundary results for parking spaces in the sample preset area; According to the first difference between the sample parking space position result and the sample parking space position label, and the second difference between the sample parking space boundary result and the sample parking space boundary label, the preset model is trained, and the initial parking space position detection network after the training is used as the parking space position detection network, the initial parking space boundary detection network after the training is used as the parking space boundary detection network, and the feature extraction network in the preset model after the training is used as the feature extraction model.

3. The parking area detection method according to claim 2, characterized in that: The step of obtaining sample image data within a preset sample area includes: Acquire the sample image data within the sample preset area; the sample image data is annotated with the sample parking space position label of the parking space within the sample preset area; Determine, according to the sample parking space position label, a sample parking space line of the parking spaces in the sample preset area and a sample parking space entrance direction corresponding to the sample parking space line; According to the sample parking space line, determining a sample processed parking space line corresponding to the sample parking space line; Determining a sample parking space corner point corresponding to the sample parking space line according to the sample parking space entrance direction corresponding to the sample parking space line; Determine the sample parking space corner point corresponding to the sample processed parking space line according to the sample processed parking space line corresponding to the sample parking space line and the sample parking space corner point corresponding to the sample parking space line; The sample parking space boundary label is determined according to the sample processed parking space line and the sample parking space corner point corresponding to the sample processed parking space line.

4. The parking area detection method according to claim 2, characterized in that: The method includes training the preset model according to the first difference between the sample parking space position result and the sample parking space position label, and the second difference between the sample parking space boundary result and the sample parking space boundary label, using the initial parking space position detection network after the training as the parking space position detection network, using the initial parking space boundary detection network after the training as the parking space boundary detection network, and using the feature extraction network in the preset model after the training as the feature extraction model, including: determining first loss information according to the first difference between the sample parking space position result and the sample parking space position label; determining second loss information according to the second difference between the sample parking space boundary result and the sample parking space boundary label; Determining target loss information according to the first loss information and the second loss information; According to the target loss information, the parameters of the preset model are adjusted, and the initial parking space position detection network after training is used as the parking space position detection network, the initial parking space boundary detection network after training is used as the parking space boundary detection network, and the feature extraction network in the preset model after training is used as the feature extraction model.

5. The parking area detection method according to claim 1, characterized in that: The feature extraction model includes the parking space position detection network and the parking space boundary detection network. The inputting the image to be processed into the feature extraction model to extract parking space features to obtain parking space position features and parking space boundary features includes: Inputting the image to be processed into the parking space line detection subnetwork in the parking space position detection network to perform parking space line detection processing to obtain the parking space lines in the image to be processed; Inputting the image to be processed into the parking space entrance direction detection subnetwork in the parking space position detection network to perform parking space entrance direction detection processing, and obtaining the parking space entrance direction corresponding to the parking space line in the image to be processed; Determining the parking space position feature according to the parking space line and the parking space entrance direction corresponding to the parking space line; Inputting the image to be processed into the processed parking space line detection subnetwork in the parking space boundary detection network to perform processed parking space line detection processing to obtain the processed parking space lines in the image to be processed; Inputting the image to be processed into the parking space corner point detection subnetwork in the parking space boundary detection network to perform parking space corner point detection processing, and obtaining parking space corner points corresponding to the processed parking space lines in the image to be processed; The parking space boundary feature is determined according to the processed parking space line and the parking space corner point corresponding to the processed parking space line.

6. The parking area detection method according to claim 1, characterized in that: The step of determining the predicted position of the parking space within the preset area according to the parking space position feature includes: Determine target coordinate information of the parking spaces in the preset area and target parking space images of the parking spaces in the preset area according to the target parking space lines and the target parking space entrance directions of the parking spaces in the preset area; The predicted position of the parking space in the preset area is determined according to the target coordinate information of the parking space in the preset area and the target parking space image of the parking space in the preset area.

7. The parking area detection method according to claim 6, characterized in that: Determining the predicted boundary of the parking space in the preset area according to the parking space boundary feature includes: Determine a target boundary image of the parking spaces in the preset area according to the target processed parking space lines and target parking space corner points of the parking spaces in the preset area; The predicted boundary of the parking space in the preset area is determined according to the target boundary image of the parking space in the preset area.

8. The parking area detection method according to claim 7, characterized in that: The determining the parking space area within the preset area according to the predicted positions of the parking spaces within the preset area and the predicted boundaries of the parking spaces within the preset area includes: Projecting the target boundary image of the parking spaces in the preset area onto the target parking space image of the parking spaces in the preset area to obtain a target parking space image to be processed; Determine the parking space line difference and the parking space position difference according to the target parking space image to be processed; the parking space line difference is determined based on the difference between the target processed parking space line and the target parking space line; the parking space position difference is determined based on the difference between the target parking space corner point and the target parking space entrance direction; According to the parking space line difference, the direction of the target parking space line on the to-be-processed parking space image is corrected to the direction of the target processed parking space line to obtain an updated target parking space line direction; According to the parking space position difference, the position of the target parking space entrance direction on the parking space image to be processed is corrected to the position of the target parking space corner point to obtain an updated target parking space entrance direction; The parking space area within the preset area is determined according to the updated target parking space line direction and the updated target parking space entrance direction.

9. The parking area detection method according to claim 1, characterized in that: The method further comprises: Extracting parking space status features from the image to be processed in the preset area to obtain parking space status features; the parking space status features represent the status of the parking space in the preset area; Determine the parking space status of the parking spaces in the preset area according to the parking space status characteristics; the parking space status includes an idle state and an occupied state; The parking space in the preset area whose parking space state is the idle state is set as the target parking space.

10. A parking area detection device, characterized in that: The device comprises: The to-be-processed image acquisition module is used to acquire the to-be-processed image in a preset area; the preset area includes a parking area; A feature extraction module is used to input the image to be processed into a feature extraction model to extract parking space features, so as to obtain parking space position features and parking space boundary features; the parking space position features represent the position information of the parking spaces in the preset area; the parking space boundary features represent the boundary information of the parking spaces in the preset area; A predicted position determination module, used to determine the predicted position of a parking space in the preset area according to the parking space position characteristics; A predicted boundary determination module, used to determine the predicted boundary of the parking space in the preset area according to the parking space boundary characteristics; The parking space area determination module is used to determine the parking space area in the preset area according to the predicted positions of the parking spaces in the preset area and the predicted boundaries of the parking spaces in the preset area.

Images