A parking space recognition method and device

Abstract

The application relates to a parking space recognition method and device, which comprises the following steps: extracting global semantic features and local semantic features contained in a parking lot image by using a target neural network model; the target parking space is an idle parking space available for parking; adjusting initial entrance corner point information based on prior parking space entrance information included in the target neural network model and the local semantic features to obtain target entrance corner point information; the prior parking space entrance information comprises a standard entrance corner point direction and a standard entrance width of a standard parking space; and the target entrance corner point direction and the target entrance width are marked in the target parking space to guide a vehicle to park. According to the scheme, the prior parking space entrance information is used to guide parking space entrance recognition, the calculation amount of the target neural network model is reduced, the model reaches a real-time operation speed, the parking space entrance recognition precision can be improved, and the problems of parking space misrecognition and inaccurate direction can be effectively avoided.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a parking space identification method and device. Background Art

[0002] As autonomous driving matures, automated parking technology is becoming increasingly important in the intelligent driving field. Automated parking refers to the process where a car automatically parks itself without human intervention, using perception and control algorithms.

[0003] Currently, automated parking algorithms typically use neural network models for parking space recognition. Specifically, a parking lot image is fed into the neural network model for image processing, obtaining the center coordinates and probability of the parking space, the coordinates and probability of the corner points, and the offset of the corner points relative to the center coordinates. Secondary coordinates of the corner points are then calculated based on the center coordinates and their offsets. The second coordinates are then used to correct the first coordinates to obtain the actual coordinates and probability of the corner points for parking space detection.

[0004] However, this algorithm has two problems: First, it requires a lot of computation, and given the limited computing power of the onboard processor, it cannot achieve real-time processing speed. Second, in real-world road conditions, the type, color, and corner point style of parking spaces often vary greatly, leading to false parking space detection and inaccurate directions. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention provides a parking space identification method in a first aspect, the method comprising:

[0006] Obtaining a parking lot image of a parking space to be identified;

[0007] Extracting global semantic features and local semantic features contained in the parking lot image using a target neural network model; the global semantic features are used to determine initial entrance corner point information of a target parking space; the target parking space is an available parking space;

[0008] Based on the prior parking space entrance information included in the target neural network model and the local semantic features, the initial entrance corner point information is modified to obtain target entrance corner point information, the target entrance corner point information including a target entrance corner point direction and a target entrance width; the prior parking space entrance information includes a standard entrance corner point direction and a standard entrance width of a standard parking space;

[0009] The target entrance corner direction and the target entrance width are marked in the target parking space to guide the vehicle to park.

[0010] Optionally, the target neural network model is trained by the following method:

[0011] Input the parking lot sample image dataset and the pre-acquired prior parking space entrance information into the initial neural network model;

[0012] Extracting sample global feature information of each cell included in the parking lot sample image using the feature backbone network of the initial neural network model; the sample global feature information includes: whether the cell is located inside a sample target parking space, the category of the sample target parking space, sample initial entrance corner point information of the sample target parking space, and whether the target parking space is occupied;

[0013] Utilizing the feature backbone network, extracting local feature information of the samples of each cell, wherein the local feature information of the samples is used to correct the initial entry corner point information of the samples to obtain a predicted entry corner point direction and a predicted entry width;

[0014] Based on the predetermined target priori parking space entrance information and the actual entrance information of the sample target parking space, respectively determining a first residual corresponding to the predicted entrance corner direction and a second residual corresponding to the predicted entrance width;

[0015] The initial neural network model is trained based on the first residual and the second residual to obtain a target neural network model.

[0016] Optionally, determining a first residual corresponding to the predicted entry corner point direction includes:

[0017] Obtaining the standard entrance corner point direction and the standard entrance width included in the prior parking space entrance information, and obtaining the real entrance corner point direction and the real entrance width included in the real entrance corner point information;

[0018] Determine the value of an exponential function with e as the base and the direction of the predicted entry corner point as the exponent to obtain a direction index value;

[0019] Determine the product of the standard entry corner point direction and the directional index value to obtain a directional product;

[0020] The square of the difference between the direction product and the true entrance corner point direction is determined to obtain the first residual.

[0021] Optionally, determining a second residual corresponding to the predicted entry width includes:

[0022] Determine a value of an exponential function with e as the base and the predicted entrance width as the exponent to obtain a distance index value;

[0023] determining a product of the standard entrance width and the distance index value to obtain a distance product;

[0024] The second residual is obtained by determining the square of the difference between the distance product and the true entrance width.

[0025] Optionally, before extracting the global semantic features and local semantic features contained in the parking lot image using the target neural network model, the method further includes:

[0026] Acquire a parking space sample image dataset; the parking space sample image dataset includes various types of parking space images currently existing;

[0027] executing a clustering algorithm on the parking space sample image dataset to obtain a target number of clusters, wherein each cluster includes parking spaces of similar types;

[0028] The entrance corner direction and entrance width of each parking space in the cluster are respectively obtained to obtain the target number group of prior parking space entrance information.

[0029] Optionally, before respectively determining the first residual corresponding to the predicted entry corner point direction and the second residual corresponding to the predicted entry width, the method further includes:

[0030] Matching the target number group's prior parking space entrance information with the real entrance corner point information corresponding to the sample target parking space to obtain a target matching degree;

[0031] From the target number group of a priori parking space entrance information, the a priori parking space entrance information with the target matching degree greater than or equal to a preset matching degree threshold is selected as the target a priori parking space entrance information.

[0032] Optionally, the target entrance width is the Euclidean distance between the left entrance corner point and the right entrance corner point of the target parking space.

[0033] A second aspect of the present invention provides a parking space recognition device, comprising:

[0034] An image acquisition module, used to acquire a parking lot image of a parking space to be identified;

[0035] a feature extraction module for extracting global semantic features and local semantic features contained in the parking lot image using a target neural network model; the global semantic features are used to determine initial entrance corner point information of a target parking space; the target parking space is an available parking space;

[0036] a target entrance corner point information acquisition module, configured to modify the initial entrance corner point information based on the prior parking space entrance information included in the target neural network model and the local semantic features to obtain target entrance corner point information, wherein the target entrance corner point information includes a target entrance corner point direction and a target entrance width; the prior parking space entrance information includes a standard entrance corner point direction and a standard entrance width of a standard parking space;

[0037] The marking module is used to mark the target entrance corner point direction and the target entrance width in the target parking space to guide the vehicle to park.

[0038] Optionally, the device further includes a model training module, wherein the model training module is configured to:

[0039] Input the parking lot sample image dataset and the pre-acquired prior parking space entrance information into the initial neural network model;

[0040] Extracting sample global feature information of each cell included in the parking lot sample image using the feature backbone network of the initial neural network model; the sample global feature information includes: whether the cell is located inside a sample target parking space, the category of the sample target parking space, sample initial entrance corner point information of the sample target parking space, and whether the target parking space is occupied;

[0041] Utilizing the feature backbone network, extracting local feature information of the samples of each cell, wherein the local feature information of the samples is used to correct the initial entry corner point information of the samples to obtain a predicted entry corner point direction and a predicted entry width;

[0042] Based on the predetermined target priori parking space entrance information and the actual entrance information of the sample target parking space, respectively determining a first residual corresponding to the predicted entrance corner direction and a second residual corresponding to the predicted entrance width;

[0043] The initial neural network model is trained based on the first residual and the second residual to obtain a target neural network model.

[0044] Optionally, the model training module is specifically used to:

[0045] Obtaining the standard entrance corner point direction and the standard entrance width included in the prior parking space entrance information, and obtaining the real entrance corner point direction and the real entrance width included in the real entrance corner point information;

[0046] Determine the value of an exponential function with e as the base and the direction of the predicted entry corner point as the exponent to obtain a direction index value;

[0047] Determine the product of the standard entry corner point direction and the directional index value to obtain a directional product;

[0048] The difference between the direction product and the true entry corner point direction is determined to obtain the first residual.

[0049] Optionally, the model training module is specifically used to:

[0050] Determine a value of an exponential function with e as the base and the predicted entrance width as the exponent to obtain a distance index value;

[0051] determining a product of the standard entrance width and the distance index value to obtain a distance product;

[0052] The difference between the distance product and the true entrance width is determined to obtain the second residual.

[0053] Optionally, the device further includes:

[0054] A sample image acquisition module is used to acquire a parking space sample image dataset; the parking space sample image dataset includes various types of parking space images currently existing;

[0055] a clustering module, configured to execute a clustering algorithm on the parking space sample image dataset to obtain a target number of clusters, wherein each cluster includes parking spaces of similar types;

[0056] The size acquisition module is used to respectively acquire the entrance corner direction and entrance width of each parking space in the cluster to obtain the prior parking space entrance information of the target number group.

[0057] Optionally, the device further includes:

[0058] a matching module, configured to match the target number group's prior parking space entrance information with the actual entrance corner point information corresponding to the sample target parking space to obtain a target matching degree;

[0059] The selection module is configured to select, from the target number group of prior parking space entrance information, prior parking space entrance information having the target matching degree greater than or equal to a preset matching degree threshold, as target prior parking space entrance information.

[0060] A third aspect of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the parking space recognition method as described in the first aspect.

[0061] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the parking space recognition method as described in the first aspect.

[0062] The embodiments of the present invention have the following beneficial effects:

[0063] A parking space recognition method provided by an embodiment of the present invention obtains a parking lot image of a parking space to be recognized; uses a target neural network model to extract global semantic features and local semantic features contained in the parking lot image; the global semantic features are used to determine initial entrance corner information of a target parking space; the target parking space is a vacant parking space available for parking; based on prior parking space entrance information included in the target neural network model and the local semantic features, the initial entrance corner information is corrected to obtain target entrance corner information, including a target entrance corner direction and a target entrance width; the prior parking space entrance information includes a standard entrance corner direction and a standard entrance width for a standard parking space; the target entrance corner direction and the target entrance width are annotated in the target parking space to guide vehicle parking. This solution uses prior parking space entrance information as a guide for parking space entrance recognition, reducing the computational complexity of the target neural network model and enabling the model to achieve real-time operation speed. Furthermore, by constraining the prediction results in the form of prior knowledge, the accuracy of parking space entrance recognition can be improved, effectively avoiding problems such as false parking space detection and inaccurate parking direction.

[0064] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] To more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0066] Figure 1 A flowchart of the steps of a first parking space identification method provided by an embodiment of the present invention;

[0067] Figure 2 A schematic diagram of a parking space marking image provided by an embodiment of the present invention;

[0068] Figure 3 A flowchart of the steps of a target neural network model training method provided by an embodiment of the present invention;

[0069] Figure 4 A flowchart of the steps of a second parking space identification method provided by an embodiment of the present invention;

[0070] Figure 5 This is a structural block diagram of a parking space recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0072] This specification provides method operation steps as described in the embodiments or flowcharts, but more or fewer operation steps may be included based on routine or non-creative work. When implemented in an actual system or server product, the methods shown in the embodiments or figures may be executed sequentially or in parallel (for example, in a parallel processor or multi-threaded processing environment).

[0073] Figure 1 This is a flowchart of the steps of the first parking space identification method provided by an embodiment of the present invention. The method may include the following steps:

[0074] Step 101: Acquire a parking lot image of a parking space to be identified.

[0075] After the vehicle enters the parking lot, the vehicle body camera can collect a real-time parking lot image around the vehicle, which may include occupied parking spaces and vacant parking spaces. Specifically, the parking lot image can be a bird's-eye view.

[0076] The parking lot image is input into a processor of the vehicle, which performs semantic analysis on the parking lot image and identifies vacant parking spaces contained in the parking lot image.

[0077] Step 102: extract global semantic features and local semantic features contained in the parking lot image using a target neural network model; the global semantic features are used to determine initial entrance corner point information of a target parking space; the target parking space is an empty parking space available for parking.

[0078] The embodiment of the present invention adopts an end-to-end single-stage target neural network model. This method can simultaneously obtain the global semantic features and local semantic features of parking spaces through a convolutional neural network. By fusing the two information, the information of parking spaces on a bird's-eye view can be obtained.

[0079] Specifically, the parking lot image is first divided into a grid by rows and columns, and the corresponding global semantic features and local semantic features are extracted for each cell through a convolutional neural network. The final detection result is the entrance corner point information of the vacant parking space obtained by combining the global semantic features and the local semantic features. The entrance corner point information is deduplicated and then output.

[0080] The output of the global semantic feature extraction branch consists of four parts: the first part indicates whether each cell is inside a parking space; the second part indicates the parking space category predicted by the internal parking space cells; the third part indicates the offset between the two corner points of a parking space entrance and the center point of each internal parking space cell; and the fourth part indicates whether the internal parking space cells predict whether the parking space is occupied by another vehicle.

[0081] Among them, the entrance corner point direction and entrance width can be obtained from the offset of the two corner points of a parking space entrance relative to the center point of each cell inside the parking space, which represents the initial entrance corner point information of the target parking space.

[0082] The parking space type and occupied information obtained by the global semantic extraction branch are accurate, but the predicted parking space entrance edge may not be accurate. This is because the cells at different locations in different parking spaces are at different distances from the parking space entrance corner points. To further improve the positioning accuracy of the parking space entrance corner points, we also need to use the local semantic information of the parking space and the prior parking space entrance information to adjust the initial entrance corner point information.

[0083] Step 103: Based on the prior parking space entrance information and the local semantic features included in the target neural network model, the initial entrance corner point information is corrected to obtain target entrance corner point information, where the target entrance corner point information includes a target entrance corner point direction and a target entrance width; the prior parking space entrance information includes a standard entrance corner point direction and a standard entrance width of a standard parking space.

[0084] The output of the local semantic extraction branch consists of three parts: the first part represents whether each cell contains a parking space entrance corner point, the second part represents the offset of a parking space entrance corner point relative to the center point of the cell to which it belongs, and the third part represents the direction of the entrance corner point.

[0085] After fusing local semantic information, the predicted position of the parking space corner point is more accurate than the position obtained using global semantic information. This is because the position is predicted by the cell containing the parking space corner point obtained by local information.

[0086] In order to further reduce the computational complexity of the neural network model and solve the problems of false parking space detection and inaccurate direction caused by changes in the parking space style, after extracting local semantic information, the embodiment of the present invention uses the prior parking space entrance information pre-stored in the target neural network model to constrain the parking space entrance corner point information prediction results of the neural network model.

[0087] Specifically, during the training of the target neural network model, the residual of the predicted parking space entrance information is determined based on the prior parking space entrance information and the actual parking space entrance information, and the model is trained based on this residual. This results in the target neural network model incorporating the prior parking space entrance information when performing parking space prediction tasks. This reduces the search range for prediction results during the calculation process, improves search and prediction efficiency, reduces computational effort, and enhances the model's prediction accuracy.

[0088] Prior parking space entrance information includes the standard entrance corner direction and standard entrance width for standard parking spaces. There are generally three types of standard parking spaces: parallel parking spaces (parallel to the vehicle), perpendicular parking spaces (perpendicular to the vehicle), and diagonal parking spaces. These three types of parking spaces have uniform and fixed dimensions. The entrance corner direction and entrance width of each type of parking space are obtained to obtain the standard entrance corner direction and standard entrance width.

[0089] In a possible implementation, the target entrance width is the Euclidean distance between a left entrance corner point and a right entrance corner point of the target parking space.

[0090] A parking space has four corner points. The two ends of the entrance edge are the left and right entrance corner points. The distance between the left and right entrance corner points is the entrance width. This distance is generally calculated using Euclidean distance.

[0091] Step 104: Mark the target entrance corner direction and the target entrance width in the target parking space to guide the vehicle to park.

[0092] The entrance corner direction and entrance width are marked in the target parking space in the parking lot image so that the vehicle's automatic parking system can use them as a parking position reference to automatically park the vehicle in the target parking space.

[0093] Figure 2 A schematic diagram of a parking space marking image provided by an embodiment of the present invention.

[0094] Reference Figure 2 ,The target neural network model identified two target parking spaces, among which parking space 1 is an oblique parking space and parking space 2 is a horizontal parking space.

[0095] A and B show the two entrance corner points of parking space 1 respectively, direction vector AC is the entrance direction of entrance corner point A, direction vector BD is the entrance direction of entrance corner point B, the line AB between entrance corner point A and entrance corner point B is the entrance edge, and the distance between entrance corner point A and entrance corner point B is the entrance width.

[0096] In summary, the parking space recognition method provided by an embodiment of the present invention obtains a parking lot image of a parking space to be recognized; utilizes a target neural network model to extract global and local semantic features contained in the parking lot image; uses the global semantic features to determine initial entrance corner information of a target parking space; the target parking space is an available parking space; based on prior parking space entrance information included in the target neural network model and the local semantic features, the initial entrance corner information is corrected to obtain target entrance corner information, including a target entrance corner direction and a target entrance width; the prior parking space entrance information includes a standard entrance corner direction and a standard entrance width for a standard parking space; and the target entrance corner direction and the target entrance width are annotated in the target parking space to guide vehicle parking. This solution utilizes prior parking space entrance information as a guide for parking space entrance recognition, reducing the computational complexity of the target neural network model and enabling the model to achieve real-time operation speed. Furthermore, by constraining the prediction results in the form of prior knowledge, the accuracy of parking space entrance recognition can be improved, effectively avoiding problems such as false parking space detection and inaccurate parking direction.

[0097] Figure 3 A flowchart of a target neural network model training method provided by an embodiment of the present invention. The method may include the following steps:

[0098] Step 201: Input the parking lot sample image dataset and the pre-acquired prior parking space entrance information into the initial neural network model.

[0099] The parking lot sample images can be taken in real scenes or downloaded from a network database. The dataset includes annotation information of real parking spaces, such as the actual entrance corner direction and actual entrance width of the real parking spaces.

[0100] The prior parking space entrance information is the entrance corner point information of the standard parking space obtained through a clustering algorithm, including the standard entrance corner point direction vector and the standard entrance width, which is the distance between the left corner point and the right corner point.

[0101] The parking lot sample image dataset and the prior parking space entrance information are input into the initial neural network model as training data.

[0102] The model uses MobilenetV2 as its feature backbone. During downsampling of the deep learning network by 2x, 4x, 8x, 16x, and 32x, the shallow layers of the network learn image features based on texture, color, and boundary information, while the deeper layers learn semantic features. Because parking space entrance information is high-level, the model extracts features using feature maps downsampled to 32x.

[0103] Of course, the feature backbone network can also be replaced accordingly according to the specific application scenario, for example, using VGG16 as the feature backbone network.

[0104] Step 202: Utilize the feature backbone network of the initial neural network model to extract sample global feature information of each cell included in the parking lot sample image; the sample global feature information includes: whether the cell is located inside a sample target parking space, the category of the sample target parking space, sample initial entrance corner point information of the sample target parking space, and whether the target parking space is occupied.

[0105] The embodiment of the present invention can simultaneously obtain the global semantic features and local semantic features of parking spaces through the feature backbone network, and the information of parking spaces in the parking lot image can be obtained by fusing the two information.

[0106] Specifically, the parking lot image is first divided into a grid by rows and columns, and a convolutional neural network is used to extract the corresponding global and local semantic features for each cell.

[0107] The output of the global semantic feature extraction branch consists of four parts: the first part indicates whether each cell is inside a parking space; the second part indicates the parking space category predicted by the internal parking space cells; the third part indicates the offset between the two corner points of a parking space entrance and the center point of each internal parking space cell; and the fourth part indicates whether the internal parking space cells predict whether the parking space is occupied by another vehicle.

[0108] Step 203: Utilize the feature backbone network to extract the sample local feature information of each cell. The sample local feature information is used to correct the sample initial entry corner point information to obtain the predicted entry corner point direction and the predicted entry width.

[0109] The output of the local semantic extraction branch consists of three parts: the first part represents whether each cell contains a parking space entrance corner point, the second part represents the offset of a parking space entrance corner point relative to the center point of the cell to which it belongs, and the third part represents the direction of the entrance corner point.

[0110] The distance between the left entrance corner point and the right entrance corner point can be calculated by the offset of the left entrance corner point relative to the center point of the cell to which it belongs, and thus the entrance width can be obtained.

[0111] In this way, the predicted entrance corner direction and predicted entrance width are obtained through the local semantic extraction branch.

[0112] Step 204 : Match the target number group's prior parking space entrance information with the actual entrance corner point information corresponding to the sample target parking space to obtain a target matching degree.

[0113] Since there are many types of parking spaces, there are multiple sets of prior parking space entrance information, corresponding to parking space entrances of different sizes and directions. The current sample target parking space may only correspond to one type of parking space entrance. Therefore, it is necessary to select the prior parking space entrance information that matches the current sample target parking space.

[0114] Match multiple sets of prior parking space entrance information with the actual entrance information corresponding to the sample target parking space. Specifically, match the entrance corner directions and entrance widths of the prior parking space and the sample target parking space, and finally obtain a target matching degree.

[0115] Step 205 : Selecting, from the target number group of a priori parking space entrance information, a priori parking space entrance information having a target matching degree greater than or equal to a preset matching degree threshold as target a priori parking space entrance information.

[0116] A matching degree threshold is preset, specifically, the matching degree threshold is a percentage, for example, 80%. Priori parking space entrance information greater than or equal to the target matching degree threshold is selected as target priori parking space entrance information.

[0117] Step 206 : Based on the predetermined target a priori parking space entrance information and the actual entrance information of the sample target parking space, respectively determine a first residual corresponding to the predicted entrance corner direction and a second residual corresponding to the predicted entrance width.

[0118] After extracting local semantic features, the embodiment of the present invention uses the prior parking space entrance information pre-stored in the initial neural network model to constrain the parking space entrance corner point information prediction result of the initial neural network model.

[0119] Based on the target parking space entrance information and the actual entrance information of the sample target parking space, the residual loss of the predicted entrance corner information output by the initial neural network model is jointly determined. Specifically, the residual loss includes: a first residual corresponding to the predicted entrance corner direction and a second residual corresponding to the predicted entrance width.

[0120] Step 207: Train the initial neural network model based on the first residual and the second residual to obtain a target neural network model.

[0121] Determine the size relationship between the first residual and the second residual and the preset residual threshold. When the first residual or the second residual is greater than the residual threshold, use the first residual and the second residual to modify the parameters of the model, and then continue to use the data set for training until the first residual and the second residual are both less than the residual threshold, or until the predicted entry corner point information obtained by the final prediction meets the preset accuracy. At this time, the training is completed and the target neural network model is obtained.

[0122] Experiments have shown that the a priori parking space entrance information of the embodiment of the present invention can effectively help the network achieve better accuracy.

[0123] In summary, this embodiment of the present invention integrates prior parking space entrance information into the initial neural network model in the form of prior knowledge. This prior knowledge guides the model to search for suitable parking space entrances, preventing blind searches during training and helping the model converge quickly. The algorithm identifies a length and shape similar to the prior parking space entrance in each cell, thereby constraining the prediction range and improving the model's convergence speed and accuracy.

[0124] In a possible implementation, determining the first residual corresponding to the predicted entry corner point direction includes the following steps 2041 to 2044:

[0125] Step 2041: Obtain the a priori entrance corner point direction and the a priori entrance width included in the target a priori parking space entrance information, and obtain the actual entrance corner point direction and the actual entrance width included in the actual entrance corner point information;

[0126] Step 2042: determine the value of an exponential function with e as the base and the direction of the predicted entry corner point as the exponent to obtain a direction index value;

[0127] Step 2043: Determine the product of the a priori entry corner point direction and the directional index value to obtain a directional product;

[0128] Step 2044: Determine the square of the difference between the direction product and the true entry corner point direction to obtain the first residual.

[0129] In steps 2041 to 2044, the following formula is specifically used to calculate the first residual φ1:

[0130]

[0131] in, is the first residual, bl is the true entrance corner direction, p l is the prior entrance corner direction in the target prior parking space entrance information, and tl is the predicted entrance corner direction output by the neural network model. tl is the direction index value, p l *e tl is the direction product.

[0132] In a possible implementation, determining the second residual corresponding to the predicted entry corner point direction includes the following steps 2045 to 2047:

[0133] Step 2045: Determine the value of an exponential function with base e and the predicted entrance width as an exponent to obtain a distance index value;

[0134] Step 2046: Determine the product of the a priori entrance width and the distance index value to obtain a distance product;

[0135] Step 2047: Determine the square of the difference between the distance product and the actual entrance width to obtain the second residual.

[0136] In steps 2045 to 2047, the second residual φ2 is calculated using the following formula:

[0137]

[0138] in, is the second residual, b θ is the actual entrance width, p θ is the prior entrance width in the target prior parking space entrance information, and tθ is the predicted entrance width output by the neural network model. tθ is the distance index value, p θ *e tθ is the distance product.

[0139] In one possible implementation, Figure 4 As shown, before step 102, the following steps are also included:

[0140] Step 301: Acquire a parking space sample image dataset; the parking space sample image dataset includes various types of parking space images currently existing.

[0141] In the embodiments of the present invention, even though the type, color, and corner pattern of parking spaces can vary significantly in practice, there are generally a limited number of parking space types, such as vertical, horizontal, and diagonal parking spaces. Therefore, the size and direction of parking space entrances are generally fixed. Consider organizing and utilizing this fixed information, such as the size and direction of parking space entrances, as prior parking space entrance information to reduce the computational complexity of the neural network model used for parking space recognition. Using this prior parking space entrance information as a constraint can improve parking space recognition accuracy.

[0142] Clustering algorithms can be used to organize existing parking space images of various types to obtain the size information of various common standard parking space entrances.

[0143] Step 302: Execute a clustering algorithm on the parking space sample image dataset to obtain a target number of clusters, wherein each cluster includes parking spaces of similar types.

[0144] Specifically, the k-means algorithm can be used to cluster the parking space sample images.

[0145] The idea of ​​the k-means algorithm is to divide the samples into K clusters based on the sample differences of the parking space entrances in a specific parking space sample data set, so that similar points are linked together within the cluster, so that the distance between clusters is larger and the distance within the cluster is smaller.

[0146] Calculate the parking space dataset and divide the clusters into the target number of clusters, for example, three clusters (Cl1, Cl2, Cl3), and optimize the objective function E:

[0147]

[0148] where μ i is cluster Cl i The mean vector of , also called the centroid, is expressed as:

[0149]

[0150] In the above example, three clusters of different lengths are pre-set to obtain three sets of prior parking space sizes. Since there are generally three types of parking spaces, the prior parking space sizes of the three clusters can effectively classify the common parking spaces in the dataset.

[0151] Step 303: Obtain the entrance corner direction and entrance width of each parking space in the cluster respectively to obtain the target number of parking space entrance information.

[0152] In this embodiment of the present invention, the size and direction information of the parking space entrance edges in each cluster is obtained, specifically including the direction vector of the parking space entrance corner point and the width of the entrance edge. The target number of sets of prior parking space entrance information can effectively represent several common sets of standard parking space entrance dimensions.

[0153] In summary, the present invention uses statistics and a k-means algorithm to generate prior parking space entrances within the training data, thereby obtaining a priori parking space entrances common to the target group. This prior parking space entrance information is integrated with the neural network using prior knowledge. This prior knowledge guides the model to search for suitable parking space entrances, preventing blind searches during training and helping the model converge quickly.

[0154] Figure 5 This is a block diagram of a parking space identification device provided by an embodiment of the present invention. The device 400 includes:

[0155] An image acquisition module 401 is used to acquire a parking lot image of a parking space to be identified;

[0156] A feature extraction module 402 is configured to extract global semantic features and local semantic features contained in the parking lot image using a target neural network model; the global semantic features are used to determine initial entrance corner information of a target parking space; the target parking space is an available parking space;

[0157] A target entrance corner point information acquisition module 403 is configured to modify the initial entrance corner point information based on the prior parking space entrance information included in the target neural network model and the local semantic features to obtain target entrance corner point information, wherein the target entrance corner point information includes a target entrance corner point direction and a target entrance width; the prior parking space entrance information includes a standard entrance corner point direction and a standard entrance width of a standard parking space;

[0158] The marking module 404 is configured to mark the target entrance corner point direction and the target entrance width in the target parking space to guide the vehicle to park.

[0159] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0160] In another embodiment provided by the present invention, a device is also provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the parking space recognition method described in the embodiment of the present invention.

[0161] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the parking space recognition method described in the embodiment of the present invention.

[0162] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0163] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0164] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For related parts, refer to the description of the method embodiment.

[0165] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A parking space identification method, characterized in that: The method comprises: Obtaining a parking lot image of a parking space to be identified; Extracting global semantic features and local semantic features contained in the parking lot image using a target neural network model; the global semantic features are used to determine initial entrance corner point information of a target parking space; the target parking space is an available parking space; Based on the prior parking space entrance information included in the target neural network model and the local semantic features, the initial entrance corner point information is modified to obtain target entrance corner point information, the target entrance corner point information including a target entrance corner point direction and a target entrance width; the prior parking space entrance information includes a standard entrance corner point direction and a standard entrance width of a standard parking space; Marking the target entrance corner direction and the target entrance width in the target parking space to guide the vehicle to park; The standard parking spaces include a parallel parking space parallel to the vehicle, a perpendicular parking space perpendicular to the vehicle, and an oblique parking space, wherein the parallel parking space, the perpendicular parking space, and the oblique parking space respectively have fixed size standards.

2. The method according to claim 1, characterized in that The target neural network model is trained by the following method: Input the parking lot sample image dataset and the pre-acquired prior parking space entrance information into the initial neural network model; Utilizing the feature backbone network of the initial neural network model, extracting sample global feature information of each cell included in the parking lot sample image; The sample global feature information includes: whether the cell is located inside the sample target parking space, the category of the sample target parking space, the sample initial entrance corner point information of the sample target parking space, and whether the target parking space is occupied; Utilizing the feature backbone network, extracting local feature information of the samples of each cell, wherein the local feature information of the samples is used to correct the initial entry corner point information of the samples to obtain a predicted entry corner point direction and a predicted entry width; Based on the predetermined target priori parking space entrance information and the actual entrance information of the sample target parking space, respectively determining a first residual corresponding to the predicted entrance corner direction and a second residual corresponding to the predicted entrance width; The initial neural network model is trained based on the first residual and the second residual to obtain a target neural network model.

3. The method according to claim 2, characterized in that The determining of the first residual corresponding to the direction of the predicted entry corner point includes: Obtaining the standard entrance corner point direction and the standard entrance width included in the prior parking space entrance information, and obtaining the real entrance corner point direction and the real entrance width included in the real entrance corner point information; Determine the value of an exponential function with e as the base and the direction of the predicted entry corner point as the exponent to obtain a direction index value; Determine the product of the standard entry corner point direction and the directional index value to obtain a directional product; The square of the difference between the direction product and the true entrance corner point direction is determined to obtain the first residual.

4. The method according to claim 3, characterized in that The determining the second residual corresponding to the predicted entry width includes: Determine a value of an exponential function with e as the base and the predicted entrance width as the exponent to obtain a distance index value; determining a product of the standard entrance width and the distance index value to obtain a distance product; The second residual is obtained by determining the square of the difference between the distance product and the true entrance width.

5. The method according to claim 2, characterized in that Before extracting the global semantic features and local semantic features contained in the parking lot image using the target neural network model, the method further includes: Acquire a parking space sample image dataset; the parking space sample image dataset includes various types of parking space images currently existing; executing a clustering algorithm on the parking space sample image dataset to obtain a target number of clusters, wherein each cluster includes parking spaces of similar types; The entrance corner direction and entrance width of each parking space in the cluster are respectively obtained to obtain the target number group of prior parking space entrance information.

6. The method according to claim 5, characterized in that Before respectively determining the first residual corresponding to the predicted entry corner point direction and the second residual corresponding to the predicted entry width, the method further includes: Matching the target number group's prior parking space entrance information with the real entrance corner point information corresponding to the sample target parking space to obtain a target matching degree; From the target number group of a priori parking space entrance information, the a priori parking space entrance information with the target matching degree greater than or equal to a preset matching degree threshold is selected as the target a priori parking space entrance information.

7. The method according to claim 1, characterized in that The target entrance width is the Euclidean distance between the left entrance corner point and the right entrance corner point of the target parking space.

8. A parking space recognition device, characterized in that: The device comprises: An image acquisition module, used to acquire a parking lot image of a parking space to be identified; a feature extraction module for extracting global semantic features and local semantic features contained in the parking lot image using a target neural network model; the global semantic features are used to determine initial entrance corner point information of a target parking space; the target parking space is an available parking space; a target entrance corner point information acquisition module, configured to modify the initial entrance corner point information based on the prior parking space entrance information included in the target neural network model and the local semantic features to obtain target entrance corner point information, wherein the target entrance corner point information includes a target entrance corner point direction and a target entrance width; the prior parking space entrance information includes a standard entrance corner point direction and a standard entrance width of a standard parking space; a marking module, configured to mark the target entrance corner direction and the target entrance width in the target parking space to guide the vehicle to park; The standard parking spaces include a parallel parking space parallel to the vehicle, a perpendicular parking space perpendicular to the vehicle, and an oblique parking space, wherein the parallel parking space, the perpendicular parking space, and the oblique parking space respectively have fixed size standards.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the parking space recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the parking space recognition method according to any one of claims 1 to 7.

Images