Mechanical parking space detection method and device, electronic equipment, medium and product

Abstract

The invention provides a mechanical parking space detection method and device, electronic equipment, a medium and a product. According to one embodiment of the invention, the method comprises the following steps: detecting multiple frames of parking space images, and obtaining point and line information in the multiple frames of parking space images; wherein the multi-frame parking space image is obtained by shooting a mechanical parking space through a monocular camera, the point-line information comprises an entrance corner point of the mechanical parking space and a parking space inner line, and the direction of the parking space inner line is the direction from the entrance of the parking space to the end of the parking space; matching the entrance corner points with the inner line of the parking space to obtain point-line pairs of the mechanical parking space; according to the method, the point-line pair is subjected to rectangularization processing, the parking space line of the mechanical parking space is obtained, the parking space line of the mechanical parking space can be accurately recognized through multi-frame image processing and point-line matching, and therefore the parking space detection precision is improved.

Description

Technical Field

[0001] The present application relates to the field of assisted driving technology, and in particular to a method, device, electronic equipment, medium and product for detecting a mechanical parking space. Background Art

[0002] With the acceleration of urbanization, land resources are becoming increasingly scarce. Mechanical parking spaces, as a parking method that efficiently utilizes space resources, have been widely used in various parking lots. However, the design of mechanical parking spaces is often compact and complex in structure. The intervals between parking spaces are narrow and often accompanied by obstructions such as columns and beams.

[0003] Monocular cameras are widely used in parking space detection due to their relatively low cost and easy installation. However, the field of view of monocular cameras is limited, and they can only capture image information within a certain range of the direction the camera is facing. There are various obstructions around mechanical parking spaces, which makes it impossible to obtain complete parking space information, resulting in a decrease in parking space detection accuracy. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present application provides a detection method, device, electronic equipment, medium and product for a mechanical parking space.

[0005] According to a first aspect of any embodiment of the present application, a method for detecting a mechanical parking space is provided, the method comprising: Detecting multiple frames of parking space images to obtain point and line information in the multiple frames of parking space images; wherein the multiple frames of parking space images are obtained by photographing the mechanical parking space with a monocular camera, and the point and line information includes: an entrance corner point of the mechanical parking space and an inner line of the parking space, wherein the direction of the inner line of the parking space is the direction from the parking space entrance to the end of the parking space; Matching the entrance corner point with the inner edge of the parking space to obtain a point-line pair of the mechanical parking space; The point-line pairs are rectangularized to obtain the parking space line of the mechanical parking space.

[0006] According to a second aspect of any embodiment of the present application, a detection device for a mechanical parking space is provided, the device comprising: A detection module is used to detect multiple frames of parking space images to obtain point and line information in the multiple frames of parking space images; wherein the multiple frames of parking space images are obtained by shooting the mechanical parking space with a monocular camera, and the point and line information includes: an entrance corner point of the mechanical parking space and an inner line of the parking space, wherein the direction of the inner line of the parking space is the direction from the parking space entrance to the end of the parking space; A matching module, used for matching the entrance corner point with the inner edge of the parking space to obtain a point-line pair of the mechanical parking space; The processing module is used to perform rectangular processing on the point-line pair to obtain the parking space line of the mechanical parking space.

[0007] According to a third aspect of any embodiment of the present application, there is provided an electronic device, including: processor; a memory for storing processor-executable instructions; The processor implements the method described in any embodiment of the present application by running the executable instructions.

[0008] According to a fourth aspect of any embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the method described in any embodiment of the present application is implemented.

[0009] According to a fifth aspect of any embodiment of the present application, a computer program product is provided, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the method described in any embodiment of the present application is implemented.

[0010] The technical solution provided by this application may have the following beneficial effects: According to the above embodiments, by detecting multiple frames of parking space images, point and line information in the multiple frames of parking space images is obtained, the entrance corner points in the point and line information are matched with the inner edges of the parking space to obtain the point and line pairs of the mechanical parking space, and the point and line pairs are rectangularized to obtain the parking space lines of the mechanical parking space. Through multi-frame image processing and point and line matching, the parking space lines of the mechanical parking space can be accurately identified, thereby improving the accuracy of parking space detection.

[0011] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in the specification and constitute a part of this application, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0013] Figure 1 is a flow chart of a method for detecting a mechanical parking space according to an exemplary embodiment of the present application; Figure 2 is a schematic diagram showing point and line information in a multi-frame parking space image according to an exemplary embodiment of the present application; Figure 3 This is a flow chart of detecting a line inside a parking space according to an exemplary embodiment of the present application; Figure 4 is a schematic diagram of a maximum viewing angle difference according to an exemplary embodiment of the present application; Figure 5 is a schematic diagram of a matching point line pair according to an exemplary embodiment of the present application; Figure 6 is a schematic diagram of searching for matching point line pairs according to an exemplary embodiment of the present application; Figure 7 is a schematic diagram of a rectangular processing of a point-line pair according to an exemplary embodiment of the present application; Figure 8 is a flow chart of another method for detecting a mechanical parking space according to an exemplary embodiment of the present application; Fig. 9 is a structural schematic diagram of an electronic device according to an exemplary embodiment of the present application; Fig.10 It is a block diagram of a detection device for a mechanical parking space shown in the present application according to an exemplary embodiment. DETAILED DESCRIPTION

[0014] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0015] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0016] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0017] At present, due to various obstructions around mechanical parking spaces, the field of view of monocular cameras is limited by their installation position and angle, and they are often unable to obtain complete parking space information. This affects the accuracy of parking space detection and is prone to misjudgment or missed judgment.

[0018] In order to solve the above problems, the present application proposes a method for detecting a mechanical parking space. To further illustrate the present application, the following embodiments are provided: See also Figure 1 , Figure 1 This is a flow chart of a method for detecting a mechanical parking space according to an exemplary embodiment of the present application. The method for detecting a mechanical parking space can be executed by a parking space detection system, which can be applied to a vehicle or to a cloud service end.

[0019] The detection method of the mechanical parking space may include the following steps: Step 102: Detect multiple frames of parking space images to obtain point and line information in the multiple frames of parking space images; wherein the multiple frames of parking space images are obtained by photographing the mechanical parking space with a monocular camera, and the point and line information includes: the entrance corner point of the mechanical parking space and the inner edge line of the parking space, and the direction of the inner edge line of the parking space is the direction from the parking space entrance to the end of the parking space.

[0020] In this step, the parking space detection system can shoot the mechanical parking space through a monocular camera to obtain multiple frames of parking space images. The parking space detection system can use a deep learning model to perform fusion detection on multiple frames of parking space images, detect points and lines in the multiple frames of parking space images, remove duplicate points and lines, and obtain point and line information in the multiple frames of parking space images.

[0021] The parking space detection system can also use image processing technologies such as edge detection, corner detection, and straight line detection to identify point and line information in multiple frames of parking space images.

[0022] When detecting multiple frames of parking space images, the parking space detection system can also detect the relative position of point and line information in the mechanical parking space by analyzing the spatial position, direction and relative relationship between the points and lines. For example, it can determine whether the entrance corner point and the inner edge of the parking space belong to the left or right side of the mechanical parking space.

[0023] Among them, the multi-frame parking space image is an image sequence obtained by taking multiple shots of the mechanical parking space by a monocular camera at different time points or different viewing angles, and is used to obtain parking space information such as the position, size and shape of the mechanical parking space.

[0024] A monocular camera is a camera system that captures image information through only one lens, captures light through an image sensor and generates a two-dimensional image. A monocular camera can be a monocular camera, a pinhole camera, etc.

[0025] Point and line information refers to key points and line information extracted from multiple frames of parking space images. Point and line information can include: entrance corner points and inner edges of mechanical parking spaces. Entrance corner points are feature points at the entrance of parking spaces, and entrance corner points are used to locate the entrance position of mechanical parking spaces.

[0026] The inner edge of the parking space is a line from the entrance to the end of the parking space. The inner edge of the parking space is used to indicate the length and direction of the mechanical parking space. The direction of the inner edge of the parking space is from the entrance to the end of the parking space. The inner edge of the parking space can include: the left inner edge and the right inner edge of the mechanical parking space.

[0027] By fusing and detecting the point and line information in multiple frames of parking space images, the inner edge line information on both sides of the mechanical parking space can be obtained, avoiding the inability to simultaneously see the inner edges of both sides of the parking space due to occlusion, thereby improving the accuracy of parking space detection.

[0028] The method for deduplicating points and lines in multiple frames of parking space images may be, for example, based on the Euclidean distance or Manhattan distance between points. When the distance between two points is less than a threshold, the two points are determined to be duplicated and one of the points is retained; based on the angle, slope or position relationship between the two lines, if the two lines have similar slopes and overlap in position, the two lines are determined to be duplicated and one of the lines is retained.

[0029] For another example, feature descriptors of the detected points and lines may be extracted, such as feature point descriptors such as Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), and feature descriptors based on line segments. A feature matching algorithm is used to match points and lines in different frames. If the feature descriptors of two points or lines are very similar, it is determined that the two points or lines are repeated.

[0030] See also Figure 2 , Figure 2 A schematic diagram of point and line information in a multi-frame parking space image is shown. Exemplarily, the parking space detection system can obtain multi-frame parking space images by photographing the mechanical parking space with a monocular camera, and detect the left inner edge line 21 (blue line) and the right inner edge line 22 (yellow line) in the multi-frame parking space images, as well as the left entrance corner point 23 (blue dot) and the right entrance corner point 24 (yellow dot).

[0031] Step 104: Match the entrance corner points and the inner edges of the parking spaces to obtain point-line pairs of the mechanical parking spaces.

[0032] In this step, the parking space detection system can extract point feature information such as the position coordinates and direction of the entrance corner point, and extract line feature information such as the slope, line segment start point, and line segment end point of the line inside the parking space. Based on the point feature information and line feature information, the entrance corner point and the line inside the parking space are matched to obtain multiple point-line pairs of the mechanical parking space.

[0033] For example, the parking space detection system can calculate the distance from the entrance corner point to each point along the parking space, and select the point with the smallest distance as the matching point. It can also determine the spatial relationship between the entrance corner point and the parking space inner line. The entrance corner point should be located at or near the starting end of the parking space inner line, and the parking space inner line should extend from the entrance corner point to the end of the parking space.

[0034] Among them, the point-line pair is an information unit that can reflect the basic shape and position of the mechanical parking space, and is used to reflect an entrance and extension direction of the mechanical parking space. The point-line pair can be the left point-line pair of the mechanical parking space or the right point-line pair of the mechanical parking space.

[0035] The left point-line pair is a point-line pair consisting of the left inner edge line and the left entrance corner point of the mechanical parking space, and the right point-line pair is a point-line pair consisting of the right inner edge line and the right entrance corner point of the mechanical parking space.

[0036] Please continue reading Figure 2 The parking space detection system can match the entrance corner points and the inner edge lines of the parking space in multiple frames of parking space images to obtain the left point line pair and the right point line pair of the mechanical parking space. The left point line pair includes the left inner edge line 21 and the left entrance corner point 23, and the right point line pair includes the right inner edge line 22 and the right entrance corner point 24.

[0037] Step 106: rectangularize the point-line pairs to obtain the parking space line of the mechanical parking space.

[0038] In this step, the parking space detection system performs rectangular processing on the multiple matched point-line pairs to obtain the parking space lines of the mechanical parking space.

[0039] For example, the left point line pair and the right point line pair of the same mechanical parking space are rectangularized, and the entrance corner points and the inner edge lines of the left point line pair and the right point line are converted into standard rectangular shapes to more accurately describe and locate the parking space and obtain the parking space line of the mechanical parking space.

[0040] The detection method of the mechanical parking space in the present embodiment detects multiple frames of parking space images to obtain point and line information in the multiple frames of parking space images, matches the entrance corner points in the point and line information with the inner edges of the parking space to obtain the point and line pairs of the mechanical parking space, and rectangularizes the point and line pairs to obtain the parking space lines of the mechanical parking space. Through multi-frame image processing and point and line matching, the parking space lines of the mechanical parking space can be accurately identified, thereby improving the accuracy of parking space detection.

[0041] In the above-mentioned embodiment, it is introduced that the parking space line of the mechanical parking space is accurately detected by performing point and line detection, matching and rectangularization processing on multiple frames of parking space images. In the following embodiment, the detection process of point and line information will be described in more detail and can be applied to any of the above embodiments.

[0042] In one embodiment, Figure 3 FIG. 4 shows a flow chart of detecting the inner edge of a parking space. Figure 3 As shown, the parking space detection system can input each frame of parking space image into the point and line detection model, and use the backbone network (Backbone) such as ResNet and MobileNet in the point and line detection model to perform convolution and pooling operations on the parking space image to extract the inner edge line features in multiple frames of parking space images.

[0043] One or more convolutional layers in the point and line detection model are used to generate a first heat map corresponding to the inner edge feature, and each pixel value on the first heat map represents the probability that the corresponding position is the inner edge of the parking space.

[0044] In the first heat map, the first candidate points along the line in the parking space are selected, and the pixel points in the first heat map that are higher than the first preset threshold are used as the first candidate points. The number of the first candidate points can also be further reduced by non-maximum suppression and other methods to eliminate redundant and overlapping points in the first heat map.

[0045] The parking space detection system can also use the point-line detection model to generate a first offset corresponding to the inner edge feature, and add the first offset to the original predicted position of the first candidate point in the first heat map to obtain an adjusted first candidate point.

[0046] Clustering algorithms such as K-means and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) are used to group the first candidate points, and the first candidate points are grouped into a point set along the line of the mechanical parking space.

[0047] The point set along the line in the parking space is fitted using fitting algorithms such as polynomial fitting and least squares fitting to obtain the two-dimensional line along the line in the parking space output by the point line detection model.

[0048] The inner edge feature is the feature information of the inner edge of the parking space. The first candidate point is the pixel point that constitutes the inner edge of the parking space in the first heat map. The point line detection model is a time series model based on deep learning.

[0049] The first heat map is a heat map that intuitively represents the first candidate point along the inner edge of the parking space, which is used to reflect the probability that each pixel in the parking space image belongs to the inner edge of the parking space. The first offset is the offset of the predicted position of the first candidate point relative to the actual position, which is used to adjust the predicted position of the first candidate point to the actual inner edge position.

[0050] As described above, by extracting the inner line features in multiple frames of parking space images, generating the first heat map and the first offset corresponding to the inner line features, and determining the first candidate point of the inner line of the parking space based on the first heat map and the first offset, the position of the inner line can be refined, and the first candidate point can be clustered and fitted to obtain the inner line of the parking space, which can effectively reduce the false detection and missed detection of the inner line and improve the reliability of parking space detection.

[0051] In one embodiment, please continue to refer to Figure 3 ,The parking space detection system utilizes the backbone network in the point and line ,detection model, performs convolution and pooling operations on the parking space image and ,extracts point features in multiple frames of parking space images.

[0052] One or more convolutional layers in the point and line detection model can be used to generate a second heat map corresponding to the point features, and each pixel value on the second heat map represents the probability that the corresponding position is an entry corner point.

[0053] In the second heat map, a second candidate point of the entry corner point is selected, and the pixel points in the second heat map that are higher than the second preset threshold are used as the second candidate points.

[0054] The parking space detection system can also use the point-line detection model to generate a second offset corresponding to the point feature, and add the second offset to the original predicted position of the second candidate point in the second heat map to obtain an adjusted second candidate point.

[0055] The non-maximum suppression method is used to delete duplicate points in the second candidate points, and the second candidate points are used as entry corner points to obtain the two-dimensional entry corner points output by the point and line detection model.

[0056] The point feature is the feature information of the entry corner point. The second candidate point is the pixel point that constitutes the entry corner point in the second thermal map.

[0057] The second heat map is a heat map of the second candidate point of the entrance corner point, which is used to reflect the probability that each pixel point in the parking space image belongs to the entrance corner point. The second offset is the offset of the predicted position of the second candidate point relative to the actual position, which is used to adjust the predicted position of the second candidate point to the actual entrance corner point position.

[0058] As described above, by extracting point features from multiple frames of parking space images and generating a second heat map and a second offset corresponding to the point features, the positioning error caused by factors such as image noise and distortion can be reduced; based on the second heat map and the second offset, the second candidate points of the entry corner points are determined, the duplicate points in the second candidate points are deleted, and the second candidate points are used as the entry corner points, which can avoid repeated calculations or misjudgments in subsequent processing, thereby improving the accuracy of parking space detection.

[0059] In one embodiment, the parking space detection system can perform distortion correction on the detected two-dimensional inner edge line and entrance corner points of the parking space, and map the inner edge line and entrance corner points of the parking space in the distorted parking space image to the undistorted image through geometric transformation to obtain the corrected inner edge line and entrance corner points of the parking space.

[0060] According to the intrinsic parameter matrix of the monocular camera, the inverse matrix of the intrinsic parameter matrix is ​​calculated through the matrix inversion algorithm. The inverse matrix of the intrinsic parameter matrix is ​​used to transform the inner edge and entrance corner points of the parking space from the image coordinate system to the camera coordinate system. The image coordinates of the inner edge and entrance corner points of the parking space are projected onto the normalized plane to obtain the normalized inner edge and entrance corner points of the parking space that are not affected by the camera intrinsic parameters and distortion.

[0061] As mentioned above, by performing distortion correction on the detected inner edge lines and entrance corner points of the parking space, the influence of distortion on the inner edge lines and entrance corner points of the parking space can be reduced. According to the inverse matrix of the intrinsic parameter matrix of the monocular camera, the image coordinates of the inner edge lines and entrance corner points of the parking space are projected onto the normalized plane to obtain the normalized inner edge lines and entrance corner points of the parking space, thereby reducing the computational complexity and error sources, making the subsequent detection of the parking space lines easier and more accurate.

[0062] In one embodiment, before matching the entrance corner point with the inner line of the parking space, the parking space detection system can obtain the position, orientation and other posture information of the vehicle corresponding to the parking space image based on the timestamp of the parking space image through the vehicle positioning system, map matching algorithm and other methods.

[0063] According to the posture information of the vehicle and the external parameters of the monocular camera, the point and line triangulation algorithm is used to convert the two-dimensional points or lines in the camera's perspective into three-dimensional positions or directions in the world coordinate system to determine the three-dimensional position of the point and line information.

[0064] The parking space detection system can use a point triangulation algorithm to determine the three-dimensional position of the entrance corner point. For example, by minimizing the projection error, the two-dimensional entrance corner points observed from multiple frames of parking space images are matched to the same three-dimensional point to determine the three-dimensional position of the entrance corner point.

[0065] The parking space detection system can use a line triangulation algorithm to determine the three-dimensional position of the line inside the parking space. For example, the points on the line inside the parking space can be triangulated first, and then a three-dimensional straight line can be fitted by a least square method or the like. Alternatively, the geometric constraints of the linear features in the parking space image can be directly used to estimate the three-dimensional direction or position of the line inside the parking space.

[0066] Since the vehicle and the camera are constantly moving during driving, the observation angle of the same point and line information will change. The parking space detection system analyzes the observation angle of multiple frames of parking space images based on the three-dimensional position of the entrance corner point and the inner line of the parking space, and determines the maximum difference in the observation angle between the entrance corner point and the inner line of the parking space in any two frames.

[0067] When the maximum viewing angle difference is greater than the preset threshold, it means that the point and line information has obvious position changes between the parking space images in different frames, which is more likely caused by real movement or structural changes. The point and line information can be retained, and the point and line information maintains relatively stable and significant features under different viewing angles.

[0068] When the maximum viewing angle difference is less than or equal to a preset threshold, the point and line information is deleted, and the point and line information in the multiple frames of parking space images is not used for point and line matching and rectangularization processing.

[0069] Among them, the external parameters of the monocular camera refer to the position and posture of the monocular camera in the world coordinate system, which are used to define the transformation relationship between the camera coordinate system and the world coordinate system. The external parameters can be fixed or obtained through camera calibration.

[0070] The maximum viewing angle difference is the maximum angle difference formed by the observation viewing angles of multiple frames of parking space images. The preset threshold is used to distinguish whether the viewing angle change is large enough to affect the accuracy of parking space detection.

[0071] See also Figure 4 , Figure 4 A schematic diagram of a maximum viewing angle difference is shown. Exemplarily, the observation frames of the multiple parking space images include observation frame A and observation frame B, and the viewing angle difference formed by observation frame A and observation frame B for the left entrance corner point 23 is the maximum viewing angle difference.

[0072] The parking space detection system compares the maximum viewing angle difference with a preset threshold. If the maximum viewing angle difference is greater than the preset threshold, the point and line information in the multiple frames of parking space images is used for subsequent point and line matching and rectangularization processing.

[0073] As described above, by determining the three-dimensional position of the point and line information based on the posture information of the vehicle and the external parameters of the monocular camera, and determining the maximum viewing angle difference of the point and line information based on the three-dimensional position, and retaining the point and line information when the maximum viewing angle difference is greater than a preset threshold, the visibility and consistency of the point and line information under different viewing angles can be more accurately evaluated, thereby improving the accuracy of subsequent point and line matching.

[0074] In the above-mentioned embodiment, it is introduced that the accuracy of detecting the parking space line of the mechanical parking space is further improved by using the heat map and offset corresponding to the point and line features, and verifying the maximum viewing angle difference of the point and line information. In the following embodiment, the point and line matching and rectangularization processing process will be described in more detail and can be applied to any of the above embodiments.

[0075] In one embodiment, the entrance corner point may include: a first entrance corner point and a second entrance corner point. The parking space inner edge line may include: a first inner edge line and a second inner edge line, the first inner edge line is the parking space inner edge line closest to the first entrance corner point, and the second inner edge line is the parking space inner edge line closest to the second entrance corner point.

[0076] The parking space detection system can obtain the inner edge of the parking space closest to each entrance corner point. The distance value between the entrance corner point and the inner edge of the parking space closest to the entrance corner point is detected, and the distance value is compared with the distance threshold to obtain a matching point-line pair.

[0077] When the first distance value between the first entry corner point and the first inner line is less than the distance threshold, the first entry corner point and the first inner line are combined into a point-line pair. When the second distance value between the second entry corner point and the second inner line is less than the distance threshold, the second entry corner point and the second inner line are combined into a point-line pair.

[0078] The first distance value is the distance value between the first entry corner point and the first inner edge line. The first distance value may include at least one of the following: a horizontal distance value, a vertical distance value, and a distance value between the first entry corner point and the nearest endpoint of the first inner edge line.

[0079] The second distance value is the distance value between the second entry corner point and the second inner edge line. The second distance value may include at least one of the following: a horizontal distance value, a vertical distance value, and a distance value between the second entry corner point and the nearest endpoint of the second inner edge line.

[0080] The first inner edge line is a line inside the mechanical parking space close to one side boundary. The direction of the first inner edge line is from the parking space entrance to the end of the parking space. The first inner edge line may be the left inner edge line of the mechanical parking space.

[0081] The second inner edge line is a line inside the mechanical parking space close to the other side boundary. The direction of the second inner edge line is from the parking space entrance to the end of the parking space. The second inner edge line can be the right inner edge line of the mechanical parking space.

[0082] The first inner edge line and the second inner edge line are relatively parallel or approximately parallel to each other, and jointly define the inner space of the mechanical parking space.

[0083] The first entrance corner point is a corner point at the entrance of the mechanical parking space, marking the beginning of one side boundary of the parking space. The first entrance corner point may be the left entrance corner point of the mechanical parking space. The second entrance corner point corresponds to the first entrance corner point, and the second entrance corner point is another corner point at the entrance of the mechanical parking space, marking the beginning of the other side boundary of the parking space. The second entrance corner point may be the right entrance corner point of the mechanical parking space.

[0084] See also Figure 5 , Figure 5 A schematic diagram of a matching point line pair is shown. Exemplarily, the first entry corner point is the left entry corner point 23, and the second entry corner point is the right entry corner point 24. The first inner edge line closest to the left entry corner point 23 is the left inner edge line 21, and the second inner edge line closest to the right entry corner point 24 is the right inner edge line 22.

[0085] When the horizontal distance value and the vertical distance value between the left entrance corner point 23 and the left inner line 21 are both less than the distance threshold, the left entrance corner point 23 and the left inner line 21 form a left point line pair. When the horizontal distance value and the vertical distance value between the right entrance corner point 24 and the right inner line 22 are both less than the distance threshold, the right entrance corner point 24 and the right inner line 22 form a right point line pair.

[0086] As described above, when the first distance value between the first entrance corner point and the first inner edge line is less than the distance threshold, the first entrance corner point and the first inner edge line that is closest to it are formed into a left point line pair; when the second distance value between the second entrance corner point and the second inner edge line is less than the distance threshold, the second entrance corner point and the second inner edge line that is closest to it are formed into a right point line pair. This allows the corner points to be accurately paired with the corresponding inner edges, filters out some erroneous pairings caused by errors, and more accurately determines the boundaries of mechanical parking spaces, thereby improving the accuracy of parking space detection.

[0087] In one embodiment, the point-line pairs may include: a plurality of first point-line pairs and a plurality of second point-line pairs. The first point-line pairs may be point-line pairs consisting of a first entry corner point and a first inner edge line, i.e., a left point-line pair. The second point-line pairs may be point-line pairs consisting of a second entry corner point and a second inner edge line, i.e., a right point-line pair.

[0088] The parking space detection system can match multiple first point-line pairs and multiple second point-line pairs, and combine the matched first point-line pairs and second point-line pairs into matched point-line pairs of the same mechanical parking space.

[0089] See also Figure 6 , Figure 6A schematic diagram of searching for matching point-line pairs is shown. For each first point-line pair, the closest second point-line pair is searched to the right, and for each second point-line pair, the closest first point-line pair is searched to the left. The closest first point-line pair and the second point-line pair are taken as matching point-line pairs.

[0090] The parking space detection system can also check whether the distance value between the nearest first point line pair and the second point line pair exceeds the preset distance value. If the distance value between the nearest first point line pair and the second point line pair exceeds the preset distance value, the nearest first point line pair and the second point line pair are used as matching point line pairs.

[0091] The parking space detection system can perform rectangular processing on the first point-line pair and the second point-line pair in each matching point-line pair to obtain the parking space lines of the mechanical parking spaces corresponding to each matching point-line pair.

[0092] The matching point-line pair is composed of a first point-line pair and a second point-line pair of the same mechanical parking space, and is used to identify the position and boundary information of the mechanical parking space.

[0093] As described above, by searching for the closest second point line pair to the right for each first point line pair, and searching for the closest first point line pair to the left for each second point line pair, and using the closest first point line pair and second point line pair as matching point line pairs, it is possible to reduce mismatches due to factors such as noise, occlusion, or changes in viewing angle; rectangularizing the first point line pair and the second point line pair in each matching point line pair, respectively, to obtain the parking space lines corresponding to the matching point line pairs, thereby ensuring that each mechanical parking space is detected completely and accurately.

[0094] In one embodiment, see Figure 7 , Figure 7 A schematic diagram of a rectangular processing of a point-line pair is shown. The parking space detection system can obtain the average value of the angle between the first point-line pair and the second point-line pair constituting the same mechanical parking space, and use the direction of the average angle as the parking space direction of the mechanical parking space.

[0095] Use the parking space direction to replace the direction of the first point line pair and the second point line pair. Take the entrance corner point of the first point line pair as the reference point, rotate the first point line pair to obtain the left boundary line parallel to the parking space direction. Take the entrance corner point of the second point line pair as the reference point, rotate the second point line pair to obtain the right boundary line parallel to the parking space direction.

[0096] The first entrance corner point in the first point line pair is projected to the parking space direction to obtain a left projection point 71. The second entrance corner point in the second point line pair is projected to the parking space direction to obtain a right projection point 72. The midpoint between the left projection point and the right projection point is taken as the projection midpoint 73.

[0097] Based on the projection midpoint, the left boundary line and the right boundary line, the front boundary line is constructed. The nearest point from the projection midpoint 73 to the left boundary line is used as the final left entrance corner point, the nearest point from the projection midpoint to the right boundary line is used as the final right entrance corner point, and the line segment between the final left entrance corner point and the right entrance corner point is used as the front boundary line.

[0098] The angle mean is the mean of the angles formed by the first point-line pair and the second point-line pair in the point-line pair. The projection midpoint is the midpoint between the left projection point and the right projection point. The parking space line may include: the front boundary line, the left boundary line and the right boundary line of the mechanical parking space.

[0099] As described above, by taking the direction of the average value of the angle formed by the first point line pair and the second point line pair in the point line pair as the parking direction of the mechanical parking space, the first point line pair and the second point line pair are rotated respectively to obtain the left boundary line and the right boundary line parallel to the parking space direction, thereby reducing the recognition error caused by the angle deviation; the first entrance corner point and the second entrance corner point are projected to the parking space direction respectively to obtain the left projection point and the right projection point, and the front boundary line is constructed based on the projection midpoint between the left projection point and the right projection point, the left boundary line and the right boundary line, so as to provide an accurate reference point for constructing the front boundary line, thereby improving the detection accuracy of the mechanical parking space.

[0100] In one embodiment, when the vehicle enters a mechanical parking space, the parking space detection system may utilize deep learning models, image processing, and other methods to continuously detect the end corner points and wheel chock positions of the mechanical parking space in the latest multi-frame parking space images.

[0101] Based on the dead end corner point and the wheel block position, a rear boundary line is determined. For example, the rear boundary line is constructed by connecting the dead end corner point and the wheel block position using a linear interpolation or curve fitting method.

[0102] The end corner point is the intersection of the inner line of the parking space and the rear boundary line of the mechanical parking space, which is used as the end point of the mechanical parking space in the length direction. The end corner point may include: the left end corner point and the right end corner point of the mechanical parking space. The wheel block position is the specific position of the wheel block in the parking space image.

[0103] As described above, by detecting the end corner points in multiple frames of parking space images, the rear boundary line of the mechanical parking space is determined based on the end corner points and the wheel block position of the mechanical parking space. The rear boundary is marked with the end corner points, and combined with the wheel block position, the rear boundary line of the mechanical parking space can be determined more accurately.

[0104] In order to further introduce the detection method of mechanical parking spaces, Figure 8 A flow chart of another method for detecting a mechanical parking space is shown. The method for detecting a mechanical parking space may include the following steps: Step 802: Based on the inner edge line features in the multiple frames of parking space images, generate a first heat map and a first offset corresponding to the inner edge line features.

[0105] In this step, the parking space detection system inputs each frame of parking space image into the point and line detection model, uses the point and line detection model to fuse multiple frames of parking space images for high-precision inner line detection, extracts the inner line features in the multiple frames of parking space images, and generates the first heat map and first offset corresponding to the inner line features.

[0106] Step 804: clustering and fitting the first candidate points along the inner line of the parking space to obtain the inner line of the parking space.

[0107] In this step, the parking space detection system selects a first candidate point along the inner line of the parking space in the first heat map, adjusts the first candidate point using a first offset, clusters and fits the adjusted first candidate point, and obtains the inner line of the parking space.

[0108] Step 806: Based on the point features in the multiple frames of parking space images, generate a second heat map and a second offset corresponding to the point features.

[0109] In this step, the parking space detection system uses the point-line detection model to perform high-precision corner point detection on multiple frames of parking space images, extracts point features in the multiple frames of parking space images, and generates a second heat map and a second offset corresponding to the point features.

[0110] Step 808: Delete duplicate points in the second candidate point of the entry corner point, and use the second candidate point as the entry corner point.

[0111] In this step, the parking space detection system selects a second candidate point for the entrance corner point in the second heat map, adjusts the second candidate point using a second offset, deletes duplicate points in the adjusted second candidate point, and uses the second candidate point as the entrance corner point.

[0112] Step 810: When the maximum viewing angle difference is greater than a preset threshold, retain the entrance corner point and the inner edge of the parking space.

[0113] In this step, the parking space detection system determines the three-dimensional position of the point and line information according to the position information of the vehicle and the external parameters of the monocular camera. Based on the three-dimensional position of the point and line information, the maximum viewing angle difference formed by the observation viewing angles of the multiple frames of parking space images is determined.

[0114] The point and line information is filtered based on the maximum viewing angle difference. When the maximum viewing angle difference is greater than a preset threshold, the entrance corner points and the inner edges of the parking space in multiple frames of parking space images are retained.

[0115] Step 812: When the first distance value is less than the distance threshold, the first entry corner point and the first inner edge line are combined into a point-line pair.

[0116] In this step, when the first distance value between the first entrance corner point and the first inner edge line is less than the distance threshold, the parking space detection system forms a point-line pair with the first entrance corner point and the first inner edge line.

[0117] Step 814: When the second distance value is less than the distance threshold, the second entry corner point and the second inner edge line are combined into a point-line pair.

[0118] In this step, when the second distance value between the second entrance corner point and the second inner edge line is less than the distance threshold, the parking space detection system forms a point-line pair with the second entrance corner point and the second inner edge line.

[0119] Step 816: The first point-line pair and the second point-line pair with the closest distance are used as matching point-line pairs.

[0120] In this step, the parking space detection system searches for the closest second point-line pair to the right for each first point-line pair, and searches for the closest first point-line pair to the left for each second point-line pair. When the distance between the closest first point-line pair and the second point-line pair exceeds the preset distance value, the closest first point-line pair and the second point-line pair are used as matching point-line pairs.

[0121] Step 818: rectangularize the first point-line pair and the second point-line pair in the matching point-line pairs to obtain parking space lines corresponding to the matching point-line pairs.

[0122] In this step, the parking space detection system uses the direction of the average of the angles between the first point line pair and the second point line pair in the matching point line pair as the parking space direction of the mechanical parking space. The first point line pair is rotated to obtain a left boundary line parallel to the parking space direction. The second point line pair is rotated to obtain a right boundary line parallel to the parking space direction.

[0123] Project the first entrance corner point in the first point line pair to the parking space direction to obtain the left projection point, and project the second entrance corner point in the second point line pair to the parking space direction to obtain the right projection point.

[0124] The nearest point from the projection midpoint to the left boundary line is taken as the final left entrance corner point, the nearest point from the projection midpoint to the right boundary line is taken as the final right entrance corner point, and the line segment between the final left entrance corner point and the right entrance corner point is taken as the front boundary line.

[0125] Step 820: Determine the rear boundary line based on the end corner point and the wheel block position.

[0126] In this step, when the vehicle enters the mechanical parking space, the parking space detection system continuously detects the end corner points and the wheel block positions of the mechanical parking space in the latest multi-frame parking space images, and determines the rear boundary line based on the end corner points and the wheel block positions.

[0127] Fig. 9 1 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present application. The electronic device may be, for example, a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a personal digital assistant, a server, a smart home appliance, a car computer, etc. Fig. 9 At the hardware level, the electronic device includes a processor 902, an internal bus 904, a network interface 906, a memory 908, and a non-volatile memory 910, and may also include hardware required for other services. The processor 902 reads the corresponding computer program from the non-volatile memory 910 into the memory 908 and then runs it, forming a detection device for a mechanical parking space at the logical level. Of course, in addition to the software implementation, this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0128] Fig.10 is a block diagram of a detection device for a mechanical parking space according to an exemplary embodiment of the present application. Fig.10 , the device may include: a detection module 1002, a matching module 1004 and a processing module 1006, wherein: The detection module 1002 is used to detect multiple frames of parking space images to obtain point and line information in the multiple frames of parking space images; wherein the multiple frames of parking space images are obtained by shooting the mechanical parking space with a monocular camera, and the point and line information includes: the entrance corner point of the mechanical parking space and the inner line of the parking space, and the direction of the inner line of the parking space is the direction from the parking space entrance to the end of the parking space; The matching module 1004 is used to match the entrance corner point with the inner line of the parking space to obtain the point-line pair of the mechanical parking space; The processing module 1006 is used to perform rectangular processing on the point-line pair to obtain the parking space line of the mechanical parking space.

[0129] In one example, the matching module 1004, before being used to match the entrance corner point and the inner line of the parking space to obtain the point-line pair of the mechanical parking space, further includes: determining the three-dimensional position of the point-line information according to the posture information of the vehicle and the external parameters of the monocular camera; based on the three-dimensional position, determining the maximum viewing angle difference of the point-line information, the maximum viewing angle difference is the maximum angle difference formed according to the observation viewing angle of the multiple frames of parking space images; when the maximum viewing angle difference is greater than a preset threshold, retaining the point-line information.

[0130] In one example, the detection module 1002, when used to detect multiple frames of parking space images and obtain point and line information in the multiple frames of parking space images, includes: extracting inner line features in the multiple frames of parking space images, generating a first heat map and a first offset corresponding to the inner line features; determining a first candidate point on the inner line of the parking space based on the first heat map and the first offset; and clustering and fitting the candidate points to obtain the inner line of the parking space.

[0131] In one example, the detection module 1002, when used to detect multiple frames of parking space images and obtain point and line information in the multiple frames of parking space images, includes: extracting point features in the multiple frames of parking space images, generating a second heat map and a second offset corresponding to the point features; determining a second candidate point for the entry corner point based on the second heat map and the second offset; deleting duplicate points in the second candidate points, and using the second candidate points as the entry corner points.

[0132] In one example, the entrance corner point includes: a first entrance corner point and a second entrance corner point; the inner edge line of the parking space includes: a first inner edge line and a second inner edge line, the first inner edge line is the inner edge line of the parking space closest to the first entrance corner point, and the second inner edge line is the inner edge line of the parking space closest to the second entrance corner point; the matching module 1004, when used to match the entrance corner point and the inner edge line of the parking space to obtain the point-line pair of the mechanical parking space, includes: when the first distance value is less than a distance threshold, the first entrance corner point and the first inner edge line are combined into the point-line pair; when the second distance value is less than the distance threshold, the second entrance corner point and the second inner edge line are combined into the point-line pair; wherein the first distance value is the distance value between the first entrance corner point and the first inner edge line, and the second distance value is the distance value between the second entrance corner point and the second inner edge line.

[0133] In one example, the point-line pairs include: multiple first point-line pairs and multiple second point-line pairs; the processing module 1006, before being used to rectangularize the point-line pairs to obtain the parking space lines of the mechanical parking space, also includes: for each first point-line pair, searching to the right for the second point-line pair that is closest; for each second point-line pair, searching to the left for the first point-line pair that is closest; using the first point-line pair and the second point-line pair that are closest as matching point-line pairs; the processing module 1006, when being used to rectangularize the point-line pairs to obtain the parking space lines of the mechanical parking space, includes: rectangularizing the first point-line pair and the second point-line pair in each matching point-line pair to obtain the parking space lines corresponding to each matching point-line pair.

[0134] In one example, the processing module 1006 is also used to detect the end corner point in the multiple frames of parking space images and the wheel chock position of the mechanical parking space; the end corner point is the intersection of the inner line of the parking space and the rear boundary line of the mechanical parking space; based on the end corner point and the wheel chock position, the rear boundary line is determined.

[0135] In one example, the parking space line includes: a front boundary line, a left boundary line and a right boundary line; the processing module 1006, when used to rectangularize the point-line pair to obtain the parking space line of the mechanical parking space, includes: taking the direction of the mean angle as the parking space direction of the mechanical parking space; wherein the mean angle is the mean of the angle formed by the first point-line pair and the second point-line pair in the point-line pair; rotating the first point-line pair to obtain the left boundary line parallel to the parking space direction; rotating the second point-line pair to obtain the right boundary line parallel to the parking space direction; projecting the first entry corner point in the first point-line pair to the parking space direction to obtain a left projection point; projecting the second entry corner point in the second point-line pair to the parking space direction to obtain a right projection point; constructing the front boundary line based on the projection midpoint, the left boundary line and the right boundary line, the projection midpoint being the midpoint between the left projection point and the right projection point.

[0136] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0137] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0138] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions. The above instructions can be executed by a processor of a detection device for a mechanical parking space to implement a method as described in any of the above embodiments.

[0139] The non-temporary computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc., and the present application does not limit this.

[0140] In an exemplary embodiment, a computer program product including a computer program / instruction is further provided. The computer program / instruction can be executed by a processor of a detection device for a mechanical parking space to implement any of the methods described in the above embodiments.

[0141] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0142] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the inventions claimed herein. The present application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0143] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for detecting a mechanical parking space, characterized in that: The method comprises: Detecting multiple frames of parking space images to obtain point and line information in the multiple frames of parking space images; wherein the multiple frames of parking space images are obtained by photographing the mechanical parking space with a monocular camera, and the point and line information includes: an entrance corner point of the mechanical parking space and an inner line of the parking space, wherein the direction of the inner line of the parking space is the direction from the parking space entrance to the end of the parking space; Matching the entrance corner point with the inner edge of the parking space to obtain a point-line pair of the mechanical parking space; The point-line pairs are rectangularized to obtain the parking space line of the mechanical parking space.

2. The method according to claim 1, characterized in that: Before matching the entrance corner point with the inner line of the parking space to obtain the point-line pair of the mechanical parking space, the method further includes: Determine the three-dimensional position of the point and line information according to the posture information of the vehicle and the external parameters of the monocular camera; Based on the three-dimensional position, determining a maximum viewing angle difference of the point and line information, wherein the maximum viewing angle difference is a maximum angle difference formed according to observation viewing angles of the multiple frames of parking space images; When the maximum viewing angle difference is greater than a preset threshold, the point and line information is retained.

3. The method according to claim 1, characterized in that: The detecting of the multiple frames of parking space images to obtain point and line information in the multiple frames of parking space images includes: Extracting inner edge line features from the multiple frames of parking space images, and generating a first heat map and a first offset corresponding to the inner edge line features; Determine a first candidate point along the line in the parking space based on the first heat map and the first offset; The candidate points are clustered and fitted to obtain the inner line of the parking space.

4. The method according to claim 1, characterized in that The detecting of the multiple frames of parking space images to obtain point and line information in the multiple frames of parking space images includes: Extracting point features from the multiple frames of parking space images, and generating a second heat map and a second offset corresponding to the point features; Determine a second candidate point for the entry corner point based on the second heat map and the second offset; Duplicate points in the second candidate points are deleted, and the second candidate points are used as the entry corner points.

5. The method according to claim 1, characterized in that The entrance corner point includes: a first entrance corner point and a second entrance corner point; the parking space inner edge line includes: a first inner edge line and a second inner edge line, the first inner edge line is the parking space inner edge line closest to the first entrance corner point, and the second inner edge line is the parking space inner edge line closest to the second entrance corner point; The step of matching the entrance corner point with the inner edge of the parking space to obtain the point-line pair of the mechanical parking space includes: When the first distance value is less than the distance threshold, the first entry corner point and the first inner edge line form the point-line pair; When the second distance value is less than the distance threshold, the second entry corner point and the second inner edge line form the point-line pair; The first distance value is the distance value between the first entry corner point and the first inner edge line, and the second distance value is the distance value between the second entry corner point and the second inner edge line.

6. The method according to claim 1, characterized in that The point-line pairs include: a plurality of first point-line pairs and a plurality of second point-line pairs; Before performing rectangular processing on the point-line pair to obtain the parking space line of the mechanical parking space, the method further includes: For each first point line pair, search rightward for the second point line pair that is closest to it; For each second point line pair, search to the left for the first point line pair that is closest to it; The first point line pair and the second point line pair that are closest to each other are taken as matching point line pairs; The rectangularization process is performed on the point-line pair to obtain the parking space line of the mechanical parking space, including: Rectangularization is performed on the first point-line pair and the second point-line pair in each matching point-line pair to obtain parking space lines corresponding to each matching point-line pair.

7. The method according to claim 1, characterized in that The method further comprises: Detecting the end corner points in the multiple frames of parking space images and the wheel block position of the mechanical parking space; the end corner point is the intersection of the inner line of the parking space and the rear boundary line of the mechanical parking space; The rear boundary line is determined based on the end corner point and the wheel block position.

8. The method according to claim 1, characterized in that: The parking space lines include: a front boundary line, a left boundary line and a right boundary line; The rectangularization process is performed on the point-line pair to obtain the parking space line of the mechanical parking space, including: The direction of the mean angle is used as the parking direction of the mechanical parking space; wherein the mean angle is the mean of the angles formed by the first point-line pair and the second point-line pair in the point-line pair; Rotating the first point-line pair to obtain the left boundary line parallel to the parking space direction; Rotating the second point line pair to obtain the right boundary line parallel to the parking space direction; Projecting the first entrance corner point in the first point line pair to the parking space direction to obtain a left projection point; Projecting the second entrance corner point in the second point line pair to the parking space direction to obtain a right projection point; The front boundary line is constructed based on a projection midpoint, the left boundary line and the right boundary line, wherein the projection midpoint is the midpoint between the left projection point and the right projection point.

9. A detection device for a mechanical parking space, characterized in that: The device comprises: A detection module is used to detect multiple frames of parking space images to obtain point and line information in the multiple frames of parking space images; wherein the multiple frames of parking space images are obtained by shooting the mechanical parking space with a monocular camera, and the point and line information includes: an entrance corner point of the mechanical parking space and an inner line of the parking space, wherein the direction of the inner line of the parking space is the direction from the parking space entrance to the end of the parking space; A matching module, used for matching the entrance corner point with the inner edge of the parking space to obtain a point-line pair of the mechanical parking space; The processing module is used to perform rectangular processing on the point-line pair to obtain the parking space line of the mechanical parking space.

10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

11. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product having a computer program / instructions stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images