Parking space detection method and device and electronic equipment

Abstract

The invention provides a parking space detection method and device and electronic equipment, and the method comprises the steps: obtaining an environment image of a vehicle, determining a parking space line instance, a parking space region instance and a parking space corner point in the environment image through a segmentation model and a detection model which are trained in advance, obtaining a first candidate region according to the parking space line instance, and carrying out the detection of the parking space. And obtaining a second candidate area according to the parking space area instance, obtaining a third candidate area according to the parking space angular point, and finally obtaining a detected parking space area according to the parking space confidence degrees corresponding to the first candidate area, the second candidate area and the third candidate area. Under the condition that the parking space line is not clear or incomplete, even if the determined parking space confidence of the first candidate area is low, the parking space area can be determined through the second candidate area and the third candidate area with higher confidence.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving, and particularly relates to a parking space detection method, device, and electronic device. Background Art

[0002] With the continuous development of intelligent driving technology, vehicles equipped with intelligent driving systems are increasingly widely used in people's lives. In the field of intelligent driving, one of the problems that has received much attention is how to accurately detect a parking space when the vehicle needs to park, so as to park the vehicle in the parking space.

[0003] Currently, a common parking space detection method is to obtain the environmental image of the vehicle, and then use a segmentation model to determine the parking space lines from the environmental image, group the detected parking space lines, determine the parking space lines belonging to the same parking space, and then determine the parking space based on the parking space lines belonging to the same parking space.

[0004] However, when the boundary lines of the parking space in the environmental image are not clear enough, the parking space cannot be accurately determined. Summary of the Invention

[0005] In view of this, embodiments of the present invention are committed to providing a parking space detection method, device, and electronic device, which can simultaneously determine the parking space lines, the internal area of the parking space, and at least two parking space corner points included in the environmental image, so as to accurately determine the parking space, and solve the problem that the parking space cannot be accurately determined in the prior art when the boundary lines of the parking space are not clear enough.

[0006] On the one hand, the present invention provides a parking space detection method, including:

[0007] Determine the environmental image of the vehicle, where the environmental image is synthesized from the image data collected by the acquisition device arranged around the vehicle;

[0008] Take the environmental image as the input, and input it into a pre-trained segmentation model and a detection model respectively, to obtain the parking space line instances and parking space area instances in the environmental image output by the segmentation model, and the parking space corner points included in the environmental image output by the detection model;

[0009] According to the parking space line instances, obtain the first candidate region, according to the parking space area instances, obtain the second candidate region, and according to the parking space corner points, obtain the third candidate region;

[0010] Determine the detected parking space area according to the parking space confidence levels corresponding to the first candidate region, the second candidate region, and the third candidate region respectively.

[0011] Optionally, the segmentation model is a multi-object segmentation model;

[0012] Taking the environmental image as an input and inputting it into a segmentation model to obtain the parking space line instances and parking space area instances in the environmental image output by the segmentation model, including:

[0013] Taking the environmental image as an input and inputting it into the feature extraction layer of the segmentation model to obtain the image features output by the feature extraction layer;

[0014] Inputting the image features into the first instance segmentation layer and the second instance segmentation layer of the segmentation model respectively to obtain the parking space line instances output by the first instance segmentation layer and the parking space area instances output by the second instance segmentation layer.

[0015] Optionally, the number of parking space corner points included in the environmental image output by the detection model is at least two;

[0016] Obtaining a third candidate region according to the parking space corner points, including:

[0017] Determining the interior angle of the parking space according to at least one of the first candidate region and the second candidate region;

[0018] Constructing a third candidate region according to the interior angle of the parking space and at least two parking space corner points.

[0019] Optionally, the border line of the first candidate region is determined by the middle line of the parking space line instance, the border line of the second candidate region is determined by the edge line of the parking space area instance, and the parking space corner points are the corner points of the border formed by the middle line between the inner and outer edges of the parking space line in the environmental image;

[0020] The method further includes:

[0021] Correcting the second candidate region according to the position of the border line of the first candidate region,

[0022] Or,

[0023] Correcting the second candidate region according to the corner point position of the third candidate region.

[0024] Optionally, it further includes:

[0025] For each of the first candidate region, the second candidate region and the third candidate region, determining the parking space confidence of the candidate region according to at least one of the length confidence of each border line of the candidate region and the corner point confidence of each corner point in the candidate region.

[0026] Optionally, determining the parking space confidence of the candidate region according to at least one of the length confidence of each border line of the candidate region and the corner point confidence of each corner point in the candidate region includes:

[0027] For each of the first candidate region, the second candidate region and the third candidate region, determining the length of each border line of the candidate region;

[0028] Determine the length confidence levels corresponding to the respective border lines of the candidate regions according to the lengths of the respective border lines of the candidate regions and a preset border threshold;

[0029] Determine the vehicle position confidence level of the candidate region according to the length confidence levels corresponding to the respective border lines.

[0030] Optionally, determine the vehicle position confidence level of the candidate region according to at least one of the length confidence levels of the respective border lines of the candidate region and the corner confidence levels of the respective corner points in the candidate region, including:

[0031] For each of the first candidate region, the second candidate region, and the third candidate region, determine the positions of the respective corner points of the candidate region;

[0032] For each corner point among the respective corner points of the candidate region, determine the border line corresponding to the corner point;

[0033] Determine the corner confidence level of the corner point according to the position of the corner point, the position of the border line corresponding to the corner point, and a preset distance threshold;

[0034] Weight the corner confidence levels corresponding to the respective corner points to obtain the vehicle position confidence level of the candidate region.

[0035] Optionally, the environmental image includes multiple candidate parking spaces.

[0036] Determine the detected parking space regions according to the vehicle position confidence levels corresponding to the first candidate region, the second candidate region, and the third candidate region respectively, including:

[0037] Determine the first candidate region, the second candidate region, and the third candidate region corresponding to each candidate parking space according to the intersection over union between the respective first candidate regions, second candidate regions, and third candidate regions detected in the environmental image;

[0038] For each candidate parking space, determine the target region corresponding to the candidate parking space according to the vehicle position confidence levels corresponding to the first candidate region, the second candidate region, and the third candidate region corresponding to the candidate parking space;

[0039] Take the target regions corresponding to the respective candidate parking spaces as the detected parking space regions.

[0040] This specification provides a parking space detection device, including:

[0041] An acquisition module, configured to determine an environmental image of a vehicle, where the environmental image is synthesized from image data collected by an acquisition device arranged around the vehicle;

[0042] An instance determination module, configured to take an environmental image as an input and input it into a pre-trained segmentation model and a detection model respectively, to obtain a parking space line instance and a parking space area instance in the environmental image output by the segmentation model, and to obtain parking space corner points included in the environmental image output by the detection model;

[0043] A region determination module, configured to obtain a first candidate region according to the parking space line instance, obtain a second candidate region according to the parking space area instance, and obtain a third candidate region according to the parking space corner points;

[0044] A parking space area determination module, configured to determine the detected parking space area according to the parking space confidence levels corresponding to the first candidate region, the second candidate region, and the third candidate region respectively.

[0045] This specification provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned parking space detection method is implemented.

[0046] This specification provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the processor executes the program, the above-mentioned parking space detection method is implemented.

[0047] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0048] In the parking space detection method provided in this specification, by acquiring an environmental image of a vehicle, and then through a pre-trained segmentation model and a detection model, determining a parking space line instance, a parking space area instance, and parking space corner points in the environmental image, and further obtaining a first candidate region according to the parking space line instance, obtaining a second candidate region according to the parking space area instance, and obtaining a third candidate region according to the parking space corner points, and finally obtaining the detected parking space area according to the parking space confidence levels corresponding to the first candidate region, the second candidate region, and the third candidate region respectively. In the case where the parking space line is unclear or incomplete, even if the parking space confidence level of the determined first candidate region is low, the parking space area can be determined through the second candidate region and the third candidate region with higher confidence levels. That is to say, the situation where the parking space line is unclear or incomplete has little impact on the detection result of the parking space area, ensuring the reliability of the determined parking space area. Description of the Drawings

[0049] Figure 1 Shown is a schematic diagram of a parking space line instance and a parking space area instance provided in this specification;

[0050] Figure 2 Shown is a flowchart of the parking space detection method provided in this specification;

[0051] Figure 3 The figure shows a schematic structural diagram of a parking space line example and a parking space area example provided in this specification;

[0052] Figure 4A The figure shows a schematic structural diagram of a first candidate area provided in this specification;

[0053] Figure 4B The figure shows a schematic structural diagram of a second candidate area provided in this specification;

[0054] Figure 4C The figure shows a schematic structural diagram of a first candidate area provided in this specification;

[0055] Figure 5 The figure shows a schematic structural diagram of a candidate area provided in this specification;

[0056] Figure 6 The figure shows a schematic flowchart of a parking space detection method provided in this specification;

[0057] Figure 7 The figure shows a schematic structural diagram of a parking space detection device provided in this specification;

[0058] Figure 8 The figure shows a schematic structural diagram of an electronic device provided in this specification. Detailed implementation manners

[0059] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0060] It should be noted that all actions of obtaining signals, information, or data in this specification are carried out on the premise of complying with the corresponding data protection regulations and policies of the location and with the authorization given by the owner of the corresponding device.

[0061] An embodiment of the present invention provides a parking space detection method, which acquires an environmental image of a vehicle, and then determines a parking space line instance, a parking space area instance, and a parking space corner point in the environmental image through a pre-trained segmentation model and a detection model. Furthermore, according to the parking space line instance, a first candidate area is obtained, according to the parking space area instance, a second candidate area is obtained, and according to the parking space corner point, a third candidate area is obtained. Finally, a target area is obtained according to the vehicle position confidences corresponding to the first candidate area, the second candidate area, and the third candidate area respectively.

[0062] In an example, specifically, as shown in Figure 1 shown.Figure 1 A schematic diagram of the parking space line example and the parking space area example provided for this application. Taking the parking space including the border line of the black rectangle with width shown in the figure and the white rectangle area surrounded by the border line as an example, according to this parking space, the parking space line example and the parking space area example can be determined.

[0063] In one example, the parking space line example is the example corresponding to the border line of the parking space, and the parking space area example is the area inside the parking space border line, that is, the example corresponding to the internal area of the parking space.

[0064] In one example, the parking space line example and the parking space area example can be Mask masks with the same size but different pixel distributions. In the parking space line example, the area where the parking space line with width is located, that is, the shaded part shown in the parking space line example has different pixels from other areas in the example, so as to distinguish the parking space line. In the parking space area example, the internal area of the parking space can be as Figure 1 shown by the shaded part in the parking space area example in [reference], and this shaded part has different pixels from other areas in the parking space area example, so as to distinguish this internal area.

[0065] In one example, the first candidate area is the parkable area of the vehicle in the parking space determined according to the parking space line, the second candidate area is the internal area in the parking space area example output according to the segmentation model. The third candidate area is the area constructed according to the parking space corner points.

[0066] The following will combine the accompanying drawings to detail the technical solutions provided by each embodiment of this specification.

[0067] Figure 2 A flowchart of a parking space detection method in this specification, specifically including the following steps:

[0068] S100: Determine the environmental image of the vehicle, and the environmental image is synthesized from the image data collected by the acquisition device set around the vehicle.

[0069] In one or more embodiments provided by this application, this parking space detection method can be applied to a vehicle equipped with an intelligent driving system, or can be applied to a server communicating with the vehicle. The vehicle can be an unmanned vehicle or a manned vehicle. For the convenience of description, the execution process of this parking space detection method executed by the vehicle will be used as an example for subsequent description.

[0070] Generally, when a vehicle needs to park in a parking space, it is usually necessary to determine the area where the vehicle itself is located and the area where the parking space is located in the surrounding environment of the vehicle. After determining the above two areas, the vehicle is controlled to adjust its pose to park in the area where the parking space is located. In the panoramic image captured with the vehicle as the coordinate origin, the area where the vehicle is located can be regarded as the origin position. Therefore, only by knowing the area where the parking space is located in the surrounding environment of the vehicle can the vehicle be parked in the parking space. The area where the parking space is located in the surrounding environment of the vehicle can be obtained from the environmental image of the vehicle. Therefore, the environmental image of the vehicle can be determined first.

[0071] Specifically, a collection device can be arranged around the vehicle. The collection device can be a wide-angle lens or a fish-eye lens. Thus, the images in different directions of the vehicle can be collected through the collection device arranged around the vehicle.

[0072] After obtaining the image data collected by each collection device simultaneously, the image data collected by each collection device simultaneously can be synthesized to obtain the environmental image of the vehicle. Among them, the environmental image is a panoramic image, and the environmental information in all directions of the vehicle can be included in the environmental image.

[0073] In one example, the collection device can be a fish-eye lens, the number of the collection devices can be four, and the collection devices are respectively arranged in the front, rear, left, and right directions of the vehicle.

[0074] In one example, the collection device can be arranged at four positions: the left front corner, the right front corner, the left rear corner, and the right rear corner of the vehicle.

[0075] S102: Taking the environmental image as the input, input it into the pre-trained segmentation model and detection model respectively, to obtain the parking space line instance and the parking space area instance in the environmental image output by the segmentation model, and the parking space corner points included in the environmental image output by the detection model.

[0076] In one or more embodiments provided by the present application, as described above, the inventive point of the parking space detection method is to simultaneously determine the parking space line, the internal area of the parking space, and the parking space corner points in the surrounding environment of the vehicle, and then, through the above three items, simultaneously determine the first candidate area, the second candidate area, and the third candidate area corresponding to the parking space, and further determine the parking space area according to the vehicle position confidence degrees corresponding to the first candidate area, the second candidate area, and the third candidate area respectively.

[0077] Therefore, the parking space line, the internal area, and the parking space corner points in the surrounding environment of the vehicle can be determined first.

[0078] Specifically, as Figure 3 shown, Figure 3A structural schematic diagram of the parking space line example and the parking space area example provided in this specification. In the figure, taking the case where the environmental image contains the parking space area shown on the left side of the figure as an example, when the environmental image is input into the pre-trained segmentation model, the parking space line example and the parking space area example output by the segmentation model can be obtained. Among them, the parking space line example is the example (Mask 1) corresponding to the border line of the parking space, the parking space area example is the example (Mask 2) corresponding to the area within the parking space border line, and the parking space corner point is the boundary point of the parking space.

[0079] It should be understood that the parking space line example can be the example corresponding to the border formed by combining the four border lines of the parking space, such as Mask1. In other embodiments, the parking space line example can also be the example corresponding to each border line respectively, that is, a parking space can have four parking space line examples. The following takes the case where the parking space line example is specifically the example corresponding to the border formed by combining the four border lines of the parking space as an example for description.

[0080] At the same time, the environmental image can also be input into the detection model to obtain the parking space corner points output by the detection model.

[0081] Among them, the segmentation model and the detection model can be trained in the following manner:

[0082] First, the server for training the model determines the sample parking spaces included in the sample environmental image.

[0083] Then, the server determines the parking space lines, internal areas, and parking space corner points corresponding to the sample parking spaces as the first annotation, second annotation, and third annotation of the sample environmental image.

[0084] After that, the server inputs the sample environmental image into the segmentation model and the detection model respectively to obtain the sample parking space line example and the sample parking space area example output by the segmentation model, and the sample corner points output by the detection model.

[0085] Finally, the server can train the segmentation model according to the gap between the sample parking space line example and the first annotation, and the gap between the sample parking space area example and the second annotation. At the same time, train the detection model according to the gap between the sample corner points and the third annotation.

[0086] In an example, the server can determine the first loss according to the gap between the sample parking space line example and the first annotation, and the gap between the sample parking space area example and the second annotation, and take minimizing the first loss as the optimization goal to adjust the model parameters of the segmentation model to complete the training of the segmentation model.

[0087] In one example, the server can determine a second loss based on the gap between the sample corner points and the third annotation, and use minimizing the second loss as the optimization objective to adjust the model parameters of the detection model to complete the training of the detection model.

[0088] The trained segmentation model can be used to determine the parking space line instances and the parking space area instances included in the vehicle environment image, and the trained detection model can be used to determine the parking space corner points.

[0089] It should be noted that the server used for training the model can be a server communicating with the vehicle or the vehicle itself.

[0090] S104: Obtain a first candidate region according to the parking space line instance, obtain a second candidate region according to the parking space area instance, and obtain a third candidate region according to the parking space corner points.

[0091] In one or more embodiments provided in this specification, in the environmental image of the vehicle, it is inevitable that the parking space lines in the parking space area are not clear or complete enough, and it is difficult to detect the parking space corner points. In this case, it is not enough to only determine the parking space line instance and obtain the parking space area according to the parking space line instance. In order to still obtain a relatively accurate parking space area in the above situation, different candidate regions can be constructed according to the parking space line instance, the parking space area instance, and the parking space corner points, and then the parking space area can be determined according to the constructed different candidate regions.

[0092] Specifically, taking the parking space line instance as Mask1 shown in Figure 3 as an example, the edge of the parking space line can be determined according to the parking space line instance, and then the border of the first candidate region can be obtained according to the edge of the parking space line.

[0093] In one embodiment, according to the parking space area instance, taking the parking space area instance as Mask2 shown in Figure 3 as an example, the edge feature points of the internal area of the parking space can be determined, and by fitting the edge feature points of the internal area, the border lines of the second candidate region can be obtained, and then the border of the second candidate region can be determined according to the border lines.

[0094] In one embodiment, the number of parking space corner points output by the detection model can be four, and then the third candidate region can be constructed according to the four parking space corner points output by the detection model. However, in actual use, due to reasons such as obstruction by obstacles, the detection model often cannot output exactly four parking space corner points, but can only output exactly two or three parking space corner points. Therefore, in order to make the determined third candidate region more accurate, the interior angle of the parking space can also be determined according to the first candidate region and the second candidate region, and then the third candidate region can be constructed according to the determined interior angle of the parking space and the parking space corner points output by the detection model.

[0095] In one embodiment, according to at least one of the first candidate region and the second candidate region, the interior angle of the parking space can be obtained. According to the obtained interior angle of the parking space and the parking space corner points output by the detection model, a third candidate region can be constructed.

[0096] In one example, for each of the first candidate region, the second candidate region, and the third candidate region, the border of the candidate region is used to represent the boundary range corresponding to the candidate region, and the border is composed of border lines, and the border lines are the line segments that make up the border.

[0097] In one example, the edge of the parking space line can be the inner edge of the parking space line or the outer edge of the parking space line.

[0098] In one example, obtaining the border of the first candidate region according to the edge of the parking space line can be using the inner edge, the outer edge, or the intermediate line between the inner edge and the outer edge of the parking space line as the border of the first candidate region. The following takes using the intermediate line between the inner edge and the outer edge of the parking space line as the border of the first candidate region as an example for illustration.

[0099] In one example, when the parking space is rectangular, the interior angle of the parking space can be the included angle between the long side and the short side of the parking space. Taking the parking space shown in Figure 3 as an example, the interior angle of the parking space can be 90 degrees.

[0100] In one example, generally, the shape of the parking space is a parallelogram or a rectangle. Therefore, when knowing one interior angle of the parking space, the angles corresponding to the four interior angles included in the parking space can be determined. Then, according to the positions of two or more parking space corner points, at least one border line corresponding to the third candidate region can be determined, and then according to the interior angle of the third candidate region and the at least one border line, the other border lines of the parking space can be constructed, so as to construct the third candidate region.

[0101] In one example, the interior angle of the parking space determined by the first candidate region and / or the second candidate region can be one or more of the four interior angles corresponding to the parking space.

[0102] In another embodiment, when determining the parking space corner points, the interior angle of the parking space can also be determined according to the environmental image, and then the third candidate region can be constructed according to the parking space corner points and the interior angle of the parking space. That is to say, determining the interior angle of the parking space according to at least one of the first candidate region and the second candidate region is only one of the embodiments for determining the interior angle of the parking space.

[0103] S106: Determine the detected parking space region according to the parking space confidence levels corresponding to the first candidate region, the second candidate region, and the third candidate region respectively.

[0104] In one or more embodiments provided in this specification, after obtaining a plurality of candidate regions, the detection result of the parking space region may be determined from the plurality of candidate regions according to the confidence levels respectively corresponding to the candidate regions, and used as the detected parking space region.

[0105] Thus, for each candidate region, the confidence level of the vehicle position corresponding to the candidate region may be determined. Among them, when the segmentation model outputs an instance, it may simultaneously output the instance confidence level corresponding to the instance. Thus, the first instance confidence level corresponding to the parking space line instance may be determined as the vehicle position confidence level of the first candidate region, and the second instance confidence level corresponding to the parking space region instance may be determined as the vehicle position confidence level of the second candidate region. At the same time, the confidence level of the parking space corner points output by the detection model is determined as the vehicle position confidence level of the third candidate region.

[0106] According to the vehicle position confidence levels respectively corresponding to the first candidate region, the second candidate region, and the third candidate region, the candidate region with the highest confidence level may be determined as the detected parking space region.

[0107] Based on Figure 2 The described parking space detection method obtains the environmental image of the vehicle, then determines the parking space line instance and the parking space region instance in the environmental image through a pre-trained segmentation model, determines the parking space corner points in the environmental image through a pre-trained detection model, and then obtains the first candidate region according to the parking space line instance, obtains the second candidate region according to the parking space region instance, and obtains the third candidate region according to the parking space corner points. Finally, according to the vehicle position confidence levels respectively corresponding to the first candidate region, the second candidate region, and the third candidate region, the detected parking space region is obtained. In the case where the parking space line is unclear or incomplete, even if the vehicle position confidence level of the determined first candidate region is low, the parking space region may be determined through the second candidate region and the third candidate region with higher confidence levels. That is to say, the situation where the parking space line is unclear or incomplete has little impact on the detection result of the parking space region, ensuring the reliability of the determined parking space region.

[0108] In one embodiment, the segmentation model is a multi-object segmentation model, and the segmentation model includes a feature extraction layer, a first instance segmentation layer, and a second instance segmentation layer. Thus, in step S102, the environmental image may be used as an input and input into the feature extraction layer of the segmentation model to obtain the image features output by the feature extraction layer. Then, the image features may be respectively input into the first instance segmentation layer and the second instance segmentation layer of the segmentation model to obtain the parking space line instance output by the first instance segmentation layer and the parking space region instance output by the second instance segmentation layer.

[0109] The segmentation model based on the multi-object segmentation model structure only needs to determine the image features of the environmental image, and then perform different segmentations on the image features to obtain the parking space line instance and the parking space area instance. Compared with the method of determining the first image features of the environmental image through the first segmentation model, and then obtaining the parking space line instance according to the first image features, and at the same time determining the second image features of the environmental image through the second segmentation model, and then obtaining the parking space area instance according to the second image features, the required computing resources are less. In the case where the computing resources in the vehicle are limited, the execution efficiency of this parking space detection method is further ensured.

[0110] In one embodiment, the first candidate area may be the parkable area of the vehicle in the parking space. Usually, for a parking space, the parkable area of the vehicle in the parking space should actually be the area enclosed by the center line of the parking space line, as Figure 4A shown, Figure 4A which is the structural schematic diagram of the first candidate area provided in this specification. It can be seen that for each orientation of the parking space area, the vehicle can determine the center line (as shown by the dotted line in the figure) located in the center of the parking space line and at the same distance from the inner edge and the outer edge of the parking space line located in that orientation as the border line of the first candidate area in that orientation. And after determining the border lines of the first candidate area in each orientation, combine the border lines to obtain the border of the first candidate area (the rectangle enclosed by the dotted line shown in the figure), and then determine the first candidate area. In this application, the center line of the inner edge and the outer edge of the parking space line is taken as an example to illustrate the border of the first candidate area.

[0111] In this case, the second candidate area may be the area enclosed by the edge lines of the parking space area in the parking space area instance, that is, as Figure 3 shown by the shaded area in Mask 2 in the figure. It can be seen that the edge corresponding to the inner area in the parking space area instance can be directly used as the border of the second candidate area. Obviously, the difference between the determined second candidate area and the actual parkable area of the vehicle is half of the width of the parking space line. Therefore, before performing step S106, the second candidate area can also be corrected according to the first candidate area. Among them, the border of the second candidate area in this application is the edge line corresponding to the inner area in the parking space area instance.

[0112] Specifically, the border lines of the first candidate area and the second candidate area located in the same orientation of the parking space can be determined first.

[0113] After that, the distance between the border lines located in the same orientation of the parking space can be determined.

[0114] If the distance is less than or equal to a preset first threshold, it can be considered that the difference between the border lines of the first candidate region and the second candidate region in this orientation is small. In this case, the width of the parking space line instance corresponding to the first candidate region can be determined, and the border line of the second candidate region can be translated to a position outside the second candidate region according to half of the width of the parking space line. After re-determining the border lines of the second candidate region located in each orientation of the parking space, the corrected second candidate region can be obtained according to the re-determined border lines, which completes the process of correcting the second candidate region. Further, subsequently, the parking position confidence levels corresponding to the first candidate region, the corrected second candidate region, and the third candidate region can be directly determined, and the detected parking space region can be determined according to the determined parking position confidence levels.

[0115] In one example, the first threshold can be slightly greater than or approximately equal to half of the width of the parking space line. In this way, the second candidate region with a small difference from the first candidate region can be corrected based on this first threshold. Of course, the specific value of this first threshold can be set as needed, and this specification does not limit it.

[0116] If the distance is greater than the preset first threshold, it can be considered that the difference between the border lines of the first candidate region and the second candidate region in this orientation is large. The reason for this situation is usually that there are obstacles in the parking space region occupying the parking space region or the parking space line, resulting in low confidence levels of the determined first candidate region and / or second candidate region. Therefore, in this case, the second candidate region is not corrected. Further, subsequently, the parking position confidence levels corresponding to the first candidate region, the second candidate region, and the third candidate region can be directly determined, and the detected parking space region can be determined according to the determined parking position confidence levels.

[0117] In one example, for an actual parking space, it usually includes three types of corner points: first, the corner points corresponding to the border of the inner edge of the parking space line; second, the corner points corresponding to the border of the outer edge of the parking space line; third, the corner points of the border formed by the middle line between the inner edge and the outer edge of the parking space line. In order to be able to determine a third candidate region that better matches the actual parking space, during the training process of the detection model, the parking space corner points included in the sample environmental image are usually the third type in the above situation. Therefore, the parking space corner points output by the detection model can be the corner points corresponding to the border formed by the middle line between the inner and outer edges of the parking space line in the environmental image. In this way, the difference between the third candidate region constructed based on the parking space corner points and the parking space interior angle and the first candidate region should be small. That is to say, before performing step S106, the second candidate region can also be corrected according to the third candidate region.

[0118] In one example, the four corner points included in the second candidate region and the four corner points included in the third candidate region can be determined. Then, for the corner points in the same orientation of the parking space (for example, the corner point on the southeast side of the parking space in the second candidate region and the corner point on the southeast side of the parking space in the third candidate region), the gap between the position of the corner point in the second candidate region and the position of the corner point in the third candidate region can be determined.

[0119] If the gap is less than or equal to a preset second threshold, it can be considered that the gap between the corner point of the second candidate region and the corner point of the third candidate region is small in this orientation. In this case, the corner point of the third candidate region can be directly used as the corner point of the second candidate region in this orientation. After the corner points of the second candidate region in each orientation of the parking space are re-determined, the second candidate region can be re-determined according to the re-determined corner points. In this way, the correction of the second candidate region is completed. Subsequently, the vehicle position confidence levels corresponding to the first candidate region, the corrected second candidate region, and the third candidate region can be directly determined, and the detected parking space region can be determined according to the determined vehicle position confidence levels.

[0120] In one example, the second threshold can be slightly greater than or approximately equal to a specific value. The specific value can be the length of the hypotenuse of an isosceles right triangle with half of the width of the parking space line as the right-angle side. In this way, the second candidate region with a small gap from the third candidate region can be corrected based on the second threshold. Of course, the specific value of the second threshold can be set as needed, and this specification does not limit it.

[0121] If the distance is greater than the preset second threshold, it can be considered that the gap between the corner points of the second candidate region and the third candidate region in this orientation is large. The reason for this situation is usually that there are obstacles in the parking space region occupying the parking space region or the parking space corner points, resulting in low confidence levels of the determined second candidate region and / or third candidate region. Therefore, in this case, the second candidate region is not corrected. Similar to the foregoing, subsequently, the vehicle position confidence levels corresponding to the first candidate region, the second candidate region, and the third candidate region can be directly determined, and the detected parking space region can be determined according to the determined vehicle position confidence levels.

[0122] In one example, before performing step S106, the second candidate region can be corrected according to at least one of the first candidate region and the third candidate region.

[0123] The process of correcting the second candidate region according to the corner positions can be referred to the above content. However, when actually correcting the second candidate region, the positions of the border lines of the third candidate region can also be determined according to the corner points of the third candidate region, and then the second candidate region can be corrected according to the border line positions. Specifically, the process of correcting the second candidate region according to the border line positions of the third candidate region can refer to the description of correcting the second candidate region according to the first candidate region above, and this specification will not elaborate on it anymore.

[0124] In this way, it can be ensured that the area of the regions corresponding to the first candidate region, the second candidate region, and the third candidate region are similar. At the same time, in the case where the environmental image contains multiple candidate parking spaces, the parking space region of one candidate parking space can be detected as the first candidate region, the second candidate region, and the third candidate region at the same time. Based on the above content of correcting the second candidate region, the intersection over union ratio between the first candidate region, the third candidate region, and the second candidate region corresponding to the same candidate parking space can be further increased, ensuring the accuracy of determining the first candidate region, the second candidate region, and the third candidate region corresponding to the same candidate parking space based on the intersection over union ratio.

[0125] In one example, the determined parking space line instances in the above figure are all complete and continuous parking space lines. However, in fact, among the determined parking space line instances, the included parking space lines may be discontinuous or incomplete parking space lines, such as Figure 4C shown. Figure 4C This is a schematic flowchart for determining the first candidate region provided by this application. Similar to Figure 4A , after obtaining the parking space line instance, for each orientation of the parking space region, according to the inner edge and outer edge of the parking space line located in this orientation, the middle line located in the center of the parking space line and at the same distance from the inner edge and outer edge can be determined (such as Figure 4C the dotted lines in the middle part shown), as the border line of the first candidate region in this orientation. The difference is that Figure 4A the border lines in are complete. By combining the border lines of the first candidate region in each orientation, the border of the first candidate region can be obtained. However, in Figure 4C , each border line is not complete. Therefore, each border line can be extended respectively, and the region enclosed by the extension lines of each border line is taken as the first candidate region (such as Figure 4C shown in the right part). Figure 4C In the right part, the solid lines represent the border lines of the first candidate region, specifically the middle lines located at the inner edge and outer edge of the parking space line determined according to the parking space line instance output by the segmentation model. The dotted lines are the extension lines of each border line (solid line), which are not the output of the segmentation model. The rectangular boundary (including the solid line part and the dotted line part) of the first candidate region is the border of this first candidate region.

[0126] In one embodiment, generally, when there are no obstacles in the parking space area or the parking space lines are relatively clear, the length of the border lines of each candidate area determined should not have too large a difference from the length of the actual border lines of the parking space area. However, when there are obstacles occupying the parking space area or the parking space lines, or the parking space lines are not clear enough, the border lines of the first candidate area and the second candidate area determined may be affected by the obstacles or clarity, resulting in a shorter length of the border lines of the candidate area determined. Obviously, the length of the border lines can, to a certain extent, represent the confidence level of the parking space. Therefore, before step S106, the parking space confidence levels corresponding to each candidate area can also be determined according to the lengths of the border lines corresponding to each candidate area.

[0127] Specifically, for each of the first candidate area, the second candidate area, and the third candidate area, the lengths of the border lines of the candidate area can be determined. A preset border threshold can be determined, and according to the comparison result between the lengths of the border lines of the candidate area and the preset border threshold, the length confidence levels corresponding to the border lines can be determined. Among them, the preset border threshold can include a first border threshold and a second border threshold.

[0128] In one example, when the parking lot plans the parking spaces, their corresponding lengths and widths usually have corresponding ranges. Taking a standard rectangular parking space as an example, the length of the rectangular parking space is usually set to be 5 - 6 meters, and the width of the rectangular parking space is usually set to be 2.3 - 2.5 meters. Therefore, the first border threshold and the second border threshold can be set according to the ranges of the lengths and widths set during the current parking space planning. Among them, the first border threshold is greater than the second border threshold. Taking the first border threshold as longTH = 2m and the second border threshold as shortTH = 1m as an example.

[0129] For each border line of the candidate area, if the border line is the long side of the rectangle, the length of the border line is compared with the first border threshold. If the border line is the short side of the rectangle, the length of the border line is compared with the second border threshold.

[0130] In one example, if the length of the border line is greater than or equal to its corresponding border threshold, it can be considered that the parking space line corresponding to the border line is relatively clear, or there is no occlusion in the parking space area. Thus, the length confidence level of the border line can be determined to be 1. If the length of the border line is less than its corresponding border threshold, it can be considered that there is an unclear situation with the parking space line corresponding to the border line, or there is a situation where the parking space area is partially occluded. Thus, the length confidence level of the border line can be determined to be less than 1.

[0131] In one example, when the length of the border line is less than its corresponding border threshold, the length confidence of the border line can specifically be the ratio of the length of the border line to its corresponding border threshold. Taking this border line as the short side, for example, the length confidence of the border line can be shortL / shortTH. Wherein, shortL is the length of the short side, and shortTH is the border threshold corresponding to the short side, that is, the second border threshold.

[0132] According to the length confidences respectively corresponding to the border lines in the candidate region, the vehicle position confidence of the candidate region can be determined. Among them, the weights of the border lines can be preset, and according to the weights of the border lines, the length confidences respectively corresponding to the border lines are weighted to obtain the vehicle position confidence of the candidate region. The weights of the border lines can be the same, for example, all are 0.25, or can be different, and the specific values corresponding to them can be set as needed. This specification does not limit this.

[0133] In this way, the vehicle position confidences of the candidate regions can be determined according to the lengths of the border lines of the candidate regions themselves, and then a relatively reliable parking space region can be determined according to the vehicle position confidences of the candidate regions.

[0134] In one embodiment, for each candidate region, each candidate region has its corresponding corner points. Among them, for each candidate region, the corner points of the candidate region are usually the intersection points of the straight lines where the border lines of the long side and the short side corresponding to the candidate region are located. However, during the process of determining the candidate region, especially during the process of determining the first candidate region according to the parking space line instance, there may be a situation where the corner points are not on the border lines of the long side or the short side, specifically as Figure 5 shown.

[0135] Figure 5 This is a schematic structural diagram of the candidate region provided in this specification. Figure 5 Similar to the right part in Figure 4C The black solid lines are the border lines of the candidate region, specifically the intermediate lines located between the inner edge and the outer edge of the parking space line determined according to the parking space line instance output by the segmentation model. The black dotted lines are the extension lines respectively corresponding to the border lines, which are not the outputs of the segmentation model. The black origin points are the corner points. In the figure, points A and B are taken as examples of corner points for illustration. It can be seen that point A is not on the long border line nor on the short border line, and point B is on the long border line but not on the short border line. The closer the distance between the corner point and the border line corresponding to the corner point, the higher the probability that the corner point is a real parking space corner point, and vice versa. Therefore, the probability that point A is a real parking space corner point is lower than the probability that point B is a real parking space corner point. If for each corner point, the probability that it is a real parking space corner point is used as the corner point confidence of the corner point, then according to the corner point confidences of the corner points, the vehicle position confidence corresponding to the candidate region can be determined.

[0136] In one example, for each of the first candidate region, the second candidate region, and the third candidate region, the positions of the corner points of the candidate region can be determined. Furthermore, based on the positions of the corner points, the confidence of the corner points can be determined according to the positions of the corner points. Furthermore, the vehicle position confidence corresponding to the candidate region can be determined according to the confidence of the corner points.

[0137] In one example, the straight line where the border line of the long side corresponding to the candidate region is located and the straight line where the border line of the short side corresponding to the candidate region is located can be determined, and the intersection point of the above two straight lines is used as the corner point of the candidate region. Thus, for each corner point, the border line on the straight line where the corner point is located can be used as the border line corresponding to the corner point.

[0138] In one example, for each border line corresponding to the corner point, if the corner point is on the border line, the confidence of the corner point corresponding to the border line is 1.

[0139] In one example, if the corner point is not on the border line and the minimum distance between the corner point and the end point of the border line is less than or equal to a preset distance threshold, the confidence of the corner point corresponding to the border line is 1.

[0140] In one example, if the corner point is not on the border line and the minimum distance between the corner point and the end point of the border line is greater than the preset distance threshold, the confidence of the corner point corresponding to the border line is less than 1, specifically 1 - minDist / distTH. Where minDist is the minimum distance between the corner point and the end point of the border line, and distTH is the preset distance threshold.

[0141] In one example, according to the confidence of the corner point corresponding to the two border lines respectively, the confidence of the corner point can be determined. The confidence of the corner point can be the mean value of the confidence of the corner point corresponding to the two border lines that make up the corner point respectively. Among them, the confidence of the corner point corresponding to the long side that makes up the corner point is proportional to the confidence of the corner point, and the confidence of the corner point corresponding to the short side that makes up the corner point is proportional to the confidence of the corner point.

[0142] Finally, according to the confidence of the corner points corresponding to each corner point in the candidate region, the vehicle position confidence of the candidate region can be determined.

[0143] Similarly, the weights of each corner point can be preset, and according to the weights of each corner point, the confidence of the corner points corresponding to each corner point is weighted to obtain the vehicle position confidence of the candidate region. The weights of each corner point can be the same, for example, all are 0.25, or they can be different, and the specific values corresponding to them can be set as needed, and this specification does not limit this.

[0144] In this way, the vehicle position confidence of each candidate region can be determined according to the positional relationship between the corner points of each candidate region and the border lines forming the corner points. Furthermore, a relatively reliable parking space region can be determined based on the vehicle position confidence of each candidate region.

[0145] In one embodiment, for each of the first candidate region, the second candidate region, and the third candidate region, after determining the length confidence of each border line and the corner point confidence of each corner point in the candidate region, the length confidence and the corner point confidence can be weighted to determine the vehicle position confidence of this candidate region.

[0146] It should be noted that this application is actually a method for determining all the parking spaces existing in the environmental image. As for determining the occupied and unoccupied parking spaces from the detected parking spaces, or determining the parking spaces that the vehicle needs to park in from the detected parking spaces, these are all subsequent steps of the method described in this application.

[0147] In one embodiment, when there are multiple parking spaces included in the environmental image, all the parking spaces existing in the environmental image are regarded as candidate parking spaces. Then, the environmental image may include multiple candidate parking spaces. For each candidate parking space, according to the first candidate region, the second candidate region, and the third candidate region corresponding to each candidate parking space, the target region corresponding to each candidate parking space is determined.

[0148] In one example, the intersection-over-union ratios corresponding to each pair of the first candidate regions, the second candidate regions, and the third candidate regions can be determined, and based on the determined intersection-over-union ratios, the first candidate region, the second candidate region, and the third candidate region belonging to the same candidate parking space are determined.

[0149] Among them, for each candidate region, if there is another candidate region whose intersection-over-union ratio with this candidate region is higher than a preset intersection-over-union ratio threshold, then this other candidate region corresponds to the same candidate parking space as this candidate region.

[0150] Then, for each candidate parking space, based on the vehicle position confidence corresponding to the first candidate region, the second candidate region, and the third candidate region of this candidate parking space respectively, the target region corresponding to this candidate parking space is determined.

[0151] Among them, if there are two or more candidate regions among the candidate regions corresponding to each candidate parking space, the candidate region with the highest confidence is selected from the candidate regions corresponding to each candidate parking space as the target region corresponding to this candidate parking space. If there is only one candidate region corresponding to this candidate parking space, then this candidate region is directly used as the target region corresponding to this candidate parking space.

[0152] Finally, according to the target regions respectively corresponding to each candidate parking space, each parking space region included in the environmental image can be determined, that is, the target regions respectively corresponding to each candidate parking space can be used as each parking space region included in the detected environmental image. Among them, each candidate parking space is each parking space existing in the environmental image. For each candidate parking space, the target region of the candidate parking space is the parking space region of the candidate parking space. For each candidate region, if there is another candidate region whose intersection over union ratio with the candidate region is higher than a preset intersection over union ratio threshold, then the other candidate region and the candidate region correspond to the same candidate parking space.

[0153] In one example, for each first candidate region, the intersection over union ratio between each second candidate region and the first candidate region can be determined, and when there is a second candidate region whose intersection over union ratio with the first candidate region is greater than the preset intersection over union ratio threshold, it is determined that the second candidate region corresponds to the same candidate parking space as the first candidate region.

[0154] In one example, after determining the first candidate region and the second candidate region corresponding to the same candidate parking space, for each third candidate region, the intersection over union ratio between each third candidate region and the first candidate region, and the intersection over union ratio between each third candidate region and the second candidate region can be determined respectively. When there is a third candidate region whose intersection over union ratio with the first candidate region is greater than the preset intersection over union ratio threshold and whose intersection over union ratio with the second candidate region is greater than the preset intersection over union ratio threshold, it is determined that the third candidate region corresponds to the same candidate parking space as the first candidate region and the second candidate region.

[0155] In one example, the above takes the case of simultaneously determining three candidate regions corresponding to the same candidate parking space as an example for illustration. However, in actual determination of the candidate regions corresponding to the same candidate parking space, there are often situations where only two candidate regions match each other, or for a certain candidate region, there is no other candidate region whose intersection over union ratio with the candidate region is higher than the intersection over union ratio threshold. Therefore, when determining the candidate regions corresponding to the same candidate parking space, the above-mentioned two mutually matching candidate regions can also be used as the candidate regions corresponding to the same candidate parking space, or the candidate region for which there is no other candidate region with an intersection over union ratio greater than the intersection over union ratio threshold can be used as the candidate region corresponding to a candidate parking space. That is to say, for each candidate parking space, the number of candidate regions corresponding to the candidate parking space can be one or more.

[0156] In another example, when the third candidate region is determined based on the first candidate region and / or the second candidate region, the third candidate region can be directly used as the third candidate region corresponding to the same candidate parking space as the first candidate region and / or the second candidate region used to determine the third candidate region.

[0157] Therefore, the intersection-and-union ratios corresponding to each first candidate area and each second candidate area can be determined, and the first candidate area and the second candidate area belonging to the same candidate parking space can be determined according to the determined intersection-and-union ratios. If the intersection-and-union ratio is higher than a preset intersection-and-union ratio threshold, the first candidate area and the second candidate area belong to the same candidate parking space.

[0158] Next, if the third candidate area is determined by the first candidate area, it can be directly considered that the first candidate area and the third candidate area belong to the same candidate parking space. Therefore, the second candidate area belonging to the same candidate parking space as the first candidate area can be directly determined based on the intersection-and-union ratio between the first candidate area and each second candidate area.

[0159] Finally, the first candidate area, the second candidate area, and the third candidate area belonging to the same candidate parking space can be directly determined.

[0160] It should be understood that, in the case of determining the third candidate area through the second candidate area, technical means similar to those described above may be used to determine the first candidate area, the second candidate area, and the third candidate area belonging to the same candidate parking space.

[0161] In another example, before constructing the third candidate area, the first candidate area and the second candidate area that match each other can be directly determined according to the intersection-over-union ratio, that is, the first candidate area and the second candidate area that belong to the same candidate parking space, and then the parking space inner angle is determined according to the first candidate area and the second candidate area that match each other. Finally, the third candidate area is constructed according to the determined parking space inner angle and the parking space corner point output by the aforementioned detection model. In this way, the third candidate area that belongs to the same candidate parking space as the first candidate area and the second candidate area can be obtained.

[0162] In one example, when determining the inner angle of a parking space according to the first candidate area and the second candidate area belonging to the same candidate parking space, the inner angle of the parking space may be the inner angle of the parking space of any one of the first candidate area and the second candidate area.

[0163] In addition, after determining the parking space areas included in the environment image, the target parking space can be selected from the parking space areas, and the vehicle can be controlled to drive to the target parking space by the position of the target parking space in the environment image and the position of the vehicle itself. Among them, the relative position between the vehicle and the target parking space can be determined according to the position of the target parking space in the environment image and the vehicle's own position, so as to control the vehicle to drive to the target parking space. The spatial three-dimensional coordinates of the target parking space can also be determined according to the position of the target parking space in the environment image, and then the vehicle can be controlled to drive to the target parking space according to the spatial three-dimensional coordinates of the target parking space and the spatial three-dimensional coordinates of the vehicle.

[0164] Figure 6Schematic flowchart of the parking space detection method provided in this specification. The parking space detection method may include the following steps:

[0165] S200: Synthesize a panoramic image based on the image data collected by the acquisition device arranged around the vehicle, and use it as the environmental image of the vehicle.

[0166] S202: Input the environmental image into the pre-trained segmentation model and detection model respectively to obtain the parking space line instances, parking space area instances, and parking space corner points in the environmental image.

[0167] S204: Determine the edge feature points of the parking space line instances, and based on the edge feature points of the parking space line instances, fit the middle lines of each parking space line instance, group each parking space line instance, determine the parking space line instances belonging to the same parking space area, and then construct the first candidate area based on the middle lines of the parking space line instances belonging to the same parking space.

[0168] S206: Determine the edge feature points of the parking space area instance, and based on the edge feature points of the parking space area instance, fit the border line of the second candidate area to obtain the second candidate area.

[0169] S208: Determine the interior angles of the parking space based on the first candidate area and / or the second candidate area, and construct the third candidate area based on the interior angles of the parking space and the parking space corner points.

[0170] S210: Modify the second candidate area based on the first candidate area and / or the third candidate area.

[0171] S212: Determine the parking space confidence levels corresponding to the first candidate area, the modified second candidate area, and the third candidate area respectively, and determine the target area based on the parking space confidence levels corresponding to the first candidate area, the modified second candidate area, and the third candidate area respectively. Determine the target area as the detected parking space area.

[0172] In one or more embodiments provided in this specification, for the process of steps S200 to S212, reference may be made to the descriptions of the above steps S100 to S106, and this specification will not repeat them here.

[0173] The above is the parking space detection method provided in one or more embodiments of this specification. Based on the same idea, this specification also provides a parking space detection device, as Figure 7 shown.

[0174] Figure 7 Schematic structural diagram of the parking space detection device provided in this specification. The parking space detection device may include:

[0175] An acquisition module 300 is configured to determine an environmental image of a vehicle, which is synthesized from image data collected by acquisition devices arranged around the vehicle.

[0176] An instance determination module 302 is configured to use the environmental image as an input and input it into a pre-trained segmentation model and a detection model respectively, to obtain a parking space line instance and a parking space area instance in the environmental image output by the segmentation model, and to obtain parking space corner points included in the environmental image output by the detection model.

[0177] A region determination module 304 is configured to obtain a first candidate region according to the parking space line instance, obtain a second candidate region according to the parking space area instance, and obtain a third candidate region according to the parking space corner points.

[0178] A parking space area determination module 306 is configured to determine a target region as the detected parking space area according to the parking space confidence levels respectively corresponding to the first candidate region, the second candidate region, and the third candidate region.

[0179] Optionally, the segmentation model is a multi-object segmentation model; the instance determination module 302 is configured to: use the environmental image as an input, input it into the feature extraction layer of the segmentation model, obtain image features output by the feature extraction layer, and input the image features into the first instance segmentation layer and the second instance segmentation layer of the segmentation model respectively, to obtain the parking space line instance output by the first instance segmentation layer and the parking space area instance output by the second instance segmentation layer.

[0180] Optionally, the number of parking space corner points included in the environmental image output by the detection model is at least two; the region determination module 304 is configured to: determine an interior angle of the parking space according to at least one of the first candidate region and the second candidate region; and construct the third candidate region according to the interior angle of the parking space and at least two parking space corner points.

[0181] Optionally, the border line of the first candidate region is determined by the middle line of the parking space line instance, the border line of the second candidate region is determined by the edge line of the parking space area instance, and the parking space corner points are the corner points of the border formed by the middle line between the inner and outer edges of the parking space line in the environmental image; the region determination module 304 is configured to: correct the second candidate region according to the position of the border line of the first candidate region, or correct the second candidate region according to the position of the corner points of the third candidate region.

[0182] Optionally, the parking space area determination module 306 is configured to: for each of the first candidate region, the second candidate region, and the third candidate region, determine the parking space confidence level of the candidate region according to at least one of the length confidence levels of the border lines of the candidate region and the corner confidence levels of the corner points in the candidate region.

[0183] Optionally, the parking space area determination module 306 is configured to: for each of the first candidate area, the second candidate area, and the third candidate area, determine the lengths of the respective border lines of the candidate area; according to the lengths of the respective border lines of the candidate area and a preset border threshold, determine the length confidence levels respectively corresponding to the respective border lines; and according to the length confidence levels respectively corresponding to the respective border lines, determine the parking space confidence level of the candidate area.

[0184] Optionally, the parking space area determination module 306 is configured to: for each of the first candidate area, the second candidate area, and the third candidate area, determine the positions of the respective corner points of the candidate area; for each corner point among the respective corner points of the candidate area, determine the border line corresponding to the corner point; according to the position of the corner point, the position of the border line corresponding to the corner point, and a preset distance threshold, determine the corner point confidence level of the corner point; and perform weighting on the corner point confidence levels respectively corresponding to the respective corner points to obtain the parking space confidence level of the candidate area.

[0185] Optionally, the environmental image includes a plurality of candidate parking spaces; the parking space area determination module 306 is configured to: according to the intersection-over-union ratio between each of the first candidate areas, each of the second candidate areas, and each of the third candidate areas detected in the environmental image, determine the first candidate area, the second candidate area, and the third candidate area respectively corresponding to each candidate parking space; for each candidate parking space, according to the parking space confidence levels respectively corresponding to the first candidate area, the second candidate area, and the third candidate area corresponding to the candidate parking space, determine the target area corresponding to the candidate parking space; and use the target areas respectively corresponding to each candidate parking space as the detected parking space areas.

[0186] The specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-mentioned parking space detection method.

[0187] The present specification also provides Figure 8 a schematic structural diagram of the electronic device shown. As Figure 8 described above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory. Of course, it may also include other hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned parking space detection method. Of course, in addition to the software implementation, the present specification does not exclude other implementation manners, such as a logic device or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logical unit, and may also be hardware or a logic device.

[0188] The systems, devices, modules, or units described in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0189] For the convenience of description, when describing the above devices, they are described as various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0190] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0191] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0192] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device realizes the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0193] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 steps of the functions specified in one block or multiple blocks.

[0194] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0195] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0196] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0197] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, commodity or device including the said element.

[0198] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0199] This specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification can also be practiced in a distributed computing environment where tasks are performed by remote processing nodes connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage nodes.

[0200] The embodiments in this specification are described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference can be made to the corresponding parts of the method embodiments for the relevant content.

[0201] The above description is only for the embodiments of this specification and is not intended to limit this specification. For those skilled in the art, various changes and modifications can be made to this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included within the scope of the claims of this specification.

Claims

1. A parking space detection method, characterized in that: include: Determine an environmental image of the vehicle, wherein the environmental image is synthesized from image data collected by a collection device arranged around the vehicle; The environment image is used as input, and is input into a pre-trained segmentation model and a detection model respectively, to obtain parking space line instances and parking space area instances in the environment image output by the segmentation model, and to obtain parking space corner points contained in the environment image output by the detection model; According to the parking space line instance, a first candidate region is obtained, according to the parking space region instance, a second candidate region is obtained, and according to the parking space corner point, a third candidate region is obtained; The detected parking space area is determined according to the vehicle position confidences respectively corresponding to the first candidate area, the second candidate area and the third candidate area.

2. The parking space detection method according to claim 1, characterized in that: The segmentation model is a multi-objective segmentation model; The environmental image is used as an input and input into the segmentation model to obtain the parking space line instance and the parking space area instance in the environmental image output by the segmentation model, including: Taking the environment image as input, inputting it into the feature extraction layer of the segmentation model, and obtaining the image features output by the feature extraction layer; The image features are respectively input into the first instance segmentation layer and the second instance segmentation layer of the segmentation model to obtain the parking space line instance output by the first instance segmentation layer and the parking space area instance output by the second instance segmentation layer.

3. The parking space detection method according to claim 1, characterized in that: The number of parking space corner points contained in the environment image output by the detection model is at least two; According to the parking space corner point, the third candidate area is obtained, including: determining an inner angle of the parking space according to at least one of the first candidate area and the second candidate area; The third candidate area is constructed according to the parking space inner angle and at least two parking space corner points.

4. The parking space detection method according to claim 1, characterized in that: The border line of the first candidate area is determined by the middle line of the parking space line instance, the border line of the second candidate area is determined by the edge line of the parking space area instance, and the parking space corner point is a corner point of the border formed by the middle line between the inner and outer edges of the parking space line in the environment image output by the detection model; The method further comprises: According to the border line position of the first candidate area, the second candidate area is corrected. or, The second candidate region is modified according to the corner point positions of the third candidate region.

5. The parking space detection method according to claim 1, characterized in that: Also includes: For each candidate area in the first candidate area, the second candidate area and the third candidate area, the vehicle position confidence of the candidate area is determined according to at least one of the length confidence of each border line of the candidate area and the corner point confidence of each corner point in the candidate area.

6. The parking space detection method according to claim 5, characterized in that: Determining the vehicle position confidence of the candidate area according to at least one of the length confidence of each border line of the candidate area and the corner point confidence of each corner point in the candidate area includes: For each candidate area among the first candidate area, the second candidate area and the third candidate area, determining the length of each border line of the candidate area; Determining the length confidences corresponding to the respective border lines of the candidate area according to the lengths of the respective border lines and a preset border threshold; The vehicle position confidence of the candidate area is determined according to the length confidence corresponding to each of the border lines.

7. The parking space detection method according to claim 5, characterized in that: Determining the vehicle position confidence of the candidate area according to at least one of the length confidence of each border line of the candidate area and the corner point confidence of each corner point in the candidate area includes: For each candidate area among the first candidate area, the second candidate area and the third candidate area, determining positions of corner points of the candidate area; For each corner point of the candidate area, determining a border line corresponding to the corner point; Determining a corner point confidence of the corner point according to the position of the corner point, the position of the border line corresponding to the corner point, and a preset distance threshold; The corner point confidences corresponding to the corner points are weighted to obtain the vehicle position confidence of the candidate area.

8. The parking space detection method according to any one of claims 1 to 7, characterized in that: The environment image contains a plurality of candidate parking spaces. Determining the detected parking space area according to the parking position confidences respectively corresponding to the first candidate area, the second candidate area, and the third candidate area includes: Determine the first candidate area, the second candidate area, and the third candidate area corresponding to each candidate parking space, respectively, according to the intersection-and-union ratios among each first candidate area, each second candidate area, and each third candidate area detected in the environment image; For each candidate parking space, determine the target area corresponding to the candidate parking space according to the parking position confidences corresponding to the first candidate area, the second candidate area, and the third candidate area respectively corresponding to the candidate parking space; The target areas corresponding to the candidate parking spaces are taken as the detected parking space areas.

9. A parking space detection device, characterized in that: include: An acquisition module, used to determine an environmental image of the vehicle, wherein the environmental image is synthesized from image data acquired by acquisition devices arranged around the vehicle; An instance determination module is used to input the environment image as an input into a pre-trained segmentation model and a detection model, respectively, to obtain parking space line instances and parking space area instances in the environment image output by the segmentation model, and to obtain parking space corner points contained in the environment image output by the detection model; An area determination module, used to obtain a first candidate area according to the parking space line instance, obtain a second candidate area according to the parking space area instance, and obtain a third candidate area according to the parking space corner point; The parking area determination module is used to determine the detected parking area according to the car position confidences corresponding to the first candidate area, the second candidate area and the third candidate area respectively.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

Images