Parking space detection method, device and equipment and storage medium

Abstract

The invention discloses a parking space detection method and device, equipment and a storage medium. The parking space detection method comprises the following steps: acquiring a plurality of frames of images; performing parking space extraction processing on each image to obtain at least one piece of parking space information in each image; and carrying out fusion processing on the parking space information belonging to the same parking space in the plurality of frames of images to obtain target parking space information of each parking space. According to the scheme, the accuracy of obtaining the detected target parking space information of each parking space can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a parking space detection method, apparatus, device, and storage medium. Background Technology

[0002] Currently, parking space detection can be achieved through non-visual inspection methods. These methods primarily use ultrasonic sensors, short-range radar, or laser scanners to directly determine the parking space. However, non-visual inspection methods cannot accurately identify parking space information when there are no cars in adjacent spaces, resulting in low accuracy in determining parking space information.

[0003] Therefore, a parking space detection method is urgently needed. Summary of the Invention

[0004] This application provides at least one parking space detection method, apparatus, device, and storage medium.

[0005] This application provides a parking space detection method, including: acquiring several frames of images; performing parking space extraction processing on each image to obtain at least one parking space information in each image; and fusing the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information for each parking space.

[0006] In some embodiments, the parking space information includes candidate corner information of the parking space. The step of performing parking space extraction processing on each image to obtain at least one parking space information in each image includes: performing the following steps on each frame image: obtaining the segmentation result of the image; performing line extraction on the segmentation result to obtain the line extraction result of each parking space in the image, wherein the line extraction result includes the initial corner information of the parking space; adjusting the initial corner information to obtain candidate corner information of the parking space.

[0007] In some embodiments, the step of extracting straight lines from the segmentation results to obtain the straight line extraction results for each parking space in the image includes: determining the parking stage of the vehicle, where the parking stage includes a parking entry stage or a parking space search stage; determining the pixel search range for each parking space in the image based on the parking stage; and for each parking space, extracting straight lines from the segmentation results within the pixel search range of the parking space to obtain the initial corner point information corresponding to the parking space.

[0008] In some embodiments, the step of determining the pixel search range of each parking space in the image based on the parking stage includes: in response to the parking stage being in the parking space search stage, using the image range corresponding to the detection box of each parking space in the image as the pixel search range of each parking space; in response to the parking stage being in the parking entry stage, using the image range corresponding to the historical corner point information of each parking space as the pixel search range of each parking space.

[0009] In some embodiments, the straight line extraction result further includes line segments of the parking space in at least one direction, and the initial corner information includes at least one corner point in the parking space. The step of adjusting the initial corner information to obtain candidate corner information of the parking space includes: performing the following steps for each corner point in the parking space: in response to the number of line segments passing through the corner point being two, using the initial corner information as candidate corner information of the parking space; in response to the number of line segments passing through the corner point being less than two, determining the preset corner information corresponding to the corner point based on the parking stage of the vehicle, and using the preset corner information as candidate corner information of the parking space.

[0010] In some embodiments, the straight line extraction result further includes line segments of the parking space in at least two directions. The initial corner point information includes the entry line corner point and the non-entry line corner point in the parking space. The candidate corner point information includes the target corner point corresponding to the non-entry line corner point. The step of adjusting the initial corner point information to obtain the candidate corner point information of the parking space includes: obtaining the line segment angle of each line segment in the straight line extraction result; determining the adjustment angle of the parking space based on the line segment angle; projecting the non-entry line corner point onto the baseline to obtain a new non-entry line corner point, and using the new non-entry line corner point as the target corner point corresponding to the non-entry line corner point. The baseline represents a straight line with the entry line corner point as the origin and the adjustment angle as the line segment angle.

[0011] In some embodiments, the step of fusing parking space information belonging to the same parking space in several frames of images to obtain target parking space information for each parking space includes: performing the following steps for each parking space: taking the parking space information belonging to the parking space in several frames of images as the parking space to be fused information set, the parking space to be fused information set includes candidate corner point information corresponding to at least one frame of images; determining the weight of each candidate corner point information in the to-be-fused information set; and performing weighted fusion of each candidate corner point information in the to-be-fused information set based on the weight of each candidate corner point information in the to-be-fused information set to obtain the target corner point information corresponding to the parking space.

[0012] This application provides a parking space detection device, including: an acquisition module, an extraction module, and a fusion module; the acquisition module is used to acquire several frames of images; the extraction module is used to perform parking space extraction processing on each image to obtain at least one parking space information in each image; the fusion module is used to fuse the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information of each parking space.

[0013] This application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-described parking space detection method.

[0014] This application provides a computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the above-described parking space detection method.

[0015] Compared to non-visual detection methods, the above-mentioned scheme has lower accuracy in detecting parking spaces. This application performs parking space extraction processing on each frame of the acquired images to obtain at least one parking space information in each image. By fusing the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information of each parking space, the accuracy of the target parking space information of each parking space can be improved.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0018] Figure 1 This is a flowchart illustrating an embodiment of the parking space detection method of this application;

[0019] Figure 2 yes Figure 1 Sub-flowchart of step S12;

[0020] Figure 3 yes Figure 2 Sub-flowchart of step S22;

[0021] Figure 4 yes Figure 2 Sub-flowchart of step S23;

[0022] Figure 5 yes Figure 1 Sub-flowchart of step S13;

[0023] Figure 6a This is a schematic diagram of the framework of an embodiment of the parking space detection method of this application;

[0024] Figure 6b This is a schematic diagram of a new non-entry line corner point in one embodiment of the parking space detection method of this application;

[0025] Figure 7 This is a schematic diagram of the structure of an embodiment of the parking space detection device of this application;

[0026] Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device of this application;

[0027] Figure 9 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0028] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0029] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0030] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0031] This application provides several parking space detection methods and devices. The application scenarios of these parking space detection methods include, but are not limited to, detecting parking spaces in the environment where vehicles are located. The executing entity of the parking space detection method can be a parking space detection device. For example, the parking space detection device can be installed in a terminal device, server, other processing equipment, or vehicle. The terminal device can be a device for parking space detection, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle equipment, etc. In some possible implementations, the parking space detection method can be implemented by a processor calling computer-readable instructions stored in memory.

[0032] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the parking space detection method of this application.

[0033] Specifically, parking space detection methods may include the following steps:

[0034] Step S11: Acquire several frames of images.

[0035] Each image can be an image captured by at least one image acquisition device on the vehicle. Several frames of images can be images randomly acquired by the image acquisition device within a preset time period or acquired at predetermined time intervals. For example, several frames of images are consecutive frames of images captured by the image acquisition device. The vehicle can be a vehicle that needs to perform parking space detection in its surrounding environment. At least one can refer to one or more. The image acquisition device is a device located outside the vehicle and capable of capturing images of the vehicle's surrounding environment. Different image acquisition devices are located at different positions outside the target vehicle. It is understood that the images captured by different image acquisition devices are images captured from different perspectives. Several frames of images are images captured by each image acquisition device at the current moment regarding the vehicle's surrounding environment. The number of several frames of images is less than or equal to the number of the at least one image acquisition device. The vehicle's surrounding environment can refer to the physical space around the vehicle. For example, when there are multiple image acquisition devices, a single frame of image can include sub-images captured by each image acquisition device at the same time. For example, step S11 can be that at the current moment, the vehicle's corresponding in-vehicle system can trigger each image acquisition device to perform image acquisition as needed. For example, in an autonomous driving system, each image acquisition device can work continuously to capture images of the target's environment in real time. For instance, the image acquisition devices may be triggered to capture images when the vehicle is in a specific driving state (such as when the vehicle is parking or searching for a parking space).

[0036] Step S12: Perform parking space extraction processing on each image to obtain at least one parking space information in each image.

[0037] Parking space extraction processing can involve extracting relevant information about parking spaces from each image to obtain at least one parking space information in each image. Each image may include at least one parking space information. Each parking space information corresponds to the relevant information of any parking space at the time the image was captured. Parking space information represents the relevant information of any parking space at the time the image was captured. In some application scenarios, parking space information includes candidate corner point information. Candidate corner point information includes four candidate corner points and their position information in the image. The line segment obtained by connecting any two adjacent candidate corner points is used as the parking space line in any direction in the image. In other application scenarios, parking space information includes candidate line segment information and candidate corner point information related to the parking space. Candidate line segment information includes four candidate line segments and their position information. The position of each candidate line segment can be the position of the parking space line in any direction in the image obtained by connecting any two adjacent candidate corner points.

[0038] In some application scenarios, step S12 above can be performed on each image as follows: Preset feature extraction is performed on the image to obtain the image segmentation result. Line extraction is then performed on the segmentation result to obtain the line extraction results for each parking space in the image, and these line extraction results are directly used as the parking space information for each parking space in the image. Preset feature extraction can involve inputting the image into a preset feature extraction module. The preset feature extraction module is used to extract features from the input image. Specifically, the preset feature extraction module can be equipped with a preset feature extraction network. The preset feature extraction network can be a convolutional neural network, a recurrent neural network, a long short-term memory network, a gated recurrent unit, a BiGRU neural network, an attention mechanism, or other feature extraction networks.

[0039] Step S13: Merge the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information for each parking space.

[0040] Parking space information belonging to the same parking space in several frames of images is taken as the parking space information set corresponding to that parking space. The parking space information set corresponding to each parking space represents at least a portion of the parking space information corresponding to that parking space, resulting in a parking space information sequence. It can be understood that each parking space corresponds to at least one parking space information, and any parking space information corresponding to each parking space includes candidate corner point information and / or candidate line segment information. Each parking space corresponds to one target parking space information, which includes target line segment information and / or target corner point information. The target line segment information includes four target line segments and the position information of each target line segment. The target corner point information includes four target corner points and the position information of each target corner point. For any target parking space information, the line segment formed by connecting any two adjacent candidate corner points is the target line segment of the parking space in any direction.

[0041] In some application scenarios, the parking space information set corresponding to each parking space includes parking space information contained in at least one image. For example, the parking space information set may contain all candidate corner point information corresponding to the same parking space. Step S13 above may be: for each parking space information set, the candidate corner point information of that parking space in the parking space information set is fused to obtain the target corner point information of that parking space. Specifically, for each parking space information set, the candidate corner point information of each direction of that parking space in the parking space information set is fused to obtain the target corner point information corresponding to that direction. And the target corner point information corresponding to all directions is taken as the target corner point information of that parking space.

[0042] Compared to non-visual detection methods, the above-mentioned scheme has lower accuracy in detecting parking spaces. This application performs parking space extraction processing on each frame of the acquired images to obtain at least one parking space information in each image. By fusing the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information of each parking space, the accuracy of the target parking space information of each parking space can be improved.

[0043] Please see Figure 2 , Figure 2 yes Figure 1 The sub-flowchart of step S12.

[0044] In some embodiments, the parking space information includes candidate corner point information for parking spaces, and step S12 may include the following steps: performing the following steps on each frame of the image: Figure 2 The following steps are shown:

[0045] Step S21: Obtain the segmentation result of the image.

[0046] The segmentation result represents the image as divided into multiple regions with visual features related to parking spaces, each region consisting of a set of consecutive pixels. It can be understood that the image segmentation result includes image features of all parking spaces contained within the image. The image segmentation result can be obtained by inputting the image into the aforementioned preset feature extraction module for feature extraction, and using the output of the preset feature extraction module as the image segmentation result.

[0047] Step S22: Extract straight lines from the segmentation results to obtain the straight line extraction results for each parking space in the image.

[0048] The line extraction result includes the initial corner point information of the parking space. The line extraction result characterizes the attribute information of the line segments and corner points obtained by extracting lines from the image segmentation results. The attribute information includes quantity and position information. The initial corner point information includes relevant information about the corner points of any parking space obtained by extracting lines from the image segmentation results.

[0049] In some application scenarios, step S22 above may include the following steps: extracting lines from the segmentation results of the image according to at least one of several preset line extraction algorithms to obtain several detection line segments; determining each predicted line segment based on the several detection line segments; and determining the line extraction results of each parking space in the image based on each predicted line segment.

[0050] In some application scenarios, based on several detected line segments, each predicted line segment is determined, including: determining the average line width and the average line length corresponding to several detected line segments; retaining the detected line segments that do not meet the line segment selection criteria, obtaining at least one retained line segment, and using each retained line segment as a predicted line segment. The line segment selection criteria are related to the line width and / or the line length of the detected line segments.

[0051] Specifically, the method for extracting lines from the image segmentation results to obtain several detected line segments can be achieved by using a line extraction module to extract lines from the image segmentation results input to the line extraction module. The input to the line extraction module is the image segmentation results. A preset line extraction algorithm is set on the line extraction module. The line extraction result can contain the position information of at least one predicted or detected line segment in the image. The preset line extraction algorithm can be a Hough transform or a line segment detection algorithm (LSD, LSD: a Line Segment Detector). When using multiple preset line extraction algorithms, the line extraction results from different preset line extraction algorithms are fused and used as the line extraction results for each parking space in the image.

[0052] Understandably, after line extraction, each detected line segment can be directly used as a predicted line segment. In other application scenarios, after line extraction, the detected line segments can be filtered to obtain the predicted line segments.

[0053] Please see Figure 3 , Figure 3 yes Figure 2 The sub-flowchart of step S22.

[0054] In some embodiments, step S22 above may include the following steps:

[0055] Step S31: Determine the parking phase of the vehicle.

[0056] The parking process includes either the parking entry phase or the parking space search phase. The parking phase represents the vehicle's current driving state. The parking entry phase represents the vehicle being in the process of entering a parking space. The parking space search phase represents the vehicle being in the process of searching for a parking space.

[0057] In some application scenarios, environmental detection is performed on several frames of images to obtain environmental detection results. These results characterize the environment in which the vehicle is located. If the vehicle's environment corresponds to the parking phase, the parking phase is determined to be the parking phase. If the vehicle's environment corresponds to the parking space search phase, the parking phase is determined to be the parking space search phase. In other application scenarios, the parking phase is determined based on the type of parking-related instruction received. The parking instruction characterizes the vehicle's initiation instruction for parking space detection. The types of parking instructions include those corresponding to the parking phase or the parking space search phase. If the type of parking instruction corresponds to the parking phase, the parking phase is determined to be the parking phase. If the type of parking instruction corresponds to the parking space search phase, the parking phase is determined to be the parking phase.

[0058] Step S32: Based on the parking phase, determine the pixel search range for each parking space in the image.

[0059] The image includes the pixel search range for each parking space contained within it. The pixel search range varies depending on the number of parking spaces in the image. The pixel search range for each parking space represents the image region corresponding to that parking space and is used for straight-line extraction of the parking space.

[0060] In some application scenarios, a pre-established association is established between the parking stage and several preset pixel search ranges. When the parking stage is in the parking entry phase, the preset image range corresponding to the parking stage is used as the pixel search range for each parking space. When the parking stage is in the parking space search phase, the preset image range corresponding to the parking space search phase is used as the pixel search range for each parking space.

[0061] In some embodiments, step S32 may include the following steps: In response to the parking stage being in the parking space search stage, the image range corresponding to the detection box of each parking space in the image is used as the pixel search range for each parking space. In response to the parking stage being in the parking entry stage, the image range corresponding to the historical corner point information of each parking space is used as the pixel search range for each parking space.

[0062] Object detection is performed on the image to obtain detection boxes for each parking space. The image range containing the detection box for each parking space, or the adjusted image range containing the detection box, is used as the pixel search range for that parking space. Object detection can be performed by inputting the image into the aforementioned preset feature extraction module, which outputs the detection boxes for each parking space. The adjusted image range containing the detection box can be larger or smaller than the actual image range of the detection box. The image range corresponding to the detection box for each parking space represents the image region where the detection box for that parking space is located.

[0063] For each image, the historical corner information of the parking space represents the target corner information of the previous frame image, or the corner information randomly selected from the target corner information of several historical frames, or selected according to the weight of the target corner information of the historical frames. It is understood that this application uses the example of historical corner information representing the target corner information with the highest weight among the target corner information of several historical frames. For each parking space in each image, when the parking phase is in the parking space search phase, the image range corresponding to the detection box of that parking space is used as the pixel search range for that parking space. When the parking phase is in the parking entry phase, the image range corresponding to the historical corner information of that parking space is used as the pixel search range for that parking space.

[0064] Step S33: For each parking space, extract straight lines from the segmentation results within the pixel search range of the parking space to obtain the initial corner information corresponding to the parking space.

[0065] In some application scenarios, for each parking space in each image, during the parking phase (when searching for a parking space), straight lines are extracted from the segmentation results within the image range corresponding to the detection box of that parking space to obtain the initial corner information of that parking space. In other application scenarios, for each parking space in each image, during the parking phase (when entering the parking space), straight lines are extracted from the segmentation results within the image range corresponding to the historical corner information of that parking space to obtain the initial corner information of that parking space. It is understood that the specific straight line extraction method can refer to the above content and use at least one of several preset straight line extraction algorithms to extract straight lines from the segmentation results within the image range corresponding to the parking phase / parking space search phase, obtaining several detection line segments; based on these detection line segments, predictive line segments are determined; and based on these predictive line segments, the straight line extraction result for each parking space in the image is determined. This will not be elaborated further here.

[0066] Step S23: Adjust the initial corner information to obtain the candidate corner information of the parking space.

[0067] Candidate corner information represents the relevant information of the corner obtained by adjusting the initial corner information of the parking space according to preset rules.

[0068] In some application scenarios, the initial corner information is adjusted using a first adjustment method and / or a second adjustment method. The first adjustment method is related to the number of line segments passing through the corner. The second adjustment method is related to the angles of each line segment in the line extraction result.

[0069] In some embodiments, the straight line extraction result of the parking space also includes line segments of the parking space in at least one direction, and the initial corner information includes at least one corner point in the parking space. Step S23 above may include the following steps: For each corner point in the parking space, perform the following steps: In response to the number of line segments passing through the corner point being two, use the initial corner information as candidate corner information for the parking space. In response to the number of line segments passing through the corner point being less than two, determine the preset corner information corresponding to the corner point based on the parking stage of the vehicle, and use the preset corner information as candidate corner information for the parking space.

[0070] The number of line segments passing through a corner point is not two, indicating that the number of line segments passing through the corner point is one or no line segments passing through the corner point. The preset corner point information represents the corner point information in the detection box of the parking space in the image or the corner point information in the historical corner point information of the parking space.

[0071] In some application scenarios, the preset corner information corresponding to corner points is determined based on the parking stage of the vehicle. This includes: for each parking space in each image, when the parking stage is in the parking space search phase, the corner information in the detection box of that parking space in the image is used as the preset corner information corresponding to that parking space. When the parking stage is in the parking entry phase, the corner information in the historical corner information of that parking space is used as the preset corner information corresponding to that parking space.

[0072] The first adjustment method mentioned above includes: traversing each corner point in the initial corner point information of the parking space; if the number of line segments passing through the corner point is two, using the initial corner point information as candidate corner point information for the parking space; if the number of line segments passing through the corner point is not two and the parking stage is in the parking space search stage, using the corner point information in the detection box of the parking space in the image as the preset corner point information corresponding to the parking space; if the number of line segments passing through the corner point is not two and the parking stage is in the parking entry stage, using the corner point information in the historical corner point information of the parking space as the preset corner point information corresponding to the parking space.

[0073] Please see Figure 4 , Figure 4 yes Figure 2 The sub-flowchart of step S23.

[0074] In some embodiments, the straight line extraction result also includes line segments of the parking space in at least two directions. The initial corner point information includes the entry line corner points and non-entry line corner points in the parking space, and the candidate corner point information includes the target corner points corresponding to the non-entry line corner points. Step S23 may include the following steps: Step S41: Obtain the line segment angles of each line segment in the straight line extraction result. Step S42: Determine the adjustment angle of the parking space based on the line segment angles. Step S43: Project the non-entry line corner points onto the baseline to obtain new non-entry line corner points, and use the new non-entry line corner points as the target corner points corresponding to the non-entry line corner points.

[0075] The direction of the parking space refers to the direction corresponding to the line segment connecting two adjacent corner points. The entry line corner point represents at least one of the corner points corresponding to the entry line when the vehicle is parking, as shown in the initial corner point information. The non-entry line corner point represents at least one of the corner points corresponding to the line segment far from the entry line in the initial corner point information. The new non-entry line corner point represents the corner point obtained by projecting the non-entry line corner point onto the baseline. The line segment angle is the angle between the line segment and a preset line segment in the coordinate system of the image. For example, the preset line segment could be the entry line when the vehicle is parking. The parking space adjustment angle represents the angle between the center of the vehicle and the entry line when the vehicle is expected to complete parking. The baseline represents a straight line with the entry line corner point as its origin and the adjustment angle as the line segment angle. It is understood that the above second adjustment method includes steps S41 to S43.

[0076] In other application scenarios, adjustment angles are generated for each line segment based on its angle. These adjustment angles are then weighted and fused to obtain the parking space's adjustment angle. The adjustment angle of a line segment represents the angle between the center of the vehicle and the entry line when the vehicle is expected to complete parking. Specifically, for each line segment in the straight line extraction results, the following steps are performed: If the line segment is an entry line or a line segment far from the entry line, the angle value of the line segment is added to a preset angle value to obtain the adjustment angle of that line segment. For example, the preset angle value is 90°. If the line segment is adjacent to the entry line, its angle is directly used as the adjustment angle of that line segment.

[0077] In other application scenarios, before weighted fusion of the adjustment angles corresponding to each line segment to obtain the adjustment angle of the parking space, the weight of each line segment is determined based on the number of pixels that intersect with the segmentation result. Then, weighted fusion is performed using the weights of each line segment and the corresponding adjustment angle to obtain the fusion result, which is used as the adjustment angle of the parking space. In other application scenarios, step S23 can be performed sequentially using the first adjustment method and the second adjustment method to obtain the candidate corner point information of the parking space output by the second adjustment method.

[0078] Please see Figure 5 , Figure 5 yes Figure 1 The sub-flowchart of step S13.

[0079] In some embodiments, step S13 above may include the following steps: performing the following procedure on each parking space: Figure 5The following steps are shown: Step S51: Use the parking space information belonging to the parking space in several frames of images as the information set to be fused. Step S52: Determine the weight of each candidate corner point information in the information set to be fused. Step S53: Based on the weight of each candidate corner point information in the information set to be fused, perform weighted fusion on each candidate corner point information in the information set to be fused to obtain the target corner point information corresponding to the parking space.

[0080] The parking space information set to be fused includes candidate corner information corresponding to at least one image frame. The weight of each candidate corner information in the information set to be fused represents the importance of fusion for each candidate corner information in the information set to be fused. The greater the weight of a candidate corner information, the higher its importance in the fusion of the candidate corner information in the information set to be fused.

[0081] In some application scenarios, step S52 above may include the following steps: For each candidate corner information in each set of information to be fused, perform the following steps: Based on the distance between the midpoint of the parking space entry line corresponding to the entry line corner in the candidate corner information and the vehicle at the acquisition time of the image corresponding to the candidate corner information, determine the first weight of the candidate corner information, where the parking space entry line represents the parking space line where the vehicle enters the parking space. And / or, based on the angle between the vehicle and the candidate corner information at the acquisition time of the image corresponding to the candidate corner information, determine the second weight of the candidate corner information. And / or, based on the distance between the first position of the vehicle at the acquisition time of the image corresponding to the candidate corner information and the second position of the vehicle at the start time of the parking phase, determine the third weight of the candidate corner information. Determine the weight corresponding to the candidate corner information based on at least one of the first weight, the second weight, and the third weight.

[0082] In some embodiments, prior to step S52, the parking space detection method may include the following steps: In response to the number of candidate corner information in the information set to be fused exceeding a preset number, filtering the candidate corner information in the information set to be fused based on the weights of each candidate corner information in the information set to be fused, to obtain a filtered information set to be fused. Step S52 may include the following steps: performing weighted fusion of the candidate corner information in the filtered information set to be fused based on the weights of each candidate corner information in the filtered information set to be fused, to obtain the target corner information corresponding to the parking space.

[0083] Please see Figure 6a , Figure 6a This is a schematic diagram of the framework of an embodiment of the parking space detection method of this application.

[0084] like Figure 6aThe perception module shown can be used to extract features from each image to obtain the image segmentation result and the detection boxes for each parking space in the image. The perception module can be the aforementioned preset feature extraction module. Each image is input into the perception module, which obtains the image segmentation result and the detection boxes for each parking space in the image. The adjustment module uses the parking space corner position as the initial value, searches for the segmentation results of pixels in the vicinity, and uses at least one straight line extraction algorithm to extract the straight line of the parking space. The accuracy of the parking space corner point and the orientation of the parking space are optimized based on the extracted parking space line. The fusion module can combine the parking space information corresponding to multiple frames of images belonging to the same parking space to obtain the current optimal perception position of the parking space, and output it to the downstream automatic parking planning and control module.

[0085] For example, a bird's-eye view obtained after distortion removal from four fisheye images is input as a frame into the perception module, which outputs the image positions of the four corner points of the parking space and the image segmentation results. Based on requirements, the parking process is divided into a parking entry stage and a parking space search stage. For the parking space search stage, a larger perception range is needed to obtain parking spaces at a distance. Therefore, for the parking space search stage, we input an image with a larger field of view. For the parking entry stage, the target parking space has been determined and is near the vehicle; to improve perception accuracy, a small field of view with high precision is input.

[0086] In the parking space search phase of the adjustment module, the detection bounding box output by the perception module is used as the initial value to determine the pixel search range for each parking space. In the parking phase, the historical optimal fusion position of the target corner information from several historical frames of each parking space is used as the initial value to determine the pixel search range for each parking space. Within the pixel search range, based on the segmentation results output by the perception module, the Hough line extraction algorithm is used to extract the straight lines of the parking spaces to obtain the initial corner information corresponding to each parking space. In some application scenarios, for the parts located in the overlapping field of view of several cameras, the result of the camera that is closest to the historical fusion result among the target corner information of several historical frames is used to complete the initial corner information. In some application scenarios, the intersection points of adjacent line segments in the initial corner information are calculated to optimize the position information of the corresponding corner points of the line segments and / or optimize the line segment angles in the initial corner information.

[0087] Specifically, the first and second adjustment methods described above are executed sequentially. When line segments are detected at the positions of two adjacent line segments of any corner point in the initial corner point information, the position information of that corner point in the initial corner point information is retained. The position information of that corner point is obtained by intersecting the two extracted line segments. When only one line segment or no line segment is detected at the positions of two adjacent line segments of any corner point in the initial corner point information, in the parking stage when the vehicle is in the parking space search stage, the corner point information in the detection box of that parking space in the image is used as the preset corner point information corresponding to that parking space. In the parking stage when the vehicle is in the parking entry stage, the corner point information in the historical corner point information to which the parking space belongs is used as the preset corner point information corresponding to that parking space. The preset corner point information is used as the candidate corner point information of the parking space to obtain the new initial corner point information corresponding to each parking space. Then, the second adjustment method described above is executed. Step S41 may be to obtain the line segment angle of each line segment in the new initial corner point information. The adjustment angle of the parking space represents the heading angle of the parking space corresponding to the expected vehicle parking.

[0088] After updating the corner point positions according to the first adjustment method described above, new initial corner point information for each parking space is obtained. In each new initial corner point information, the number of pixels representing the effective parking space lines determined when extracting parking space lines from each side of the parking space is used as the weight. Based on prior information about the angles between adjacent line segments, the parking space heading is further optimized to obtain the adjustment angle of the parking space. The more parking space line pixels a line segment passes through in the segmentation result in the new initial corner point information, the greater its impact on the parking space heading angle, and the greater the weight of the corresponding line segment.

[0089] Based on the prior conditions corresponding to the initial corner information or the prior conditions corresponding to the new initial corner information, and the type of the entry line, the adjustment angle corresponding to each line segment is generated. This application takes the sequential execution of the first adjustment method and the second adjustment method described above as an example. The prior conditions include at least one of the following: opposite sides of two line segments are parallel, adjacent line segments are perpendicular, the line segment adjacent to the entry line is parallel to the heading angle, and the line segment opposite to the entry line is parallel to the heading angle. The line segment adjacent to the entry line is the line segment away from the entry line. The type of entry line includes long or short line segments of the parking space. For example, when reversing into a parking space, the type of entry line is a short line segment. When parallel parking, the type of entry line is a long line segment.

[0090] Please see Figure 6b , Figure 6b This is a schematic diagram of a new non-entry line corner point in one embodiment of the parking space detection method of this application.

[0091] Among them, such as Figure 6bPoints A, B, C, and D are shown as four corner points in the new initial corner point information. The line segment between points C and D is the entry line. L represents the vehicle's heading angle; the angle between L and the line segment between points C and D is the adjustment angle for the determined parking space. Points A and B are non-entry line corner points. Points C and D are entry line corner points. Points a and b are new non-entry line corner points. Based on the prior conditions corresponding to the new initial corner point information and the type of entry line, the adjustment angles corresponding to each line segment are generated. The adjustment angles corresponding to each line segment are weighted and fused to obtain the adjustment angle of the parking space. The non-entry line corner points (such as points A or B) are projected onto the baseline to obtain new non-entry line corner points (such as points a or b), and these new non-entry line corner points are used as the target corner points corresponding to the non-entry line corner points. Using the entry line corner points as base points, the positions of the non-entry line corner points are adjusted within a certain threshold range according to the adjustment angle of the parking space. The heading of the parking space is the direction of its longer side, which is also the orientation of the vehicle when it enters the target parking position. Specifically, the angle of the line segment of the entry line between points C and D is angle 1, and the corresponding adjustment angle is Q1+90°. The line segment between points A and B is opposite to the entry line, with an angle of Q2, and the adjustment angle is Q2+90°. The line segment between points A and C is adjacent to the entry line, with an angle of Q3, and the adjustment angle is Q3. The line segment between points B and D is adjacent to the entry line, with an angle of Q4, and the adjustment angle is Q4. The weighted average of the angles of each line segment in the parking space is used to obtain the estimated value e of the overall heading angle of the parking space (i.e., the adjustment angle of the parking space). Two baselines are determined, passing through points C and D on the entry line and with a heading angle of e. Points A and B, which are far from the entry line, are projected onto the two baselines to obtain point a corresponding to point A and point b corresponding to point B in the updated position based on the heading angle.

[0092] The fusion module establishes a tracking sequence (i.e., a set of information to be fused) of up to 10 for each parking space. The average value of the parking space information corresponding to at least one frame within the tracking sequence is used as the output result for that parking space, which is the aforementioned target parking space information. Matching is performed based on the intersection-over-union ratio (IOU) between the target parking space information and the parking space information in the image. If the IOU meets a threshold, the match is successful, and the space is placed in the set of information to be fused corresponding to the successfully matched target parking space. If the IOU does not meet the threshold, the match fails, and a new set of information to be fused for the new parking space is created. The weights of each candidate corner point in the set of information to be fused are calculated based on the perceived position of the parking space in the image, the angle between the vehicle and the target parking space, and the distance the vehicle has traveled since the moment the parking space was perceived. In some application scenarios, when the number of candidate corner point information in the set of information to be fused is less than 10, the successfully matched parking space information is directly added to the set of information to be fused. When the number of candidate corner information in the information set to be merged is equal to 10, the weight of the successfully matched parking space information is used as a benchmark, and compared with the weight of the number of candidate corner information in the information set to be merged. The item with the lowest weight among the candidate corner information in the information set to be merged is replaced, or the successfully matched parking space information is directly deleted. In other application scenarios, the average of all candidate corner information in the information set to be merged is used to obtain the optimal fusion result for the parking spaces corresponding to that information set. The optimal fusion result can be used as input to the downstream planning and control module or as input to the adjustment module. The corner information in the optimal fusion result is used as the aforementioned preset corner information.

[0093] In other application scenarios, the weights of each candidate corner information in the information set to be fused can be calculated using at least one of the first, second, and third methods. The first method determines the first weight of the candidate corner information based on the distance between the midpoint of the parking space entry line corresponding to the candidate corner information and the vehicle at the time the image corresponding to the candidate corner information was captured. The parking space entry line represents the parking space line where the vehicle enters the parking space. For example, the smaller the distance, the larger the first weight. The second method determines the second weight of the candidate corner information based on the angle between the vehicle and the candidate corner information at the time the image corresponding to the candidate corner information was captured. The smaller the angle value, the larger the second weight. For example, the smaller the difference between the vehicle's heading angle to the parking space corresponding to the candidate corner information and a line segment angle with a value of 0° or 90° relative to the entry line is determined. The smaller the angle value corresponding to the smaller value, the larger the second weight. The third method determines the third weight of the candidate corner information based on the distance the vehicle moves between its first position at the moment the image corresponding to the candidate corner information is acquired and its second position at the start of the parking phase. The greater the moving distance, the greater the third weight. It can be understood that for any parking space in the next frame image acquired after the aforementioned several frames, the weight of each candidate corner information in the fusion set belonging to that parking space can represent the weight of the historical corner information corresponding to each historical frame image when the next frame image is the current image.

[0094] In other application scenarios, the candidate corner information with the largest weight is selected from the weights of each candidate corner information in the information set to be fused as the historical corner information of the parking space to which the information set to be fused belongs when the next frame image is the current image, so as to determine the pixel search range of the parking space in the next frame image based on the historical corner information when the vehicle is in the parking stage.

[0095] Compared to non-visual detection methods, the above-mentioned scheme has lower accuracy in detecting parking spaces. This application performs parking space extraction processing on each frame of the acquired images to obtain at least one parking space information in each image. By fusing the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information of each parking space, the accuracy of the target parking space information of each parking space can be improved.

[0096] Please see Figure 7 , Figure 7This is a schematic diagram of an embodiment of the parking space detection device of this application. The parking space detection device 70 includes an acquisition module 71, an extraction module 72, and a fusion module 73; the acquisition module 71 is used to acquire several frames of images; the extraction module 72 is used to perform parking space extraction processing on each image to obtain at least one parking space information in each image; the fusion module 73 is used to fuse the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information of each parking space.

[0097] Compared to non-visual detection methods, the above-mentioned scheme has lower accuracy in detecting parking spaces. This application performs parking space extraction processing on each frame of the acquired images to obtain at least one parking space information in each image. By fusing the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information of each parking space, the accuracy of the target parking space information of each parking space can be improved.

[0098] Please refer to the parking space detection method for the functions performed by each module; they will not be repeated here.

[0099] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 80 includes a memory 81 and a processor 82. The processor 82 is used to execute program instructions stored in the memory 81 to implement the steps in the above-described parking space detection method embodiment. In a specific implementation scenario, the electronic device 80 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 80 may also include mobile devices such as laptops and tablets, which are not limited here.

[0100] Specifically, processor 82 controls itself and memory 81 to implement the steps in the above-described parking space detection method embodiment. Processor 82 can also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 82 can be implemented using integrated circuit chips.

[0101] Compared to non-visual detection methods, the above-mentioned scheme has lower accuracy in detecting parking spaces. This application performs parking space extraction processing on each frame of the acquired images to obtain at least one parking space information in each image. By fusing the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information of each parking space, the accuracy of the target parking space information of each parking space can be improved.

[0102] Please see Figure 9 , Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 90 stores program instructions 901 thereon, which, when executed by a processor, implement the steps in any of the above-described parking space detection method embodiments.

[0103] Compared to non-visual detection methods, the above-mentioned scheme has lower accuracy in detecting parking spaces. This application performs parking space extraction processing on each frame of the acquired images to obtain at least one parking space information in each image. By fusing the parking space information belonging to the same parking space in several frames of images to obtain the target parking space information of each parking space, the accuracy of the target parking space information of each parking space can be improved.

[0104] In some embodiments, the system provided in this disclosure may have functions or include modules that can be used to execute the methods described in the above method embodiments. The specific implementation of these methods can be referred to the description in the above method embodiments, and for the sake of brevity, they will not be repeated here.

[0105] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0107] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A parking space detection method, characterized in that, The method includes: Acquire several frames of images; Parking space extraction processing is performed on each image to obtain at least one parking space information in each image; The parking space information belonging to the same parking space in the several frames of images is fused to obtain the target parking space information for each parking space.

2. The method according to claim 1, characterized in that, The parking space information includes candidate corner point information of the parking space. The step of performing parking space extraction processing on each image to obtain at least one parking space information in each image includes: Perform the following steps for each frame of the image: Obtain the segmentation result of the image; Line extraction is performed on the segmentation results to obtain the line extraction results of each parking space in the image. The line extraction results include the initial corner point information of the parking space. The initial corner information is adjusted to obtain the candidate corner information of the parking space.

3. The method according to claim 2, characterized in that, The step of extracting straight lines from the segmentation results to obtain the straight line extraction results for each parking space in the image includes: Determine the parking phase of the vehicle, which includes either the parking phase or the parking space search phase; Based on the parking phase, determine the pixel search range for each parking space in the image; For each parking space, a straight line is extracted from the segmentation result within the pixel search range of the parking space to obtain the initial corner information corresponding to the parking space.

4. The method according to claim 3, characterized in that, The step of determining the pixel search range of each parking space in the image based on the parking stage includes: In response to the parking phase being in the parking space search phase, the image range corresponding to the detection box of each parking space in the image is used as the pixel search range of each parking space. In response to the parking phase being in the parking entry phase, the image range corresponding to the historical corner point information of each parking space is used as the pixel search range for each parking space.

5. The method according to claim 2, characterized in that, The straight line extraction result also includes line segments of the parking space in at least one direction, and the initial corner information includes at least one corner point of the parking space. The step of adjusting the initial corner information to obtain candidate corner information of the parking space includes: For each corner point in the parking space, perform the following steps: In response to the fact that there are two line segments passing through the corner point, the initial corner point information is used as the candidate corner point information for the parking space; In response to the fact that the number of line segments passing through the corner point is not two, the preset corner point information corresponding to the corner point is determined based on the parking stage of the vehicle, and the preset corner point information is used as the candidate corner point information of the parking space.

6. The method according to claim 2, characterized in that, The straight line extraction result also includes line segments of the parking space in at least two directions. The initial corner point information includes the corner points of the parking space entering the line and the corner points of the parking space not entering the line. The candidate corner point information includes the target corner points corresponding to the corner points of the non-entry line. The step of adjusting the initial corner point information to obtain the candidate corner point information of the parking space includes: Obtain the line segment angles of each line segment in the line extraction result; The adjustment angle of the parking space is determined based on the angle of each line segment; The non-entry line corner point is projected onto the baseline to obtain a new non-entry line corner point, and the new non-entry line corner point is used as the target corner point corresponding to the non-entry line corner point. The baseline represents a straight line with the entry line corner point as the origin and the adjustment angle as the line segment angle.

7. The method according to claim 1, characterized in that, The step of fusing parking space information belonging to the same parking space in the plurality of frames of images to obtain target parking space information for each parking space includes: For each of the aforementioned parking spaces, perform the following steps: The parking space information belonging to the parking space in the aforementioned frames of images is used as the set of information to be fused for the parking space. The set of information to be fused for the parking space includes candidate corner point information corresponding to at least one frame of the image. Determine the weights of each candidate corner point in the information set to be fused; Based on the weights of each candidate corner information in the information set to be fused, the candidate corner information in the information set to be fused is weighted and fused to obtain the target corner information corresponding to the parking space.

8. A parking space detection device, characterized in that, include: The acquisition module is used to acquire several frames of images; The extraction module is used to extract parking spaces from each image to obtain at least one parking space information from each image. The fusion module is used to fuse parking space information belonging to the same parking space in the several frames of images to obtain the target parking space information for each parking space.

9. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to perform the method as claimed in any one of claims 1-7.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they are used to implement the method as described in any one of claims 1-7.

Images