Parking space detection method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a parking space detection method, apparatus, device, and storage medium, which relate to the field of data detection technology to solve the problem of low parking space detection accuracy in the prior art. Specifically, the method comprises: acquiring a first image; wherein the first image comprises a first parking space corner point and a parking space line corresponding to the first parking space corner point; inputting the first image into a detection model to obtain position information of the first parking space corner point and a parking space line corresponding to the first parking space corner point; wherein the detection model comprises a heat map prediction branch and a direction prediction branch, wherein the heat map prediction branch is used to obtain a heat map including the position information of the parking space corner point, and the direction prediction branch is used to obtain a feature map including the direction information of the parking space line; according to the heat map, the position information of the first parking space corner point is obtained, and according to the feature map, the parking space line corresponding to the first parking space corner point is obtained.

Description

Technical Field

[0001] The present application relates to the field of data detection technology, and specifically to a parking space detection method, device, equipment and storage medium. Background Art

[0002] Currently, the parking space detection process involves identifying parking space corner points in images captured by a car's camera, then cutting out these corner points for template matching. Existing templates include "T"-shaped, "L"-shaped, and "X"-shaped parking space corner points at various angles. The matching results derive the corner point locations and parking space line angles. Because the template angles are fixed, the matching results are not very accurate. Summary of the Invention

[0003] The present application provides a parking space detection method, apparatus, device and storage medium to solve the problem of low parking space detection accuracy in the prior art.

[0004] To achieve the above objectives, this application adopts the following technical solutions:

[0005] In a first aspect, the present application provides a parking space detection method, the method comprising: acquiring a first image; wherein the first image comprises a first parking space corner point and a parking space line corresponding to the first parking space corner point; inputting the first image into a detection model to obtain position information of the first parking space corner point and a parking space line corresponding to the first parking space corner point; wherein the detection model comprises a heat map prediction branch and a direction prediction branch, the heat map prediction branch is used to obtain a heat map including the position information of the parking space corner point, the direction prediction branch is used to obtain a feature map including the direction information of the parking space line, according to the heat map, the position information of the first parking space corner point is obtained, and according to the feature map, the parking space line corresponding to the first parking space corner point is obtained.

[0006] As can be seen from the above, the parking space detection method provided by this application uses a detection model to detect the acquired image and obtain the position information of the first parking space corner point in the image and the parking space line corresponding to the first parking space corner point. The detection model can also output the detailed angle of the parking space line corresponding to the first parking space corner point. Compared with the approximate angle obtained by template matching in related technologies, this application improves the accuracy of the first parking space corner point and the parking space line corresponding to the first parking space corner point.

[0007] In one possible implementation, the parking space corner point position information includes the coordinate information of each pixel point and at least one thermal cluster, each thermal cluster includes multiple thermal points and the confidence that each thermal point is the first parking space corner point. According to the heat map, the position information of the first parking space corner point is obtained, including: based on the confidence that each thermal point is the first parking space corner point, screening out the first thermal point with a confidence greater than a first threshold; and using the coordinate information of the pixel point corresponding to the first thermal point as the position information of the first parking space corner point.

[0008] In one possible implementation, the parking space line direction information includes the coordinate information of each pixel point and the confidence of each pixel point at each angle in a preset angle range. According to the feature map, the parking space line corresponding to the first parking space corner point is obtained, including: comparing the position information of the first parking space corner point with the coordinate information of the pixel point in the parking space line direction information to determine the first pixel point corresponding to the first parking space corner point; performing a Gaussian transformation on the confidence of each angle of the first pixel point in the preset angle range to obtain a Gaussian distribution curve for characterizing the correspondence between the angle and the confidence; on the Gaussian distribution curve, through a preset sliding window, screen out the target angle, and use the parking space line indicated by the target angle as the parking space line corresponding to the first parking space corner point.

[0009] In one possible implementation, a target angle is screened out on a Gaussian distribution curve through a preset sliding window, including: overlapping the sliding window with the Gaussian distribution curve, determining the intersection point of the sliding window and the Gaussian distribution curve after overlapping, and using the angle corresponding to the intersection point as the target angle.

[0010] In one possible implementation, before inputting the first image into the detection model, the method also includes: obtaining multiple sample images and target parking space corner point identifiers in the multiple sample images; generating multiple heat maps based on the multiple sample images and target parking space corner point identifiers in the multiple sample images, and determining the thermal value of each heat point in the multiple heat maps; wherein, a heat point is used to characterize a parking space corner point, and the thermal value of a heat point is used to characterize the confidence that the parking space corner point corresponding to the heat point is the target parking space corner point of the sample image; using the multiple sample images and the thermal value of each heat point in the multiple heat maps, the detection model to be trained is trained to generate a trained detection model.

[0011] In one possible implementation, before inputting the first image into the detection model, the method also includes: obtaining multiple sample images and parking line angle identifications of target parking space corner points in the multiple sample images, and setting the confidence of the parking line angle identifications of the target parking space corner points to 1; using multiple sample images, the parking line angle identifications of the target parking space corner points in the multiple sample images, and the confidence of the parking line angle identifications of the target parking space corner points, to train the detection model to be trained to generate a trained detection model.

[0012] In a possible implementation, the angle of the target parking space line is the direction of the target parking space line relative to the horizontal direction of the first image.

[0013] In one possible implementation, the method also includes: inputting multiple images into a detection model to obtain multiple parking space corner points and parking space lines corresponding to the multiple parking space corner points; the multiple images all contain the same object as the first image; based on the first image and the multiple images, performing information tracking processing on the same object in the first image and the multiple images to obtain a target image, and the target image includes a location tag of the same object; based on the first parking space corner point, the parking space line corresponding to the first parking space corner point, multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points and the number of vehicles in the target image, determining that the target image contains undetected parking space corner points and parking space lines; using the first parking space corner point, the parking space line corresponding to the first parking space corner point, multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points and parking space marking rules, determine the undetected parking space corner points and parking space lines.

[0014] In one possible implementation, the first image is input into a detection model to obtain position information of a first parking space corner point and a parking space line corresponding to the first parking space corner point, including: inputting the first image into a detection model to obtain position information of multiple parking space corner points in the first image and parking space lines corresponding to the multiple parking space corner points; wherein the multiple parking space corner points include the first parking space corner point; the method also includes: based on the position information of each parking space corner point among the multiple parking space corner points in the first image and the number of vehicles in the first image, determining the parking space corner points that have not been detected in the first image; and using the position information of each parking space corner point and parking space line marking rules to determine the position information of the parking space corner points that have not been detected in the first image.

[0015] In a possible implementation, the method also includes: inputting the first image into a detection model to obtain the position information of the first parking space corner point, the parking space line corresponding to the first parking space corner point, the position information of the third parking space corner point and the parking space line corresponding to the third parking space corner point, where the first parking space corner point and the third parking space corner point are adjacent; using the target intersection point, the position information of the first parking space corner point, the parking space line corresponding to the first parking space corner point, the position information of the third parking space corner point and the parking space line corresponding to the third parking space corner point, to determine the area where the parking space including the first parking space corner point and the third parking space corner point is located; the target intersection point is the intersection of the midline of the first angle and the midline of the second angle, the first angle is obtained based on the first parking space line and the second parking space line of the first parking space corner point, the second angle is obtained based on the third parking space line and the fourth parking space line of the third parking space corner point, the direction of the first parking space line is opposite to the direction of the third parking space line, and the angle of the second parking space line is the same as the angle of the fourth parking space line.

[0016] In a second aspect, a parking space detection device is provided, comprising an acquisition unit and a processing unit. The acquisition unit is configured to acquire a first image, wherein the first image includes a first parking space corner point and a parking space line corresponding to the first parking space corner point. The processing unit is configured to input the first image into a detection model to obtain position information of the first parking space corner point and the parking space line corresponding to the first parking space corner point. The detection model includes a heat map prediction branch and a direction prediction branch. The heat map prediction branch is configured to obtain a heat map including position information of the parking space corner point, and the direction prediction branch is configured to obtain a feature map including direction information of the parking space line. Based on the heat map, the position information of the first parking space corner point is obtained, and based on the feature map, the parking space line corresponding to the first parking space corner point is obtained.

[0017] In one possible implementation, the processing unit is further used to screen out the first thermal point with a confidence greater than a first threshold based on the confidence that each thermal point is the first parking space corner point; the processing unit is further used to use the coordinate information of the pixel point corresponding to the first thermal point as the position information of the first parking space corner point.

[0018] In one possible implementation, the processing unit is further used to compare the position information of the first parking space corner point and the coordinate information of the pixel points in the parking space line direction information to determine the first pixel point corresponding to the first parking space corner point; the processing unit is further used to perform a Gaussian transformation on the confidence level of each angle of the first pixel point in a preset angle range to obtain a Gaussian distribution curve for characterizing the correspondence between the angle and the confidence level; the processing unit is further used to screen out the target angle on the Gaussian distribution curve through a preset sliding window, and use the parking space line indicated by the target angle as the parking space line corresponding to the first parking space corner point.

[0019] In a possible implementation, the processing unit is further configured to overlap the sliding window with the Gaussian distribution curve, determine an intersection point between the overlapping sliding window and the Gaussian distribution curve, and use an angle corresponding to the intersection point as the target angle.

[0020] In one possible implementation, the acquisition unit is further used to acquire multiple sample images and target parking space corner point identifiers in multiple sample images; the processing unit is further used to generate multiple heat maps based on multiple sample images and target parking space corner point identifiers in multiple sample images, and determine the thermal value of each heat point in the multiple heat maps; wherein, a heat point is used to characterize a parking space corner point, and the thermal value of a heat point is used to characterize the confidence that the parking space corner point corresponding to the heat point is the target parking space corner point of the sample image; the processing unit is further used to use the thermal value of each heat point in multiple sample images and multiple heat maps to train the detection model to be trained and generate a trained detection model.

[0021] In one possible implementation, the acquisition unit is further used to acquire multiple sample images and the parking line angle identification of the target parking space corner point in the multiple sample images, and set the confidence of the parking line angle identification of the target parking space corner point to 1; the processing unit is further used to train the detection model to be trained using multiple sample images, the parking line angle identification of the target parking space corner point in the multiple sample images, and the confidence of the parking line angle identification of the target parking space corner point to generate a trained detection model.

[0022] In a possible implementation, the angle of the target parking space line is the direction of the target parking space line relative to the horizontal direction of the first image.

[0023] In one possible implementation, the processing unit is further used to input multiple images into a detection model to obtain multiple parking space corner points and parking space lines corresponding to the multiple parking space corner points; the multiple images all contain the same object as the first image; the processing unit is further used to perform information tracking processing on the same object in the first image and the multiple images based on the first image and the multiple images to obtain a target image, and the target image includes a location tag of the same object; the processing unit is further used to determine whether the target image contains undetected parking space corner points and parking space lines based on the first parking space corner point, the parking space line corresponding to the first parking space corner point, multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points and the number of vehicles in the target image; the processing unit is further used to determine the undetected parking space corner points and parking space lines using the first parking space corner point, the parking space line corresponding to the first parking space corner point, multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points and parking space marking rules.

[0024] In one possible implementation, the processing unit is further used to input the first image into a detection model to obtain position information of multiple parking space corner points in the first image and parking space lines corresponding to the multiple parking space corner points; wherein the multiple parking space corner points include the first parking space corner point; the processing unit is further used to determine that the first image contains undetected parking space corner points based on the position information of each parking space corner point in the multiple parking space corner points in the first image and the number of vehicles in the first image; the processing unit is further used to use the position information of each parking space corner point and the parking space line marking rules to determine the position information of the undetected parking space corner points in the first image.

[0025] In a possible implementation, the processing unit is further used to input the first image into the detection model to obtain the position information of the first parking space corner point, the parking space line corresponding to the first parking space corner point, the position information of the third parking space corner point and the parking space line corresponding to the third parking space corner point, and the first parking space corner point and the third parking space corner point are adjacent; the processing unit is further used to use the target intersection point, the position information of the first parking space corner point, the parking space line corresponding to the first parking space corner point, the position information of the third parking space corner point and the parking space line corresponding to the third parking space corner point to determine the area where the parking space including the first parking space corner point and the third parking space corner point is located; the target intersection point is the intersection of the midline of the first angle and the midline of the second angle, the first angle is obtained based on the first parking space line and the second parking space line of the first parking space corner point, the second angle is obtained based on the third parking space line and the fourth parking space line of the third parking space corner point, the direction of the first parking space line is opposite to the direction of the third parking space line, and the angle of the second parking space line is the same as the angle of the fourth parking space line.

[0026] In a third aspect, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute instructions to implement the parking space detection method provided in the first aspect above.

[0027] In a fourth aspect, the present application provides a computer-readable storage medium comprising instructions. When the instructions are executed on a computer, the computer executes the parking space detection method provided in the first aspect.

[0028] In a fifth aspect, the present application provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the parking space detection method provided in the first aspect above.

[0029] It should be noted that the above-mentioned computer instructions may be stored in whole or in part on a first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the access network terminal device, or may be packaged separately from the processor of the access network terminal device, and this application does not limit this.

[0030] The description of the second, third, fourth and fifth aspects in this application can refer to the detailed description of the first aspect; and the beneficial effects described in the second, third, fourth and fifth aspects can refer to the analysis of the beneficial effects of the first aspect, which will not be repeated here.

[0031] In this application, the above names do not limit the terminal devices or functional modules themselves. In actual implementation, these terminal devices or functional modules may appear with other names. As long as the functions of each terminal device or functional module are similar to those of this application, they fall within the scope of the claims of this application and their equivalents.

[0032] These and other aspects of the present application will become more readily apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a structural diagram of a parking space detection system provided in an embodiment of the present application;

[0034] Figure 2 This is one of the flow charts of a parking space detection method provided in an embodiment of the present application;

[0035] Figure 3 is a schematic diagram of an application detection model provided in an embodiment of the present application;

[0036] Figure 4 This is a second flow chart of a parking space detection method provided in an embodiment of the present application;

[0037] Figure 5a is a schematic diagram of a heat map provided in an embodiment of the present application;

[0038] Figure 5b is a schematic diagram of a Gaussian distribution curve provided in an embodiment of the present application;

[0039] Figure 5c This is a schematic diagram of a confidence result of a parking space line provided in an embodiment of the present application;

[0040] Figure 6 This is the third flow chart of a parking space detection method provided in an embodiment of the present application;

[0041] Figure 7 This is one of the schematic diagrams of a parking space detection scenario provided in an embodiment of the present application;

[0042] Figure 8 This is a fourth flow chart of a parking space detection method provided in an embodiment of the present application;

[0043] Figure 9 This is the fifth flow chart of a parking space detection method provided in an embodiment of the present application;

[0044] Figure 10 This is the sixth flow chart of a parking space detection method provided in an embodiment of the present application;

[0045] Figure 11 This is the second schematic diagram of a parking space detection scenario provided by an embodiment of the present application;

[0046] Figure 12 This is the seventh flow chart of a parking space detection method provided in an embodiment of the present application;

[0047] Figure 13This is the third schematic diagram of a parking space detection scenario provided by an embodiment of the present application;

[0048] Figure 14 Schematic diagram of the structure of a parking space detection device provided in an embodiment of the present application;

[0049] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0050] Figure 16 It is a structural diagram of a computer program product of the parking space detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0052] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity or execution order.

[0053] Currently, the parking space detection process involves identifying the parking space corners in a captured image, then cutting out these corners for template matching. Existing templates include "T"-shaped, "L"-shaped, and "X"-shaped parking space corners at various angles. Based on the matching results, the template with the highest degree of match for the current parking space corner is determined. However, since the angles of existing templates are fixed, matching accuracy is low.

[0054] Furthermore, because images are mostly captured by cameras mounted near the headlights and taillights of vehicles, this detection method can only detect four parking corner points for empty spaces. For occupied spaces, at most the two outer corner points can be detected; the inner two corner points cannot be detected due to obstruction by parked vehicles. Therefore, the applicability of this detection method is relatively limited.

[0055] To address the above issues, embodiments of the present application provide a parking space detection method that utilizes a detection model to detect images captured by a multimedia acquisition device, obtaining parking space corner points and corresponding parking space lines in the image. This improves the accuracy of the determined parking space corner points.

[0056] The parking space detection method provided in the embodiment of the present application can be applied to a parking space detection system. Figure 1 Figure 2 shows a structure of the parking space detection system. Figure 1 As shown, the system includes an electronic device 1 and multiple multimedia capture devices 2. Electronic device 1 is connected to the multimedia capture devices 2 via any possible connection method, such as a wired network or a wireless network. Multiple multimedia capture devices 2 capture images in real time and transmit the captured images to electronic device 1 in real time. After receiving the images, electronic device 1 detects the images using a detection model to obtain parking space corner points in the images and the parking space lines corresponding to the parking space corner points.

[0057] The electronic device may be a tablet computer, desktop computer, laptop computer, notebook computer, netbook computer, virtual reality terminal, augmented reality terminal, wireless terminal in unmanned driving, etc. The multimedia acquisition device 2 is a device for acquiring images or videos, such as a camera in a car.

[0058] The electronic device 1 and the multimedia acquisition device 2 may be integrated into one device, or located in two independent devices. The embodiment of the present application does not impose any limitation on the positional relationship between the electronic device 1 and the multimedia acquisition device 2.

[0059] It should be noted that the parking space detection method provided in the embodiments of the present application can be applied to electronic devices. The execution subject of the parking space detection method provided in the embodiments of the present application can also be a parking space detection device. The parking space detection device can be an electronic device, or a central processing unit (CPU) in the electronic device, or a control module in the electronic device for executing the parking space detection method.

[0060] The parking space detection method provided in the embodiment of the present application is described in detail below using an electronic device as an example.

[0061] Specifically, please refer to Figure 2 , is a flow chart of a parking space detection method provided in an embodiment of the present application. Figure 2 As shown, the method may include S201-S202.

[0062] S201: The electronic device acquires a first image, wherein the first image includes a first parking space corner point and a parking space line corresponding to the first parking space corner point.

[0063] The electronic device receives an image captured by a connected multimedia capture device. The first image may be any image acquired by the electronic device that includes a first parking space corner point and a parking space line corresponding to the first parking space corner point. For example, the captured image may be captured in a parking scenario (e.g., a smart parking scenario), a violation detection scenario, a parking lot management scenario, or an intelligent assisted driving scenario.

[0064] In smart parking scenarios, the multimedia capture device can be a camera on the vehicle. The camera captures images of the parking situation and sends them to an electronic device. The electronic device uses the returned images to determine whether there is an empty parking space at the car's current location. If there is an empty space, parking is performed based on the detected specific location, thus achieving smart parking.

[0065] In traffic violation detection scenarios, multimedia acquisition devices can be cameras on vehicles or cameras installed on certain fixed roads in cities. The cameras capture images of parked vehicles and send them to electronic devices. Based on the images transmitted back, the electronic devices determine whether the parked vehicles in the images are in violation of traffic regulations, such as parking over a line.

[0066] In the parking lot management scenario, the multimedia acquisition device is a camera installed in the parking lot. The camera takes pictures to describe the parking situation of the vehicle and sends the pictures to the electronic device. The electronic device detects the parking situation of the parking lot in real time based on the pictures and obtains the location of the empty parking spaces at the current moment, so that when the customer has parking needs, the electronic device can actively indicate the location and direction of available parking.

[0067] In the intelligent assisted driving scenario, since the driver cannot accurately judge the position of the parking space corner behind the car based on the front-view mirror, the multimedia acquisition device is the camera on the vehicle. The camera takes pictures containing the parking space corners in real time and sends the pictures to the electronic device. The electronic device determines the position of the parking space corners behind the car based on the returned pictures, and displays the determined position to the driver through the display device in the car system, thereby realizing intelligent assisted driving.

[0068] S202: The electronic device inputs the first image into a detection model to obtain position information of a first parking space corner point and a parking space line corresponding to the first parking space corner point.

[0069] Among them, the detection model includes a heat map prediction branch and a direction prediction branch. The heat map prediction branch is used to obtain a heat map including the position information of the parking space corner points, and the direction prediction branch is used to obtain a feature map including the direction information of the parking space line. According to the heat map, the position information of the first parking space corner point is obtained, and according to the feature map, the parking space line corresponding to the first parking space corner point is obtained.

[0070] The electronic device inputs the first image into the detection model. If the first image contains only one parking space corner point, the detection model will detect the parking space corner point and the parking space line corresponding to the parking space corner point. If the first image contains multiple parking space corner points, the detection model can detect each of the multiple parking space corner points and the parking space line corresponding to each parking space corner point. The position information of each detected parking space corner point is the coordinates of the parking space corner point, and the position information of the parking space line corresponding to each parking space corner point is specifically the angle of the parking space line relative to a reference direction. Optionally, the reference direction can be the horizontal direction of the first image.

[0071] Specifically, because the lines marking the actual parking space corner points in the first image have a certain width, after the detection model detects multiple parking space corner points, it is necessary to evaluate each corner point and determine the first parking space corner point based on the evaluation results. Exemplarily, the parking space corner point evaluation specifically involves evaluating the confidence level of each parking space corner point. A higher confidence level indicates a higher probability that the parking space corner point is the first parking space corner point. The first parking space corner point is specifically determined by selecting a parking space corner point with a confidence level greater than a first preset threshold as the first parking space corner point.

[0072] After determining the first parking space corner point, there are countless parking space lines connected to it. Therefore, it is necessary to evaluate all parking space lines connected to the first parking space corner point. A specific evaluation method can be to use the first parking space corner point as the origin, and determine the confidence level of each parking space line at each angle. For example, the confidence level of a 30° parking space line is 80%, and the confidence level of a 60° parking space line is 90%.

[0073] After determining the confidence level of the parking space lines at all angles, the parking space lines are screened according to a preset condition to select parking space lines that meet the preset condition. The preset condition is that the confidence level of the parking space line's directional information is greater than a second threshold. The values ​​of the first and second thresholds are determined based on the image performance in the actual scene, and this application does not impose any restrictions on the specific values.

[0074] Furthermore, the position information of the first parking space corner point and the parking space line corresponding to the first parking space corner point are specifically determined by the detection model. Figure 3 As shown, the detection model includes a deep learning unit, a first decoding unit, a query unit, a second decoding unit and an information fusion unit.

[0075] When the first image is input into the detection model, the deep learning unit in the detection model processes the first image to obtain a heat map and a feature map. Specifically, the deep learning unit includes a feature extraction branch, a heat map prediction branch, and a direction prediction branch. The first image is input into the feature extraction branch, and the feature extraction branch extracts features from the first image. After the feature information is extracted, the feature information is input into the heat map prediction branch and the direction prediction branch. The heat map prediction branch generates a heat map based on the feature information, and the heat map includes the position information of the parking space corner points. The direction prediction branch generates a feature map based on the feature information, and the feature map includes the direction information of the parking space lines.

[0076] After obtaining the heat map and feature map, the deep learning unit sends the heat map to the first decoding unit. The first decoding unit decodes the coordinate information of the parking space corner points in the heat map to obtain a parking space corner point coordinate sequence. The parking space corner point coordinate sequence includes the coordinate information of multiple parking space corner points. Then, the first parking space corner point is determined based on the confidence level of each parking space corner point among the multiple parking space corner points.

[0077] After determining the first parking space corner point, the first parking space corner point is sent to a query unit. Based on the corner point coordinates of the first parking space corner point, the query unit determines coded direction information for multiple parking space lines with the first parking space corner point as the origin. The query unit then sends the coded direction information for the multiple parking space lines to a second decoding unit. The second decoding unit decodes the coded direction information for the multiple parking space lines to obtain the directions of the multiple parking space lines connected to the first parking space corner point. The second decoding unit then calculates confidence levels for the directions of the multiple parking space lines connected to the first parking space corner point to obtain confidence levels for the directions of the multiple parking space lines with the first parking space corner point as the origin. Based on the confidence levels for the directions of each parking space line, the direction information for the parking space line corresponding to the first parking space corner point is determined.

[0078] After determining the coordinate information of the first parking space corner point and the direction information of the parking space line corresponding to the first parking space corner point, the information fusion unit outputs the coordinate information of the first parking space corner point and the parking space line corresponding to the first parking space corner point.

[0079] The position of the current parking space can be determined based on the coordinate information of the first parking space corner point and the direction information of the parking space line corresponding to the first parking space corner point. The first parking space corner point and the parking space line corresponding to the first parking space corner point can form a variety of shapes, such as "L" shape, "T" shape and "cross" shape. When the shape formed by the first parking space corner point and the parking space line corresponding to the first parking space corner point is "L" shape, it means that the parking space corner point is located at both ends of multiple parking spaces. When the first parking space corner point and the parking space line corresponding to the first parking space corner point form a "T"-shaped parking space angle, it means that the parking space corner point is located in the middle of multiple parking spaces. When the first parking space corner point and the parking space line corresponding to the first parking space corner point form a "cross"-shaped parking space angle, it means that the parking space corner point is located in the middle of multiple parking spaces and there are at least two rows of parking spaces.

[0080] Example, combined Figure 4 In step S202, the position information of the first parking space corner point is obtained according to the heat map, including:

[0081] S2021. The electronic device selects a first heat point having a confidence level greater than a first threshold based on the confidence level that each heat point is a first parking space corner point.

[0082] S2022. The electronic device uses the coordinate information of the pixel point corresponding to the first thermal point as the position information of the first parking space corner point.

[0083] The parking space corner point position information includes coordinate information of each pixel point and at least one thermal cluster, each thermal cluster includes multiple thermal points and the confidence level of each thermal point being the first parking space corner point.

[0084] After the first image is input into the feature extraction branch in the deep learning unit, the feature information of the first image is obtained. The feature information of the first image is then input into the heat map prediction branch, which outputs a heat map including the position information of the parking space corner points. Figure 5a , Figure 5a The white part in the middle is used to represent the thermal mass, which is composed of Figure 5a As can be seen, the heat map includes multiple heat clusters, where each heat cluster can correspond to a parking space corner point. Then, based on the confidence level of the heat points in each heat cluster, a heat point that meets the requirements is determined. The heat point that meets the requirements can be the first heat point whose confidence level is greater than a first threshold. The first heat point is then used as the first parking space corner point, and the coordinate information of the first heat point is the coordinate information of the first parking space corner point.

[0085] Example, combined Figure 4 In step S202, the parking space line corresponding to the first parking space corner point is obtained according to the feature map, including:

[0086] S2023. The electronic device compares the position information of the first parking space corner point with the coordinate information of the pixel points in the parking space line direction information to determine a first pixel point corresponding to the first parking space corner point.

[0087] S2024. The electronic device performs Gaussian transformation on the confidence level of each angle of the first pixel point in the preset angle range to obtain a Gaussian distribution curve for characterizing the corresponding relationship between the angle and the confidence level.

[0088] S2025. The electronic device selects a target angle on the Gaussian distribution curve through a preset sliding window, and uses the parking space line indicated by the target angle as the parking space line corresponding to the first parking space corner point.

[0089] The feature map includes parking space line direction information, and the parking space line direction information includes coordinate information of each pixel point and the confidence level of each pixel point at each angle within a preset angle range.

[0090] Combined with S2022, the coordinate information of the first parking space corner point can be obtained, and then the coordinate information of the first parking space corner point is compared with the coordinate information of each pixel point in the feature map containing the parking space line direction information to obtain the pixel point corresponding to the first parking space corner point.

[0091] Since the feature map contains the confidence level of each pixel point in the preset angle range, after determining the pixel point corresponding to the first parking space corner point, the confidence level of each angle in the preset angle range can be obtained. Then, the confidence level of each angle in the preset angle range of the pixel point corresponding to the first parking space corner point is Gaussian transformed to obtain a Gaussian distribution curve. The Gaussian distribution curve is used to characterize the correspondence between angles and confidence levels. Figure 5b As shown, [θ-5°, θ+5°] is used as an angle interval, the preset angle range is [0, 2π], and approximately 36 Gaussian distribution curves are generated.

[0092] For example, the electronic device selects a sliding window with a length of 11° and a height of 80% to screen the target angle on the Gaussian distribution curve. Specifically, the above sliding window is used to sequentially pass through the Gaussian distribution curve to screen the target angle. The screened target angle is the angle of the parking space line corresponding to the first parking space corner point. The screened target angle is specifically an angle with a confidence level greater than a preset threshold. The sliding window can be a way to achieve the screening of the target angle. Other methods can also be used to screen the target angle, and this application does not limit this.

[0093] Optional, see Figure 4 , S2025 also includes: S20251, overlapping the sliding window and the Gaussian distribution curve, determining the intersection point of the overlapping sliding window and the Gaussian distribution curve, and using the angle corresponding to the intersection point as the target angle.

[0094] For example, see Figure 5b , overlap the sliding window with the Gaussian distribution curve. After overlap, if the sliding window intersects the Gaussian distribution curve, the angle corresponding to the intersection point can be used as the target angle. Alternatively, the angle corresponding to the intersection point and points within the Gaussian curve outside the intersection point can be used as the target angle, or the angle corresponding to points within the Gaussian curve outside the intersection point can be used as the target angle. Specifically, the method for selecting the target angle is limited by actual needs and is not limited in this application.

[0095] In the above embodiment, the detection model detects the captured image to obtain the position information of the first parking space corner point and the parking space line corresponding to the first parking space corner point. The detection model can also output the detailed angle of the parking space line corresponding to the first parking space corner point. Compared with the approximate angle obtained by template matching in related technologies, the present application improves the accuracy of the first parking space corner point and the parking space line corresponding to the first parking space corner point.

[0096] Example, combined Figure 6 Before inputting the first image into the detection model in S202, the method further includes:

[0097] S2011. The electronic device obtains multiple sample images and target parking space corner point identifiers in the multiple sample images.

[0098] The electronic device obtains multiple sample images, and each of the multiple sample images is marked with a target parking space corner mark. The sample images are mostly images captured by a camera when the car is parked. In the sample images, the parking space corners and parking space lines are manually marked to obtain parking space corner mark and parking space line mark. For example, Figure 7 As shown in (a), the parking space corner point is marked as P0, and the parking space lines are marked as line segments P0P1 and P0P2.

[0099] After the electronic device obtains the sample image containing the parking space corner point mark, in order to make the created detection model applicable to various types of images including bird's-eye view, front view, fisheye view, etc., this method creates a coordinate system for the marked parking space corner point mark, based on which the position information of the parking space corner point is more accurately described. Since the sample image is taken by a car camera, the size of the captured image is mostly rectangular. For example, Figure 7 As shown in (a), the parking space corner point and parking space line form an "L" shape, which includes the parking space corner point P0, parking space line P1 and parking space line P2. A polar coordinate system is established with the parking space corner point P0 as the polar coordinate origin. The parking space corner point P0 is represented by coordinates, and the parking space lines P1 and P2 are represented by polar angles. The parking space line P1 corresponds to Parking line P2 corresponds The angle is formed by the line ρ that is parallel to the horizontal direction of the sample image and passes through the parking space corner point P0, with P0 as the origin and the parking space line P1 as the end edge. The angle formed by the line ρ parallel to the horizontal direction of the sample image and passing through the parking corner point P0 as the starting edge and the parking line P2 as the ending edge is P0 as the origin. The parking corner point and parking line can be expressed as Similarly, if Figure 7 As shown in (b), the parking space corner points and parking space lines can be expressed as like Figure 7As shown in (c) in [reference], the parking space corner points and parking space lines can be represented as

[0100] Furthermore, the direction information of the parking space line corresponding to the first parking space corner point is the direction of the parking space line corresponding to the first parking space corner point relative to the horizontal direction of the first image.

[0101] Exemplarily, the images captured by the multimedia acquisition device are usually rectangular pictures. Describing the direction information of the parking space line in the above way will not result in a situation where it cannot be obtained due to low pixel count and insufficient clarity of the pictures. This description method has a wider application range.

[0102] S2012. The electronic device generates multiple heatmaps based on multiple sample images and the target parking space corner point identifiers in the multiple sample images, and determines the heat values of each heat point in the multiple heatmaps.

[0103] Among them, one heat point is used to represent one parking space corner point, and the heat value of one heat point is used to represent the confidence that the parking space corner point corresponding to the heat point is the target corner point of the sample image.

[0104] After the electronic device inputs the multiple sample images into the detection model to be trained, the heatmap prediction branch in the detection model to be trained can input multiple heatmaps according to the multiple sample images, and output the heat values of each heat point in the heatmaps.

[0105] S2013. The electronic device uses the multiple sample images and the heat values of each heat point in the multiple heatmaps to train the detection model to be trained, and generates a trained detection model.

[0106] Specifically, the detection model to be trained can output the input multiple heatmaps, and output the heat values of each heat point in the heatmaps.

[0107] When not relying on the model, the calculation of the heat value of the heat point satisfies the following expression:

[0108]

[0109] Among them, r is the distance from the heat point on the heatmap to the parking space corner point P0, r < R, R = min(max(d, 2), 30), d is the radius of the heatmap calibrated for each parking space corner point, d = min(P0P1, P0P2, P0P3,...), P0P1, P0P2, P0P3 are the distances marked manually.

[0110] When the thermal value output by the detection model to be trained is different from the calculated thermal value, the loss function is used to adjust the detection model to be trained until the difference between the thermal value output by the detection model and the calculated thermal value meets the requirements. For example, the loss function satisfies the following expression:

[0111] L heatmap =E{-log[1-(heatmap-target_heatmap) 2 ]}

[0112] Among them, heatmap is the thermal value output by the detection model, and target_heatmap is the calculated thermal value.

[0113] In the above embodiment, the detection model is trained using a large number of sample images and parking space corner point markers to obtain a detection model that meets the requirements. When the detection model is applied to vehicle image detection, the obtained parking space corner position information has a high accuracy.

[0114] Example, combined Figure 8 Before inputting the first image into the detection model in S202, the method further includes:

[0115] S2014: The electronic device obtains a plurality of sample images and the parking space line angle identification of the target parking space corner point in the plurality of sample images, and sets the confidence level of the parking space line angle identification of the target parking space corner point to 1.

[0116] Referring to S2011, the electronic device acquires a plurality of sample images, and each of the plurality of sample images is marked with a parking space line mark corresponding to the target parking space corner point, and the confidence level of the angle corresponding to the parking space line of the target parking space corner point is set to 1. Figure 5c As shown in the figure, the highest confidence level of the parking space line is 1. The closer to the parking space line, the higher the confidence level, which are 0.95 and 0.9 respectively. The farther away from the parking space line, the lower the confidence level.

[0117] S2015. The electronic device trains the detection model to be trained using multiple sample images, the parking space line angle identifications of the target parking space corner points in the multiple sample images, and the confidence levels of the parking space line angle identifications of the target parking space corner points to generate a trained detection model.

[0118] After the electronic device inputs multiple sample images into the detection model to be trained, the direction prediction branch in the detection model to be trained can output multiple feature maps based on the multiple sample images, and the output feature maps include each pixel point and the confidence results of each angle in the preset angle range.

[0119] When the output confidence is inconsistent with the pre-labeled confidence, the loss function is used to adjust the detection model to be trained until the gap between the confidence output by the detection model and the pre-labeled confidence meets the requirements. For example, the loss function satisfies the following expression:

[0120] L cofidence =E{-log[1-(pt) 2 ]}

[0121] Among them, p is the confidence result of the parking space corner point at a certain angle, and t is the standard confidence result.

[0122] Example, combined Figure 9 , the method further comprises:

[0123] S901: The electronic device inputs a plurality of images into a detection model to obtain a plurality of parking space corner points and parking space lines corresponding to the plurality of parking space corner points.

[0124] The multiple images contain the same object as that in the first image.

[0125] The electronic device inputs multiple images into a detection model to obtain multiple parking space corner points and parking space lines corresponding to the multiple parking space corner points. The multiple images are images that are consecutive frames of the first image, and the multiple images are likely to contain objects consistent with the first image. The objects can be vehicles, trees, clouds, etc. Exemplarily, the objects consistent with the first image in the multiple images can be determined using target tracking technology.

[0126] S902: The electronic device performs information tracking processing on the same object in the first image and the multiple images based on the first image and the multiple images to obtain a target image, where the target image includes a location tag of the same object.

[0127] When it is determined that the first image and the multiple images contain the same object, information of the same object in the first image and the multiple images is tracked to determine the position of the same object in each image, and generate a target image containing a position tag of the same object.

[0128] S903. The electronic device determines whether the target image contains undetected parking corner points and parking lines based on the first parking corner point, the parking line corresponding to the first parking corner point, multiple parking corner points, the parking lines corresponding to the multiple parking corner points, and the number of vehicles in the target image.

[0129] After obtaining the target image, the electronic device can determine whether the target image contains undetected parking space corner points and parking space lines based on the detected parking space corner points, parking space lines and the number of vehicles in the target image.

[0130] For example, the detection model detected two parking corner points, and the target image contained two cars. Based on the common parking space marking rule (one car is surrounded by at least two parking corner points), we know that, in addition to the parking corner points blocked by the inner side of the car body, at least three parking corner points can be detected for two cars. Therefore, the target image contains undetected parking corner points.

[0131] S904: The electronic device determines undetected parking corner points and parking lines by using the first parking corner point, the parking line corresponding to the first parking corner point, multiple parking corner points, the parking lines corresponding to the multiple parking corner points, and parking line marking rules.

[0132] The electronic device supplements the undetected parking corner points based on the detected parking space corner points, parking space lines and parking space marking rules, and determines whether the vehicle is illegally parked based on the supplemented results.

[0133] Example, combined Figure 10 S202 inputs the first image into the detection model to obtain the position information of the first parking space corner point and the parking space line corresponding to the first parking space corner point, including:

[0134] S1001: The electronic device inputs a first image into a detection model to obtain position information of a plurality of parking space corner points in the first image and parking space lines corresponding to the plurality of parking space corner points, wherein the plurality of parking space corner points includes a first parking space corner point.

[0135] The first image contains position information of multiple parking space corner points. The electronic device obtains the position information of all parking space corner points in the first image by detecting the model.

[0136] S1002: The electronic device determines that the first image includes undetected parking corner points based on the position information of each parking corner point among the plurality of parking corner points in the first image and the number of vehicles in the first image.

[0137] S1003: The electronic device determines the position information of the parking space corner points that are not detected in the first image by using the position information of each parking space corner point and the parking space marking rule.

[0138] Combining S903 and S904, we can see that undetected parking corner points can appear in the target image, or even in a single image. After determining the location information of undetected parking corner points in the image, the location information of the undetected parking corner points is supplemented using the existing locations of the parking corner points, the number of vehicles, and the vehicle marking rules, thereby obtaining the locations of all parking corner points in the image.

[0139] For example, Figure 11 As shown, Figure 11The image in can be a target image or an image. This image includes the detected parking space corner point a at the upper left of parking space A, the parking space corner point b at the upper left of parking space B, the parking space corner point c at the lower left of parking space D, and the parking space corner point d at the upper left. Since there are two cars between parking spaces B and D, we know that there is another parking space C between them, and that there is an undetected parking space corner point e at the upper left of parking space C. Then, based on parking space corner point b, parking space corner point d, and the parking space marking rules, we can determine the parking space corner point e at the upper left of parking space C.

[0140] For example, combined Figure 12 , the method further comprises:

[0141] S1201. The electronic device inputs the first image into a detection model to obtain position information of a first parking space corner point, a parking space line corresponding to the first parking space corner point, position information of a third parking space corner point, and a parking space line corresponding to the third parking space corner point.

[0142] Among them, the first parking space corner point and the third parking space corner point are adjacent.

[0143] Typically, a parking space has four corner points, located at the top left, bottom left, top right, and bottom right corners of the space. The first and third corner points can be the top left and bottom left, top left and top right, top right and bottom right, or bottom left and bottom right.

[0144] S1202. The electronic device determines an area where the parking space including the first parking space corner point and the third parking space corner point is located by using the target intersection point, the position information of the first parking space corner point, the parking space line corresponding to the first parking space corner point, the position information of the third parking space corner point, and the parking space line corresponding to the third parking space corner point.

[0145] Among them, the target intersection point is the intersection of the midline of the first angle and the midline of the second angle. The first angle is obtained based on the first parking line and the second parking line of the first parking corner point, and the second angle is obtained based on the third parking line and the fourth parking line of the third parking corner point. The direction of the first parking line is opposite to the direction of the third parking line, and the angle of the second parking line is the same as the angle of the fourth parking line.

[0146] For example, Figure 13As shown, the first parking space corner point includes parking space corner point P0, parking space line P1, parking space line P2, and parking space line P3. The third parking space corner point includes parking space corner point Q0, parking space line Q1, parking space line Q2, and parking space line Q3. Parking space corner point P0, parking space line P1, and parking space line P2 form a first angle, while parking space corner point Q0, parking space line Q1, and parking space line Q3 form a second angle. Draw the center line P0P4 of the first angle and the center line Q0Q4 of the second angle. The intersection point S of the center line P0P4 and the center line Q0Q4 is the center point of the parking space. Using the position of the center point, the position information of the first parking space corner point, the parking space line corresponding to the first parking space corner point, the position information of the third parking space corner point, and the parking space line corresponding to the third parking space corner point, the area where the entire parking space is located can be obtained.

[0147] The above describes the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven hardware manner depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0148] The following, combined Figure 14 The parking space detection device provided in the embodiment of the present application is described in detail. It should be understood that the description of the device embodiment corresponds to the description of the method embodiment. Therefore, for matters not described in detail, please refer to the method embodiment above. For the sake of brevity, they will not be repeated here.

[0149] Figure 14 This is a schematic diagram of the structure of a parking space detection device provided in an embodiment of the present application. Figure 14 As shown, the parking space detection device is used to execute the above parking space detection method. The parking space detection device may include an acquisition unit 141 and a processing unit 142.

[0150] The acquisition unit 141 is configured to acquire a first image; wherein the first image includes a first parking space corner point and a parking space line corresponding to the first parking space corner point; for example, in combination with Figure 2 , the acquisition unit 141 can be used to execute S201.

[0151] The processing unit 142 is used to input the first image into the detection model to obtain the position information of the first parking space corner point and the parking space line corresponding to the first parking space corner point; wherein the detection model includes a heat map prediction branch and a direction prediction branch, the heat map prediction branch is used to obtain a heat map including the position information of the first parking space corner point, and the direction prediction branch is used to obtain a feature map including the direction information of the parking space line, obtain the position information of the first parking space corner point according to the heat map, and obtain the parking space line corresponding to the first parking space corner point according to the feature map. For example, combined with Figure 2 , the processing unit 142 can be used to execute S202.

[0152] Optionally, the processing unit is further configured to filter out the first heat point with a confidence level greater than a first threshold based on the confidence level of each heat point being the first parking space corner point. Figure 4 , the processing unit 142 can be used to execute S2021.

[0153] The processing unit is further configured to use the coordinate information of the pixel point corresponding to the first thermal point as the position information of the first parking space corner point. Figure 4 , the processing unit 142 can be used to execute S2022.

[0154] Optionally, the processing unit is further configured to compare the position information of the first parking space corner point with the coordinate information of the pixel point in the parking space line direction information to determine the first pixel point corresponding to the first parking space corner point; for example, in combination with Figure 4 , the processing unit 142 can be used to execute S2023.

[0155] The processing unit is further configured to perform Gaussian transformation on the confidence level of each angle of the first pixel point in the preset angle range to obtain a Gaussian distribution curve for characterizing the corresponding relationship between the angle and the confidence level; for example, in combination with Figure 4 , the processing unit 142 can be used to execute S2024.

[0156] The processing unit is further configured to filter out the target angle on the Gaussian distribution curve through a preset sliding window, and use the parking space line indicated by the target angle as the parking space line corresponding to the first parking space corner point. Figure 4 , the processing unit 142 can be used to execute S2025.

[0157] Optionally, the processing unit is further configured to overlap the sliding window with the Gaussian distribution curve, determine the intersection point of the overlapping sliding window and the Gaussian distribution curve, and use the angle corresponding to the intersection point as the target angle. Figure 4 , the processing unit 142 can be used to execute S20251.

[0158] Optionally, the acquisition unit is further configured to acquire multiple sample images and target parking space corner point identifiers in the multiple sample images; for example, in combination with Figure 6 , the acquisition unit 141 can be used to execute S2011.

[0159] The processing unit is further configured to generate multiple heat maps based on multiple sample images and target parking space corner point identifiers in the multiple sample images, and determine the heat value of each heat point in the multiple heat maps; wherein a heat point is used to represent a parking space corner point, and the heat value of a heat point is used to represent the confidence that the parking space corner point corresponding to the heat point is the target parking space corner point of the sample image; for example, combined with Figure 6 , the processing unit 142 can be used to execute S2012.

[0160] The processing unit is further configured to train the detection model to be trained using the thermal values ​​of each thermal point in the multiple sample images and multiple thermal maps, and generate a trained detection model. Figure 6 , the processing unit 142 can be used to execute S2013.

[0161] Optionally, the acquisition unit is further configured to acquire multiple sample images and parking space line angle identifiers of target parking space corner points in the multiple sample images, and set the confidence level of the parking space line angle identifier of the target parking space corner point to 1; for example, in combination with Figure 8 , the acquisition unit 141 can be used to execute S2014.

[0162] The processing unit is further configured to train the detection model to be trained using the multiple sample images, the parking line angle identification of the target parking space corner point in the multiple sample images, and the confidence level of the parking line angle identification of the target parking space corner point to generate a trained detection model. Figure 8 , the processing unit 142 can be used to execute S2015.

[0163] Optionally, the angle of the parking space line corresponding to the first parking space corner point is the direction of the parking space line corresponding to the first parking space corner point relative to the horizontal direction of the first image.

[0164] Optionally, the processing unit is further configured to input multiple images into the detection model to obtain multiple parking space corner points and parking space lines corresponding to the multiple parking space corner points; the multiple images all contain the same object as the first image; for example, combined with Figure 9 , the processing unit 142 can be used to execute S901.

[0165] The processing unit is further configured to perform information tracking processing on the same object in the first image and the multiple images based on the first image and the multiple images to obtain a target image, wherein the target image includes a position tag of the same object; for example, in combination with Figure 9 , the processing unit 142 can be used to execute S902.

[0166] The processing unit is further configured to determine whether the target image contains undetected parking space corner points and parking space lines based on the first parking space corner point, the parking space line corresponding to the first parking space corner point, multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points, and the number of vehicles in the target image; for example, in combination with Figure 9 , the processing unit 142 can be used to execute S903.

[0167] The processing unit is further configured to determine undetected parking space corner points and parking space lines using the first parking space corner point, the parking space line corresponding to the first parking space corner point, multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points, and parking space marking rules. Figure 9 , the processing unit 142 can be used to execute S904.

[0168] Optionally, the processing unit is further configured to input the first image into the detection model to obtain position information of multiple parking space corner points in the first image and parking space lines corresponding to the multiple parking space corner points; wherein the multiple parking space corner points include the first parking space corner point; for example, in combination with Figure 10 , the processing unit 142 can be used to execute S1001.

[0169] The processing unit is further configured to determine whether the first image contains undetected parking corner points based on the position information of each parking corner point among the plurality of parking corner points in the first image and the number of vehicles in the first image; for example, in combination with Figure 10 , the processing unit 142 can be used to execute S1002.

[0170] The processing unit is further configured to determine the position information of the parking space corner points that are not detected in the first image by using the position information of each parking space corner point and the parking space marking rule. Figure 10 , the processing unit 142 can be used to execute S1003.

[0171] Optionally, the processing unit is further configured to input the first image into the detection model to obtain the position information of the first parking space corner point, the parking space line corresponding to the first parking space corner point, the position information of the third parking space corner point and the parking space line corresponding to the third parking space corner point, wherein the first parking space corner point and the third parking space corner point are adjacent to each other; for example, in combination with Figure 10 , the processing unit 142 can be used to execute S1201.

[0172] The processing unit is further configured to determine the area where the parking space including the first parking space corner point and the third parking space corner point is located by using the target intersection point, the position information of the first parking space corner point, the parking space line corresponding to the first parking space corner point, the position information of the third parking space corner point and the parking space line corresponding to the third parking space corner point; the target intersection point is the intersection of the center line of the first angle and the center line of the second angle, the first angle is obtained based on the first parking space line and the second parking space line of the first parking space corner point, the second angle is obtained based on the third parking space line and the fourth parking space line of the third parking space corner point, the direction of the first parking space line is opposite to the direction of the third parking space line, and the angle of the second parking space line is the same as the angle of the fourth parking space line. For example, in combination Figure 10 , the processing unit 142 can be used to execute S1202.

[0173] Of course, the parking space detection device provided in the embodiment of the present application includes but is not limited to the above modules. For example, the search intention determination device may also include a storage unit 143. The storage unit 143 may be used to store the program code of the search intention determination device, and may also be used to store data generated during the operation of the search intention determination device, such as data in a write request.

[0174] Figure 15 This is a structural diagram of an electronic device provided in an embodiment of the present application. The electronic device may include: at least two processors, a memory 152, a communication interface 153 and a communication bus 154.

[0175] The following combination Figure 15 The following is a detailed introduction to the various components of the terminal equipment overload detection device:

[0176] The processor 151 is the control center of the terminal device overload detection device and can be a single processor or a collective term for multiple processing elements. For example, the processor 151 is a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application, such as one or more DSPs or one or more field programmable gate arrays (FPGAs).

[0177] In a specific implementation, as an embodiment, the processor 151 may include one or more CPUs, such as Figure 15 Furthermore, as an embodiment, the terminal device overload detection device may include multiple processors, such as Figure 151 and 155 are shown in FIG. Each of these processors can be a single-core processor (Single-CPU) or a multi-core processor (Multi-CPU). The processor here can refer to one or more terminal devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0178] The memory 152 may be a read-only memory (ROM) or other type of static storage terminal device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage terminal device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage terminal device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 152 may exist independently and communicate with the processor 151 through the communication bus 154 for overload detection. The memory 152 may also be integrated with the processor 151.

[0179] In a specific implementation, the memory 152 is used to store the data in this application and execute the software program of this application. The processor 151 can perform various functions of the air conditioner by running or executing the software program stored in the memory 152 and calling the data stored in the memory 152.

[0180] The communication interface 153 uses any transceiver or other device for communicating with other terminal devices or communication networks, such as radio access networks (RAN), wireless local area networks (WLAN), terminal devices, and the cloud. The communication interface 153 may include an acquisition unit to implement an acquisition function and a transmission unit to implement a transmission function.

[0181] The communication bus 154 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0182] Another embodiment of the present application further provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a computer, the computer executes the method shown in the above method embodiment.

[0183] In some embodiments, the disclosed methods may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or on other non-transitory media or articles of manufacture.

[0184] Figure 16 A conceptual partial view of a computer program product provided by an embodiment of the present application is schematically shown, where the computer program product includes a computer program for executing a computer process on a computing terminal device.

[0185] In one embodiment, the computer program product is provided using a signal bearing medium 1610. The signal bearing medium 1610 may include one or more program instructions that, when executed by one or more processors, may provide the above-described Figure 2 Thus, for example, reference to Figure 2 In the embodiment shown in , one or more features of S201-S202 may be undertaken by one or more instructions associated with the signal bearing medium 1610. In addition, Figure 16 The program instructions in also describe example instructions.

[0186] In some examples, signal bearing medium 1610 may include computer readable medium 1611 such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM), a random access memory (RAM), and the like.

[0187] In some embodiments, the signal bearing medium 1610 may include a computer recordable medium 1612 such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, or the like.

[0188] In some embodiments, signal bearing medium 1610 may include communication medium 1613 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0189] The signal bearing medium 1610 may be communicated by a wireless form of communication medium 1613. The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.

[0190] In some examples, such as for Figure 2 The described data writing device may be configured to provide various operations, functions, or actions in response to one or more program instructions via computer-readable media 1611 , computer-recordable media 1612 , and / or communication media 1613 .

[0191] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete the full classification or partial functions described above.

[0192] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication overload detection shown or discussed can be through some interfaces, indirect coupling of devices or units or communication overload detection, which can be electrical, mechanical or other forms.

[0193] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0194] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0195] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the full classification part or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a terminal device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute the full classification part or part S of the method of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk.

[0196] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A parking space detection method, characterized in that: include: Acquire a first image; wherein the first image includes a first parking space corner point and a parking space line corresponding to the first parking space corner point; Inputting the first image into a detection model to obtain position information of the first parking space corner point and a parking space line corresponding to the first parking space corner point; The detection model includes a heat map prediction branch and a direction prediction branch. The heat map prediction branch is used to obtain a heat map including the location information of parking space corner points, and the direction prediction branch is used to obtain a feature map including the direction information of parking space lines. The location information of the first parking space corner points is obtained based on the heat map. The parking space line direction information includes the coordinate information of each pixel point and the confidence level of each pixel point at each angle within a preset angle range. Comparing the position information of the first parking space corner point with the coordinate information of the pixel points in the parking space line direction information to determine a first pixel point corresponding to the first parking space corner point; Performing a Gaussian transformation on the confidence level of each angle of the first pixel point within a preset angle range to obtain a Gaussian distribution curve for characterizing the corresponding relationship between the angle and the confidence level; On the Gaussian distribution curve, a target angle is screened out through a preset sliding window, and the parking space line indicated by the target angle is used as the parking space line corresponding to the first parking space corner point.

2. The method according to claim 1, characterized in that The parking space corner point location information includes coordinate information of each pixel point and at least one thermal cluster, each thermal cluster includes multiple thermal points and a confidence level that each thermal point is the first parking space corner point. The obtaining of the location information of the first parking space corner point based on the thermal map includes: Based on the confidence level of each heat point being the first parking space corner point, screening out the first heat point having a confidence level greater than a first threshold; The coordinate information of the pixel point corresponding to the first thermal point is used as the position information of the first parking space corner point.

3. The method according to claim 1, characterized in that The target angle is screened out on the Gaussian distribution curve through a preset sliding window, including: The sliding window is overlapped with the Gaussian distribution curve, an intersection point between the sliding window and the Gaussian distribution curve after the overlap is determined, and an angle corresponding to the intersection point is used as the target angle.

4. The method according to claim 1, wherein Before inputting the first image into the detection model, the method further includes: Acquire multiple sample images and multiple target parking space corner point identifiers in the sample images; Based on multiple sample images and multiple target parking space corner point identifiers in the sample images, multiple heat maps are generated, and a heat value of each heat point in the multiple heat maps is determined; wherein a heat point is used to represent a parking space corner point, and the heat value of a heat point is used to represent the confidence level that the parking space corner point corresponding to the heat point is the target parking space corner point in the sample image; The detection model to be trained is trained using the multiple sample images and the thermal value of each thermal point in the multiple thermal maps to generate the trained detection model.

5. The method according to claim 1, wherein Before inputting the first image into the detection model, the method further includes: Acquire multiple sample images and parking space line angle identifiers of target parking space corner points in the multiple sample images, and set the confidence level of the parking space line angle identifier of the target parking space corner point to 1; The detection model to be trained is trained using the plurality of sample images, the parking space line angle identifiers of the target parking space corner points in the plurality of sample images, and the confidence levels of the parking space line angle identifiers of the target parking space corner points to generate the trained detection model.

6. The method according to any one of claims 1 to 5, characterized in that The angle of the parking space line corresponding to the first parking space corner point is the angle of the parking space line corresponding to the first parking space corner point relative to the horizontal direction of the first image.

7. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Inputting a plurality of images into the detection model to obtain a plurality of parking space corner points and parking space lines corresponding to the plurality of parking space corner points; wherein the plurality of images all contain the same object as that in the first image; Based on the first image and the multiple images, performing information tracking processing on the same object in the first image and the multiple images to obtain a target image, wherein the target image includes a position tag of the same object; determining, based on the first parking space corner point, the parking space line corresponding to the first parking space corner point, the multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points, and the number of vehicles in the target image, whether the target image includes undetected parking space corner points and parking space lines; The undetected parking space corner points and parking space lines are determined by using the first parking space corner point, the parking space line corresponding to the first parking space corner point, the multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points, and parking space marking rules.

8. The method according to any one of claims 1 to 5, characterized in that Inputting the first image into a detection model to obtain position information of the first parking space corner point and a parking space line corresponding to the first parking space corner point includes: Inputting the first image into a detection model to obtain position information of a plurality of parking space corner points in the first image and parking space lines corresponding to the plurality of parking space corner points; wherein the plurality of parking space corner points includes the first parking space corner point; The method further comprises: determining, based on position information of each parking space corner point among a plurality of parking space corner points in the first image and the number of vehicles in the first image, whether the first image includes an undetected parking space corner point; The position information of each parking space corner point that is not detected in the first image is determined by using the position information of each parking space corner point and the parking space marking rule.

9. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Inputting the first image into a detection model, obtaining position information of the first parking space corner point, a parking space line corresponding to the first parking space corner point, position information of a third parking space corner point, and a parking space line corresponding to the third parking space corner point, wherein the first parking space corner point and the third parking space corner point are adjacent to each other; Determine an area where the parking space including the first parking space corner point and the third parking space corner point is located by using the target intersection point, the position information of the first parking space corner point, the parking space line corresponding to the first parking space corner point, the position information of the third parking space corner point, and the parking space line corresponding to the third parking space corner point; the target intersection point is the intersection of a midline of a first angle and a midline of a second angle, the first angle is obtained based on the first parking space line and the second parking space line of the first parking space corner point, the second angle is obtained based on the third parking space line and the fourth parking space line of the third parking space corner point, the direction of the first parking space line is opposite to the direction of the third parking space line, and the angle of the second parking space line is the same as the angle of the fourth parking space line; The second parking space line angle is an angle of the second parking space line relative to the horizontal direction of the first image, and the fourth parking space line angle is an angle of the fourth parking space line relative to the horizontal direction of the first image.

10. A parking space detection device, characterized in that: include: An acquisition unit, configured to acquire a first image; wherein the first image includes a first parking space corner point and a parking space line corresponding to the first parking space corner point; a processing unit, configured to input the first image into a detection model to obtain position information of the first parking space corner point and a parking space line corresponding to the first parking space corner point; wherein the detection model includes a heat map prediction branch and a direction prediction branch, the heat map prediction branch is configured to obtain a heat map including position information of the parking space corner point, and the direction prediction branch is configured to obtain a feature map including direction information of the parking space line; the position information of the first parking space corner point is obtained based on the heat map, and the parking space line corresponding to the first parking space corner point is obtained based on the feature map; The processing unit is further configured to input a plurality of images into the detection model to obtain a plurality of parking space corner points and parking space lines corresponding to the plurality of parking space corner points; the plurality of images all contain the same object as that in the first image; The processing unit is further configured to perform information tracking processing on the same object in the first image and the multiple images based on the first image and the multiple images to obtain a target image, wherein the target image includes a location tag of the same object; The processing unit is further configured to determine whether the target image includes undetected parking space corner points and parking space lines based on the first parking space corner point, the parking space line corresponding to the first parking space corner point, the multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points, and the number of vehicles in the target image; The processing unit is further used to determine the undetected parking space corner points and parking space lines by using the first parking space corner point, the parking space line corresponding to the first parking space corner point, the multiple parking space corner points, the parking space lines corresponding to the multiple parking space corner points and parking space marking rules.

11. The device according to claim 10, characterized in that The processing unit is further configured to screen out first thermal points having a confidence level greater than a first threshold based on the confidence level of each thermal point being the first parking space corner point; the parking space corner point position information includes coordinate information of each pixel point and at least one thermal cluster, each thermal cluster including multiple thermal points and a confidence level of each thermal point being the first parking space corner point; The processing unit is further configured to use the coordinate information of the pixel point corresponding to the first thermal point as the position information of the first parking space corner point; The processing unit is further configured to compare the position information of the first parking space corner point with the coordinate information of the pixel points in the parking space line direction information to determine a first pixel point corresponding to the first parking space corner point; The processing unit is further configured to perform Gaussian transformation on the confidence level of each angle of the first pixel point within a preset angle range to obtain a Gaussian distribution curve for characterizing the corresponding relationship between the angle and the confidence level; The processing unit is further configured to screen a target angle on the Gaussian distribution curve through a preset sliding window, and use the parking space line indicated by the target angle as the parking space line corresponding to the first parking space corner point; The processing unit is further configured to overlap the sliding window with the Gaussian distribution curve, determine an intersection point between the sliding window and the Gaussian distribution curve after overlap, and use an angle corresponding to the intersection point as the target angle; The acquisition unit is further configured to acquire a plurality of sample images and target parking space corner point identifiers in the plurality of sample images; The processing unit is further configured to generate a plurality of heat maps based on a plurality of sample images and a plurality of target parking space corner point identifiers in the sample images, and determine a heat value for each heat point in the plurality of heat maps; wherein a heat point is used to represent a parking space corner point, and a heat value of a heat point is used to represent a confidence level that the parking space corner point corresponding to the heat point is the target parking space corner point in the sample image; The processing unit is further configured to train the detection model to be trained using the plurality of sample images and the thermal value of each thermal point in the plurality of thermal maps to generate the trained detection model; The acquisition unit is further configured to acquire a plurality of sample images and parking space line angle identifiers of target parking space corner points in the plurality of sample images, and set the confidence level of the parking space line angle identifiers of the target parking space corner points to 1; The processing unit is further configured to train the detection model to be trained by using the plurality of sample images, the parking space line angle identifiers of the target parking space corner points in the plurality of sample images, and the confidence levels of the parking space line angle identifiers of the target parking space corner points to generate the trained detection model; The processing unit is further configured to input the first image into a detection model to obtain position information of a plurality of parking space corner points in the first image and parking space lines corresponding to the plurality of parking space corner points; wherein the plurality of parking space corner points includes the first parking space corner point; The processing unit is further configured to determine whether the first image includes undetected parking space corner points based on the position information of each parking space corner point among the plurality of parking space corner points in the first image and the number of vehicles in the first image; The processing unit is further configured to determine the position information of parking space corner points that are not detected in the first image by using the position information of each parking space corner point and parking space marking rules; The processing unit is further configured to input the first image into a detection model to obtain position information of the first parking space corner point, a parking space line corresponding to the first parking space corner point, position information of a third parking space corner point, and a parking space line corresponding to the third parking space corner point, wherein the first parking space corner point and the third parking space corner point are adjacent to each other; The processing unit is further configured to determine an area where a parking space including the first parking space corner point and the third parking space corner point is located by using a target intersection point, position information of the first parking space corner point, a parking space line corresponding to the first parking space corner point, position information of a third parking space corner point, and a parking space line corresponding to the third parking space corner point; the target intersection point is an intersection of a center line of a first angle and a center line of a second angle, the first angle is obtained based on the first parking space line and the second parking space line of the first parking space corner point, the second angle is obtained based on the third parking space line and the fourth parking space line of the third parking space corner point, a direction of the first parking space line is opposite to a direction of the third parking space line, and an angle of the second parking space line is the same as an angle of the fourth parking space line; The second parking space line angle is an angle of the second parking space line relative to the horizontal direction of the first image, and the fourth parking space line angle is an angle of the fourth parking space line relative to the horizontal direction of the first image.

12. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the parking space detection method according to any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that The method comprises instructions, which, when executed by an electronic device, enable the electronic device to execute the parking space detection method according to any one of claims 1 to 9.

Images