Parking space occupancy detection method, detection device and computer readable storage medium

Abstract

The application discloses a parking space occupation detection method, a detection device and a computer readable storage medium. The detection method comprises the following steps: acquiring a detection image, the detection image comprising at least one parking space region; determining a target detection frame corresponding to a target vehicle in the detection image and a rotation angle by using a target detection model; wherein the target detection frame is a rectangle, and the rotation angle represents an angle between a geometric feature line of the vehicle and a geometric feature line of the target detection frame; determining a vehicle region from the target detection frame based on the rotation angle, the vehicle region being smaller than the target detection frame; and determining a parking space occupation result based on the vehicle region and the parking space region. In the foregoing manner, the accuracy of determining the vehicle region can be improved, and the parking space occupation result can be determined, so that the correspondence between the parking space and the vehicle is more accurate, and the accuracy of the parking space occupation judgment is improved.

Description

Technical Field

[0001] This application relates to the field of parking space occupancy technology, and in particular to methods, devices and computer-readable storage media for detecting parking space occupancy. Background Technology

[0002] With the continuous increase in the number of cars and the rapid development and application of computer vision algorithms, video analysis algorithms are widely used in practical tasks such as parking space monitoring, improving the efficiency of parking lot management.

[0003] The inventors' long-term research revealed that the planning of parking spaces and the setting of monitoring angles could lead to errors in judgment, resulting in a decrease in the accuracy of binding parking space status. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a method, device, and computer-readable storage medium for detecting parking space occupancy, which can improve the accuracy of determining vehicle areas, thereby determining the parking space occupancy result, making the correspondence between parking spaces and vehicles more accurate, and improving the accuracy of parking space occupancy judgment.

[0005] To address the aforementioned problems, this application provides a method for detecting parking space occupancy. This method includes: acquiring a detection image, which includes at least one parking space region; using a target detection model to determine the target detection box corresponding to a target vehicle in the detection image and its rotation angle; wherein the target detection box is rectangular, and the rotation angle represents the angle between the geometric feature lines of the vehicle and the geometric feature lines of the target detection box; determining a vehicle region from the target detection box based on the rotation angle, wherein the vehicle region is smaller than the target detection box; and determining the parking space occupancy result based on the vehicle region and the parking space region.

[0006] The rotation angle includes a first rotation angle and a second rotation angle; determining the vehicle region from the target detection box based on the rotation angle includes: determining the diagonal of the target detection box; controlling the diagonal to rotate according to the first rotation angle and the second rotation angle to intersect with the target detection box; and taking the area formed by the points intersecting with the target detection box as the vehicle region.

[0007] The target detection box includes a first side, a second side, a third side, and a fourth side connected in sequence. Controlling the diagonal to rotate according to a first rotation angle and a second rotation angle, intersecting the target detection box, includes: controlling the diagonal to rotate clockwise around the center point according to the first rotation angle, intersecting the first side at a first intersection point, and intersecting the third side at a second intersection point; determining the line connecting the first and second intersection points; controlling the line to rotate clockwise around the second intersection point according to the second rotation angle, intersecting the second side at a third intersection point; and controlling the line to rotate counterclockwise around the first intersection point according to the second rotation angle, intersecting the fourth side at a fourth intersection point; and using the area formed by the points intersecting the target detection box as the vehicle area, includes: connecting the first, third, second, and fourth intersection points, and using the area formed by the first, third, second, and fourth intersection points as the vehicle area.

[0008] The method further includes: acquiring training images; wherein the training images are labeled with target detection boxes and ground truth information of the target vehicle, and the ground truth information includes the intersection of the ground truth box and the target detection box of the target vehicle; using a target detection model to detect the training images to obtain the detection information of the target vehicle, wherein the detection information of the target vehicle includes the final detection box of the target vehicle; and adjusting the network parameters of the target detection model according to the difference between the ground truth information and the detection information of the target vehicle.

[0009] The target detection model includes a feature extraction network and a classification layer. The target detection model is used to detect training images to obtain the detection information of target vehicles, including: inputting the training images into the feature extraction network to obtain multi-dimensional feature maps; and inputting the multi-dimensional feature maps into the classification layer to obtain the detection information of target vehicles in the training images.

[0010] The process involves inputting the training image into a feature extraction network to obtain a multidimensional feature map, including: using the feature extraction network to sequentially downsample the training image N times to obtain an N-dimensional initial feature map, where N is greater than 2; for the N-dimensional initial feature map, performing the (i+1)th upsampling on the N-th initial feature map based on the Ni-dimensional initial feature map to obtain the (i+1)th final feature map; where i is an integer from 0 to N-1.

[0011] The feature extraction network is a Feature Pyramid Network (FPN), which includes multiple upsampling layers and downsampling layers corresponding to the multiple upsampling layers. Each upsampling or downsampling layer includes convolutions with different resolutions.

[0012] The process of determining parking space occupancy based on vehicle area and parking space area includes: determining at least one target parking space area corresponding to the vehicle area; determining the intersection-merge ratio of the vehicle area and the at least one target parking space area; and determining the parking space occupancy result based on the intersection-merge ratio.

[0013] Determining at least one target parking space area corresponding to the vehicle area includes: acquiring at least one pre-configured target parking space area; or, determining at least one target parking space area using an image processing algorithm; or, determining at least one target parking space area using a target detection model.

[0014] The process of determining the intersection ratio between a vehicle area and at least one target parking space area includes: determining the intersection ratio between a vehicle area and each target parking space area; and determining the parking space occupancy result based on the intersection ratio, including: if the intersection ratio is greater than or equal to a first preset value, then the target parking space area is determined to be occupied; if the intersection ratio is less than the first preset value, and there are two intersection ratios greater than a second preset value, then two adjacent target parking space areas are determined to be occupied; the second preset value is less than the first preset value.

[0015] To address the aforementioned issues, another technical solution adopted in this application is to provide a parking space occupancy detection device, which includes a processor and a memory coupled to the processor; wherein the memory is used to store a computer program, and the processor is used to execute the computer program to implement the method provided by the above technical solution.

[0016] To address the aforementioned problems, another technical solution adopted in this application is to provide a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the method provided by the above technical solution.

[0017] The beneficial effects of this application are as follows: Unlike existing technologies, this application provides a parking space occupancy detection method. This method includes: acquiring a detection image, which includes at least one parking space region; using a target detection model to determine the target detection box corresponding to a target vehicle in the detection image and its rotation angle; wherein the target detection box is rectangular, and the rotation angle represents the angle between the geometric feature lines of the vehicle and the geometric feature lines of the target detection box; determining a vehicle region from the target detection box based on the rotation angle, where the vehicle region is smaller than the target detection box; and determining the parking space occupancy result based on the vehicle region and the parking space region. By using the determined rotation angle to determine the vehicle region from the target detection box, the accuracy of determining the vehicle region can be improved, thereby determining the parking space occupancy result, making the correspondence between parking spaces and vehicles more accurate, and improving the accuracy of parking space occupancy judgment. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an embodiment of the parking space occupancy detection method provided in this application;

[0019] Figure 2 This is a schematic diagram of an embodiment of the detection image provided in this application when there is no vehicle.

[0020] Figure 3 and Figure 4 This is a schematic diagram illustrating an application scenario of the parking space occupancy detection method provided in this application;

[0021] Figure 5 and Figure 6 This is a schematic diagram illustrating another application scenario of the parking space occupancy detection method provided in this application;

[0022] Figure 7 This is a flowchart illustrating another embodiment of the parking space occupancy detection method provided in this application;

[0023] Figure 8 This is a flowchart illustrating an embodiment of step 55 provided in this application;

[0024] Figure 9 This is a schematic diagram illustrating another application scenario of the parking space occupancy detection method provided in this application;

[0025] Figure 10 This is a flowchart illustrating another embodiment of the parking space occupancy detection method provided in this application;

[0026] Figure 11 This is a flowchart illustrating an embodiment of steps 85 and 86 provided in this application;

[0027] Figure 12 This is a schematic diagram illustrating another application scenario of the parking space occupancy detection method provided in this application;

[0028] Figure 13 This is a flowchart illustrating another embodiment of the parking space occupancy detection method provided in this application;

[0029] Figure 14 This is a flowchart illustrating an embodiment of the target detection model training process provided in this application;

[0030] Figure 15 This is a flowchart illustrating an embodiment of step 122 provided in this application;

[0031] Figure 16 This is a schematic diagram of the structure of an embodiment of the feature extraction network provided in this application;

[0032] Figure 17 This is a schematic diagram of an embodiment of a convolutional layer in the feature extraction network provided in this application;

[0033] Figure 18 This is a schematic diagram illustrating another application scenario of the parking space occupancy detection method provided in this application;

[0034] Figure 19 This is a schematic diagram illustrating another application scenario of the parking space occupancy detection method provided in this application;

[0035] Figure 20 This is a schematic diagram of the non-silent angle loss training results provided in this application;

[0036] Figure 21 This is a schematic diagram of the silent angle loss training results provided in this application;

[0037] Figure 22 This is a schematic diagram of the silent angle loss training results provided in this application;

[0038] Figure 23 This is a flowchart illustrating an embodiment of the parking space occupancy detection device provided in this application;

[0039] Figure 24 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are only for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts related to this application are shown in the accompanying drawings, not all structures. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0041] The terms "first," "second," etc., used in this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0042] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0043] The inventors, through long-term research, discovered two scenarios in outdoor parking lots. Based on the camera's installation angle and the multi-area distribution of parking areas, one scenario, as seen in the image, involves parking lines that are horizontal to the camera's viewpoint, while the other involves parking lines that are not horizontal. When the viewpoint is horizontal, the front, rear, or body of the vehicle faces the camera, and the vehicle's bounding rectangle matches the height of the parking space's line. When the viewpoint is not horizontal, there is obstruction between vehicles, and the parking line changes from a horizontal rectangle to an inclined quadrilateral. The larger the non-horizontal angle, the more severe the obstruction, and the greater the inclination of the parking line. This causes the vehicle's bounding rectangle to span multiple parking lines. In this case, whether the parking line is bounded by the center point of the vehicle's bounding rectangle or by the center point of the parking line, there is a risk of misjudgment, leading to a decrease in the accuracy of binding the parking space status. Based on this, this application proposes the following technical solution to address either of the above technical problems.

[0044] See Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the parking space occupancy detection method provided in this application. The method includes:

[0045] Step 11: Obtain the detection image, which includes at least one parking space area.

[0046] In some embodiments, image acquisition devices can be used for image acquisition. For example, a camera can be used to acquire images of a target area, and the acquired images can be used as detection images. At least one parking space is provided within the target area. Figure 2 As shown, the detected image includes parking space areas A, B, C, D, E, F, G, H, I, J, K, and L. Vehicles can be parked in each parking space area. Figure 2 In some cases, the parking lines in the parking areas are not consistent, meaning that the orientation of the vehicles after parking is also inconsistent.

[0047] Step 12: Use the object detection model to determine the target detection box and rotation angle corresponding to the target vehicle in the detection image; where the target detection box is a rectangle, and the rotation angle represents the angle between the geometric feature lines of the vehicle and the geometric feature lines of the target detection box.

[0048] If a vehicle is parked within the area corresponding to the detected image, the target detection box and rotation angle corresponding to the target vehicle in the detected image can be determined.

[0049] In some implementations, object detection models can be used to determine the bounding box and rotation angle corresponding to the target vehicle in the detected image. For example, the rotation angle can be the angle between the vehicle's diagonal and the rectangle's diagonal, or the angle between one side of the vehicle and one side of the rectangle.

[0050] Step 13: Determine the vehicle region from the target detection box based on the rotation angle. The vehicle region is smaller than the target detection box.

[0051] The target detection bounding box is rectangular, meaning it can be the bounding rectangle of the target vehicle. The actual area of ​​the target vehicle can be determined based on the rotation angle.

[0052] In some implementations, combined with Figure 3 and Figure 4 Explanation:

[0053] The rectangle abcd of the target detection box is as follows Figure 3 As shown, the plane includes the first side ab, the second side bc, the third side cd, and the fourth side da, which are connected in sequence. The diagonal is ac.

[0054] Control the diagonal to rotate clockwise around the center point O, intersecting the first side ab at the first intersection point a′, and intersecting the third side cd at the second intersection point c′.

[0055] Control the diagonal to rotate counterclockwise around the center point O, intersecting the second side bc at the third intersection point b′, and intersecting the fourth side da at the fourth intersection point d′.

[0056] like Figure 4 As shown, the first intersection point a′, the third intersection point b′, the second intersection point c′, and the fourth intersection point d′ are connected sequentially. The area formed by these four intersection points is designated as the vehicle area. That is, quadrilateral a′b′c′d′ is the vehicle area.

[0057] In other embodiments, combined with Figure 5 and Figure 6 Explanation:

[0058] The rectangle abcd of the target detection box is as follows Figure 5 As shown, the plane includes the first side ab, the second side bc, the third side cd, and the fourth side da, which are connected in sequence. The diagonal is ac.

[0059] Control the diagonal to rotate clockwise around the center point O, intersecting the first side ab at the first intersection point a′, and intersecting the third side cd at the second intersection point c′.

[0060] Then find a point b' on the second side bc and a point d' on the fourth side da, such that c'b'⊥a'b' and c'd'⊥a'd', as shown below. Figure 6As shown. Connect a′, b′, c′, and d′ sequentially, and use the rectangular area formed by a′, b′, c′, and d′ as the vehicle area. That is, the quadrilateral a′b′c′d′ is used as the vehicle area.

[0061] Step 14: Determine the parking space occupancy result based on the vehicle area and parking space area.

[0062] The images of the parking space area are acquired in advance after the parking spaces are planned. Once the vehicle area is determined, the parking space area within that vehicle area can be identified. Specifically, this can be determined based on image coordinates. For example, if the image acquisition device is fixed, the image coordinates of the parking space area are fixed, and the image coordinates of the vehicle area are also fixed after the vehicle area is determined. Therefore, the overlap between the parking space area and the vehicle area can be used to determine whether the parking space is occupied.

[0063] In one application scenario, this embodiment is applied to a parking lot. A display screen is installed at the entrance of the parking lot, showing the parking space information, such as XX available spaces and YY occupied spaces. When a vehicle enters the parking lot and parks itself, the steps described in the above embodiment are used to determine the specific parking space occupied by the vehicle. If the vehicle is not parked in a designated parking area, a warning is issued. The parking lot can also be divided into zones, such as three floors. Each floor has a certain number of parking spaces. After determining the remaining parking spaces on each floor using the above method, newly entering vehicles can be prompted to park on the floor with more remaining spaces.

[0064] In this embodiment, a detection image is acquired, which includes at least one parking space area. A target detection model is used to determine the target detection box corresponding to the target vehicle in the detection image and its rotation angle. The target detection box is a rectangle, and the rotation angle represents the angle between the geometric feature lines of the vehicle and the geometric feature lines of the target detection box. Based on the rotation angle, a vehicle area is determined from the target detection box, and the vehicle area is smaller than the target detection box. This method of determining parking space occupancy based on the vehicle area and the parking space area, by using the determined rotation angle to determine the vehicle area from the target detection box, improves the accuracy of determining the vehicle area, thereby determining the parking space occupancy result. This makes the correspondence between parking spaces and vehicles more accurate and improves the accuracy of parking space occupancy judgment.

[0065] See Figure 7 , Figure 7 This is a flowchart illustrating another embodiment of the parking space occupancy detection method provided in this application. The method includes:

[0066] Step 51: Obtain the detection image, which includes at least one parking space area.

[0067] Step 52: Use the object detection model to determine the target detection box and rotation angle corresponding to the target vehicle in the detection image; where the target detection box is a rectangle, and the rotation angle represents the angle between the geometric feature lines of the vehicle and the geometric feature lines of the target detection box.

[0068] Steps 51 and 52 have the same or similar technical solutions as the above embodiments, and will not be described in detail here.

[0069] Step 53: Determine the diagonal of the target detection box.

[0070] Since the target detection box is rectangular, its diagonal can be determined.

[0071] Step 54: Control the diagonal to rotate according to the first rotation angle and the second rotation angle, so that it intersects with the target detection box.

[0072] Step 55: The region formed by the points that intersect with the target detection box is taken as the vehicle region.

[0073] In some embodiments, see Figure 8 Steps 54 and 55 can be implemented using the following process:

[0074] Step 61: Control the diagonal to rotate clockwise around the center point according to the first rotation angle, intersecting the first side at the first intersection point, and intersecting the third side at the second intersection point.

[0075] The target detection box consists of a first side, a second side, a third side, and a fourth side connected in sequence.

[0076] Step 62: Determine the line connecting the first intersection point and the second intersection point.

[0077] Step 63: Control the connection line to rotate clockwise with the second intersection point as the reference and according to the second rotation angle, intersecting the second side at the third intersection point; and control the connection line to rotate counterclockwise with the first intersection point as the reference and according to the second rotation angle, intersecting the fourth side at the fourth intersection point.

[0078] Step 64: Connect the first intersection point, the third intersection point, the second intersection point, and the fourth intersection point in sequence, and use the area formed by the first intersection point, the third intersection point, the second intersection point, and the fourth intersection point as the vehicle area.

[0079] In one application scenario, the rectangle abcd of the target detection box is as follows: Figure 9 As shown, the plane includes the first side ab, the second side bc, the third side cd, and the fourth side da, which are connected in sequence. The diagonal is ac.

[0080] Control the diagonal to rotate clockwise around the center point O, according to the first rotation angle α, intersecting the first side ab at the first intersection point a′, and intersecting the third side cd at the second intersection point c′.

[0081] Determine the line a′c′ connecting the first and second intersection points.

[0082] The control line a′c′ is rotated clockwise with the second intersection point c′ as the reference and the second rotation angle β, intersecting the second side bc at the third intersection point b′; and the control line a′c′ is rotated counterclockwise with the first intersection point a′ as the reference and the second rotation angle β, intersecting the fourth side da at the fourth intersection point d′.

[0083] Connect the first intersection point a′, the third intersection point b′, the second intersection point c′, and the fourth intersection point d′ in sequence. The area formed by these four intersection points is designated as the vehicle area. That is, quadrilateral a′b′c′d′ is designated as the vehicle area.

[0084] In the above process, combined with Figure 9 Explanation: By rotating clockwise by a first rotation angle α, the line intersects the first side ab at the first intersection point a′, and the third side cd at the second intersection point c′. This forms a line a′c′ connecting the first and second intersection points. Controlling this line a′c′ to rotate clockwise around the second intersection point c′, by a second rotation angle β, the line intersects the second side bc at the third intersection point b′, thus forming b′c′ and a′b′. Similarly, controlling this line a′c′ to rotate counterclockwise around the first intersection point a′, by a second rotation angle β, the line intersects the fourth side da at the fourth intersection point d′, thus forming c′d′ and a′d′. This results in a quadrilateral a′b′c′d′.

[0085] Among these issues, the image acquisition devices at different angles and the planning angle of the parking space area mean that, after image acquisition, the actual vehicle area in the image is not necessarily a rectangle, but a quadrilateral. Therefore, the vehicle area determined by using two rotation angles is more accurate.

[0086] In other embodiments, steps 54 and 55 may be the following process:

[0087] The diagonal is controlled to rotate clockwise around the center point according to a first rotation angle, intersecting the first side at a first intersection point and the third side at a second intersection point; a line is determined connecting the first and second intersection points; this line is controlled to rotate clockwise around the second intersection point according to a second rotation angle, intersecting the second side at a third intersection point; wherein the second and first rotation angles are complementary. The line is also controlled to rotate counterclockwise around the first intersection point according to a second rotation angle, intersecting the fourth side at a fourth intersection point; the first, third, second, and fourth intersection points are connected, and the area formed by these points is designated as the vehicle area.

[0088] It is understandable that there is no mandatory requirement for the order of rotation of the first rotation angle and the second rotation angle in the above instruction manual.

[0089] Step 56: Determine the parking space occupancy result based on the vehicle area and parking space area.

[0090] In some embodiments, parking space occupancy can be determined based on the overlap between the vehicle area and the parking space area. For example, if the vehicle area and the parking space area overlap by 70%, it is determined that the parking space in that area is occupied by the vehicle. If the vehicle area and the parking space area overlap by 20%, it is determined that the vehicle is illegally parked, and an early warning can be issued.

[0091] In this embodiment, by acquiring a detection image, which includes at least one parking space area; determining the target detection box corresponding to the target vehicle in the detection image, a first rotation angle, and a second rotation angle; determining the vehicle area from the target detection box based on the first rotation angle and the second rotation angle; and determining the parking space occupancy result based on the vehicle area and the parking space area, the method of determining the vehicle area from the target detection box using the determined first rotation angle and the second rotation angle can improve the accuracy of determining the vehicle area, thereby determining the parking space occupancy result, making the correspondence between parking spaces and vehicles more accurate, and improving the accuracy of parking space occupancy judgment.

[0092] See Figure 10 , Figure 10 This is a flowchart illustrating another embodiment of the parking space occupancy detection method provided in this application. The method includes:

[0093] Step 81: Obtain the detection image, which includes at least one parking space area.

[0094] Step 82: Use the object detection model to determine the target detection box and rotation angle corresponding to the target vehicle in the detection image; where the target detection box is a rectangle, and the rotation angle represents the angle between the geometric feature line of the vehicle and the geometric feature line of the target detection box.

[0095] Step 83: Determine the vehicle region from the target detection box based on the rotation angle. The vehicle region is smaller than the target detection box.

[0096] In this embodiment, steps 81-83 have the same or similar technical solutions as any of the above embodiments, and will not be described in detail here.

[0097] Step 84: Determine at least one target parking space area corresponding to the vehicle area.

[0098] In some embodiments, at least one target parking space area may be pre-configured. For example, the area information of the corresponding parking space area.

[0099] In some embodiments, an image processing algorithm is used to determine at least one target parking space area. For example, when no vehicle is parked, the image processing algorithm determines the area information of the target parking space area, or simultaneously detects the area of ​​the parking space area in the detected image.

[0100] In some embodiments, an object detection model is used to determine at least one target parking space area.

[0101] Step 85: Determine the intersection-merge ratio of the vehicle area with at least one target parking space area.

[0102] The intersection-union ratio is the ratio of the intersection to the union of a vehicle area and at least one target parking space area.

[0103] Step 86: Determine the parking space occupancy result based on the intersection-merge ratio.

[0104] In some embodiments, see Figure 11 Steps 85 and 86 can be implemented using the following process:

[0105] Step 91: Determine the intersection-combination ratio of the vehicle area and each target parking space area.

[0106] Step 92: If the intersection-to-merger ratio is greater than or equal to the first preset value, then the target parking space area is determined to be occupied.

[0107] For example, if the first preset value is set to 0.5, and there is an intersection-union ratio (IU) greater than or equal to 0.5, then it is determined that the target parking space area for which the IU is calculated with the vehicle area is occupied. Then, the vehicle and the parking space can be linked, and the parking space status can be updated from empty to occupied. In other embodiments, the first preset value can be set according to the specific area of ​​the parking space area in the detection image. For example, the first preset value can be set to 0.52 or 0.54.

[0108] Step 93: If the intersection-to-merge ratio is less than the first preset value, and there are two intersection-to-merge ratios greater than the second preset value, then it is determined that two adjacent target parking space areas are occupied; the second preset value is less than the first preset value.

[0109] For example, the first preset value is set to 0.5, and the second preset value is set to 0.2. If the intersection-union ratio is less than 0.5 and the second intersection-union ratio is greater than 0.2, then it is determined that two adjacent target parking space areas are occupied. At this time, the vehicle is illegally parked and occupies two parking spaces.

[0110] Combination Figure 12 Explanation:

[0111] like Figure 12As shown, the detected image contains vehicle regions C1 and C2. Furthermore, the detected image contains parking space regions A, B, C, D, E, F, G, H, I, J, K, and L.

[0112] Determine the intersection / merger ratios of vehicle area C1 and vehicle area C2 with these parking space areas respectively. For example... Figure 10 As shown, the intersection-to-merger ratio of vehicle area C1 and parking areas A, B, E, F, I, J, K and L is 0.

[0113] The overlap ratio between vehicle area C1 and parking space area C is greater than 0.5; the overlap ratio between vehicle area C1 and parking space area D is less than 0.5; the overlap ratio between vehicle area C1 and parking space area G is less than 0.5; and the overlap ratio between vehicle area C1 and parking space area H is less than 0.5. If any overlap ratio is greater than 0.5, then parking space area C is determined to be occupied. Then, a vehicle can be bound to this parking space, and the parking space status can be updated from empty to occupied.

[0114] Continue reading Figure 12 The intersection / merger ratio of vehicle area C2 with parking areas A, B, C, D, E, F, G, H, I, and J is 0. The intersection / merger ratio of vehicle area C2 with parking area K is less than 0.5 and greater than 0.2, and the intersection / merger ratio of vehicle area C2 with parking area L is less than 0.5 and greater than 0.2. Therefore, parking areas K and L are determined to be occupied. At this time, the vehicle is illegally parked, occupying two parking spaces.

[0115] In this embodiment, a detection image is acquired, which includes at least one parking space area; a target detection box corresponding to the target vehicle in the detection image is determined, along with a first rotation angle and a second rotation angle; a vehicle area is determined from the target detection box based on the first and second rotation angles; and the parking space occupancy result is determined based on the intersection-union ratio (IUU) of the vehicle area and the parking space area. By using the determined rotation angle to determine the vehicle area from the target detection box, the accuracy of determining the vehicle area can be improved. Furthermore, the parking space occupancy result is determined using the IUU, making the correspondence between parking spaces and vehicles more accurate and improving the accuracy of parking space occupancy judgment.

[0116] See Figure 13 , Figure 13 This is a flowchart illustrating another embodiment of the parking space occupancy detection method provided in this application. The method includes:

[0117] Step 111: Obtain the detection image, which includes at least one parking space area.

[0118] Step 112: Input the detected image into the trained target detection model to obtain the target detection box and the rotation angle; wherein, the target detection box is a rectangle, and the rotation angle represents the angle between the geometric feature line of the vehicle and the geometric feature line of the target detection box.

[0119] Step 113: Determine the vehicle region from the target detection box based on the rotation angle. The vehicle region is smaller than the target detection box.

[0120] Steps 111-113 have the same or similar technical solutions as any of the above embodiments, and will not be described in detail here.

[0121] Step 114: Determine the parking space occupancy result based on the vehicle area and parking space area.

[0122] See Figure 14 The training process of the object detection model is explained below:

[0123] Step 121: Obtain training images; wherein, the training images are labeled with the target detection boxes of the target vehicles and the ground truth information, and the ground truth information includes the intersection points of the ground truth boxes and the target detection boxes of the target vehicles.

[0124] Step 122: Use the object detection model to detect the training images and obtain the detection information of the target vehicles, where the detection information of the target vehicles includes the final detection box of the target vehicles.

[0125] In some embodiments, the object detection model includes a feature extraction network and a classification layer. See also Figure 15 Step 122 can be the following process:

[0126] Step 1221: Input the training image into the feature extraction network to obtain a multidimensional feature map.

[0127] Specifically, the training image is downsampled N times sequentially using a feature extraction network to obtain an N-dimensional initial feature map, where N is greater than 2; for the N-dimensional initial feature map, the N-th initial feature map is upsampled (i+1) times based on the Ni-dimensional initial feature map to obtain the (i+1)-th final feature map; where i is an integer from 0 to N-1.

[0128] The feature extraction network is a Feature Pyramid Network (FPN), which includes multiple upsampling layers and downsampling layers corresponding to the multiple upsampling layers. Each upsampling or downsampling layer includes convolutions with different resolutions.

[0129] Combination Figure 16 and Figure 17 Explanation:

[0130] The feature extraction network is a Feature Pyramid Network (FPN), which consists of multiple upsampling layers and corresponding downsampling layers. For example... Figure 14 As shown, the Feature Pyramid Network (FPN) consists of three upsampling layers and three corresponding downsampling layers. The training image is downsampled three times sequentially using the feature extraction network to obtain the corresponding 3D initial feature map.

[0131] For example, the training image Y is input into the first downsampling layer 141 for downsampling processing, and the first initial feature map is output. The first initial feature map is input into the second downsampling layer 142 for downsampling processing, and the second initial feature map is output. The second initial feature map is input into the third downsampling layer 143 for downsampling processing, and the third initial feature map is output.

[0132] The third initial feature map is input into the first upsampling layer 144 for upsampling processing, and the first final feature map is output. The first final feature map is input into the second upsampling layer 145 for upsampling processing, and the second final feature map is output. The second final feature map is input into the third upsampling layer 146 for upsampling processing, and the third final feature map is output.

[0133] See Figure 17 Each upsampling or downsampling layer includes convolutions at different resolutions, such as 1x1, 3x3, and 5x5 convolutions. These 1x1, 3x3, and 5x5 convolutions are used to convolve the feature maps, resulting in corresponding feature maps. These feature maps are then fused to obtain the final feature map. Using different convolution operations on the feature maps increases the receptive field, allowing more feature information to be learned.

[0134] Step 1222: Input the multidimensional feature map into the classification layer to obtain the detection information of the target vehicle in the training image.

[0135] The final feature map is input into the classification layer to obtain the detection information of the target vehicle in the training image.

[0136] Step 123: Adjust the network parameters of the target detection model based on the difference between the actual information and the detection information of the target vehicle.

[0137] In some embodiments, the number of training iterations of the defect detection model can be adjusted based on the difference between the real information and the detected information of the target vehicle, thereby adjusting the network parameters of the target detection model. For example, if the real information is A and the detected information is B, the number of training iterations of the target detection model can be adjusted, thereby adjusting the network parameters of the target detection model; if the real information is A and the detected information is B, but the confidence level is lower than a set threshold, the number of training iterations of the target detection model can be adjusted, thereby adjusting the network parameters of the target detection model.

[0138] In some embodiments, the network parameters of the target detection model can be adjusted based on the difference between the real information and the detection information of the target vehicle. For example, if the target detection model contains a convolutional neural network, the number of convolutional kernels, stride, padding, activation function, and pooling layer parameters can be set.

[0139] In some embodiments, the loss value can also be calculated using the real information and detection information of the target vehicle. If the loss value differs from the preset loss threshold, the network parameters of the target detection model can be adjusted.

[0140] In one application scenario, the YOLO algorithm can be used to train an object detection model. Based on the quintuple, a second dimension is added to complete the aforementioned rotation angle prediction. The object detection model uses a pyramid network structure. For different scenarios involving large and small objects, the deep network identifies large objects, and the features from the deep network are fused with those from the shallow network to identify small objects. This ensures that the deep network focuses on optimizing the recognition of large objects, while the shallow network focuses on optimizing the recognition of small objects. The training image material covers 100,000 images in a 10:1 ratio of horizontal to non-horizontal scenes with camera and parking lines. The feature pyramid network contains three detection layers. The loss function of each detection layer consists of foreground prediction, classification prediction, background prediction, and coordinate group regression, supplemented by two angle predictions. The weights of each are set as follows, and the loss function is:

[0141]

[0142] Before training began, since the angle range was 0-90 degrees, to reduce the impact of large-scale values ​​in the loss function on the regression of other prediction terms, the angles were normalized, and the updated angle values ​​were between 0 and 1. During the first 100,000 training iterations, the weight of λ5 was limited to 0, focusing on optimizing the accuracy of the bounding rectangle detection box in the initial stage. After another 100,000 training iterations, when updating λ5, it was ensured that its proportion in the weights of each λ component was consistent with that of λ4. A total of 200,000 training iterations were performed to complete the model training.

[0143] This involves pre-calibrating the training images. For example, calibrating the bounding rectangle of the rotating target (vehicle area) and simultaneously calibrating the three corner points A, B, and C. Figure 17As shown. The corner point marking sequence is as follows: mark the intersection of the vehicle and the outer rectangle, taking the right edge, left edge, and bottom edge respectively.

[0144] Based on the angle regression algorithm that maps rotation direction and width and height scale variations, two additional angles, α and β, are added, such as... Figure 18 As shown. Angle α ranges from 0 to 90 degrees, and angle β ranges from 0 to 90 degrees. Integer values ​​are used. α maps to the same target, showing the angle change before and after rotation; β maps to the same target, showing the change in width and height dimensions before and after rotation. For calculating the two angles, the angle β between the diagonals of the actual rotated bounding box is calculated by connecting the first and second corner points, and then connecting the second and third corner points, to obtain the angle between the two sides. The angle α between the diagonals of the circumscribed rectangle is calculated by using the lines connecting the lower left and upper right corner points, and the lines connecting the first and second corner points, when selecting the diagonals. The algorithm model regresses the two angles to complete the rotating target detection.

[0145] By predicting the circumscribed rectangle of the target vehicle, take its diagonal. Using the center point, control the diagonal to rotate clockwise by an angle α, intersecting the circumscribed rectangle at points A and B. Starting from point B, rotate clockwise by an angle β along the direction of side BA, intersecting the rectangle at point C. Similarly, starting from point A, rotate counterclockwise by β along the direction of side AB, intersecting the rectangle at point D. Figure 19 As shown.

[0146] After identifying the vehicle area, the intersection / merger ratio is calculated for the same parking space area. Parking space areas with an intersection / merger ratio greater than 0.5 are selected, vehicles are bound to these parking spaces, and the parking space status is updated from "no car" to "car present".

[0147] In addition, if the overlap ratio between each parking space area and a vehicle is less than 0.5, and there are two overlap ratios greater than 0.2, then a vehicle is illegally parked and occupies two parking spaces, and a warning will be issued.

[0148] During the training of the object detection model, two parameter tuning methods can be used: non-silent angle weights and silent angle weights. Non-silent angle weights initialize λ4 and λ5 to the same weight from the start of training. Silent angle weights initialize λ5 to 0 at the start of training, and then update λ4 and λ5 to the same weight after the loss stabilizes. The loss function is defined as follows.

[0149]

[0150] Compare the changes in loss values ​​between the non-silent angle weighting method and the silent angle weighting method.

[0151] The training results for the loss of non-silent angle weights are as follows: Figure 20During training, the rotation angles α and β regress simultaneously with the coordinates. There are slight fluctuations over a period of time, caused by the mutual interference between the rotation angles on the bounding box and the deformation. As the number of iterations increases, the loss tends to stabilize.

[0152] Among them, the training results of the silent angle loss are as follows: Figure 21 and Figure 22 During training, the weights were dynamically tuned. At the start of training, the weights for the two angles were set to a static value of 0, focusing on regressing the center point and coordinates of the bounding rectangle. After stabilization, the loss weights for the two angles were increased. Comparing the silent and non-silent training methods, in the early stages of training, the non-silent method exhibited severe loss jitter, while the silent method showed stable loss convergence. Later, with the addition of angle losses, the loss initially increased but quickly decreased, rapidly adjusting and regressing the deformation of the bounding rectangle to rotation.

[0153] See Figure 23 , Figure 23 This is a schematic diagram of an embodiment of the parking space occupancy detection device provided in this application. The detection device 210 includes a processor 211 and a memory 212 coupled to the processor 211; wherein, the memory 212 is used to store a computer program, and the processor 211 is used to execute the computer program to implement the following method:

[0154] Acquire a detection image, which includes at least one parking space area; use an object detection model to determine the target detection box corresponding to the target vehicle in the detection image and its rotation angle; wherein, the target detection box is a rectangle, and the rotation angle represents the angle between the geometric feature line of the vehicle and the geometric feature line of the target detection box; determine the vehicle area from the target detection box based on the rotation angle, the vehicle area being smaller than the target detection box; determine the parking space occupancy result based on the vehicle area and the parking space area.

[0155] It is understood that the processor 211 is also used to execute computer programs to implement the methods of any of the above embodiments, as detailed in any of the above embodiments, which will not be repeated here.

[0156] See Figure 24 , Figure 24 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 220 is used to store a computer program 221, which, when executed by a processor, implements the following methods:

[0157] Acquire a detection image, which includes at least one parking space area; use an object detection model to determine the target detection box corresponding to the target vehicle in the detection image and its rotation angle; wherein, the target detection box is a rectangle, and the rotation angle represents the angle between the geometric feature line of the vehicle and the geometric feature line of the target detection box; determine the vehicle area from the target detection box based on the rotation angle, the vehicle area being smaller than the target detection box; determine the parking space occupancy result based on the vehicle area and the parking space area.

[0158] It is understood that when computer program 221 is executed by the processor, it is also used to implement the method of any of the above embodiments. For details, please refer to any of the above embodiments, which will not be repeated here.

[0159] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0160] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0161] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0162] If the integrated units in the other embodiments described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0163] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for detecting parking space occupancy, characterized in that, The detection method includes: Acquire a detection image, the detection image including at least one parking space area; The target detection model is used to determine the target detection box and rotation angle corresponding to the target vehicle in the detection image; wherein, the target detection box is a rectangle, and the rotation angle represents the angle between the geometric feature line of the vehicle and the geometric feature line of the target detection box; the rotation angle includes a first rotation angle and a second rotation angle; Determine the diagonal of the target detection box; the target detection box includes a first side, a second side, a third side, and a fourth side connected in sequence; The diagonal is controlled to rotate clockwise around the center point according to the first rotation angle, intersecting the first side at the first intersection point and the third side at the second intersection point; Determine the line connecting the first and second intersection points; The connecting line is controlled to rotate clockwise around the second intersection point as a reference, according to the second rotation angle, and intersects the second side at the third intersection point; And control the connecting line to rotate counterclockwise with the first intersection point as a reference, according to the second rotation angle, and intersect the fourth side at the fourth intersection point; Connect the first intersection point, the third intersection point, the second intersection point, and the fourth intersection point, and use the area formed by the first intersection point, the third intersection point, the second intersection point, and the fourth intersection point as the vehicle area, where the vehicle area is smaller than the target detection box; Determine at least one target parking space area corresponding to the vehicle area; Determine the intersection-union ratio between the vehicle area and each target parking space area; If the intersection-union ratio is greater than or equal to the first preset value, then the target parking space area is determined to be occupied; If the intersection-to-merge ratio is less than the first preset value, and there are two intersection-to-merge ratios greater than the second preset value, then it is determined that two adjacent target parking space areas are occupied; the second preset value is less than the first preset value.

2. The method according to claim 1, characterized in that, The method further includes: Acquire training images; wherein the training images are labeled with target detection boxes and ground truth information of the target vehicle, and the ground truth information includes the intersection of the ground truth box and the target detection box of the target vehicle; The training image is detected using an object detection model to obtain the detection information of the target vehicle, wherein the detection information of the target vehicle includes the final detection box of the target vehicle; The network parameters of the target detection model are adjusted based on the difference between the actual information of the target vehicle and the detection information.

3. The method according to claim 2, characterized in that, The target detection model includes a feature extraction network and a classification layer; The step of using a target detection model to detect the training images to obtain the detection information of the target vehicle includes: The training images are input into a feature extraction network to obtain a multidimensional feature map; The multidimensional feature map is input into the classification layer to obtain the detection information of the target vehicle in the training image.

4. The method according to claim 3, characterized in that, The step of inputting the training image into the feature extraction network to obtain a multidimensional feature map includes: The training image is downsampled N times sequentially using the feature extraction network to obtain an N-dimensional initial feature map, where N is greater than 2; For the N-dimensional initial feature map, the N-dimensional initial feature map is upsampled (i+1) times based on the Ni-dimensional initial feature map to obtain the (i+1)-dimensional final feature map; where i is an integer from 0 to N-1.

5. The method according to claim 3, characterized in that, The feature extraction network is a Feature Pyramid Network (FPN), which includes multiple upsampling layers and downsampling layers corresponding to the multiple upsampling layers. Each upsampling layer or downsampling layer includes convolutions with different resolutions.

6. The method according to claim 1, characterized in that, Determining at least one target parking space area corresponding to the vehicle area includes: Obtain at least one pre-configured target parking space area; or, Use image processing algorithms to determine at least one target parking space area; or, The target detection model is used to determine at least one target parking space area.

7. A parking space occupancy detection device, characterized in that, The detection device includes a processor and a memory coupled to the processor; The memory is used to store a computer program, and the processor is used to execute the computer program to implement the method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which, when executed by a processor, is used to implement the method as described in any one of claims 1-6.

Images