A method, system and device for detecting a parking state

Abstract

The application discloses a parking state detection method, system and device, acquires the image of a vehicle parked in a preset parking area, then inputs the image of the vehicle into a pre-trained neural network model to obtain a three-dimensional vehicle frame of the vehicle, the neural network model is pre-trained by a preset number of vehicle images, and finally, the parking state of the vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and preset three-dimensional parking space frames of each preset parking space in the shooting area, the parking state includes an abnormal state and a normal state, management personnel is not required to patrol each parking space, the efficiency of detecting the parking state is improved, the workload is reduced, and abnormal parking behaviors can be found in time; moreover, the parking state of the vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and the three-dimensional parking space frame, the probability that the parking state of a normally parked vehicle is misjudged as the abnormal state is reduced, and the determination accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of parking detection, in particular to a parking state detection method, system and device. BACKGROUND

[0002] Some car owners do not park their vehicles in the designated parking spaces when parking on the roadside, for example, the car owner may park the vehicle between two parking spaces or park the vehicle diagonally in a parking space. Diagonal parking will make part of the vehicle's front or rear occupy part of the road, which will hinder other vehicles from parking and affect the normal driving process of passing vehicles, causing inconvenience to road traffic. The prior art usually uses the following two methods to detect the parking state of vehicles parked on the roadside:

[0003] 1. Detecting the parking state of vehicles in parking spaces by management personnel continuously patrolling each parking space. This method not only has low efficiency in patrolling parking spaces and requires a lot of work, but also, with different car owners, the parking state of a vehicle parked in any parking space can change at any time point, which has great randomness, causing the management personnel to be unable to timely discover the abnormal parking state of a vehicle.

[0004] 2. Capturing the preset parking spaces and the surrounding area by a camera, determining the coincidence degree between the positions of each preset parking space and the position of the parked vehicle in the picture captured by the camera, and determining the parking state of the parked vehicle based on the coincidence degree. However, due to the influence of factors such as camera height, lens angle of view, distance, etc. in roadside parking, the two-dimensional image obtained by shooting cannot well represent the real parking state of the vehicle. For example, when a vehicle with a high height is normally parked in a designated parking space, according to the picture captured by the camera, it may be found that part of the chassis of the vehicle exists in one parking space, but part of the roof of the vehicle exists in another parking space, and the vehicle is misjudged as abnormal parking, which has the problem of low judgment accuracy. SUMMARY

[0005] The purpose of the present application is to provide a parking state detection method, system and device, which can not require management personnel to patrol each parking space, improve the efficiency of detecting the parking state, reduce the workload, and timely discover abnormal parking behavior; and can reduce the probability of misjudging the parking state of a normally parked vehicle as an abnormal state, and improve the judgment accuracy.

[0006] To solve the above technical problems, the present application provides a parking state detection method, comprising:

[0007] detecting a current vehicle in an image frame captured by a camera on a shooting area containing preset parking spaces, the shooting area containing a plurality of preset parking spaces;

[0008] if the current vehicle is a vehicle parked in the preset parking area, obtaining an image of the current vehicle;

[0009] inputting the image of the current vehicle into a pre-trained neural network model to obtain a three-dimensional vehicle frame of the current vehicle, the neural network model being pre-trained by a preset number of vehicle images;

[0010] determining a parking state of the current vehicle based on a positional relationship between the three-dimensional vehicle frame and a preset three-dimensional parking space frame of each of the preset parking spaces in the shooting area.

[0011] Preferably, determining the parking state of the current vehicle based on the positional relationship between the three-dimensional vehicle frame and the three-dimensional parking space frame of each of the preset parking spaces in the shooting area comprises:

[0012] determining a spatial coincidence degree between the three-dimensional vehicle frame and each of the preset three-dimensional parking space frames;

[0013] determining whether there is a spatial coincidence degree greater than a preset spatial coincidence degree;

[0014] if there is a spatial coincidence degree greater than a preset spatial coincidence degree, determining that the parking state of the current vehicle is a normal state;

[0015] if there is no spatial coincidence degree greater than a preset spatial coincidence degree, determining that the parking state of the current vehicle is an abnormal state.

[0016] Preferably, determining the spatial coincidence degree between the three-dimensional vehicle frame and each of the preset three-dimensional parking space frames comprises:

[0017] determining a coordinate of a center point of the three-dimensional vehicle frame in a preset spatial coordinate system, the preset spatial coordinate system having an X-axis as a horizontal axis, a Y-axis as a vertical axis, and a Z-axis as a vertical axis;

[0018] determining a rotation parameter of the three-dimensional vehicle frame corresponding to the preset spatial coordinate system, the rotation parameter including a rotation degree of freedom of the three-dimensional vehicle frame corresponding to the X-axis, the Y-axis and the Z-axis of the preset spatial coordinate system;

[0019] determining a shape parameter of the current vehicle based on the three-dimensional vehicle frame, the shape parameter including a length, a width and a height of the current vehicle;

[0020] determining a first ratio between a difference between a coordinate of a center point of the three-dimensional vehicle frame and a coordinate of a center point of each of the three-dimensional parking space frames and the coordinate of the center point of the three-dimensional parking space frame;

[0021] determining a second ratio between a difference between a shape parameter of the three-dimensional vehicle frame and a shape parameter corresponding to each of the three-dimensional parking space frames and the shape parameter corresponding to the three-dimensional parking space frame;

[0022] multiplying the first ratio, the second ratio and the rotation parameter by a preset weight value corresponding to each of the first ratio, the second ratio and the rotation parameter respectively, and adding the multiplied values to obtain a spatial relationship value representing the spatial coincidence degree;

[0023] determining whether the spatial coincidence degree is greater than a preset spatial coincidence degree, comprising:

[0024] determining whether the spatial relationship value is less than a preset spatial relationship value;

[0025] if the spatial relationship value is less than the preset spatial relationship value, determining that the spatial coincidence degree is greater than the preset spatial coincidence degree;

[0026] if the spatial relationship value is not less than the preset spatial relationship value, determining that the spatial coincidence degree is not greater than the preset spatial coincidence degree.

[0027] Preferably, after obtaining the three-dimensional vehicle frame of the current vehicle, the method further comprises:

[0028] determining a spatial coincidence degree between the three-dimensional vehicle frame and each of the preset three-dimensional parking space frames;

[0029] determining a preset parking space corresponding to the preset three-dimensional parking space frame with the greatest spatial coincidence degree as a parking space of the current vehicle, so as to determine that the preset parking space is occupied.

[0030] Preferably, the parking state of the current vehicle is determined based on a positional relationship between the three-dimensional vehicle frame and a preset three-dimensional parking space frame of each of the preset parking spaces in the shooting area, comprising:

[0031] determining a deflection angle between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of the parking space;

[0032] determining whether the deflection angle is greater than a preset deflection angle;

[0033] if the deflection angle is not greater than the preset deflection angle, determining that the current vehicle does not have an abnormal parking behavior;

[0034] if the deflection angle is greater than the preset deflection angle, determining that the current vehicle has an abnormal parking behavior.

[0035] Preferably, determining the deflection angle between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of the parking space comprises:

[0036] determining a connection line between a center point of a front of the current vehicle and a center point of a back of the current vehicle based on the three-dimensional vehicle frame;

[0037] mapping the connection line and a dividing line between the parking space and the driving road into a preset two-dimensional coordinate system, wherein a horizontal axis of the preset two-dimensional coordinate system is an X axis and a vertical axis is a Y axis;

[0038] taking an included angle between the connection line and the dividing line as the deflection angle.

[0039] Preferably, the neural network model is a neural network model composed of a ResNet34 network followed by an FPN network.

[0040] Preferably, determining whether the current vehicle is a vehicle parked in a preset parking area comprises:

[0041] when the current vehicle is detected in the image frame, obtaining a two-dimensional vehicle frame of the current vehicle;

[0042] determining whether a total displacement of a center point of the two-dimensional vehicle frame in consecutive N image frames is less than a preset displacement, N being an integer not less than 2;

[0043] if yes, determining that the current vehicle is a vehicle parked in a preset parking area;

[0044] if no, determining that the current vehicle is not a vehicle parked in a preset parking area.

[0045] The application also provides a parking state detection system, comprising:

[0046] a judging unit configured to determine whether a current vehicle is a vehicle parked in a preset parking area when the vehicle is detected in an image frame captured by a camera on a shooting area containing a preset parking space, the preset parking area containing a plurality of preset parking spaces; and if the current vehicle is a vehicle parked in a preset parking area, triggering an image obtaining unit;

[0047] the image obtaining unit is configured to obtain an image of the current vehicle;

[0048] a three-dimensional vehicle frame obtaining unit configured to input the image of the current vehicle into a pre-trained neural network model to obtain a three-dimensional vehicle frame of the current vehicle, the neural network model being pre-trained by a preset number of vehicle images;

[0049] A parking state determination unit is configured to determine the parking state of the current vehicle based on the positional relationship between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of each preset parking space in the shooting area.

[0050] The application also provides a parking state detection device, which comprises:

[0051] A memory is configured to store a computer program.

[0052] A processor is configured to execute the computer program to implement the steps of the parking state detection method.

[0053] The application provides a parking state detection method, system and device. When a current vehicle is detected in an image frame captured by a camera in a shooting area containing preset parking spaces, it is determined whether the vehicle is parked in a preset parking area. If the vehicle is parked in the preset parking area, the image of the vehicle is obtained, and then the image of the current vehicle is input into a pre-trained neural network model for feature extraction, and a three-dimensional vehicle frame of the vehicle is obtained. The neural network model is pre-trained by a preset number of vehicle images. Finally, the parking state of the vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of each preset parking space in the shooting area. The parking state includes an abnormal state and a normal state. The management personnel do not need to patrol each parking space, the efficiency of detecting the parking state is improved, the workload is reduced, and abnormal parking behaviors can be found in time. Moreover, the parking state of the vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and the three-dimensional parking space frame, the probability that the parking state of a normally parked vehicle is misjudged as an abnormal state is reduced, and the determination accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed in the prior art and the embodiments. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0055] Figure 1 A flowchart of a parking state detection method provided by the application is shown in the figure.

[0056] Figure 2 A schematic diagram of a three-dimensional vehicle frame provided by the application is shown in the figure.

[0057] Figure 3 A schematic diagram of a vehicle with an abnormal parking state is shown in the figure.

[0058] Figure 4A structural schematic diagram of a parking state detection system provided by the present application is shown in the figure.

[0059] Figure 5 A structural schematic diagram of a parking state detection device provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0060] The core of the present application is to provide a parking state detection method, system and device, which can not require management personnel to patrol each parking space, improve the efficiency of detecting the parking state, reduce the workload, and timely discover abnormal parking behavior; and can reduce the probability of misjudging the parking state of a normally parked vehicle as an abnormal state, and improve the determination accuracy.

[0061] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0062] Please refer to Figure 1 , Figure 1 A parking state detection method provided by the present application includes:

[0063] S11: When a current vehicle is detected in an image frame captured by a camera on a shooting area containing preset parking spaces, it is determined whether the current vehicle is a vehicle parked in a preset parking area containing a plurality of preset parking spaces.

[0064] S12: If the current vehicle is a vehicle parked in the preset parking area, an image of the current vehicle is obtained.

[0065] S13: The image of the current vehicle is input into a pre-trained neural network model to obtain a three-dimensional vehicle frame of the current vehicle, and the neural network model is pre-trained by a preset number of vehicle images.

[0066] S14: The parking state of the current vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and a preset three-dimensional parking space frame of each preset parking space in the shooting area.

[0067] When the camera captures the shooting area containing the preset parking space, in addition to the vehicle parked in the shooting area, the camera will also capture the vehicle passing by the preset parking space. Since the passing vehicle will not hinder the parking of other vehicles and will not affect the normal driving process of other passing vehicles, the passing vehicle needs to be excluded from the subsequent steps, and only the parking state of the vehicle parked in the preset parking area needs to be determined, including abnormal state and normal state.

[0068] After the vehicle is parked stably, the image of the vehicle is acquired, and then the image of the vehicle is sent into a neural network model for feature extraction. The neural network model is a monocular three-dimensional target detection model, which can output a three-dimensional vehicle frame of the vehicle. Based on the position relationship between the three-dimensional vehicle frame and the three-dimensional parking space frame of each preset parking space in the shooting area, the parking state of the vehicle is determined. Please refer to Figure 2 , Figure 2 A schematic diagram of a three-dimensional vehicle frame is provided for the present application. Specifically, a preset space coordinate system is set in the shooting area. The preset space coordinate system can take the ground as the horizontal axis-vertical axis plane, the horizontal axis is parallel to the road center line, the vertical axis is perpendicular to the road center line, and the vertical axis is perpendicular to the ground. Since the position of the preset parking space is fixed, the position of the three-dimensional parking space frame corresponding to the parking space is fixed. When setting the three-dimensional parking space frame, first, the four vertex coordinates of the parking space on the ground, i.e., the horizontal axis-vertical axis plane, are determined as point A (xa, ya), point B (xb, yb), point C (xc, yc), and point D (xd, yd). Assuming that the line connecting point A and point C and the line connecting point B and point D are the long sides of the parking space, and the line connecting point A and point B and the line connecting point C and point D are the wide sides of the parking space, based on the four vertex coordinates, the center point coordinate E (xe, ye) of the parking space on the ground can be determined. The planar position information of the parking space on the horizontal axis-vertical axis is mapped to the three-dimensional parking space frame. The parameters of the three-dimensional parking space frame of the parking space include: the position of the center point of the three-dimensional parking space frame in the preset space coordinate system (Xpark, Ypark, Zpark), the rotational degrees of freedom of the three-dimensional parking space frame with respect to the three dimensions of the preset space coordinate system (pitch-park, roll-park, yaw-park), and the shape parameters of the three-dimensional parking space frame (Lpark, Wpark, Hpark). Among them, wherein θ is the angle between the two perpendicular edges of the parking space mapped in the two-dimensional picture, Zpark can be a preset coordinate value or consistent with the Z-axis coordinate of the three-dimensional vehicle frame since the Z-axis coordinate has no effect on determining the parking state of the vehicle; pitch-park is the rotation freedom of the three-dimensional parking frame on the X-axis, roll-park is the rotation freedom of the three-dimensional parking frame on the Y-axis, and yaw-park is the rotation freedom of the three-dimensional parking frame on the Z-axis, since the parking space is usually parallel to the road center line and perpendicular to the ground, pitch-park and yaw-park are both 0; Lpark is half of the length of the three-dimensional parking frame, which is equal to half of the length of the line between point A and point C or the line between point B and point D, and Wpark is half of the width of the three-dimensional parking frame, which is equal to half of the length of the line between point A and point B and the line between point C and point D, since Hpark also has no effect on determining the parking state of the vehicle, Hpark can be a preset value or consistent with half of the height of the three-dimensional vehicle frame.

[0069] When determining the positional relationship between the three-dimensional vehicle frame and the three-dimensional parking frame, the relationship between the above parameters of the three-dimensional vehicle frame and the above parameters of the three-dimensional parking frame is determined, for example, when the corresponding parameters of the three-dimensional vehicle frame are (Xcar, Ycar, Zcar, pitch-car, yaw-car, roll-car, Lcar, Wcar, Hcar), the difference between Xcar and Xpark is determined, the smaller the difference, the closer the X-axis distance between the three-dimensional vehicle frame and the three-dimensional parking frame; the difference between pitch-car and pitch-park or the difference between yaw-car and yaw-park is determined, the smaller the two differences, the more similar the placement angle between the three-dimensional vehicle frame and the three-dimensional parking frame, since the two parameters of the three-dimensional parking frame are both 0, the closer the two parameters of the three-dimensional vehicle frame to 0, the more standard the parking of the three-dimensional vehicle frame; the difference between Lcar and Lpark is determined, since Lpark is fixed, but Lcar will change with the position of the vehicle in the shooting area, it can be seen that the smaller the difference, the closer the vehicle to the parking space.

[0070] The application provides a parking state detection method, system and device. When a current vehicle is detected in an image frame captured by a camera on a shooting area containing preset parking spaces, it is determined whether the vehicle is parked in a preset parking area. If the vehicle is parked in the preset parking area, an image of the vehicle is obtained. Then, the image of the current vehicle is input into a pre-trained neural network model for feature extraction, and a three-dimensional vehicle frame of the vehicle is obtained. The neural network model is pre-trained by a preset number of vehicle images. Finally, the parking state of the vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and preset three-dimensional parking space frames of each preset parking space in the shooting area. The parking state includes an abnormal state and a normal state. The management personnel do not need to patrol each parking space, the efficiency of detecting the parking state is improved, the workload is reduced, and abnormal parking behaviors can be found in time. Moreover, the parking state of the vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and the three-dimensional parking space frame, the probability that the parking state of a normally parked vehicle is misjudged as an abnormal state is reduced, and the determination accuracy is improved.

[0071] On the basis of the above-mentioned embodiments:

[0072] As a preferred embodiment, the parking state of the current vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and the three-dimensional parking space frame of each preset parking space in the shooting area, including:

[0073] determining the spatial coincidence degree between the three-dimensional vehicle frame and each preset three-dimensional parking space frame;

[0074] determining whether there is a spatial coincidence degree greater than a preset spatial coincidence degree;

[0075] if there is a spatial coincidence degree greater than a preset spatial coincidence degree, determining that the parking state of the current vehicle is a normal state;

[0076] if there is no spatial coincidence degree greater than a preset spatial coincidence degree, determining that the parking state of the current vehicle is an abnormal state.

[0077] The spatial coincidence degree represents a proportion of a volume of a coinciding part between the three-dimensional vehicle frame and the three-dimensional parking space frame to a total volume of the three-dimensional vehicle frame. In an actual application scenario, since a footprint of a parking space is usually larger than a footprint of a vehicle, when the vehicle is normally parked in a preset parking space corresponding to the three-dimensional parking space frame, the three-dimensional vehicle frame is completely contained in the three-dimensional parking space frame. For example, if the preset spatial coincidence degree is 90%, after the vehicle is parked, if the vehicle is normally parked in a preset parking space, at this moment, the spatial coincidence degree between the three-dimensional vehicle frame and the three-dimensional parking space frame is 100%, and the spatial coincidence degrees between the three-dimensional vehicle frame and other three-dimensional parking space frames are all 0%. Since there is a spatial coincidence degree greater than the preset spatial coincidence degree, it can be determined that the parking state of the vehicle is a normal state. If the target vehicle is parked in a position across two parking spaces, half of the vehicle is in the A parking space and the other half of the vehicle is in the B parking space. At this moment, the spatial coincidence degree between the three-dimensional vehicle frame and the three-dimensional parking space frame of the A parking space is 60%, the spatial coincidence degree between the three-dimensional vehicle frame and the three-dimensional parking space frame of the B parking space is 40%, and the spatial coincidence degrees between the three-dimensional vehicle frame and other three-dimensional parking space frames are all 0%. Since there is no spatial coincidence degree greater than the preset spatial coincidence degree, it can be determined that the parking state of the vehicle is an abnormal state. Further, if it is necessary to determine which parking space is occupied by the vehicle, the parking space numbers corresponding to all spatial coincidence degrees greater than 0% can be determined, so as to determine that the vehicle occupies the A parking space and the B parking space.

[0078] It can be seen that, in the embodiment, by calculating the spatial coincidence degrees between the three-dimensional vehicle frame and each preset three-dimensional parking space frame and comparing the spatial coincidence degrees with the preset spatial coincidence degree, it can be determined whether the parking state of the vehicle is normal, and when it is determined that the parking state of the vehicle is abnormal, for example, when the vehicle is parked across parking spaces, the parking space numbers specifically occupied by the vehicle can be determined.

[0079] As a preferred embodiment, the spatial coincidence degrees between the three-dimensional vehicle frame and each preset three-dimensional parking space frame are determined, including:

[0080] The coordinates of the center point of the three-dimensional vehicle frame in the preset spatial coordinate system are determined, the horizontal axis of the preset spatial coordinate system is the X axis, the vertical axis is the Y axis, and the vertical axis is the Z axis;

[0081] The rotation parameters of the three-dimensional vehicle frame corresponding to the preset spatial coordinate system are determined, the rotation parameters including rotation degrees of freedom of the three-dimensional vehicle frame corresponding to the X axis, the Y axis and the Z axis of the preset spatial coordinate system;

[0082] The shape parameters of the current vehicle are determined based on the three-dimensional vehicle frame, the shape parameters including the length, the width and the height of the current vehicle;

[0083] The first ratio between the difference between the coordinates of the center point of the three-dimensional vehicle frame and the coordinates of the center point of each three-dimensional parking space frame and the coordinates of the center point of the three-dimensional parking space frame is determined;

[0084] determining a second ratio between the difference value and the shape parameter corresponding to the three-dimensional parking space frame;

[0085] adding the first ratio, the second ratio and the rotation parameter multiplied by the preset weight value corresponding to the rotation parameter to obtain a value as the spatial relationship value representing the spatial coincidence degree;

[0086] judging whether the spatial coincidence degree is greater than a preset spatial coincidence degree, comprising:

[0087] judging whether the spatial relationship value is less than a preset spatial relationship value;

[0088] if the spatial relationship value is less than the preset spatial relationship value, it is determined that the spatial coincidence degree is greater than the preset spatial coincidence degree;

[0089] if the spatial relationship value is not less than the preset spatial relationship value, it is determined that the spatial coincidence degree is not greater than the preset spatial coincidence degree.

[0090] Since the position of the preset parking space is fixed, the center point coordinates and shape parameters of the three-dimensional parking space frame are also fixed, and a coordinate system can be preset. The horizontal axis of the coordinate system is the X axis, corresponding to the horizontal direction of the image, i.e., the direction deviating to the road axis. The vertical axis is the Y axis, corresponding to the vertical direction of the image, i.e., the direction parallel to the side of the parking space. The vertical axis is the Z axis, corresponding to the height of the three-dimensional space, i.e., the direction perpendicular to the ground. The corresponding relationship between the coordinate axes and angles is: X axis-Pitch angle (pitch angle), Y axis-Roll angle (roll angle), and Z axis-Yaw angle (yaw angle). The specific center point coordinates and shape parameters of each three-dimensional parking space frame, and the shape parameters of the three-dimensional vehicle frame will change with the vehicle model. The center point coordinates and rotation parameters of the three-dimensional vehicle frame will change with the parking position of the vehicle. The center point coordinates of the three-dimensional parking space frame of the preset parking space can be defined as (Xpark, Ypark, Zpark), and the shape parameters can be defined as (Lpark, Wpark, Hpark), where Lpark is half of the length of the three-dimensional parking space frame, Wpark is half of the width of the three-dimensional parking space frame, and Hpark is half of the height of the three-dimensional parking space frame. When the vehicle is parked, the center point coordinates of the three-dimensional vehicle frame are defined as (Xcar, Ycar, Zcar), and the shape parameters are defined as (Lcar, Wcar, Hcar), where Lcar is the length of the three-dimensional vehicle frame, Wcar is the width of the three-dimensional vehicle frame, and Hcar is the height of the three-dimensional vehicle frame. The rotation parameters of the three-dimensional vehicle frame are defined as (pitch-car, roll-car, yaw-car), where pitch-car is the degree of freedom of rotation of the three-dimensional vehicle frame around the X axis, roll-car is the degree of freedom of rotation of the three-dimensional vehicle frame around the Y axis, and yaw-car is the degree of freedom of rotation of the three-dimensional vehicle frame around the Z axis. As can be seen, the first ratio of the difference between the center point coordinates of the three-dimensional vehicle frame and the center point coordinates of the three-dimensional parking space frame and the center point of the three-dimensional parking space frame can determine the distance between the three-dimensional vehicle frame and the three-dimensional parking space frame, i.e., the distance, and thus the spatial coincidence degree. The second ratio of the difference between the shape parameters of the three-dimensional vehicle frame and the corresponding shape parameters of the three-dimensional parking space frame and the corresponding shape parameters of the three-dimensional parking space frame can also reflect the spatial coincidence degree to some extent, since the shape parameters of the three-dimensional vehicle frame are usually smaller than the shape parameters of the three-dimensional parking space frame. If the ratio is very small or greater than 0, it may indicate that the vehicle is not parked in the correct position or that a large vehicle is parked, etc. The rotation parameters of the three-dimensional vehicle frame can determine whether the three-dimensional vehicle frame exceeds the three-dimensional parking space frame, and thus the spatial coincidence degree.Since the first ratio, the second ratio and the rotation parameter can reflect the spatial coincidence degree to different degrees, the corresponding weight values can be set according to the reaction degree, the weight values of the first ratio are defined as a1, a2 and a3, wherein a1 is the weight corresponding to the X-axis coordinate, a2 is the weight corresponding to the Y-axis coordinate, and a3 is the weight corresponding to the Z-axis coordinate; the weight values of the rotation parameter are defined as a4, a5 and a6, wherein a4 is the weight corresponding to pitch-car, a5 is the weight corresponding to roll-car, and a6 is the weight corresponding to yaw-car; the weight values of the second ratio are defined as a7, a8 and a9, a7 is the weight corresponding to the length, a8 is the weight corresponding to the width, and a9 is the weight corresponding to the height; if the spatial relationship value between the three-dimensional vehicle frame and the three-dimensional parking space frame is set as Φ, then Φ = a1 (Xcar-Xpark) / Xpark + a2 (Ycar-Ypark) / Ypark + a3 (Zcar-Zpark) / Zpark + a4 *pitch-car + a5 *roll-car + a6 *yaw-car + a7 (Lcar-Lpark) / Lpark + a8 (Wcar-Wpark) / Wpark + a9 (Hcar-Hpark) / Hpark, when there are multiple preset parking spaces, then multiple Φs are obtained, if there are 4 preset parking spaces in total, then the Φs between the three-dimensional parking space frames of the 4 preset parking spaces and the three-dimensional vehicle frame are calculated to obtain Φ1, Φ2, Φ3 and Φ4, then it is judged whether each Φ is less than the preset spatial relationship value δ, if there is a three-dimensional parking space frame less than δ, then it is determined that the spatial coincidence degree is greater than the preset spatial coincidence degree, otherwise it is determined that the spatial coincidence degree is not greater than the preset spatial coincidence degree. In addition, the three-dimensional parking space frame with the smallest Φ value and less than δ can be taken as the parking space of the vehicle to determine that the parking space is occupied.

[0091] It also needs to be explained that considering that in actual application scenarios, the preset parking space is usually a roadside parking space, when judging whether the parking state of the vehicle is an abnormal parking state, the specific parking position of the vehicle and the position of each preset parking space are usually used for judgment, that is, the specific ground position occupied by the vehicle and the specific ground position occupied by the preset parking space are used for judgment, since there is no multi-layer parking space on the same ground position, the height of the vehicle and the parking space and the Z-axis coordinate have no influence on the judgment of the parking state, in order to facilitate calculation, the height of the vehicle and the parking space and the Z-axis coordinate can be set as the same value.

[0092] It can be seen that by calculating the spatial relationship value between the three-dimensional parking space frame and the three-dimensional vehicle frame, the spatial coincidence degree between the three-dimensional vehicle frame and each three-dimensional parking space frame can be accurately calculated, so as to determine whether the vehicle is in an abnormal parking state subsequently.

[0093] As a preferred embodiment, after obtaining the three-dimensional vehicle frame of the current vehicle, further comprising:

[0094] determining the spatial coincidence degree between the three-dimensional vehicle frame and each preset three-dimensional parking space frame;

[0095] taking the preset parking space corresponding to the preset three-dimensional parking space frame with the largest spatial coincidence degree as the parking space of the current vehicle, so as to determine that the preset parking space is occupied.

[0096] In order to determine whether the parking space is occupied, in the present application, considering that in actual situations, the state information of part of the parking spaces may be uploaded to the parking system, so as to facilitate the parking charging of the vehicle in the parking space or the search for empty parking spaces by other vehicles. After obtaining the three-dimensional vehicle frame of the current vehicle, the spatial coincidence degree between the three-dimensional vehicle frame and each preset three-dimensional parking space frame is determined. The more the overlapping part between the three-dimensional vehicle frame and the preset three-dimensional parking space frame, the higher the spatial coincidence degree. It can be seen that the spatial coincidence degree of the preset parking space where the current vehicle is located is the highest, and the spatial coincidence degree of the preset parking space without any intersection with the current vehicle is the lowest. It can be seen that by taking the preset parking space corresponding to the preset three-dimensional parking space frame with the largest spatial coincidence degree as the parking space of the current vehicle, it can be determined that the preset parking space is occupied by the vehicle, so as to facilitate the parking charging of the vehicle or the search for empty parking spaces by other vehicles.

[0097] As a preferred embodiment, the parking state of the current vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of each preset parking space in the shooting area, comprising:

[0098] determining the deflection angle between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of the parking space;

[0099] judging whether the deflection angle is greater than a preset deflection angle;

[0100] if not greater than the preset deflection angle, it is determined that the current vehicle does not have abnormal parking behavior;

[0101] if greater than the preset deflection angle, it is determined that the current vehicle has abnormal parking behavior.

[0102] In order to determine the parking state of the current vehicle, in the present application, it is considered that part of the vehicles will not have the behavior of parking across the parking spaces, but will have the behavior of parking at an angle. Please refer to Figure 3 , Figure 3 a schematic view of a vehicle with an abnormal parking state, Figure 2The vehicle in the three-dimensional vehicle frame is a diagonal parking. Since the parking spaces on the roadside are usually side parking, it can be seen that in general, the side of the vehicle is parallel to the driving road when parking, and the front and rear of the vehicle are perpendicular to the driving road. Diagonal parking refers to a part of the vehicle, such as the front or rear, being parked in a certain preset parking space, but another part of the vehicle, such as the front or rear, being parked on the driving road, that is, diagonal or transverse parking. Diagonal parking will bring great inconvenience to road traffic. At this time, the deflection angle between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of the parking space where the vehicle is located can be determined, for example, the angle between a certain edge line in the three-dimensional vehicle frame and the edge line at the same position in the preset three-dimensional parking space frame can be determined, and then it is determined whether the angle is greater than the preset deflection angle. If it is not greater than the preset deflection angle, it means that the front or rear of the vehicle does not deviate too much from the parking space area, and it is within the tolerance range. At this time, it is determined that the vehicle does not have abnormal parking behavior. If it is greater than the preset deflection angle, it means that the front or rear of the vehicle deviates too much from the parking space area. At this time, it is determined that the vehicle has abnormal parking behavior. It can be seen that by judging whether the deflection angle is greater than the preset deflection angle, it can be simply determined whether the parking state of the vehicle is the abnormal parking state of diagonal or transverse parking.

[0103] As a preferred embodiment, the deflection angle between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of the parking space is determined, comprising:

[0104] Based on the three-dimensional vehicle frame, a connecting line between the center point of the front of the vehicle and the center point of the back of the vehicle is determined.

[0105] The connecting line, the dividing line between the parking space and the driving road are all mapped into a preset two-dimensional coordinate system, the horizontal axis of the preset two-dimensional coordinate system is X axis, and the vertical axis is Y axis.

[0106] The angle between the connecting line and the dividing line is taken as the deflection angle.

[0107] In actual scenarios, the preset three-dimensional parking space frame and the three-dimensional vehicle frame are usually cuboids, the long side in the three-dimensional vehicle frame represents the long side of the side surface of the vehicle, that is, the side from the center point of the front of the vehicle to the center point of the back of the vehicle, and since the angle between the long side of the three-dimensional parking space frame and the long side of the three-dimensional vehicle frame is more susceptible to the placement position of the vehicle, the angle between the long side of the vehicle and the long side of the three-dimensional vehicle frame, that is, the angle between the connecting line and the dividing line, is used as the deflection angle to more intuitively reflect whether the parking state of the vehicle is an abnormal parking state. For example, when the parking space is a side parking space arranged on the roadside, the dividing line between the three-dimensional parking space frame and the driving road is the long side of the three-dimensional parking space frame, which is usually approximately parallel to the center line of the driving road, and when the vehicle is parked in the parking space, the side surface of the vehicle, that is, the connecting line from the center point of the front of the vehicle to the center point of the back of the vehicle, is also usually approximately parallel to the center line of the driving road, and it can be seen that when the vehicle is normally parked in the parking space, the angle between the connecting line and the dividing line is usually very small or even parallel, but when the vehicle is not normally parked in the parking space, for example, part of the front or part of the back does not enter the parking space, the angle between the connecting line and the dividing line is large, and it can be seen that using the angle between the connecting line and the dividing line as the deflection angle can better reflect whether the parking state of the vehicle is an abnormal parking state.

[0108] As a preferred embodiment, the neural network model is a neural network model composed of a ResNet34 network followed by an FPN network.

[0109] In order to reduce the error of the neural network model, in the present application, it is considered that various types of neural networks will have a common phenomenon that the training accuracy will decrease with the increase of the network depth. The reason for this phenomenon is that the deeper the network depth, the more obvious the gradient disappearance phenomenon, and during the back propagation of the usual neural network model, the gradient cannot be effectively updated to the front network layer, resulting in that the parameters of the front network layer cannot be updated, leading to poor training effect. In the ResNet34 network, a residual network is provided, which adds an identity mapping, which can directly transmit the input of the current layer network to the next layer network, equivalent to skipping the operation of the current layer, and during the back propagation of the ResNet34 network, the gradient of the next layer network is directly transmitted to the previous layer network, which alleviates the gradient disappearance problem of the deep network. On the other hand, since the FPN is a feature pyramid with good generalization ability, the method of connecting the FPN behind the ResNet34 can not only improve the accuracy of the determined three-dimensional vehicle frame, but also improve the speed of determining the three-dimensional vehicle frame. Therefore, connecting the ResNet34 network followed by the FPN network as the neural network model can reduce the accuracy of the determined three-dimensional vehicle frame and reduce the error of the neural network model.

[0110] As a preferred embodiment, judging whether the current vehicle is a vehicle parked in the preset parking area comprises:

[0111] When the current vehicle is detected in the image frame, a two-dimensional vehicle frame of the current vehicle is acquired;

[0112] Judging whether a total displacement amount of a center point of the two-dimensional vehicle frame in the continuous N image frames is less than a preset displacement amount, N being an integer not less than 2;

[0113] If yes, it is determined that the current vehicle is a vehicle parked in the preset parking area;

[0114] If no, it is determined that the current vehicle is not a vehicle parked in the preset parking area.

[0115] In order to judge whether the vehicle photographed by the camera is parked stably, in the present application, considering that the vehicle parked in the preset parking area is stationary, the detection of the parking state of the vehicle is the detection of the parking state of the stationary vehicle. When the vehicle enters the shooting area of the camera, the two-dimensional vehicle frame of the vehicle will be acquired until the vehicle leaves the shooting area or is parked stably in the shooting area, i.e., is stationary. For example, the image frame photographed by the camera can be sent to a pre-trained neural network model to acquire the two-dimensional vehicle frame of the vehicle, and the neural network model is pre-trained by a plurality of vehicle images. When the vehicle is stationary, the position in the image frame photographed by the camera is also stationary, but at this time, considering that the camera itself is light and is usually set in the air, it is easy to be disturbed by air and to shake, so that the position of the stationary vehicle in the image frame will have a slight change. Therefore, when judging whether the vehicle is parked stably, it is necessary to judge whether the total displacement amount of the center point of the two-dimensional vehicle frame of the vehicle in the continuous N image frames photographed by the camera is less than a preset displacement amount. If yes, it is indicated that the center point of the vehicle only fluctuates in a small range, which can be regarded as that the vehicle has been parked stably, and at this time, it is determined that the vehicle is a vehicle parked in the preset parking area. Otherwise, it is indicated that the vehicle is still moving, and at this time, it is determined that the vehicle is not a vehicle parked in the preset parking area. It can be seen that by detecting the total displacement amount in the N image frames, it can be accurately determined whether the vehicle in the shooting area is parked stably in the preset parking area.

[0116] Please refer to Figure 4 , Figure 4 A structure diagram of a parking state detection system provided by the present application comprises:

[0117] The judging unit 11 is configured to judge whether the current vehicle is a vehicle parked in a preset parking area when a vehicle is detected in an image frame captured by the camera on a shooting area containing preset parking spaces, and the preset parking area contains a plurality of preset parking spaces; and trigger the image acquisition unit if the current vehicle is a vehicle parked in the preset parking area.

[0118] The image acquisition unit 12 is configured to acquire an image of the current vehicle.

[0119] The three-dimensional vehicle frame acquisition unit 13 is configured to input the image of the current vehicle into a pre-trained neural network model to obtain a three-dimensional vehicle frame of the current vehicle, and the neural network model is pre-trained by a preset number of vehicle images.

[0120] The parking state determination unit 14 is configured to determine the parking state of the current vehicle based on the positional relationship between the three-dimensional vehicle frame and a preset three-dimensional parking space frame of each preset parking space in the shooting area.

[0121] For the detailed introduction of the parking state detection system provided in the present application, please refer to the above-mentioned embodiments of the parking state detection method, which will not be repeated here.

[0122] For the detailed introduction of the parking state detection system provided in the present application, please refer to the above-mentioned embodiments of the parking state detection method, which will not be repeated here. Figure 5 , Figure 5 The structure diagram of the parking state detection device provided in the present application comprises:

[0123] The memory 21 is configured to store a computer program.

[0124] The processor 22 is configured to execute the computer program to realize the steps of the above-mentioned parking state detection method.

[0125] For the detailed introduction of the parking state detection device provided in the present application, please refer to the above-mentioned embodiments of the parking state detection method, which will not be repeated here.

[0126] In the present specification, each embodiment is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0127] It also needs to be explained that in the present specification, the relational terms such as first and second and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

Claims

1. A method of detecting a parking state, characterized by, The method comprises the following steps: When a current vehicle is detected in an image frame captured by a camera on a shooting area containing preset parking spaces, it is determined whether the current vehicle is a vehicle parked in a preset parking area containing a plurality of preset parking spaces; If the current vehicle is a vehicle parked in a preset parking area, an image of the current vehicle is obtained; The image of the current vehicle is input into a pre-trained neural network model to obtain a three-dimensional vehicle frame of the current vehicle, wherein the neural network model is pre-trained by a preset number of vehicle images, and the neural network model is a monocular three-dimensional target detection model composed of a ResNet34 network followed by an FPN network; The parking state of the current vehicle is determined based on the positional relationship between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of each of the preset parking spaces in the shooting area, comprising: The spatial coincidence degree between the three-dimensional vehicle frame and each of the preset three-dimensional parking space frames is determined; It is determined whether there is a spatial coincidence degree greater than a preset spatial coincidence degree; If there is a spatial coincidence degree greater than a preset spatial coincidence degree, it is determined that the parking state of the current vehicle is normal; If there is no spatial coincidence degree greater than a preset spatial coincidence degree, it is determined that the parking state of the current vehicle is abnormal; Wherein, the spatial coincidence degree between the three-dimensional vehicle frame and each of the preset three-dimensional parking space frames is determined, comprising: The coordinates of the center point of the three-dimensional vehicle frame in a preset spatial coordinate system are determined, wherein the horizontal axis of the preset spatial coordinate system is the X axis, the vertical axis is the Y axis, and the vertical axis is the Z axis; The rotation parameters of the three-dimensional vehicle frame corresponding to the preset spatial coordinate system are determined, wherein the rotation parameters include the rotation degrees of freedom of the three-dimensional vehicle frame corresponding to the X axis, Y axis and Z axis of the preset spatial coordinate system; The shape parameters of the current vehicle are determined based on the three-dimensional vehicle frame, wherein the shape parameters include the length, width and height of the current vehicle; The first ratio between the difference between the coordinates of the center point of the three-dimensional vehicle frame and the coordinates of the center point of each of the three-dimensional parking space frames and the coordinates of the center point of the three-dimensional parking space frame is determined; The second ratio between the difference between the shape parameters of the three-dimensional vehicle frame and the shape parameters corresponding to each of the three-dimensional parking space frames and the shape parameters corresponding to the three-dimensional parking space frame is determined; The value obtained by multiplying the first ratio, the second ratio and the preset weight value corresponding to the rotation parameters and then adding them up is taken as the spatial relationship value representing the spatial coincidence degree; It is determined whether there is a spatial relationship value less than a preset spatial relationship value; If there is a spatial relationship value less than a preset spatial relationship value, it is determined that there is a spatial coincidence degree greater than a preset spatial coincidence degree; If there is no spatial relationship value less than a preset spatial relationship value, it is determined that there is no spatial coincidence degree greater than a preset spatial coincidence degree.

2. The method of claim 1, wherein the vehicle is determined to be in the parked state when the vehicle is determined to be in the stopped state and the engine is determined to be in the off state. After obtaining the three-dimensional vehicle frame of the current vehicle, the method further comprises the following steps: The spatial coincidence degree between the three-dimensional vehicle frame and each of the preset three-dimensional parking space frames is determined; The preset parking space corresponding to the preset three-dimensional parking space frame with the maximum spatial coincidence degree is taken as a parking space of the current vehicle, so as to determine that the preset parking space is occupied.

3. The method of claim 2, wherein the vehicle is determined to be in the parked state when the vehicle is determined to be in the stopped state and the engine is determined to be in the off state. The parking state of the current vehicle is determined based on a positional relationship between the three-dimensional vehicle frame and preset three-dimensional parking space frames of each preset parking space in the shooting area, and the parking state determination unit includes: determining a deflection angle between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of the parking space; determining whether the deflection angle is greater than a preset deflection angle; if the deflection angle is not greater than the preset deflection angle, it is determined that the current vehicle does not have an abnormal parking behavior; if the deflection angle is greater than the preset deflection angle, it is determined that the current vehicle has an abnormal parking behavior.

4. The method for detecting parking status as described in claim 3, characterized in that, The deflection angle between the three-dimensional vehicle frame and the preset three-dimensional parking space frame of the parking space is determined, and the determination includes: determining a connecting line between a center point of a front of the current vehicle and a center point of a back of the current vehicle based on the three-dimensional vehicle frame; mapping the connecting line and a dividing line between the parking space and a driving road into a preset two-dimensional coordinate system, wherein an X-axis of the preset two-dimensional coordinate system is a horizontal axis, and a Y-axis of the preset two-dimensional coordinate system is a vertical axis; taking an included angle between the connecting line and the dividing line as the deflection angle.

5. The method of detecting a parking state according to any one of claims 1 to 4, characterized in that, The current vehicle is determined to be a vehicle parked in a preset parking area, and the determination includes: when the current vehicle is detected in the image frame, a two-dimensional vehicle frame of the current vehicle is obtained; determining whether a total displacement of a center point of the two-dimensional vehicle frame in consecutive N image frames is less than a preset displacement, N being an integer greater than or equal to 2; if yes, the current vehicle is determined to be a vehicle parked in a preset parking area; if no, the current vehicle is determined to be not a vehicle parked in a preset parking area.

6. A parking state detection system characterized by comprising: The determination includes: a judgment unit is configured to determine whether a current vehicle is a vehicle parked in a preset parking area when the current vehicle is detected in an image frame captured by a camera shooting a shooting area containing preset parking spaces, and the preset parking area contains a plurality of preset parking spaces; if the current vehicle is a vehicle parked in a preset parking area, an image acquisition unit is triggered; the image acquisition unit is configured to acquire an image of the current vehicle; a three-dimensional vehicle frame acquisition unit is configured to input the image of the current vehicle into a pre-trained neural network model, so as to obtain a three-dimensional vehicle frame of the current vehicle, the neural network model being pre-trained by a preset number of vehicle images; the neural network model is a monocular three-dimensional target detection model composed of a ResNet34 network followed by an FPN network; a parking state determination unit is configured to determine a parking state of the current vehicle based on a positional relationship between the three-dimensional vehicle frame and preset three-dimensional parking space frames of each preset parking space in the shooting area; the parking state determination unit is specifically configured to: determine spatial coincidence degrees between the three-dimensional vehicle frame and each preset three-dimensional parking space frame; determine whether there is a spatial coincidence degree greater than a preset spatial coincidence degree; if there is a spatial coincidence degree greater than a preset spatial coincidence degree, it is determined that the parking state of the current vehicle is a normal state; If there is no spatial coincidence degree greater than the preset spatial coincidence degree, it is determined that the parking state of the current vehicle is an abnormal state; The spatial coincidence degree between the three-dimensional vehicle frame and each of the preset three-dimensional parking space frames is determined, including: The coordinate of the center point of the three-dimensional vehicle frame in a preset spatial coordinate system is determined, wherein the horizontal axis of the preset spatial coordinate system is the X axis, the vertical axis is the Y axis, and the vertical axis is the Z axis; The rotation parameters of the three-dimensional vehicle frame corresponding to the preset spatial coordinate system are determined, including the rotation degrees of freedom of the three-dimensional vehicle frame corresponding to the X axis, Y axis and Z axis of the preset spatial coordinate system; The shape parameters of the current vehicle are determined based on the three-dimensional vehicle frame, including the length, width and height of the current vehicle; The first ratio between the difference between the coordinate of the center point of the three-dimensional vehicle frame and the coordinate of the center point of each of the three-dimensional parking space frames and the coordinate of the center point of the three-dimensional parking space frame is determined; The second ratio between the difference between the shape parameters of the three-dimensional vehicle frame and the shape parameters corresponding to each of the three-dimensional parking space frames and the shape parameters corresponding to the three-dimensional parking space frame is determined; The value obtained by multiplying the first ratio, the second ratio and the preset weight value corresponding to the rotation parameters and then adding them up is taken as the spatial relationship value representing the spatial coincidence degree; It is determined whether there is a spatial relationship value less than the preset spatial relationship value; If there is a spatial relationship value less than the preset spatial relationship value, it is determined that there is a spatial coincidence degree greater than the preset spatial coincidence degree; If there is no spatial relationship value less than the preset spatial relationship value, it is determined that there is no spatial coincidence degree greater than the preset spatial coincidence degree.

7. A parking state detection device characterized by comprising: including: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the parking state detection method according to any one of claims 1 to 5.

Images