A parking management method, a parking management device, and a computer-readable storage medium

Abstract

The present application discloses a parking management method, a parking management device, and a computer-readable storage medium. The method includes: obtaining video data of a target parking scenario and parking configuration data, where the parking configuration data includes information related to at least one parking space in the target parking scenario; processing the video data to obtain a detection result of a vehicle in the target parking scenario, where the detection result includes reference point information of the vehicle, vehicle size information of the vehicle, and roof position information of the vehicle roof in the video data; determining vehicle chassis position information based on the reference point information, the vehicle size information, and the roof position information, where the vehicle chassis position information represents the position information of the vehicle chassis in the video data; and determining the driving state of the vehicle relative to a target parking space based on the vehicle chassis position information and the parking configuration data, where the target parking space is a parking space among the at least one parking space. By the above method, the present application can improve the false alarm or missed alarm of the driving state of the vehicle.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technologies, and particularly to a parking management method, a parking management device, and a computer-readable storage medium. Background Art

[0002] At present, many people own motor vehicles, resulting in an increasing demand for roadside parking. Charging for roadside parking manually can no longer meet the current needs and wastes human resources. Therefore, developing an intelligent control method for roadside parking events has become an urgent problem to be solved. Summary of the Invention

[0003] This application provides a parking management method, a parking management device, and a computer-readable storage medium, which can improve the false reporting or missed reporting of the driving state of a vehicle.

[0004] To solve the above technical problems, the technical solution adopted in this application is: providing a parking management method, which includes: obtaining video data and parking configuration data of a target parking scenario, where the parking configuration data includes information related to at least one parking space in the target parking scenario; processing the video data to obtain a detection result of a vehicle in the target parking scenario, where the detection result includes reference point information of the vehicle, vehicle size information of the vehicle, and roof position information of the vehicle roof in the video data; determining vehicle chassis position information based on the reference point information, the vehicle size information, and the roof position information, where the vehicle chassis position information represents the position information of the vehicle chassis in the video data; and determining the driving state of the vehicle relative to the target parking space based on the vehicle chassis position information and the parking configuration data, where the target parking space is a parking space among the at least one parking space.

[0005] To solve the above technical problems, another technical solution adopted in this application is: providing a parking management device, which includes a memory and a processor connected to each other. The memory is used to store a computer program, and when the computer program is executed by the processor, it is used to implement the parking management method in the above technical solution.

[0006] To solve the above technical problems, another technical solution adopted in this application is: providing a computer-readable storage medium, which is used to store a computer program, and when the computer program is executed by the processor, it is used to implement the parking management method in the above technical solution.

[0007] Through the above solution, the beneficial effects of the present application are as follows: First, obtain the video data in the target parking scenario and the parking configuration data including the relevant information of the parking spaces in the target parking scenario; then process the video data to obtain the detection result of the vehicle, which includes the reference point information and the vehicle size information of the vehicle, and the reference point information is determined based on the roof position information of the vehicle roof in the video data; then use the reference point information and the vehicle size information to calculate the vehicle chassis position information, which is the position information of the vehicle chassis in the video data; then use the vehicle chassis position information and the parking configuration data to determine the driving state of the current vehicle relative to one of all the parking spaces (i.e., the target parking space), and determine whether the current vehicle is in the driving-in state or the driving-out state; this solution uses the position at the top to estimate the position of the chassis, can accurately estimate the position of the chassis, and then realizes using the position of the chassis to judge the actual position relationship between the vehicle and the parking space, improves the recognition accuracy of the driving-in state or the driving-out state, and solves the problem of inaccurate vehicle position estimation caused by vehicle occlusion. Description of the Drawings

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. Among them:

[0009] Figure 1 is a flowchart of an embodiment of the parking management method provided by the present application;

[0010] Figure 2 is a schematic diagram of the warning area, parking space and warning line provided by the present application;

[0011] Figure 3 is a flowchart of another embodiment of the parking management method provided by the present application;

[0012] Figure 4 is a schematic diagram of the vehicle chassis estimation network provided by the present application;

[0013] Figure 5 is a schematic diagram of the monitoring image provided by the present application;

[0014] Figure 6 is Figure 5 the image generated by inputting the image in

[0015] Figure 7 is a schematic diagram of the motion state estimation queue provided by the present application;

[0016] Figure 8It is a schematic diagram of parking state judgment provided by this application;

[0017] Figure 9 It is a captured image schematic diagram of the driving-in state provided by this application;

[0018] Figure 10 It is a captured image schematic diagram of the driving-out state provided by this application;

[0019] Figure 11 It is a schematic structural diagram of an embodiment of the parking management device provided by this application;

[0020] Figure 12 It is a schematic structural diagram of another embodiment of the parking management device provided by this application;

[0021] Figure 13 It is a schematic structural diagram of an embodiment of the computer-readable storage medium provided by this application. Detailed implementation manners

[0022] The following will further describe this application in detail in conjunction with the accompanying drawings and embodiments. It should be specifically noted that the following embodiments are only used to illustrate this application, but do not limit the scope of this application. Similarly, the following embodiments are only partial embodiments of this application rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope protected by this application.

[0023] Referring to "embodiment" in this application means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0024] It should be noted that the terms "first", "second", and "third" in this application are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", and "third" may explicitly or implicitly include at least one of such features. In the description of this application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically and clearly defined. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or devices.

[0025] In the road test parking scenario, vehicles generally enter the parking space from the side. During the process of the vehicle entering the parking space, there is a phenomenon of vehicle inclination. The target detection scheme in the related technology cannot estimate the inclined edge of the vehicle, and the detection frames with high overlap also hinder target tracking in case of occlusion, making it impossible to accurately determine whether the vehicle has parked in the parking space, resulting in missed reports / false reports of the driving-in state or driving-out state. To solve these problems, the present application provides a new solution that can improve the detection accuracy of the driving-in / driving-out state of the vehicle, which will be described in detail below.

[0026] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an embodiment of the parking management method provided by the present application. The method includes:

[0027] S11: Obtain video data and parking configuration data of the target parking scenario.

[0028] A camera device can be used to shoot the target parking scenario to generate video data, or receive video data output by other devices (such as: embedded devices). Specifically, the target parking scenario is a scenario with parking spaces, such as: parking lots on both sides of the road or parking lots at other locations; the video data includes multiple frames of monitoring images, and the monitoring images include vehicles; the parking configuration data is data required to determine the state of the vehicle, which includes at least one of the coordinates of at least one parking space in the target parking scenario, the coordinates of the warning area, or the coordinates of the warning line. The warning area includes all areas where parking is located (denoted as the parking area). As Figure 2 shown, the warning line can be the edge line of the parking space.

[0029] In one embodiment, the parking configuration data includes the coordinates of at least one parking space in the target parking scenario. First, the edge line of the parking space can be determined through the coordinates of the parking space, and then, based on the edge line of the parking space and the coordinates of the parking space, it can be determined whether the vehicle is driving into the parking space or driving out of the parking space; alternatively, the parking configuration data includes the coordinates of at least one parking space and the coordinates of the warning line in the target parking scenario. By the coordinates of the warning line and the coordinates of the parking space, it can be determined whether the vehicle is driving into the parking space or driving out of the parking space; alternatively, the parking configuration data includes the coordinates of at least one parking space, the coordinates of the warning area, and the coordinates of the warning line in the target parking scenario. Through the coordination of the coordinates of the parking space, the coordinates of the warning area, and the coordinates of the warning line, it can be more accurately determined whether the vehicle is driving into the parking space or driving out of the parking space.

[0030] S12: Process the video data to obtain the detection result of the vehicle in the target parking scenario.

[0031] After obtaining the video data, the object detection method in related technologies can be used to perform object detection processing on the video data to generate a detection result, which includes reference point information, vehicle size information of the vehicle, and roof position information of the roof of the vehicle (denoted as the roof) in the video data; specifically, the reference point information includes the coordinates of the center point of the vehicle, and the vehicle size information includes the width and height of the vehicle; the roof position information includes the coordinates of the roof key points, and the roof key points include multiple corner points (key points). Since the shape of the roof is generally a quadrilateral, the key points of the roof (denoted as roof key points) are the vertices of the quadrilateral.

[0032] S13: Based on the reference point information, vehicle size information, and roof position information, determine the vehicle chassis position information.

[0033] After obtaining the detection result related to the vehicle, the test point information, vehicle size information, and roof position information in the detection result can be processed to generate vehicle chassis position information, which represents the position information of the vehicle chassis in the video data; specifically, the vehicle chassis position information includes the position where the vehicle chassis key points are located, and the vehicle chassis key points are the key points of the vehicle chassis (denoted as the vehicle chassis). Since the shape of the vehicle chassis is generally a quadrilateral, the vehicle chassis key points are the vertices of the quadrilateral.

[0034] S14: Based on the vehicle chassis position information and parking configuration data, determine the driving state of the vehicle relative to the target parking space.

[0035] After obtaining the vehicle chassis position information of the vehicle, use the vehicle chassis position information and parking configuration data to generate the driving state of the vehicle relative to the target parking space. The target parking space is a parking space in at least one parking space; specifically, the parking space that the vehicle drives into or out of is determined as the target parking space. The driving state includes the first state in which the vehicle drives into the target parking space and / or the second state in which the vehicle drives out of the target parking space. The first state can be denoted as the driving-in state, and the second state can be denoted as the driving-out state.

[0036] In a specific embodiment, the relevant information of the parking space includes the coordinates of the parking space. After obtaining the coordinates of the vehicle chassis key points of the vehicle, the coordinates of the vehicle chassis key points can be compared with the coordinates of the parking space in the parking configuration data to determine the parking space (i.e., the target parking space) where the vehicle is located. Then, by comparing the coordinates of the vehicle chassis key points with the position of the target parking space, the position of the warning area, or the position of the warning line, it is determined whether the vehicle enters the target parking space or drives out of the target parking space, so as to realize the recognition of the driving state; it can be understood that when it is determined that the driving state of the current vehicle is the driving-in state, it can be determined that the vehicle has had a driving-in event, and when it is determined that the driving state of the vehicle is the driving-out state, it can be determined that the current vehicle has had a driving-out event.

[0037] This embodiment provides a method for estimating the position of a vehicle chassis based on the position of the vehicle roof, which can accurately detect the position of the vehicle chassis using the position of the vehicle roof, and can be applied to the field of intelligent management and control of roadside parking events to solve the problem of inaccurate vehicle position estimation caused by vehicle occlusion in roadside parking scenarios; moreover, the driving-in / driving-out state of the vehicle can be further determined using the key points of the vehicle chassis, and the actual position relationship between the vehicle and the parking space can be accurately judged, which can improve the accuracy of whether a driving-in event and / or a driving-in event occurs, thereby solving the problem of missed reporting or false reporting of driving-in / driving-out events caused by inaccurate vehicle position estimation in roadside parking scenarios, and realizing high-recall and high-accuracy capture of driving-in / driving-out events.

[0038] Please refer to Figure 3 , Figure 3 which is a schematic flowchart of another embodiment of the parking management method provided by this application. The method includes:

[0039] S31: Obtain the video data and parking configuration data of the target parking scenario.

[0040] First, take the video containing the target parking scenario (such as a roadside parking scenario) or the real-time code stream output by an embedded device as the video data; then configure the warning area, warning line, and the area where each parking space is located (denoted as the parking space area) as the basis for judging driving-in / driving-out events.

[0041] S32: Process the monitoring image using the trained vehicle chassis estimation network to obtain the detection result.

[0042] This embodiment designs a method for estimating and tracking the vehicle chassis frame based on the key points of the vehicle roof to solve the problem that the target detection frame in the related technology cannot accurately estimate the actual position of the vehicle in the case of vehicle occlusion, which may lead to missed reporting / false reporting of driving-in / driving-out events in roadside parking scenarios; specifically, use the Figure 4 shown vehicle chassis estimation network to estimate the coordinates of the vehicle chassis frame. The vehicle chassis estimation network includes a detection network and a calculation network. The detection network includes a feature extraction sub-network, a feature fusion sub-network, and a target detection regression sub-network. The feature extraction sub-network can be used to process the monitoring image to obtain the first vehicle feature; the feature fusion sub-network can be used to process the first vehicle feature to obtain the second vehicle feature; the target detection regression sub-network can be used to process the second vehicle feature to obtain the detection result, which includes the coordinates of the key points on the vehicle roof, the center point of the vehicle, the width of the vehicle, and the height of the vehicle. The key points on the vehicle roof are the key points on the top of the vehicle.

[0043] In a specific embodiment, the feature extraction sub-network may be a backbone network. The backbone network adopts deep learning feature extraction methods, including but not limited to AlexNet, Visual Geometry Group Network (VGGNet), or SFNet, etc.; the feature fusion sub-network includes an encoder and a decoder. The feature fusion sub-network is a feature fusion network using deep learning methods, including but not limited to using a Multi-Dimesional Attention (MDA-Net) or a CrossStage Partial Network (CSP-Net), etc. to fuse boundary feature maps, attention mechanism feature maps, original network feature maps, saliency feature maps, or binary feature maps, etc.; the object detection regression sub-network is a key-point based object detection regression network, including but not limited to centernet or cornernet, etc.

[0044] S33: Determine the vehicle chassis position information based on the reference point information, vehicle size information, and roof position information.

[0045] The roof position information includes the coordinates of the roof key points. The roof key points include the upper corner point and the lower corner point of the roof. The coordinates of the vehicle chassis frame can be calculated using the following scheme:

[0046] 1) Based on the coordinates of the center point of the vehicle, the width of the vehicle, and the height of the vehicle, calculate the coordinates of at least two corner points on the vehicle detection frame that identifies the vehicle.

[0047] The at least two corner points include the lower left corner point and the lower right corner point of the vehicle detection frame. The vehicle chassis position information includes the coordinates of the vehicle chassis frame. The vehicle chassis frame is the detection frame where the chassis of the vehicle is located. The scheme for calculating the coordinates of the lower left corner point and the lower right corner point of the vehicle detection frame is as follows:

[0048] Calculate the difference between the abscissa of the center point and the width to obtain the abscissa of the lower left corner point of the vehicle detection frame, where the vehicle detection frame is the detection frame where the vehicle is located; calculate the difference between the ordinate of the center point and the height to obtain the ordinate of the lower left corner point of the vehicle detection frame; calculate the sum of the abscissa of the center point and the width to obtain the abscissa of the lower right corner point of the vehicle detection frame; calculate the sum of the ordinate of the center point and the height to obtain the ordinate of the lower right corner point of the vehicle detection frame. That is, use the following formula to calculate the coordinates of the lower left corner point and the lower right corner point of the vehicle detection frame:

[0049]

[0050] 2) Determine the vehicle chassis position information by using the roof position information and the coordinates of at least two corner points.

[0051] The coordinates of the vehicle chassis frame can be calculated based on the coordinates of the lower left corner point and the lower right corner point of the vehicle detection frame. Specifically, based on the magnitude relationship between the abscissa of the upper corner point of the roof and the abscissa of the corresponding lower corner point of the roof, using the corresponding pre-designed calculation rules, the coordinates of the lower left corner point of the vehicle detection frame, the coordinates of the lower right corner point of the vehicle detection frame, and the coordinates of the key points on the roof are processed to obtain the coordinates of the vehicle chassis frame, where the key points on the roof are the detection frame where the top of the vehicle is located.

[0052] In a specific embodiment, the upper corner points of the roof include the upper left corner point and the upper right corner point of the roof, and the lower corner points of the roof include the lower left corner point and the lower right corner point of the roof. The following scheme can be adopted:

[0053] (1) The first pre-designed calculation rule

[0054] When the abscissa of the upper left corner point of the roof is greater than or equal to the abscissa of the lower left corner point of the roof, determine the abscissa of the lower left corner point of the vehicle detection frame as the abscissa of the lower left corner point of the vehicle chassis frame; determine the ordinate of the lower left corner point of the vehicle detection frame as the ordinate of the lower left corner point of the vehicle chassis frame; calculate the difference between the abscissa of the lower left corner point of the roof and the abscissa of the upper left corner point of the roof to obtain the first difference; calculate the difference between the abscissa of the lower left corner point of the vehicle chassis frame and the first difference to obtain the abscissa of the upper left corner point of the vehicle chassis frame; calculate the difference between the ordinate of the lower left corner point of the roof and the ordinate of the upper left corner point of the roof to obtain the second difference; calculate the difference between the ordinate of the lower left corner point of the vehicle chassis frame and the second difference to obtain the ordinate of the upper left corner point of the vehicle chassis frame. That is, when Rul.x≥Rbl.x, the coordinates of the upper left corner point and the lower left corner point of the vehicle chassis frame are calculated using the following formula:

[0055]

[0056] Among them, Rul.x is the abscissa of the upper left corner point of the roof, Rul.y is the ordinate of the upper left corner point of the roof, Rbl.x is the abscissa of the lower left corner point of the roof, Rbl.y is the ordinate of the lower left corner point of the roof; Vbl.x is the abscissa of the lower left corner point of the vehicle detection frame, Vbl.y is the ordinate of the lower left corner point of the vehicle detection frame; VCbl.x is the abscissa of the lower left corner point of the vehicle chassis frame, VCbl.y is the ordinate of the lower left corner point of the vehicle chassis frame, VCul.x is the abscissa of the upper left corner point of the vehicle chassis frame, and VCul.y is the ordinate of the upper left corner point of the vehicle chassis frame.

[0057] (2) The second pre-designed calculation rule

[0058] When the abscissa of the upper left corner point of the vehicle roof is less than the abscissa of the lower left corner point of the vehicle roof, calculate the sum of the abscissa of the lower left corner point of the vehicle roof and the first difference to obtain the abscissa of the lower left corner point of the vehicle chassis frame; determine the ordinate of the lower left corner point of the vehicle roof as the ordinate of the lower left corner point of the vehicle chassis frame; determine the abscissa of the lower left corner point of the vehicle chassis frame as the abscissa of the upper left corner point of the vehicle chassis frame; calculate the difference between the ordinate of the lower left corner point of the vehicle chassis frame and the second difference to obtain the ordinate of the upper left corner point of the vehicle chassis frame, that is, when Rul.x < Rbl.x, use the following formula to calculate the coordinates of the upper left corner point and the lower left corner point of the vehicle chassis frame:

[0059]

[0060] (3) The third pre-designed calculation rule

[0061] When the abscissa of the upper right corner point of the vehicle roof is less than the abscissa of the lower right corner point of the vehicle roof, determine the abscissa of the lower right corner point of the vehicle roof as the abscissa of the lower right corner point of the vehicle chassis frame; determine the ordinate of the lower right corner point of the vehicle roof as the ordinate of the lower right corner point of the vehicle chassis frame; calculate the difference between the abscissa of the lower right corner point of the vehicle roof and the abscissa of the upper right corner point of the vehicle roof to obtain the third difference; calculate the difference between the abscissa of the lower right corner point of the vehicle chassis frame and the third difference to obtain the abscissa of the upper right corner point of the vehicle chassis frame; calculate the difference between the ordinate of the lower right corner point of the vehicle roof and the ordinate of the upper right corner point of the vehicle roof to obtain the fourth difference; calculate the difference between the ordinate of the lower right corner point of the vehicle chassis frame and the fourth difference to obtain the ordinate of the upper right corner point of the vehicle chassis frame, that is, when Rur.x < Rbr.x, use the following formula to calculate the coordinates of the upper right corner point and the lower right corner point of the vehicle chassis frame:

[0062]

[0063] Among them, Rbr.x is the abscissa of the lower right corner point of the vehicle roof, Rbr.y is the ordinate of the lower right corner point of the vehicle roof, Rur.x is the abscissa of the upper right corner point of the vehicle roof, Rur.y is the ordinate of the upper right corner point of the vehicle roof; Vbr.x is the abscissa of the lower right corner point of the vehicle detection frame, Vbr.y is the ordinate of the lower right corner point of the vehicle detection frame; VCur.x is the abscissa of the upper right corner point of the vehicle chassis frame, VCur.y is the ordinate of the upper right corner point of the vehicle chassis frame, and VCbr.x is the abscissa of the lower right corner point of the vehicle chassis frame, and VCbr.y is the ordinate of the lower right corner point of the vehicle chassis frame.

[0064] (4) The fourth pre-designed calculation rule

[0065] When the abscissa of the upper right corner point of the vehicle roof is greater than or equal to the abscissa of the lower right corner point of the vehicle roof, calculate the sum of the abscissa of the lower right corner point of the vehicle roof and the third difference to obtain the abscissa of the lower right corner point of the vehicle chassis frame; determine the ordinate of the lower right corner point of the vehicle roof as the ordinate of the lower right corner point of the vehicle chassis frame; determine the abscissa of the lower right corner point of the vehicle chassis frame as the abscissa of the upper right corner point of the vehicle chassis frame; calculate the difference between the ordinate of the lower right corner point of the vehicle chassis frame and the fourth difference to obtain the ordinate of the upper right corner point of the vehicle chassis frame, that is, when Rur.x≥Rbr.x, the coordinates of the upper right corner point and the lower right corner point of the vehicle chassis frame are calculated using the following formula:

[0066]

[0067] In one embodiment, as Figure 5 and Figure 6 shown, input the Figure 5 shown image into the vehicle chassis estimation network to obtain the Figure 6 shown result, that is, detect the center point c, width w, height h of the vehicle and the 4 key points of the vehicle roof (upper left corner point Rul, lower left corner point Rbl, upper right corner point Rur, and lower right corner point Rbr). Vbl and Vbr are respectively the lower left corner point and the lower right corner point of the vehicle detection frame, and are calculated through the above formula (1); according to the detection result and formulas (2)-(3), the four corner points of the vehicle chassis frame (upper left corner point VCul, lower left corner point VCbl, upper right corner point VCur, and lower right corner point VCbr) can be calculated.

[0068] In another specific embodiment, a preset tracking strategy can be adopted to determine whether the vehicle in the previous frame of the monitoring image is the same as the vehicle in the current frame of the monitoring image; if the vehicle in the previous frame of the monitoring image is the same as the vehicle in the current frame of the monitoring image, track the vehicle to obtain a tracking result, and the tracking result includes the vehicle chassis frame of the vehicle. Specifically, calculate the difference between the abscissa of the center point of the vehicle chassis frame corresponding to the previous frame of the monitoring image and the abscissa of the center point of the vehicle chassis frame corresponding to the current frame of the monitoring image to obtain a fifth difference; calculate the difference between the ordinate of the center point of the vehicle chassis frame corresponding to the previous frame of the monitoring image and the ordinate of the center point of the vehicle chassis frame corresponding to the current frame of the monitoring image to obtain a sixth difference; determine whether the intersection-over-union (IOU) of the vehicle detection frame corresponding to the previous frame of the monitoring image and the vehicle detection frame corresponding to the current frame of the monitoring image is greater than a first preset value, whether the absolute value of the fifth difference is less than the height of the vehicle, and whether the absolute value of the sixth difference is less than the height of the vehicle; if the intersection-over-union of the vehicle detection frame corresponding to the previous frame of the monitoring image and the vehicle detection frame corresponding to the current frame of the monitoring image is greater than the first preset value, the absolute value of the fifth difference is less than the height of the vehicle, and the absolute value of the sixth difference is less than the height of the vehicle, it is determined that the vehicle in the previous frame of the monitoring image is the same as the vehicle in the current frame of the monitoring image.

[0069] Furthermore, a Kalman filter tracking method based on the vehicle chassis frame is adopted to track the vehicle, and a preset tracking strategy is added. The tracking will determine that the front and rear vehicles are the same vehicle for tracking only when all the following judgment conditions are met:

[0070]

[0071] Among them, pre represents the vehicle chassis frame corresponding to the previous frame of the monitoring image (denoted as the previous vehicle chassis frame), pre_C.x is the abscissa of the center point of the previous vehicle chassis frame, and pre_C.y is the ordinate of the center point of the previous vehicle chassis frame; cur represents the vehicle chassis frame corresponding to the current frame of the monitoring image (denoted as the current vehicle chassis frame), cur_C.x is the abscissa of the center point of the current vehicle chassis frame, and cur_C.y is the ordinate of the center point of the current vehicle chassis frame; h is the height of the current vehicle chassis frame; Vth1 is the first preset value, which can be 0.2.

[0072] In other embodiments, a maximum tracking interval can also be set: a preset time interval to solve the problem of unstable vehicle detection; specifically, determine whether the time difference between the current moment and the moment when the vehicle was last detected is greater than the preset time interval; if the time difference between the current moment and the moment when the vehicle was last detected is greater than the preset time interval, that is, the time since the vehicle was last detected has exceeded the preset time interval, it is determined not to track the vehicle.

[0073] In another specific embodiment, the vehicle can also be subjected to attribute recognition processing to obtain the attribute information of the vehicle. The attribute information includes the color, model, license plate, angle, or motion state of the vehicle.

[0074] Further, a deep learning object recognition method can be used to recognize the color, model, and license plate of the vehicle. The deep learning object recognition method includes but is not limited to AlexNet, VGGNet, or SFNet, etc. The angle of the vehicle can be obtained by calculating the included angle between the vertical center line of the vehicle chassis frame and the vertical center line of the berth area. The motion state includes a stationary state, and it can be determined whether the vehicle is stationary through the motion state estimation of the vehicle. For example, a motion state estimation queue is established. The motion state estimation queue includes the intersection over union (IoU) of the vehicle chassis frames corresponding to the current frame monitoring image and a preset number of frame monitoring images after the current frame monitoring image; it is determined whether all the IoUs in the motion state estimation queue are greater than a second preset value; if all the IoUs in the motion state estimation queue are greater than the second preset value, it is determined that the motion state of the vehicle is a stationary state, and the second preset value can be 0.5.

[0075] In one implementation manner, a motion state estimation queue with a length of (n - 1) can be established, as Figure 7 shown, the IOU F1-n represents the IOU between the vehicle chassis frame corresponding to the nth frame image and the vehicle chassis frame corresponding to the first frame image. n can be set according to a set time period (such as: 3 seconds) or application requirements; if the IoUs between the vehicle chassis frames corresponding to the 2nd to nth frame images and the vehicle chassis frame corresponding to the first frame image are all greater than 0.5, it is considered that the motion state of the vehicle is a stationary state.

[0076] S34: Determine the target berth based on the coordinates of the vehicle chassis frame and the coordinates of the berth.

[0077] After obtaining the coordinates of the vehicle chassis frame, compare the coordinates of the vehicle chassis frame with the coordinates of all berths to determine which berth the coordinates of the vehicle chassis frame fall into, that is, determine which berth the vehicle drives into, and obtain the target berth. The target berth is the berth where the vehicle is located.

[0078] S35: Determine the parking state of the vehicle based on the intersection over union (IoU) between the vehicle chassis frame and the detection frame of the target berth.

[0079] The parking states include parking on the line, parking across berths, and normal parking. When the intersection ratio is greater than the third preset value and less than the fourth preset value, and there is an intersection between the vehicle chassis frame and the warning line, the parking state is determined to be parking on the line; when the intersection ratio is greater than the third preset value and less than the fourth preset value, and there is an intersection between the vehicle chassis frame and the upper edge and / or lower edge of the berth, the parking state is determined to be parking across berths; when the intersection ratio is greater than the fourth preset value, the parking state is determined to be normal parking.

[0080] In one embodiment, the third preset value can be 0.1, and the fourth preset value can be 0.7. Taking Figure 8 as an example, the parking state is judged by calculating the IOU between the vehicle chassis frame and berth 5. The calculation formula of IOU is as follows:

[0081]

[0082] where I1 is the vehicle chassis frame and I2 is the detection frame of berth 5.

[0083] The method for judging the parking state is as follows:

[0084] 1) If 0.1 < IOU < 0.7 and there is an intersection between the vehicle chassis frame and the warning line, the parking state of the vehicle is parking on the line.

[0085] 2) If 0.1 < IOU < 0.7 and there is an intersection between the vehicle chassis frame and the upper edge line and / or lower edge line of berth 5, the parking state of the vehicle is parking across berths.

[0086] 3) If IOU > 0.7, it is judged that the parking state of the vehicle is normal parking.

[0087] S36: Based on the coordinates of the vehicle chassis frame and the parking configuration data, determine the driving state of the vehicle relative to the target berth.

[0088] Obtain the image in which the center point of the vehicle chassis frame falls within the warning area to get the first captured image; obtain the image in which the center point of the vehicle chassis frame exceeds the warning line to get the second captured image; obtain the image in which the intersection ratio between the vehicle chassis frame and the detection frame of the berth is greater than the fifth preset value, the center point of the vehicle chassis frame falls within the berth, and the motion state of the vehicle is the stationary state to get the third captured image; obtain the image in which the center point of the vehicle chassis frame falls outside the warning area to get the fourth captured image; obtain the image in which the intersection ratio between the vehicle chassis frame and the detection frame of the berth is greater than the fifth preset value and the center point of the vehicle chassis frame falls within the berth to get the fifth captured image; based on the capture time of the first captured image, the second captured image, the third captured image, the fourth captured image, or the fifth captured image, determine the driving state and generate a reporting message, and the reporting message includes the driving state, the parking state of the vehicle, or the attribute information.

[0089] Furthermore, the driving state includes a first state in which the vehicle drives into the target berth and / or a second state in which the vehicle drives out of the target berth; when it is determined that the current meets the first preset condition, the driving state is determined as the first state (i.e., the driving-in state), and the first preset condition includes that the capture time of the second captured image is greater than the capture time of the first captured image and less than the capture time of the third captured image, that is, the first captured image is captured first, then the second captured image is captured, and then the third captured image is captured; and / or, when it is determined that the current meets the second preset condition, the driving state is determined as the second state (i.e., the driving-out state), and the second preset condition includes that the capture time of the second captured image is greater than the capture time of the fifth captured image and less than the capture time of the fourth captured image, that is, the fifth captured image is captured first, then the second captured image is captured, and then the fourth captured image is captured.

[0090] In one embodiment, assuming that the fifth preset value is 0.1, as Figure 9 shown, when it is determined that the center point of the vehicle chassis frame enters the warning area, the first image (i.e., the first captured image) is captured; when it is determined that the center point of the vehicle chassis frame crosses the warning line, the second image (i.e., the second captured image) is captured; when it is determined that the center point of the vehicle chassis frame falls within the berth area, the motion state of the vehicle is in a stationary state, and the IOU between the vehicle chassis frame and the berth area is > 0.1, the third image (i.e., the third captured image) is captured, and the parking state and the attribute information of the vehicle are reported at the same time; if it is determined that the order of the three captured images is the same as above, it is determined that a driving-in event has occurred.

[0091] As Figure 10 shown, when it is determined that the center point of the vehicle chassis frame is within the berth area and the IOU between the vehicle chassis frame and the berth area is > 0.1, the first image (i.e., the fifth captured image) is captured; when it is determined that the center point of the vehicle chassis frame crosses the warning line, the second image (i.e., the second captured image) is captured; when it is determined that the center point of the vehicle chassis frame is outside the warning area, the third image (i.e., the fourth captured image) is captured, and the parking state and the attribute information of the vehicle are reported at the same time; if it is determined that the order of the three captured images is the same as above, it is determined that a driving-out event has occurred.

[0092] In other embodiments, it is also possible to determine whether the vehicle corresponding to the first state and the vehicle corresponding to the second state are the same vehicle; if the vehicle corresponding to the first state and the vehicle corresponding to the second state are the same vehicle, then the first state and the second state are associated. Specifically, the first state is the driving-in state, and the second state is the driving-out state. It is possible to determine whether the vehicle has a license plate. If the vehicle has a license plate, the driving-in event and the driving-out event of the vehicle are associated by the license plate number; if the vehicle does not have a license plate, the driving-in event and the driving-out event of the vehicle can be associated by re-identification (reid) technology.

[0093] This embodiment provides a method for judging the parking state and driving state based on the vehicle chassis frame, which can accurately judge the actual position relationship between the vehicle and the parking space, so as to accurately estimate the parking state and driving state of the vehicle and improve the accurate detection of the vehicle entering and leaving the parking space.

[0094] Please refer to Figure 11 , Figure 11 which is a schematic structural diagram of an embodiment of the parking management device provided by the present application. The parking management device 110 includes a mutually connected memory 111 and a processor 112. The memory 111 is used to store a computer program, and when the computer program is executed by the processor 112, it is used to implement the parking management method in the above embodiment.

[0095] Please refer to Figure 12 , Figure 12 which is a schematic structural diagram of another embodiment of the parking management device provided by the present application. The parking management device 120 includes an input module 121, an event configuration module 122, a vehicle detection and tracking module 123, a vehicle attribute module 124, and an event judgment module 125.

[0096] The input module 121 is used to receive video data containing the target parking scenario or video data output in real time by an embedded device (not shown in the figure).

[0097] The event configuration module 122 is used to obtain parking configuration data, which includes a warning area, a warning line, and a parking space area, and is used as a basis for judging driving-in and driving-out events.

[0098] The vehicle detection and tracking module 123 is connected to the input module 121 and the event configuration module 122, and includes a vehicle chassis detection module 1231 and a tracking module 1232 that are connected to each other. The vehicle chassis detection module 1231 is used to process video data to obtain the detection result of the vehicle in the target parking scenario. The detection result includes the reference point information of the vehicle, the vehicle size information of the vehicle, and the roof position information of the vehicle roof in the video data. Based on the reference point information, the vehicle size information, and the roof position information, the vehicle chassis position information is determined, and the vehicle chassis position information represents the position information of the vehicle chassis in the video data. Based on the vehicle chassis position information and the parking configuration data, the driving state of the vehicle relative to the target parking space is determined, and the target parking space is a parking space in at least one parking space. The tracking module 1232 is used to track the vehicle.

[0099] The vehicle attribute module 124 is connected to the vehicle detection and tracking module 123, and is used to implement vehicle color recognition, vehicle type recognition, license plate recognition, vehicle angle estimation, or vehicle motion state estimation.

[0100] The event judgment module 125 is connected to the vehicle attribute module 124, and includes a parking state judgment module 1251 and an in-out event judgment module 1252. The parking state judgment module 1251 is used to judge the parking state of the vehicle, and the in-out event judgment module 1252 is used to judge whether an in event or an out event occurs.

[0101] This embodiment provides an intelligent management solution for roadside parking events with high-precision positioning, which can accurately detect whether a vehicle enters or exits the parking space area, and prevent missed reports or false reports of in-out events.

[0102] Please refer to Figure 13 , Figure 13 is a schematic structural diagram of an embodiment of a computer-readable storage medium provided by this application. The computer-readable storage medium 130 is used to store a computer program 131, and when the computer program 131 is executed by a processor, it is used to implement the parking management method in the above embodiment.

[0103] The computer-readable storage medium 130 can be various media that can store program codes, such as a server, a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc.

[0104] In several embodiments provided by the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0107] The above are only the embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A parking management method, characterized in that, it includes: obtaining video data and parking configuration data of a target parking scenario, where the parking configuration data includes information related to at least one parking space in the target parking scenario; processing the video data to obtain a detection result of a vehicle in the target parking scenario, where the detection result includes reference point information of the vehicle, vehicle size information of the vehicle, and roof position information of the vehicle in the video data, the reference point information includes the coordinates of the center point of the vehicle, and the vehicle size information includes the width and height of the vehicle; determining vehicle chassis position information based on the reference point information, the vehicle size information, and the roof position information, where the vehicle chassis position information represents the position information of the chassis of the vehicle in the video data, and includes: calculating the coordinates of at least two corner points on a vehicle detection frame identifying the vehicle based on the coordinates of the center point of the vehicle, the width, and the height of the vehicle; using the roof position information and the coordinates of the at least two corner points to determine the vehicle chassis position information; the at least two corner points include the lower left corner point and the lower right corner point of the vehicle detection frame, and the vehicle chassis position information includes the coordinates of a vehicle chassis frame, and the vehicle chassis frame is a detection frame where the chassis of the vehicle is located; determining the driving state of the vehicle relative to a target parking space based on the vehicle chassis position information and the parking configuration data, where the target parking space is a parking space among the at least one parking space.

2. The parking management method according to claim 1, characterized in that, the video data includes multiple frames of monitoring images, the monitoring images include the vehicle, and the step of processing the video data to obtain a detection result of the vehicle in the target parking scenario includes: processing the monitoring images using a trained vehicle chassis estimation network to obtain the detection result.

3. The parking management method according to claim 1, characterized in that, the roof position information includes the coordinates of roof key points, the roof key points include an upper roof corner point and a lower roof corner point, and the step of using the roof position information and the coordinates of the at least two corner points to determine the vehicle chassis position information includes: processing the coordinates of the lower left corner point of the vehicle detection frame, the coordinates of the lower right corner point of the vehicle detection frame, and the coordinates of the roof key points using a corresponding pre-designed calculation rule based on the size relationship between the abscissa of the upper roof corner point and the abscissa of the corresponding lower roof corner point to obtain the coordinates of the vehicle chassis frame, where the roof key points are key points at the top of the vehicle.

4. The parking management method according to claim 3, characterized in that, The upper corner points of the vehicle roof include the upper left corner point and the upper right corner point of the vehicle roof, and the lower corner points of the vehicle roof include the lower left corner point and the lower right corner point of the vehicle roof; the step of processing the coordinates of the lower left corner point of the vehicle detection frame, the coordinates of the lower right corner point of the vehicle detection frame, and the coordinates of the key points on the vehicle roof by using corresponding pre-designed calculation rules based on the magnitude relationship between the abscissa of the upper corner points of the vehicle roof and the abscissa of the corresponding lower corner points of the vehicle roof to obtain the coordinates of the vehicle chassis frame includes: When the abscissa of the upper left corner point of the vehicle roof is greater than or equal to the abscissa of the lower left corner point of the vehicle roof, determine the abscissa of the lower left corner point of the vehicle detection frame as the abscissa of the lower left corner point of the vehicle chassis frame; determine the ordinate of the lower left corner point of the vehicle detection frame as the ordinate of the lower left corner point of the vehicle chassis frame; calculate the difference between the abscissa of the lower left corner point of the vehicle roof and the abscissa of the upper left corner point of the vehicle roof to obtain a first difference; calculate the difference between the abscissa of the lower left corner point of the vehicle chassis frame and the first difference to obtain the abscissa of the upper left corner point of the vehicle chassis frame; calculate the difference between the ordinate of the lower left corner point of the vehicle roof and the ordinate of the upper left corner point of the vehicle roof to obtain a second difference; calculate the difference between the ordinate of the lower left corner point of the vehicle chassis frame and the second difference to obtain the ordinate of the upper left corner point of the vehicle chassis frame; When the abscissa of the upper left corner point of the vehicle roof is less than the abscissa of the lower left corner point of the vehicle roof, calculate the sum of the abscissa of the lower left corner point of the vehicle roof and the first difference to obtain the abscissa of the lower left corner point of the vehicle chassis frame; determine the ordinate of the lower left corner point of the vehicle roof as the ordinate of the lower left corner point of the vehicle chassis frame; determine the abscissa of the lower left corner point of the vehicle chassis frame as the abscissa of the upper left corner point of the vehicle chassis frame; calculate the difference between the ordinate of the lower left corner point of the vehicle chassis frame and the second difference to obtain the ordinate of the upper left corner point of the vehicle chassis frame; When the abscissa of the upper right corner point of the vehicle roof is less than the abscissa of the lower right corner point of the vehicle roof, determine the abscissa of the lower right corner point of the vehicle roof as the abscissa of the lower right corner point of the vehicle chassis frame; determine the ordinate of the lower right corner point of the vehicle roof as the ordinate of the lower right corner point of the vehicle chassis frame; calculate the difference between the abscissa of the lower right corner point of the vehicle roof and the abscissa of the upper right corner point of the vehicle roof to obtain a third difference; calculate the difference between the abscissa of the lower right corner point of the vehicle chassis frame and the third difference to obtain the abscissa of the upper right corner point of the vehicle chassis frame; calculate the difference between the ordinate of the lower right corner point of the vehicle roof and the ordinate of the upper right corner point of the vehicle roof to obtain a fourth difference; calculate the difference between the ordinate of the lower right corner point of the vehicle chassis frame and the fourth difference to obtain the ordinate of the upper right corner point of the vehicle chassis frame; When the abscissa of the upper right corner point of the vehicle roof is greater than or equal to the abscissa of the lower right corner point of the vehicle roof, calculate the sum of the abscissa of the lower right corner point of the vehicle roof and the third difference to obtain the abscissa of the lower right corner point of the vehicle chassis frame; determine the ordinate of the lower right corner point of the vehicle roof as the ordinate of the lower right corner point of the vehicle chassis frame; determine the abscissa of the lower right corner point of the vehicle chassis frame as the abscissa of the upper right corner point of the vehicle chassis frame; calculate the difference between the ordinate of the lower right corner point of the vehicle chassis frame and the fourth difference to obtain the ordinate of the upper right corner point of the vehicle chassis frame.

5. The parking management method according to claim 1, wherein, the method further includes: judging whether the vehicle in the previous frame of monitoring image is the same as the vehicle in the current frame of monitoring image by using a preset tracking strategy; if so, tracking the vehicle.

6. The parking management method according to claim 5, wherein, the step of judging whether the vehicle in the previous frame of monitoring image is the same as the vehicle in the current frame of monitoring image by using a preset tracking strategy includes: calculating the difference between the abscissa of the center point of the vehicle chassis frame corresponding to the previous frame of monitoring image and the abscissa of the center point of the vehicle chassis frame corresponding to the current frame of monitoring image to obtain a fifth difference; calculating the difference between the ordinate of the center point of the vehicle chassis frame corresponding to the previous frame of monitoring image and the ordinate of the center point of the vehicle chassis frame corresponding to the current frame of monitoring image to obtain a sixth difference; judging whether the intersection over union ratio of the vehicle detection frame corresponding to the previous frame of monitoring image and the vehicle detection frame corresponding to the current frame of monitoring image is greater than a first preset value, whether the absolute value of the fifth difference is less than the height of the vehicle, and whether the absolute value of the sixth difference is less than the height; if so, determining that the vehicle in the previous frame of monitoring image is the same as the vehicle in the current frame of monitoring image.

7. The parking management method according to claim 1, wherein, the method further includes: establishing a motion state estimation queue of the vehicle, where the motion state estimation queue includes the intersection over union ratios of the vehicle chassis frames corresponding to the current frame of monitoring image and a preset number of frames of monitoring images after the current frame of monitoring image; judging whether all the intersection over union ratios in the motion state estimation queue are greater than a second preset value; if so, determining that the motion state of the vehicle is a stationary state.

8. The parking management method according to claim 1, wherein, the relevant information includes the coordinates of the parking space, and the method further includes: determining the target parking space based on the coordinates of the vehicle chassis frame and the coordinates of the parking space; determining the parking state of the vehicle based on the intersection over union ratio of the vehicle chassis frame and the detection frame of the target parking space.

9. The parking management method according to claim 8, wherein, the parking configuration data further includes the coordinates of the warning line, the parking states include parking on the line, parking across the parking space, and normal parking, and the step of determining the parking state of the vehicle based on the intersection over union ratio of the vehicle chassis frame and the detection frame of the target parking space includes: When the intersection-to-joint ratio is greater than a third preset value and less than a fourth preset value, and the vehicle chassis frame and the warning line intersect, determining that the parking state is line-pressing parking; When the intersection-to-joint ratio is greater than the third preset value and less than the fourth preset value, and the vehicle chassis frame intersects with the upper edge and / or the lower edge of the berth, determining that the parking state is the cross-berth parking; When the intersection-to-merger ratio is greater than the fourth preset value, the parking state is determined to be the normal parking state.

10. The parking management method according to claim 1, It is characterized in that The parking configuration data further includes coordinates of a warning area. The step of determining the driving state of the vehicle relative to the target parking space based on the vehicle chassis position information and the parking configuration data includes: Acquire an image in which the center point of the vehicle chassis frame falls within the warning area to obtain a first captured image; Acquire an image in which the center point of the chassis frame of the vehicle exceeds the warning line to obtain a second captured image; Acquire an image in which an intersection-and-joint ratio of the vehicle chassis frame and the detection frame of the berth is greater than a fifth preset value, a center point of the vehicle chassis frame falls within the berth, and the motion state of the vehicle is a stationary state, to obtain a third captured image; Acquire an image in which the center point of the vehicle chassis frame falls outside the warning area to obtain a fourth captured image; Acquire an image in which the intersection-and-joint ratio of the vehicle chassis frame and the detection frame of the berth is greater than the fifth preset value and the center point of the vehicle chassis frame falls within the berth, to obtain a fifth captured image; Based on the capture time of the first captured image, the second captured image, the third captured image, the fourth captured image or the fifth captured image, the driving state is determined, and reporting information is generated, the reporting information including the driving state.

11. The parking management method according to claim 10, It is characterized in that The driving state includes a first state in which the vehicle enters the target berth and / or a second state in which the vehicle exits the target berth. The step of determining the driving state based on the capture time of the first captured image, the second captured image, the third captured image, the fourth captured image, or the fifth captured image includes: When it is determined that a first preset condition is currently satisfied, determining that the driving state is the first state, the first preset condition including that the capture time of the second snapshot image is greater than the capture time of the first snapshot image and less than the capture time of the third snapshot image; and / or When it is determined that the second preset condition is currently met, the driving state is determined to be the second state, and the second preset condition includes that the capture time of the second snapshot image is greater than the capture time of the fifth snapshot image and less than the capture time of the fourth snapshot image.

12. The parking management method according to claim 11, It is characterized in that The method further comprises: determining whether the vehicle corresponding to the first state and the vehicle corresponding to the second state are the same vehicle; If so, the first state is associated with the second state.

13. A parking management device, It is characterized in that it includes a memory and a processor which are connected to each other, wherein the memory is used for storing a computer program, and when the computer program is executed by the processor, it is used to implement the parking management method described in any one of claims 1-12.

14. A computer-readable storage medium for storing a computer program It is characterized in that when the computer program is executed by a processor, it is used to implement the parking management method described in any one of claims 1-12.

Images