Parking space occupancy state detection method and device and electronic equipment

Abstract

The invention belongs to the field of image processing, and relates to a parking space occupancy state detection method and device and electronic equipment, and the method comprises the steps: obtaining a plurality of AVM sample images in different parking scenes; determining and marking a plurality of first-class key points of each visible parking space in each AVM sample image, and determining and standardizing a plurality of second-class key points corresponding to each visible vehicle to obtain an image sample set; training the deep learning model to obtain a key point detection model; inputting the real-time AVM image of the vehicle to be parked into the key point detection model to obtain a key point detection result; and judging and detecting the occupancy state of the visible parking space according to whether the second type of key points in the key point detection result fall in the parking space visible area formed by the first type of key points, and obtaining a parking space occupancy state detection result. Whether the parking space is occupied or not is judged by judging whether the second type of key points fall in the parking space visible area formed by the first type of key points or not, and the accuracy and reliability of parking space occupation state detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, device and electronic equipment for detecting parking space occupancy status. Background Art

[0002] With the rapid development of intelligent automotive technology in recent years, automatic parking has become increasingly popular among drivers. These systems help drivers park their vehicles more conveniently and safely, significantly enhancing the driving experience. However, accurate detection of parking space occupancy is a key prerequisite for the effective operation of these systems.

[0003] In related technologies, automatic parking typically relies on images captured by a surround-view fisheye camera to detect whether a parking space is occupied. Deep learning models are typically used to detect the location of parking spaces from an AVM (Around View Monitor) image, which is a composite of images captured by multiple surround-view fisheye cameras. Traditional image processing algorithms, such as binarization, are then used to determine whether the parking space is free. Alternatively, deep learning models can be used to directly detect whether the parking space is occupied within the AVM image. In practice, the first approach, which uses simple image processing such as binarization, is simply not suitable for images captured and processed in different scenarios. For example, varying light levels can easily invalidate previously tuned threshold parameters. The second type of method has design limitations such as improper labeling of sample data used to train deep learning models (for example, labeling visually obstructed parking spaces as vacant or non-vacant is entirely based on the judgment of the labeler, and there is no reliable judgment rule that conforms to the actual occupancy of the parking spaces) or the use of only one task head to detect multiple categories of objects in the deep learning model (for example, one detection head is used to detect the front, rear, or side of a car at the same time, and different parts of the car are detected as cars. In fact, the visual features of different parts of these cars are quite different, making it difficult for the model to learn common features). As a result, the trained deep learning model has large differences in accuracy when detecting different objects, and is prone to not detecting or misidentifying objects with a small number of samples, resulting in low accuracy and reliability of parking space occupancy status detection results, which is difficult to meet actual detection needs. Summary of the Invention

[0004] The present invention provides a parking space occupancy status detection method, device and electronic equipment, which are used to solve the defects of low detection accuracy and reliability of traditional parking space occupancy status detection solutions.

[0005] In one aspect, the present invention provides a method for detecting parking space occupancy status, comprising: Obtain multiple AVM sample images under different parking scenarios; Determine and label a plurality of first-category key points corresponding to each visible parking space in each AVM sample image, and determine and label a plurality of second-category key points corresponding to each visible vehicle in each AVM sample image based on the visible part of each visible vehicle in each AVM sample image, to obtain an image sample set; Training a pre-built deep learning model using the image sample set to obtain a key point detection model; Inputting the real-time AVM image of the vehicle to be parked into the key point detection model to obtain the key point detection results output by the key point detection model; Based on whether the second type of key points in the key point detection result fall within the parking space visible area composed of the first type of key points, the occupancy status of the visible parking spaces in the real-time AVM image is discriminated and detected to obtain a parking space occupancy status detection result.

[0006] According to the parking space occupancy status detection method provided by the present invention, based on the visible parts of each visible vehicle in each AVM sample image, a plurality of second-category key points corresponding to each visible vehicle in each AVM sample image are determined, including: For any visible vehicle in the AVM sample image, if the visible part includes two side wheels, the second type of key points are the wheel contact points corresponding to the two side wheels; If the visible part includes the front of the vehicle and at most one wheel on the side, the second type of key point is the visual grounding point of the front of the vehicle; If the visible part includes the rear of the vehicle and at most one wheel on the side, the second type of key point is the rear visual grounding point corresponding to the rear of the vehicle.

[0007] According to the parking space occupancy status detection method provided by the present invention, the deep learning model includes: a feature processing network and multiple key point detection heads; The feature processing network is used to perform feature processing on the input image to obtain feature enhanced data; The multiple key point detection heads are used to detect different types of key points in the input image based on the feature enhancement data, and at least some of the multiple key point detection heads detect different numbers of key points.

[0008] According to the parking space occupancy status detection method provided by the present invention, the multiple key point detection heads include: A first key point detection head, configured to detect a plurality of first-category key points of visible parking spaces in an input image; The second key point detection head is used to detect the front of the vehicle in the input image and the corresponding front visual grounding point of the vehicle; The third key point detection head is used to detect the rear end of the vehicle in the input image and the corresponding rear visual ground point of the vehicle; The fourth key point detection head is used to detect two wheels on the side of the vehicle visible in the input image and the wheel contact points corresponding to the two wheels.

[0009] According to the parking space occupancy status detection method provided by the present invention, the feature processing network includes: The backbone network is used to extract features from the input image and obtain initial feature data; The neck network is used to perform enhancement processing on the initial feature data to obtain feature-enhanced data.

[0010] According to the parking space occupancy status detection method provided by the present invention, in the key point detection results, the first type of key points include visible parking space key points, and the second type of key points include key points of visible parts of the target; Based on whether the second type of key points in the key point detection result fall within the parking space visible area formed by the first type of key points, the occupancy status of the visible parking spaces in the real-time AVM image is discriminated and detected to obtain a parking space occupancy status detection result, including: Determining a parking space visible area of each visible parking space in the real-time AVM image based on the visible parking space key points; For any target visible parking space, if all target visible parts do not overlap with the parking space visible area of the target visible parking space, the parking space occupancy status detection result is that the target visible parking space is in an idle state; If at least one target visible part overlaps with the parking space visible area of the target visible parking space, and the key point of the visible part is within the parking space visible area of the target visible parking space, the parking space occupancy status detection result is that the target visible parking space is occupied.

[0011] If at least one target visible part overlaps with the parking space visible area of the target visible parking space, and all key points of the target visible parts are not within the parking space visible area of the target visible parking space, the parking space occupancy status detection result is that the target visible parking space is in an idle state.

[0012] According to the parking space occupancy status detection method provided by the present invention, each group of first-category key points includes four visible parking space key points.

[0013] According to the parking space occupancy status detection method provided by the present invention, determining the parking space visible area of each visible parking space in the real-time AVM image based on the visible parking space key points includes: Connect the four visible parking space key points corresponding to each visible parking space in sequence to obtain a quadrilateral with the four visible parking space key points as vertices; The quadrilateral is used as the parking space visible area of each visible parking space in the real-time AVM image.

[0014] On the other hand, the present invention also provides a parking space occupancy status detection device, comprising: An acquisition module, used to acquire multiple AVM sample images under different parking scenarios; a labeling module for determining and labeling a plurality of first-category key points corresponding to each visible parking space in each AVM sample image, and determining and labeling a plurality of second-category key points corresponding to each visible vehicle in each AVM sample image based on the visible part of each visible vehicle in each AVM sample image, to obtain an image sample set; A training module, configured to train a pre-built deep learning model using the image sample set to obtain a key point detection model; An input module, configured to input a real-time AVM image of a vehicle to be parked into the key point detection model, and obtain a key point detection result output by the key point detection model; The detection module is used to judge and detect the occupancy status of the visible parking spaces in the real-time AVM image based on whether the second type of key points in the key point detection results fall within the visible parking space area composed of the first type of key points, and obtain a parking space occupancy status detection result.

[0015] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the parking space occupancy status detection method as described above is implemented.

[0016] The parking space occupancy status detection method, device and electronic device provided by the present invention obtain multiple AVM sample images in different parking scenarios, determine and mark multiple first-category key points corresponding to each visible parking space in each AVM sample image, and determine and mark multiple second-category key points corresponding to each visible vehicle in each AVM sample image based on the visible parts of each visible vehicle in each AVM sample image to obtain an image sample set; a pre-constructed deep learning model is trained with the image sample set to obtain a key point detection model; subsequently, the real-time AVM image of the vehicle to be parked is input into the key point detection model to obtain a key point detection result; finally, based on whether the second-category key points in the key point detection result fall within the visible area of the parking space composed of the first-category key points, the occupancy status of the visible parking spaces in the real-time AVM image is discriminated and detected to obtain a parking space occupancy status detection result. Due to the sample labeling process, multiple first-category key points corresponding to visible parking spaces and multiple second-category key points corresponding to visible vehicles are used for data labeling, and whether the parking space is occupied is judged by whether the second-category key points fall within the visible area of the parking space composed of the first-category key points, thereby improving the accuracy and reliability of the parking space occupancy status detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 1 is a flow chart of a method for detecting parking space occupancy status provided by an embodiment of the present invention; Figure 2 It is a schematic diagram of parking space structures of various shapes; Figure 3 It is a schematic diagram of the distribution of key points of parking spaces; Figure 4 It is a schematic diagram of the position of the visible area of the parking space and the area where the front of the vehicle is located when the front of the vehicle is visible; Figure 5 2 is a schematic structural diagram of a parking space occupancy status detection device provided by an embodiment of the present invention; Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0020] The following combination Figures 1 to 6 The following describes the detailed scheme of the parking space occupancy status detection method, device and electronic equipment provided by the embodiments of the present invention.

[0021] Figure 1 It is a flow chart of a parking space occupancy status detection method provided by an embodiment of the present invention.

[0022] like Figure 1 As shown, the parking space occupancy status detection method provided by the embodiment of the present invention can be executed by an electronic device with data transmission, reception and data processing capabilities. The above method mainly includes the following steps: Step 110: Acquire multiple AVM sample images under different parking scenarios.

[0023] AVM, short for Around View Monitor, refers to a panoramic monitoring system. This system captures images using multiple ultra-wide-angle fisheye cameras installed around the vehicle. These images are then processed through distortion correction and stitching to create a panoramic image of the vehicle's surroundings. In this embodiment, the AVM sample images cover a variety of parking scenarios, including parking spaces of varying shapes and conditions. Figure 2 Several common parking spaces are shown as examples.

[0024] It should be noted that the AVM sample image is obtained by dedistorting and stitching multiple original images taken by multiple fisheye cameras on both sides of the vehicle.

[0025] Step 120: Determine and label multiple first-category key points corresponding to each visible parking space in each AVM sample image, and determine and label multiple second-category key points corresponding to each visible vehicle in each AVM sample image based on the visible parts of each visible vehicle in each AVM sample image to obtain an image sample set.

[0026] In this embodiment, when determining multiple first-category key points corresponding to visible parking spaces, only the two visible corner points closest to the vehicle in the parking space can be identified and labeled. Then, depending on the situation, the remaining two points can be identified and labeled. Specifically, when a vehicle is parked in the space, the two intersection points between the vehicle and the parking space line are labeled. When the space is vacant, the remaining two visible corner points are labeled. This yields multiple first-category key points corresponding to each visible parking space. In practical applications, the number of first-category key points can be four or more.

[0027] It should be noted that the multiple second-category key points corresponding to the visible vehicle need to be determined specifically based on the visible parts of the visible vehicle. Specifically, they can be the grounding points of the two side wheels, or the two visual grounding points of the front of the vehicle, or the two visual grounding points of the rear of the vehicle.

[0028] In practical applications, taking the marking of four first-class key points as an example, the four first-class key points corresponding to the visible parking spaces can be numbered in a counterclockwise direction. The two first-class key points corresponding to the edge of the parking space close to the vehicle are numbered 1 and 4, and the remaining two first-class key points are numbered 2 and 3, such as Figure 3 As shown, the status of the parking space on the left is the labeling status of the four first-category key points when the parking space is vacant, and the status of the four first-category key points on the right is the labeling status when there is a car in the parking space.

[0029] When marking the second type of key points, the wheels, rear end, or front end where the second type of key points are located can be marked with a rectangular frame, and the corresponding two key points can be marked to train the model to detect key points on different objects. It can be understood that by marking the wheels, rear end, or front end with a rectangular frame, the model's ability to detect different objects can be trained. In the actual detection process, the model first detects different objects and then detects the key points on each object. This task-decomposition, top-down design helps improve the model's ability to detect key points of different objects.

[0030] Step 140: Train the pre-built deep learning model using the image sample set to obtain a key point detection model.

[0031] After the data labeling is completed, a set of labeled image samples can be obtained, and then a key point detection model can be trained. It can be understood that the input data of the key point detection model is the input image, that is, the real-time AVM image, and the output data is the key point detection result containing key parts and key points.

[0032] Step 150: Input the real-time AVM image of the vehicle to be parked into the key point detection model to obtain the key point detection results output by the key point detection model.

[0033] It can be understood that the real-time AVM image is obtained by stitching images taken by fisheye cameras installed on the vehicle to be parked.

[0034] Step 160: Based on the key point detection results, the occupancy status of the visible parking spaces in the real-time AVM image is detected to obtain a parking space occupancy status detection result.

[0035] The solution provided by this embodiment has higher accuracy in key point detection results because the key point detection model used in the key point detection link is trained based on the image sample set generated by the improved data annotation scheme. Subsequently, whether the parking space is occupied is judged based on whether the second type of key points fall within the visible area of the parking space composed of the first type of key points. Compared with the conventional technical solution of directly annotating the parking space as vacant or occupied in the image data to train the model to detect the occupancy status of the parking space, this embodiment can avoid the false detection of the target parking space as occupied when a car in the adjacent parking space blocks the target parking space, and the detection result is more accurate and reliable.

[0036] In one embodiment, based on the visible parts of each visible vehicle in each AVM sample image, a plurality of second-category key points corresponding to each visible vehicle in each AVM sample image are determined, specifically including: For any visible vehicle in the AVM sample image, if the visible part includes two side wheels, the second type of key points are the wheel contact points corresponding to the two side wheels.

[0037] If the visible part includes the front of the vehicle and at most one wheel on the side, the second type of key point is the visual grounding point of the front of the vehicle corresponding to the front of the vehicle.

[0038] If the visible part includes the rear of the vehicle and at most one wheel on the side, the second type of key point is the rear visual grounding point corresponding to the rear of the vehicle.

[0039] It is understood that for any visible vehicle in the AVM sample image, if the visible portion includes both side wheels, that is, both side wheels of the visible vehicle are visible, then the two side wheels and their corresponding wheel contact points can be annotated. If the visible portion includes only one side wheel or does not include any side wheels, that is, only one of the two side wheels is visible or both are invisible, then the visible front of the vehicle and the front visual contact point can be annotated, or the visible rear of the vehicle and the rear visual contact point can be annotated.

[0040] It is not difficult to find that this embodiment adopts the key point labeling method, which can adapt to parking spaces of different shapes and has stronger generalization ability. As long as a certain number of AVM sample images of each shape of parking space are collected for labeling and model training, one model can stably detect parking spaces of various shapes. There is no need to spend a lot of energy to label details such as different types of line segments and line segment angles of parking spaces. The data labeling process is more convenient, efficient and accurate.

[0041] In one embodiment, the deep learning model specifically includes: a feature processing network and multiple key point detection heads.

[0042] The feature processing network is used to perform feature processing on the input image to obtain feature enhanced data.

[0043] The multiple key point detection heads are used to detect different types of key points in the input image based on the feature enhancement data, and at least some of the multiple key point detection heads detect different numbers of key points.

[0044] In a specific implementation, the multiple key point detection heads specifically include: A first key point detection head, configured to detect a plurality of first-category key points of visible parking spaces in an input image; The second key point detection head is used to detect the front of the vehicle in the input image and the corresponding front visual grounding point of the vehicle; The third key point detection head is used to detect the rear end of the vehicle in the input image and the corresponding rear visual ground point of the vehicle; The fourth key point detection head is used to detect two wheels on the side of the vehicle visible in the input image and the wheel contact points corresponding to the two wheels.

[0045] In practical applications, traditional deep learning models only support configuring a single keypoint detection task head to detect keypoints of multiple object categories, but the number of keypoints must remain consistent. However, in this embodiment, the parking space to be detected requires four keypoints, and the visual contact points of the wheels at the front, rear, or side of the vehicle require two keypoints. Therefore, traditional deep learning models cannot be directly applied to this solution.

[0046] At the same time, considering that a key point detection head detects multiple different categories of objects, there is a large difference in the detection accuracy of the trained model when detecting different categories of objects due to the different number of sample data of different categories of objects.

[0047] Accordingly, this embodiment modifies the network structure of the traditional deep learning model and configures four key point detection heads for the model, which are used to detect parking spaces, the front of the vehicle and the visual grounding point of the front of the vehicle, the rear of the vehicle and the visual grounding point of the rear of the vehicle, and the two wheels on the side and the wheel grounding points. This solves the defect that the traditional deep learning model cannot configure multiple key point detection heads at the same time, and removes the constraint that the number of key points of different objects must be the same when detecting key points of different categories of objects. This allows the configuration of multiple key point detection heads, each of which can detect key points of different categories of objects and can detect different numbers of key points. In this way, compared with the traditional deep learning model, while improving the detection accuracy of each object and its key points, it removes the unreasonable restriction that the number of key points of different objects must be the same. The above modification improves the accuracy and flexibility of the key point detection link.

[0048] In a specific implementation, the feature processing network specifically includes: The backbone network is used to extract features from the input image and obtain initial feature data; The neck network is used to enhance the initial feature data to obtain feature enhanced data.

[0049] In practical applications, the backbone network is usually composed of convolutional neural networks, such as ResNet (Residual Network) and VGG (Visual Geometry Group Network). Its function is to extract features from the input image, convert the input image into initial feature data, and extract basic features such as edges and textures from the input image.

[0050] The neck network is mainly used to receive the initial feature data output by the backbone network and further fuse and enhance these features, such as using structures such as feature pyramid networks to improve the expressiveness of features and obtain feature enhanced data.

[0051] This embodiment uses the marking and identification of the two wheels at the front, rear or side of the vehicle and their corresponding grounding points, which can relatively stably and accurately detect the visible parts of the vehicle appearing in the fisheye camera and its grounding points in the real-time AVM image.

[0052] First, labeling and identifying the front, rear, and side wheels as different categories of objects is more accurate and stable than labeling and detecting the vehicle as a whole. Second, after subdividing the front, rear, and side wheels for target detection, the deep learning model can better learn the common features of each category, and has better detection capabilities and higher accuracy when detecting their corresponding grounding points, thereby improving the detection accuracy of the key point detection model.

[0053] In one embodiment, in the key point detection results, the first type of key points includes key points of visible parking spaces, and the second type of key points includes key points of visible parts of the target.

[0054] Based on whether the second-category key points in the key point detection results fall within the visible parking space area formed by the first-category key points, the occupancy status of the visible parking spaces in the real-time AVM image is discriminated and detected, and the parking space occupancy status detection results are obtained, specifically including: In the first step, the parking space visibility area of each visible parking space in the real-time AVM image is determined based on the key points of the visible parking space.

[0055] In this embodiment, each group of first-category key points includes four visible parking space key points.

[0056] In a specific implementation, determining the parking space visible area of each visible parking space in the real-time AVM image based on the visible parking space key points specifically includes: The four visible parking space key points corresponding to each visible parking space are connected in sequence to obtain a quadrilateral with the four visible parking space key points as vertices, and the quadrilateral is used as the parking space visible area of each visible parking space in the real-time AVM image.

[0057] Figure 4 The example shows a situation where the front of the vehicle is visible, and shows a quadrilateral parking space visible area formed by four connecting lines corresponding to four visible parking space key points P1, P2, P3, and P4 of the visible parking space in the real-time AVM image.

[0058] In some embodiments, there may be more than four visible parking space key points. Accordingly, the parking space visibility area is a polygon composed of multiple parking space visible key points, and the number of sides of the polygon corresponds to the number of parking space key points in the workshop.

[0059] In one case, for any target visible parking space, if all target visible parts do not overlap with the parking space visible area of the target visible parking space, the parking space occupancy status detection result is that the target visible parking space is in an idle state.

[0060] In another case, if at least one target visible part overlaps with the parking space visible area of the target visible parking space, and the key point of the visible part is within the parking space visible area of the target visible parking space, the parking space occupancy status detection result is that the target visible parking space is occupied.

[0061] In another case, if at least one target visible part overlaps with the parking space visible area of the target visible parking space, and all key points of the target visible part are not within the parking space visible area of the target visible parking space, the parking space occupancy status detection result is that the target visible parking space is in an idle state.

[0062] like Figure 4 As shown, taking the target visible part as the front of the vehicle as an example, the rectangular box is the area where the front of the vehicle is located, which can indicate the position of the front of the vehicle. The four connecting lines corresponding to the four key points P1, P2, P3, and P4 enclose a trapezoidal parking space visible area 210. At this time, the target visible part shown in the rectangular box overlaps with the parking space visible area 210, and the two vehicle head visual grounding points P5 and P6 are both in the parking space visible area 210, indicating that the current parking space is occupied.

[0063] Considering that the distortion and stretching in the AVM images synthesized by fisheye cameras often cause a car to block several adjacent parking spaces, creating the illusion that all adjacent parking spaces are detected as occupied, the solution of using deep learning models to detect whether parking spaces are vacant after training the deep learning model with data labeled with two categories: empty parking spaces and non-empty parking spaces basically cannot solve the above problem. This is because when a parking space is partially blocked by a car in a neighboring parking space, the image features are relatively close to those when there is actually a car parked in the parking space. The model has difficulty distinguishing the feature differences between the two, and the two categories are easily confused during detection. The solution of using traditional binary image processing plus pixel value to determine whether a parking space is vacant has a higher probability of false detection.

[0064] To this end, this embodiment determines whether a visible parking space is actually occupied by a vehicle by judging whether the grounding point of the front, rear, or wheels of the vehicle is within the quadrilateral of the visible part of the parking space. Since the positions of the grounding points of the front, rear, and wheels are detected more accurately, the status of the vehicle occupying the parking space can be accurately detected, and the occupancy status of adjacent parking spaces that are partially blocked by the vehicle can be accurately determined.

[0065] In some embodiments, when a vacant parking space is detected in the parking space occupancy detection results, the vacant parking space can be used as the target parking space for further detection of any obvious foreign objects other than the vehicle within the visible area of the target parking space, such as a ground lock blocking the vehicle, a large rock, or a trash can. Because the visible area of the parking space in this embodiment is formed by a quadrilateral formed by four key points of the visible parking space, the visible range of the parking space can be precisely located from the perspective of the parked vehicle. Further foreign object detection can be performed within this visible range, thereby narrowing the image detection range, reducing the detection computation load, and improving parking safety.

[0066] In practical applications, an algorithm based on analyzing image pixel values can be used to identify foreign objects that may occupy parking spaces. For example, the pixel mean and variance of all pixels in the visible area of the target parking space can be compared with the pixel mean and variance of the corresponding area in the image of the parking space in a normal, idle state. If the difference between the pixel mean and variance in the visible area of the target parking space and the pixel mean and variance of the vacant parking space is greater than a preset difference threshold, it indicates that the color distribution of part of the visible area of the target parking space is abnormal and significantly different from the color distribution of the image of the parking space in a normal, idle state. In this case, the target parking space can be defined as an abnormal parking space, and a prompt message can be issued to prompt the driver to further check the target parking space for foreign objects other than cars.

[0067] To sum up, the parking space occupancy detection method provided in this embodiment can simultaneously detect the key points of the visible parts of each parking space in the AVM measured image and the rear, front or side wheels of each vehicle and their corresponding grounding points through a key point detection model that can detect multiple targets and their respective key points. Then, an algorithm for determining whether the parking space is occupied determines whether each parking space is occupied. The entire execution process is more efficient and faster, the detection results are more accurate than those of the usual detection methods, and the detection efficiency is effectively improved.

[0068] Based on the same inventive concept, the present invention also protects a parking space occupancy status detection device. The parking space occupancy status detection device provided by the present invention is described below. The parking space occupancy status detection device described below and the parking space occupancy status detection method described above can be referenced to each other.

[0069] like Figure 5 As shown, the parking space occupancy status detection device provided by the embodiment of the present invention specifically includes: The acquisition module 310 is used to acquire multiple AVM sample images in different parking scenarios.

[0070] The labeling module 320 is used to determine and label multiple first-category key points corresponding to each visible parking space in each AVM sample image, and determine and label multiple second-category key points corresponding to each visible vehicle in each AVM sample image based on the visible parts of each visible vehicle in each AVM sample image, to obtain an image sample set.

[0071] The training module 330 is used to train the pre-built deep learning model through the image sample set to obtain a key point detection model.

[0072] The input module 340 is used to input the real-time AVM image of the vehicle to be parked into the key point detection model to obtain the key point detection results output by the key point detection model.

[0073] The detection module 350 is used to determine the occupancy status of the visible parking spaces in the real-time AVM image based on whether the second type of key points in the key point detection results fall within the visible parking space area composed of the first type of key points, and obtain the parking space occupancy status detection result.

[0074] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0075] Figure 6 It is a structural diagram of an electronic device provided by an embodiment of the present invention.

[0076] like Figure 6As shown, the electronic device may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logic instructions in the memory 430 to execute the parking space occupancy status detection method provided by the above-mentioned embodiments, including: obtaining multiple AVM sample images under different parking scenarios; determining and labeling multiple first-category key points corresponding to each visible parking space in each AVM sample image, and determining and labeling multiple second-category key points corresponding to each visible vehicle in each AVM sample image based on the visible parts of each visible vehicle in each AVM sample image to obtain an image sample set; training a pre-built deep learning model through the image sample set to obtain a key point detection model; inputting the real-time AVM image of the vehicle to be parked into the key point detection model to obtain the key point detection results output by the key point detection model; based on whether the second-category key points in the key point detection results fall in the visible area of the parking space composed of the first-category key points, the occupancy status of the visible parking spaces in the real-time AVM image is judged and detected to obtain a parking space occupancy status detection result.

[0077] Furthermore, the logic instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0078] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the parking space occupancy status detection method provided by the above-mentioned embodiments, including: obtaining multiple AVM sample images under different parking scenarios; determining and labeling multiple first-class key points corresponding to each visible parking space in each AVM sample image, and determining and labeling multiple second-class key points corresponding to each visible vehicle in each AVM sample image based on the visible parts of each visible vehicle in each AVM sample image to obtain an image sample set; training a pre-constructed deep learning model through the image sample set to obtain a key point detection model; inputting the real-time AVM image of the vehicle to be parked into the key point detection model to obtain the key point detection result output by the key point detection model; based on whether the second-class key points in the key point detection result fall in the visible area of the parking space composed of the first-class key points, the occupancy status of the visible parking space in the real-time AVM image is judged and detected to obtain a parking space occupancy status detection result.

[0079] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the parking space occupancy status detection method provided in the above-mentioned embodiments, including: obtaining multiple AVM sample images under different parking scenarios; determining and labeling multiple first-category key points corresponding to each visible parking space in each AVM sample image, and determining and labeling multiple second-category key points corresponding to each visible vehicle in each AVM sample image based on the visible parts of each visible vehicle in each AVM sample image, to obtain an image sample set; training a pre-constructed deep learning model through the image sample set to obtain a key point detection model; inputting the real-time AVM image of the vehicle to be parked into the key point detection model to obtain a key point detection result output by the key point detection model; based on whether the second-category key points in the key point detection result fall in the visible area of the parking space composed of the first-category key points, the occupancy status of the visible parking space in the real-time AVM image is discriminated and detected to obtain a parking space occupancy status detection result.

[0080] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0081] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A parking space occupancy status detection method, characterized in that: include: Obtain multiple AVM sample images under different parking scenarios; Determine and label a plurality of first-category key points corresponding to each visible parking space in each AVM sample image, and determine and label a plurality of second-category key points corresponding to each visible vehicle in each AVM sample image based on the visible part of each visible vehicle in each AVM sample image, to obtain an image sample set; Training a pre-built deep learning model using the image sample set to obtain a key point detection model; Inputting the real-time AVM image of the vehicle to be parked into the key point detection model to obtain the key point detection results output by the key point detection model; Based on whether the second type of key points in the key point detection result fall within the parking space visible area composed of the first type of key points, the occupancy status of the visible parking spaces in the real-time AVM image is discriminated and detected to obtain a parking space occupancy status detection result.

2. The parking space occupancy status detection method according to claim 1, characterized in that: According to the visible parts of each visible vehicle in each AVM sample image, multiple second-category key points corresponding to each visible vehicle in each AVM sample image are determined, including: For any visible vehicle in the AVM sample image, if the visible part includes two side wheels, the second type of key points are the wheel contact points corresponding to the two side wheels; If the visible part includes the front of the vehicle and at most one wheel on the side, the second type of key point is the visual grounding point of the front of the vehicle; If the visible part includes the rear of the vehicle and at most one wheel on the side, the second type of key point is the rear visual grounding point corresponding to the rear of the vehicle.

3. The parking space occupancy status detection method according to claim 2, characterized in that: The deep learning model includes: a feature processing network and multiple key point detection heads; The feature processing network is used to perform feature processing on the input image to obtain feature enhanced data; The multiple key point detection heads are used to detect different types of key points in the input image based on the feature enhancement data, and at least some of the multiple key point detection heads detect different numbers of key points.

4. The parking space occupancy status detection method according to claim 3, characterized in that: The multiple key point detection heads include: A first key point detection head, configured to detect a plurality of first-category key points of visible parking spaces in an input image; The second key point detection head is used to detect the front of the vehicle in the input image and the corresponding front visual grounding point of the vehicle; The third key point detection head is used to detect the rear end of the vehicle in the input image and the corresponding rear visual ground point of the vehicle; The fourth key point detection head is used to detect two wheels on the side of the vehicle visible in the input image and the wheel contact points corresponding to the two wheels.

5. The parking space occupancy status detection method according to claim 3, characterized in that: The feature processing network includes: The backbone network is used to extract features from the input image and obtain initial feature data; The neck network is used to perform enhancement processing on the initial feature data to obtain feature-enhanced data.

6. The parking space occupancy status detection method according to claim 1, characterized in that: In the key point detection results, the first type of key points includes visible parking space key points, and the second type of key points includes key points of visible parts of the target; Based on whether the second type of key points in the key point detection result fall within the parking space visible area formed by the first type of key points, the occupancy status of the visible parking spaces in the real-time AVM image is discriminated and detected to obtain a parking space occupancy status detection result, including: Determining a parking space visible area of each visible parking space in the real-time AVM image based on the visible parking space key points; For any target visible parking space, if all target visible parts do not overlap with the parking space visible area of the target visible parking space, the parking space occupancy status detection result is that the target visible parking space is in an idle state; If at least one target visible part overlaps with the parking space visible area of the target visible parking space, and the key point of the target visible part is within the parking space visible area of the target visible parking space, then the parking space occupancy state detection result is that the target visible parking space is occupied; If at least one target visible part overlaps with the parking space visible area of the target visible parking space, and all key points of the target visible parts are not within the parking space visible area of the target visible parking space, the parking space occupancy status detection result is that the target visible parking space is in an idle state.

7. The parking space occupancy status detection method according to claim 6, characterized in that: Each group of first-category keypoints contains four visible parking space keypoints.

8. The parking space occupancy status detection method according to claim 7, characterized in that: Determining a parking space visible area of each visible parking space in the real-time AVM image based on the visible parking space key points includes: Connect the four visible parking space key points corresponding to each visible parking space in sequence to obtain a quadrilateral with the four visible parking space key points as vertices; The quadrilateral is used as the parking space visible area of each visible parking space in the real-time AVM image.

9. A parking space occupancy status detection device, characterized in that: include: An acquisition module, used to acquire multiple AVM sample images under different parking scenarios; a labeling module for determining and labeling a plurality of first-category key points corresponding to each visible parking space in each AVM sample image, and determining and labeling a plurality of second-category key points corresponding to each visible vehicle in each AVM sample image based on the visible part of each visible vehicle in each AVM sample image, to obtain an image sample set; A training module, configured to train a pre-built deep learning model using the image sample set to obtain a key point detection model; An input module, configured to input a real-time AVM image of a vehicle to be parked into the key point detection model, and obtain a key point detection result output by the key point detection model; The detection module is used to judge and detect the occupancy status of the visible parking spaces in the real-time AVM image based on whether the second type of key points in the key point detection results fall within the visible parking space area composed of the first type of key points, and obtain a parking space occupancy status detection result.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the parking space occupancy status detection method according to any one of claims 1 to 8 is implemented.

Images