Parking space occupation detection method, device and system, electronic equipment and storage medium

Through the parking space occupation detection model of the deep separable convolutional network, the parking space occupation detection in multi-camera scenarios are processed, and the inclination angle prediction box is generated, which solves the detection inaccuracy caused by rotating square boxes and improves the detection accuracy and speed.

CN119068453AInactive Publication Date: 2024-12-03CHINA TELECOM CORP LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411010018.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, when the rotating square frame is used for parking space and vehicle detection, the detection results of parking space occupation are inaccurate, especially in multi-camera scenarios, the top shot angle is not standard, resulting in poor detection results.

Method used

A parking space occupation detection model based on a depth separable convolutional network is adopted. Through feature classification and regression branch processing, a parallelogram prediction box with inclination angle is generated, and the detection accuracy is improved through interleaving and matching and non-maximum value suppression.

Benefits of technology

It improves the accuracy of parking space occupation detection, adapts to the inference speed requirements of multi-camera images, and outputs bounding boxes that meet the top shot scene, enhancing the reliability of the detection results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068453A_ABST
    Figure CN119068453A_ABST
Patent Text Reader

Abstract

The invention provides a parking space occupation detection method, device and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining a detection image of each camera in a parking lot, and the detection image comprises a camera identifier and a position mark number of each parking space under each camera; performing prediction processing on the detection image by using a parking space occupation detection model to obtain an occupation condition of each parking space under each camera and a prediction frame of the vehicle, the parking space occupation detection model being a detection model based on a depth separable convolutional network; and displaying the occupation condition of the parking space under each camera in the parking lot. Namely, in the embodiment of the invention, the parking space occupation detection model based on the depth separable convolutional network is utilized to predict the parking space occupation condition, the structure is simple, and the accuracy of the parking space occupation condition is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a parking space occupancy detection method, device, system, electronic equipment and computer-readable storage medium. Background Art

[0002] With the acceleration of urbanization, parking difficulties have become a common problem faced by major cities. Traditional parking methods can no longer meet the growing parking demand, so smart parking projects have emerged. Smart parking projects are based on advanced information technology and intelligent equipment, aiming to improve the efficiency and convenience of parking spaces.

[0003] In the related technology, in the parking space occupancy detection scenario, a parking lot often requires coverage by multiple lenses (generally 4-8 cameras), which places high demands on the model detection speed. In addition, there may be overlaps in the monitored parking spaces in these images, and sometimes the same parking space may have different detection results under different camera images. Different detection results need to be processed separately, and then merged after processing. The camera angle will affect the detection effect. In current target detection, ROT (rotating target detection model) is generally used for overhead shots. Compared with conventional target detection, it has a rotation angle and can output a square prediction box with a rotation angle. However, since there is no standard overhead shot in real scenes, there are more overhead images with tilted angles. In this scenario, the parking space and vehicle are detected by rotating the square box, resulting in inaccurate parking space occupancy detection results.

[0004] Therefore, how to improve the accuracy of parking space occupancy detection is a technical problem that needs to be solved. Summary of the invention

[0005] The present invention provides a parking space occupancy detection method, device, system, electronic device and computer-readable storage medium to at least solve the problem in the related art that the parking space and vehicle are detected by rotating a square frame, resulting in inaccurate parking space occupancy detection results. The technical solution of the present invention is as follows:

[0006] According to a first aspect of an embodiment of the present invention, a parking space occupancy detection method is provided, comprising:

[0007] Acquire detection images of each camera in the parking lot, wherein the detection images include camera identifiers and location labels of each parking space under each camera;

[0008] The detection image is predicted and processed by using a parking space occupancy detection model to obtain the occupancy status of each parking space and the prediction frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on deep separable convolutional learning;

[0009] Display the occupancy status of parking spaces under each camera in the parking lot.

[0010] Optionally, the use of a parking space occupancy detection model to perform prediction processing on the detection image to obtain the occupancy status of each parking space and the predicted frame of the vehicle under each camera includes:

[0011] Using a parking space occupancy detection model to perform feature classification processing on the detection image, to obtain a classification branch for predicting vehicles and a regression branch for cutting angles;

[0012] Performing non-maximum suppression processing on the classification branch and the regression branch to obtain multiple prediction boxes with camera logos;

[0013] The multiple prediction frames are matched with the set parking space boundary frames under the corresponding cameras to predict the occupancy status and vehicle positions of each parking space under different cameras.

[0014] Optionally, matching the multiple prediction frames with the set parking space boundary frames corresponding to each camera to predict the occupancy status and vehicle position of each parking space under different cameras includes:

[0015] Determine an intersection-over-union ratio between the multiple prediction boxes and the set parking space boundary boxes under each corresponding camera;

[0016] Based on the intersection-and-union ratio, the occupancy status of each parking space and the vehicle position under different cameras are predicted.

[0017] Optionally, determining the intersection-and-union ratios of the multiple prediction boxes and the parking space boundary boxes corresponding to the cameras includes:

[0018] Determine four linear functions between four coordinate points of each prediction box, and determine four linear functions between four coordinate points of a set parking space boundary box under the corresponding camera;

[0019] Based on the four coordinate points of each prediction box and the four coordinate points of the corresponding set parking space boundary box, respectively determine the coordinate points in the corresponding prediction box and the set parking space boundary box;

[0020] According to the relative positions of the coordinates of the points in the frame and the corresponding linear functions, the linear function inequalities of the upper, lower, left and right sides of the center point of the enclosed area between each prediction frame and the corresponding set parking space boundary frame are determined;

[0021] Determine the corresponding first clamped area equation and the second clamped area equation respectively based on the linear function inequality;

[0022] According to the discrete point set set in the image, respectively determine a first intersection of the discrete point set with the first enclosed area equation and a second intersection with the second enclosed area equation;

[0023] Determine the intersection of the first intersection and the second intersection;

[0024] The ratio of the number of points in the intersection to the number of duplicate points in the first intersection and the second intersection excluding the intersection is determined as the intersection-union ratio.

[0025] Optionally, the predicting, based on the intersection-over-union ratio, the occupancy status of each parking space and the vehicle position distributed under different cameras includes:

[0026] Obtaining the confidence of the prediction box;

[0027] Determine an average value of the intersection-over-union ratio and the confidence of the prediction box, and use the average value as the final intersection-over-union ratio;

[0028] Based on the final intersection-to-union ratio, the occupancy status of each parking space and the vehicle position under different cameras are predicted.

[0029] Optionally, the method further includes: pre-training the parking space occupancy detection model in the following manner:

[0030] Acquire a training image, wherein the training image includes: a plurality of anchor points tiled on the image, and anchor frames of different sizes and proportions stacked around each anchor point;

[0031] Inputting the training image into a mobile neural network to generate feature maps of various scales, wherein a feature point on the feature map corresponds one-to-one to each anchor point;

[0032] Calculate an intersection-over-union ratio based on a true bounding box on the training image and a corresponding anchor box;

[0033] A positive sample is determined based on the size of the intersection-over-union ratio, wherein the component of the positive sample responsible for regression is the conversion of the anchor box to its rotated parallelogram box format, and the conversion includes: long side, short side scaling, center offset, rotation angle and tilt angle;

[0034] Performing non-maximum suppression processing on the positive samples to obtain a detection result with a prediction box for each camera;

[0035] The detection result is matched with the real parking space boundary frame under the corresponding camera to obtain the occupancy status of each parking space under different cameras.

[0036] Optionally, the displaying of the occupancy status of parking spaces under each camera in the parking lot includes:

[0037] The GUI interface is used to display the occupancy status of parking spaces under each camera in the parking lot.

[0038] According to a second aspect of an embodiment of the present invention, there is provided a parking space occupancy detection device, comprising:

[0039] An acquisition module is used to acquire detection images of each camera in the parking lot, wherein the detection images include camera identifications and position labels of each parking space under each camera;

[0040] A prediction module, used to perform prediction processing on the detection image using a parking space occupancy detection model to obtain the occupancy status of each parking space and the prediction frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on a deep separable convolutional network;

[0041] The display module is used to display the occupancy status of parking spaces under each camera in the parking lot.

[0042] Optionally, the prediction module includes:

[0043] A classification processing module, used to perform feature classification processing on the detection image using a parking space occupancy detection model to obtain a classification branch for predicting vehicles and a regression branch for cutting angles;

[0044] Suppression processing is used to perform non-maximum suppression processing on the classification branch and the regression branch to obtain multiple prediction boxes with camera identification;

[0045] The matching module is used to match the multiple prediction frames with the set parking space boundary frames under the corresponding cameras, and predict the occupancy status and vehicle position of each parking space under different cameras.

[0046] Optionally, the matching module includes:

[0047] A determination module, used to determine an intersection-over-union ratio between the multiple prediction frames and a set parking space boundary frame corresponding to each camera;

[0048] The position prediction module is used to predict the occupancy status and vehicle position of each parking space under different cameras based on the intersection-to-union ratio.

[0049] Optionally, the determining module includes:

[0050] A function determination module, used to determine four linear functions between four coordinate points of each prediction box, and to determine four linear functions between four coordinate points of a set parking space boundary box under a corresponding camera;

[0051] A coordinate point determination module, used to determine the corresponding in-frame coordinate points in the prediction frame and the set parking space boundary frame respectively based on the four coordinate points of each prediction frame and the four coordinate points of the corresponding set parking space boundary frame;

[0052] A function inequality determination module, used to determine the linear function inequalities of the upper, lower, left and right four positions of the center point of the enclosed area between each prediction box and the corresponding set parking space boundary box according to the relative positions of the coordinate points in the box and the corresponding linear functions;

[0053] An equation determination module, used to determine the corresponding first clamped area equation and second clamped area equation respectively based on the linear function inequality;

[0054] A first intersection determination module, used for determining a first intersection of the discrete point set and the first enclosed area equation, and a second intersection of the discrete point set and the second enclosed area equation, respectively, according to the discrete point set set in the image;

[0055] A second intersection determination module, used to determine the intersection of the first intersection and the second intersection;

[0056] The intersection-and-union ratio determination module is used to determine the ratio of the number of points in the intersection to the first intersection and the second intersection excluding the number of repeated points in the intersection, as the intersection-and-union ratio.

[0057] Optionally, the location prediction module includes:

[0058] A confidence acquisition module, used to obtain the confidence of the prediction box;

[0059] An average value determination module, used to determine an average value of the intersection-over-union ratio and the confidence of the prediction box, and use the average value as the final intersection-over-union ratio;

[0060] The position prediction submodule is used to predict the occupancy status and vehicle position of each parking space under different cameras based on the final intersection-to-union ratio.

[0061] The device further comprises: a training module, configured to pre-train the parking space occupancy detection model in the following manner:

[0062] Acquire a training image, wherein the training image includes: a plurality of anchor points tiled on the image, and anchor frames of different sizes and proportions stacked around each anchor point;

[0063] A training module is used to input the training image into the parking space occupancy detection model for training, specifically including: the mobile neural network in the parking space occupancy detection model uses a deep separable convolutional network to generate feature maps of various scales, wherein the feature points on the feature map correspond one-to-one to each anchor point; based on the real bounding box on the training image and the corresponding anchor box, the intersection and union ratio is calculated; based on the size of the intersection and union ratio, a positive sample is determined, wherein the component responsible for regression of the positive sample is the conversion of the anchor box to its rotated parallelogram box format, and the conversion includes: long side, short side scaling, center offset, rotation angle and tilt angle; non-maximum suppression processing is performed on the positive sample to obtain a detection result with a prediction box for each camera; the detection result is matched with the real parking space bounding box under the corresponding camera to obtain the occupancy status of each parking space under different cameras.

[0064] Optionally, the display module specifically utilizes a graphical user interface (GUI) to display the occupancy status of parking spaces under each camera in the parking lot.

[0065] According to a third aspect of an embodiment of the present invention, a parking space occupancy detection system is provided, comprising: a sensing terminal, a computing server and a display, wherein the sensing terminal is connected to the computing server and the display respectively, wherein:

[0066] The sensing terminal is used to obtain detection images of each camera in the parking lot, wherein the detection images include camera identifiers and position labels of each parking space under each camera; and send the detection images to the computing server;

[0067] The computing server is used to perform prediction processing on the detection image using a parking space occupancy detection model to obtain the occupancy status of each parking space and the prediction frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on a deep separable convolutional network; and send the occupancy status of each parking space and the prediction frame of the vehicle under each camera to the display;

[0068] The display is used to display the occupancy status of the parking spaces under each camera in the parking lot when receiving the occupancy status of each parking space under each camera and the predicted frame of the vehicle.

[0069] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0070] processor;

[0071] a memory for storing instructions executable by the processor;

[0072] The processor is configured to execute the instructions to implement the parking space occupancy detection method as described above.

[0073] According to a fifth aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the parking space occupancy detection method as described above.

[0074] According to a sixth aspect of an embodiment of the present invention, there is provided a computer program product, comprising a computer program or instructions, which, when executed by a processor of an electronic device, implements the parking space occupancy detection method as described above.

[0075] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects:

[0076] In an embodiment of the present invention, a detection image of each camera in a parking lot is obtained, wherein the detection image includes a camera identifier and a position number of each parking space under each camera; a parking space occupancy detection model is used to perform predictive processing on the detection image to obtain the occupancy status of each parking space under each camera and the prediction frame of the vehicle, wherein the parking space occupancy detection model is a detection model based on deep learning; and the occupancy status of the parking spaces under each camera in the parking lot is displayed. That is to say, in an embodiment of the present invention, a parking space occupancy detection model based on a deep separable convolutional network is used to predict the parking space occupancy status, which has a simple structure and improves the accuracy of the parking space occupancy status. The parking space occupancy detection model is more adaptable to vehicle affine deformation through regression, can adapt to the inference speed requirements of multi-camera images, and outputs a bounding box that conforms to the overhead scene.

[0077] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention, and do not constitute an improper limitation of the present invention. In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0079] Figure 1 It is a flow chart of a parking space occupancy detection method provided by an embodiment of the present invention.

[0080] Figure 2 It is a schematic diagram of a parking space occupancy detection model provided by an embodiment of the present invention.

[0081] Figure 3 It is a schematic diagram of a parallelogram prediction box with an inclined angle provided by an embodiment of the present invention.

[0082] Figure 4A It is a schematic diagram of a deep convolution in MobileNet provided by an embodiment of the present invention.

[0083] Figure 4B It is a schematic diagram of residual convolution in a ResNet provided by an embodiment of the present invention.

[0084] Figure 5 It is a schematic diagram of calculating the intersection and union ratio of convex quadrilaterals provided by an embodiment of the present invention.

[0085] Figure 6 It is a block diagram of a parking space occupancy detection device provided by an embodiment of the present invention.

[0086] Figure 7 It is a block diagram of a prediction module provided by an embodiment of the present invention.

[0087] Figure 8 The present invention provides a block diagram of a parking space occupancy detection system.

[0088] Fig. 9 This is a diagram showing an application example of a parking space occupancy detection system provided by an embodiment of the present invention.

[0089] Fig.10 It is a block diagram of an electronic device provided by an embodiment of the present invention.

[0090] Fig.11 It is a block diagram of a device for parking space occupancy detection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0091] In order to enable ordinary persons in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0092] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0093] Figure 1 is a flow chart of a parking space occupancy detection method provided by an embodiment of the present invention, such as Figure 1 As shown, the parking space occupancy detection method includes the following steps:

[0094] Step 101: Acquire detection images of each camera in the parking lot, wherein the detection images include camera identifiers and position labels of each parking space under each camera.

[0095] Step 102: Use a parking space occupancy detection model to perform prediction processing on the detection image to obtain the occupancy status of each parking space and the predicted frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on a deep separable convolutional network.

[0096] Step 103: Display the occupancy status of parking spaces under each camera in the parking lot.

[0097] The parking space occupancy detection method described in the present invention can be applied to terminals, servers, etc., without limitation herein. The terminal implementation device can be electronic devices such as smart phones, laptops, tablet computers, desktop computers, personal digital assistants (PDAs) and wearable devices. The server can be an independent server or a server cluster, or a server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, intermediate services, domain name services, security services, content distribution networks, or a big data and artificial intelligence platform, etc., without limitation herein.

[0098] Combine the following Figure 1 , the specific implementation steps of a parking space occupancy detection method provided by an embodiment of the present invention are described in detail.

[0099] In step 101, detection images of various cameras in the parking lot are obtained, wherein the detection images include camera identifiers and position labels of various parking spaces under each camera.

[0100] In this step, the cameras in the parking lot are usually multiple elevated cameras, which cover the entire parking lot with a bird's-eye view as much as possible. Since the height of the deployment affects the field of view of the camera, the number of cameras required for monitoring the same parking lot using cameras at different heights is different. The present invention can adapt to different scenarios and allow N cameras to be set up at different heights. The monitoring areas of n (n<=N) cameras can also overlap. After the cameras are set up, try not to move them. And all the cameras in the parking lot are numbered, such as camera 1, camera 2, and so on.

[0101] In this embodiment, it is also necessary to set the position of the parking spaces captured by each camera image in the parking lot and the position number of the parking spaces. For example, the parking spaces captured by camera No. 1 are: parking space No. 1, parking space No. 2, parking space No. 3, ..., parking space No. 10; the parking spaces captured by camera No. 2 are: parking space No. 11, parking space No. 12, ..., parking space No. 20; the parking spaces captured by camera No. 3 are: parking space No. 21, parking space No. 22, ..., parking space No. 25, etc. Of course, individual numbered parking spaces may also be captured by different cameras at the same time, which is not limited in this embodiment.

[0102] In step 102, a parking space occupancy detection model is used to perform prediction processing on the detection image to obtain the occupancy status of each parking space and the predicted frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on a deep separable convolutional network.

[0103] Among them, in this step, the parking space occupancy detection model is used to perform feature classification processing on the detection image to obtain a classification branch for predicting the vehicle and a regression branch for the cutting angle; non-maximum suppression processing is performed on the classification branch and the regression branch to obtain multiple prediction frames with camera identification; the multiple prediction frames are matched with the set parking space boundary boxes under the corresponding cameras to predict the occupancy status and vehicle position of each parking space distributed under different cameras.

[0104] Among them, a parking space occupancy detection model provided in this embodiment is as follows Figure 2 As shown, Figure 2 A schematic diagram of a parking space occupancy detection model provided in an embodiment of the present invention.

[0105] like Figure 2 As shown, the detection images captured by N cameras (here the detection images are training images) are input into MobileNet. MobileNet decomposes the ordinary convolution into point-by-point convolution and depth convolution, that is, the original m*n calculation amount is changed to m+n, which accelerates the operation. This implementation takes three 1×1 convolutions as an example, which is not limited to this in practical applications. The classification branch and the regression branch are obtained, wherein the regression branch includes: center x offset, center y offset, width expansion, rotation angle and tilt angle, etc. Afterwards, the classification branch and the regression branch are subjected to non-large value suppression processing (this embodiment takes the convex quadrilateral lou calculation as an example) to obtain the detection result of each camera. Finally, the detection result is matched with the parking space to obtain the parking space occupancy situation.

[0106] The parking space occupancy detection model provided by the present invention is modified compared with the existing RetinaNet model. In the backbone part, a mobile neural network (MobileNet) is used to replace the deep residual network (ResNet) to improve the speed. In the neck part, the FPN structure is abandoned to simplify it, and only the features of the C4-C6 levels are used to predict the vehicle, which can achieve a faster speed. In addition, in the head part (i.e., the detection head of the model), in addition to the center offset width and height scaling, additional regression branches for rotation and bevel angles are added, so that it can generate a parallelogram prediction box with a tilt angle, such as Figure 3 As shown, Figure 3 A schematic diagram of a parallelogram prediction box with a tilt angle provided by an embodiment of the present invention. During training, the rotation angle and tilt angle are additionally provided to supervise the two components. The supervisory signal corresponding to the annotation of angle a is a / Π, Π=3.1415926. During reasoning, the angle prediction component × Π is the tilt angle or rotation angle.

[0107] Among them, MobileNet can also be understood as a lightweight mobile terminal neural network. That is to say, in this embodiment, the parking space occupancy detection model is modified compared to the existing RetinaNet model. The mobile neural network MobileNet is used in the backbone part to replace the deep residual network ResNet to improve the speed. Figure 4A Structure replacement in ResNet model Figure 4B Structure. Among them, Figure 4A It is a schematic diagram of a deep convolution in MobileNet provided by an embodiment of the present invention. Figure 4B It is a schematic diagram of residual convolution in a ResNet provided by an embodiment of the present invention.

[0108] in, Figure 4A The processing process is as follows: Mobilenet decomposes the ordinary convolution into point-by-point convolution and depth-wise convolution, and the original m*n calculation amount becomes m+n, which speeds up the operation. Among them, ReLU: is the activation function, and convbn is the ordinary convolution + normalization. Figure 4B The processing process of residual convolution is already well known in the art and will not be described in detail here.

[0109] It should be noted that the RetinaNet model consists of three parts. The first is to input the image to the backbone network to extract image features. The backbone network refers to various deep neural networks, such as VGG Net, GoogLeNet, ResNet, etc. The second is to identify the category branch of the target of interest in the image, that is, the classification task, which is based on the features output by the backbone; the third is to mark the location of the target of interest in the image with a rectangular box, that is, the regression task (localization), which is also based on the features output by the backbone. Generally, the classification and regression tasks are collectively referred to as the detection head of the model.

[0110] The model will generate anchor boxes of different sizes and proportions based on each pixel on the feature map output by the feature extraction network to cover the target on the entire original image. However, the target to be detected usually does not account for a high proportion of the entire image, and the generated anchor boxes do not frame the target to be detected. At this time, the intersection of union (IoU) will be calculated based on the generated anchor boxes and the pre-marked ground truth box, that is, the degree of overlap between the two. The larger the IoU value, the higher the degree of overlap. For training, we divide all anchor boxes into two categories, one is positive samples, whose IoU value with the real target is greater than 0.5; the other is negative samples, which do not overlap with any real target or whose IoU value with the real target is less than 0.3 (0.5, 0.3 are commonly used IoU thresholds, of course, they can also be adjusted to other values ​​according to specific applications, and this embodiment is not limited). Usually, the number of negative samples will far exceed the number of positive samples. Then, a certain proportion of the positive and negative samples are sampled and sent to the classification and regression networks to train the category information and position offset respectively, and the loss function is obtained. The picture is input during inference to finally get the prediction result.

[0111] Among them, matching the multiple prediction frames with the set parking space boundary frames under the corresponding cameras to predict the occupancy status and vehicle positions of the parking spaces distributed under different cameras includes: determining the intersection-and-union ratio of the multiple prediction frames and the set parking space boundary frames under the corresponding cameras; based on the intersection-and-union ratio, predicting the occupancy status and vehicle positions of the parking spaces distributed under different cameras.

[0112] Wherein, the determination of the intersection and union ratios of the multiple prediction frames and the corresponding parking space boundary frames of each camera includes: determining four linear functions between the four coordinate points of each prediction frame, and determining four linear functions between the four coordinate points of the parking space boundary frame under the corresponding camera; based on the four coordinate points of each prediction frame and the four coordinate points of the corresponding parking space boundary frame, respectively determine the coordinate points in the corresponding prediction frame and the setting parking space boundary frame; based on the linear function inequality, respectively determine the corresponding first enclosed area equation and the second enclosed area equation; according to the discrete point set set of the image, respectively determine the first intersection of the discrete point set and the first enclosed area equation, and the second intersection with the second enclosed area equation; determine the intersection of the first intersection and the second intersection; determine the number of points in the intersection, and the ratio of the number of duplicate points in the first intersection and the second intersection excluding the intersection as the intersection and union ratio. Specifically, Figure 5 As shown, Figure 5 A schematic diagram of calculating the intersection and union ratio of a convex quadrilateral provided in an embodiment of the present invention. This embodiment only takes the calculation of the intersection and union ratio of a convex quadrilateral as an example. In practical applications, it is not limited to the intersection of convex quadrilaterals, and can also be a parallelogram, etc., specifically including:

[0113] 1) Determine four linear functions between four coordinate points of each prediction box, and determine four linear functions between four coordinate points of the set parking space boundary box under the corresponding camera.

[0114] This embodiment takes two convex quadrilaterals as an example, that is, the above Figure 5 There are eight linear functions shown in total. Among them, the eight linear functions are expressed as follows:

[0115] y=f1(x)

[0116] y-f2(x)

[0117] y=f3(x)

[0118] y=f4(x)

[0119] y=f′1(x)

[0120] y=f′2(x)

[0121] y=f′3(x)

[0122] y=f′4(x)

[0123] 2) Based on the four coordinate points of each prediction box and the four coordinate points of the corresponding set parking space boundary box, the coordinate points in the corresponding prediction box and the set parking space boundary box are determined respectively.

[0124] Specifically, the coordinates of a point inside the two convex quadrilaterals (a point inside any quadrilateral) can be obtained as Point1 and Point2 respectively according to the coordinates of the four points of the two convex quadrilaterals;

[0125] Point1=(x1,y1)

[0126] Point2=(x2,y2)

[0127] 3) According to the relative position of the coordinates of the points in the box and the corresponding linear functions, determine the linear function inequalities of the top, bottom, left and right of the center point of the enclosed area between each prediction box and the corresponding set parking space boundary box, and determine the corresponding first enclosed area equation and second enclosed area equation respectively based on the linear function inequalities.

[0128] Specifically, according to the relative positions of the coordinates of Point1 and Point2 and the linear function, the inequality signs of the linear functions at the top, bottom, left and right of the center point are determined when obtaining the enclosed area, and the two enclosed area equations (i.e., the inequality method) are determined.

[0129] Enclosed region equation Q1

[0130] y-f1(x)>0if y1>f1(x1)else<0 ①

[0131] y-f2(x)>0 if y1>f2(x1)else<0 ②

[0132] y-f3(x)>0 if y1>f3(x1)else<0 ③

[0133] y-f4(x)>0 if y1>f4(x1)else<0 ④

[0134] The closed region equation Q2:

[0135] yf′1(x)>0if y1>f′1(x1)else<0 ①

[0136] yf′2(x)>0if y1>f′2(x1)else<0 ②

[0137] yf′3(x)>0if y1>f′3(x1)else<0 ③

[0138] yf′4(x)>0if y1>f′4(x1)else<0 ④

[0139] 4) According to the discrete point set set in the image, a first intersection of the discrete point set and the first enclosed area equation and a second intersection of the discrete point set and the second enclosed area equation are determined respectively.

[0140] Specifically, in this embodiment, based on the discrete point set pre-set for the image, the first intersection Inter1 and the second intersection Inter2 with the enclosed area equation are respectively obtained.

[0141] The discrete point set Z is a set of points with intervals tiled on the image. Assuming (x1, y1) is the point at the i-th row and j-th column on the image with height H and width W, then when the step size is 3

[0142] Z={(x1,y1)|i,j∈3×N,N is an integer}

[0143] Inter1=(Q1∩Z)

[0144] Inter2=(Q2∩Z)

[0145] 5) Determine the intersection of the first intersection and the second intersection.

[0146] Specifically, calculate the intersection of the corresponding intersections of two convex quadrilaterals, that is

[0147] Inter=Inter1∩Inter2

[0148] 6) Determine the ratio of the number of points in the intersection to the number of duplicate points in the first intersection and the second intersection excluding the intersection, as the intersection-union ratio.

[0149] Specifically, the ratio of the number of points in the first intersection and the second intersection to the number of points in the intersection corresponding to the two quadrilaterals after deduplication is the intersection-over-union ratio IoU, as shown in the following formula:

[0150] IoU=Inter / (Inter1+Inter2-Inter)

[0151] 7) After obtaining the intersection-and-union ratio, the parking space occupancy information in each camera image is basically complete. However, due to the different viewing angles of each camera and the intersection between cameras, the same parking space and the corresponding vehicle have different shapes and face different occlusions in different cameras. Therefore, it is necessary to solve the problem of self-contradiction in occupancy detection of the same parking space in two camera images. The present invention uses the average value of the intersection-and-union ratio with the agreed parking space and the confidence of the prediction box when matching the parking space to select the final result, and adopts the higher one.

[0152] Among them, based on the intersection-and-union ratio, predicting the occupancy status and vehicle position of each parking space distributed under different cameras, includes: obtaining the confidence of the prediction frame; determining the average value of the intersection-and-union ratio and the confidence of the prediction frame, and using the average value as the final intersection-and-union ratio; based on the final intersection-and-union ratio, predicting the occupancy status and vehicle position of each parking space distributed under different cameras.

[0153] In step 103, the occupancy status of parking spaces under each camera in the parking lot is displayed.

[0154] In this step, the display can be controlled to display the occupancy status of parking spaces under each camera in the parking lot using a graphical user interface (GUI). GUI generally refers to a computer operation user interface that is displayed in a graphical manner.

[0155] In an embodiment of the present invention, a detection image of each camera in a parking lot is obtained, wherein the detection image includes a camera identifier and a position number of each parking space under each camera; the detection image is predicted and processed using a parking space occupancy detection model to obtain the occupancy status of each parking space under each camera and the prediction frame of the vehicle, wherein the parking space occupancy detection model is a detection model based on deep learning; and the occupancy status of the parking spaces under each camera in the parking lot is displayed. That is to say, in an embodiment of the present invention, a parking space occupancy detection model based on deep learning is used to predict the occupancy status of parking spaces, and the structure thereof is simple and the prediction accuracy is high. The parking space occupancy detection model is more adaptable to vehicle affine deformation through regression, can adapt to the inference speed requirements of multi-camera images, and outputs a bounding box that conforms to the overhead scene.

[0156] Optionally, in another embodiment, based on the above embodiment, the method may further include: pre-training the parking space occupancy detection model in the following manner: specifically including:

[0157] A training image is obtained, wherein the training image includes: a plurality of anchor points tiled on the image, and anchor frames of different sizes and proportions stacked around each anchor point; the training image is input into the parking space occupancy detection model for training, specifically including: a mobile neural network in the parking space occupancy detection model uses a deep separable convolutional network to generate feature maps of various scales, wherein the feature points on the feature map correspond one-to-one to each anchor point; an intersection-over-union ratio is calculated based on the real bounding box on the training image and the corresponding anchor box; a positive sample is determined based on the size of the intersection-over-union ratio, wherein the component responsible for regression of the positive sample is the conversion of the anchor frame to its rotated parallelogram frame format, and the conversion includes: long side, short side scaling, center offset, rotation angle and tilt angle; a non-maximum suppression process is performed on the positive sample to obtain a detection result with a prediction frame for each camera; the detection result is matched with the real parking space bounding box under the corresponding camera to obtain the occupancy status of each parking space under different cameras.

[0158] That is to say, in the embodiment of the present invention, the existing anchor detection architecture is used, and the detection architecture is improved for training. During training, the mobile neural network MobileNet is used in the backbone part to replace ResNet to increase the training speed.

[0159] In traditional model training, a series of anchor points are pre-generated and tiled on the image, and then anchor frames of different sizes and proportions are generated with them as the center. After the image is input into the backbone network (i.e., ResNet network), feature maps of various scales are generated. Each pre-generated anchor point corresponds to a feature point on the feature map one by one. The positive and negative samples are determined by calculating the intersection and union ratio between the real bounding box and the anchor frame. Generally, I O If U>0.5, it means that the anchor box is a positive sample, otherwise it is a negative sample. The anchor box is responsible for predicting the true bounding box, and its category and regression components (center regression and width and height scaling) are used to fit the category of the true bounding box and the regression component relative to the anchor box. During inference, after the image is input, the backbone network outputs a feature map. If the category score of the feature point corresponding to the anchor box on the feature map is a positive example, the anchor box is fine-tuned (using the predicted center regression and width and height scaling components), and the prediction box is output together with the joint classification. At this time, due to anchor supervision, there are often many stacked redundant prediction boxes. After non-maximum suppression, only the box with the highest classification score is retained in each cluster of boxes, and the output result is the final detection result.

[0160] Based on the training of the above model, the embodiment of the present invention proposes to use depthwise separable convolution to replace the ordinary convolution in resnet in order to simplify the calculation in the backbone network, so as to achieve the purpose of accelerating the calculation. The pre-generated anchor frame is also a traditional rectangular frame. The real bounding box participating in the intersection and union ratio when matching positive and negative samples is its rectangular bounding box format. After determining the positive and negative samples, the component responsible for regression of the positive sample is the conversion of the anchor frame to its rotated parallelogram frame format. The conversion not only includes the scaling of the long side and short side, the center offset, but also includes the rotation angle and the tilt angle. During reasoning, as with the traditional model: the image is input into the backbone network, the feature map is output, the feature map is input into the detection head, and the category is determined according to the result after each feature point passes through the detection head. If it is a region of interest (ROI) object, then the corresponding anchor frame of the feature point is fine-tuned (using the predicted center offset length, width, scaling, rotation angle and tilt angle, etc.), and finally, after non-maximum suppression (needing to use the intersection and union ratio of convex quadrilaterals to filter out), multiple prediction frames with camera identification are obtained, and then the occupancy of each parking space and the vehicle position under different cameras are obtained.

[0161] Among them, the embodiment of the present invention is based on multi-camera scene merge detection, and the detection result is matched with the multi-camera parking space agreement to output the occupancy detection result. The multi-camera scene is in line with reality, taking into account the problem of overlapping monitoring areas. The parking space positions of each camera are agreed upon instead of obtaining the parking space positions through detection because there are a large number of conventional irregular parking spaces in reality. In order to avoid ignoring these parking spaces, the parking spaces are defined in an agreed manner.

[0162] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that this implementation disclosure is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0163] Figure 6 4 is a block diagram of a parking space occupancy detection device provided by an embodiment of the present invention. The device comprises: an acquisition module 401, a prediction module 602 and a display module 603, wherein:

[0164] The acquisition module 601 is used to acquire the detection images of each camera in the parking lot, wherein the detection images include the camera identification and the position number of each parking space under each camera;

[0165] A prediction module 602 is used to perform prediction processing on the detection image using a parking space occupancy detection model to obtain the occupancy status of each parking space and the prediction frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on a deep separable convolutional network;

[0166] The display module 603 is used to display the occupancy status of parking spaces under each camera in the parking lot.

[0167] Optionally, in another embodiment, based on the above embodiment, the prediction module 602 includes: a classification processing module 701, a suppression processing module 702 and a matching module 703, and its structural block diagram is as follows: Figure 7 As shown,

[0168] A classification processing module 701 is used to perform feature classification processing on the detection image using a parking space occupancy detection model to obtain a classification branch for predicting vehicles and a regression branch for cutting angles;

[0169] A suppression processing module 702 is used to perform non-maximum suppression processing on the classification branch and the regression branch to obtain multiple prediction boxes with camera identification;

[0170] The matching module 703 is used to match the multiple prediction frames with the set parking space boundary frames corresponding to each camera, and predict the occupancy status and vehicle position of each parking space distributed under different cameras.

[0171] Optionally, in another embodiment, based on the above embodiment, the matching module includes:

[0172] A determination module, used to determine an intersection-over-union ratio between the multiple prediction frames and a set parking space boundary frame corresponding to each camera;

[0173] The position prediction module is used to predict the occupancy status and vehicle position of each parking space under different cameras based on the intersection-to-union ratio.

[0174] Optionally, in another embodiment, based on the above embodiment, the determining module includes:

[0175] A function determination module, used to determine four linear functions between four coordinate points of each prediction box, and to determine four linear functions between four coordinate points of a set parking space boundary box under a corresponding camera;

[0176] A coordinate point determination module, used to determine the corresponding in-frame coordinate points in the prediction frame and the set parking space boundary frame respectively based on the four coordinate points of each prediction frame and the four coordinate points of the corresponding set parking space boundary frame;

[0177] A function inequality determination module, used to determine the linear function inequalities of the upper, lower, left and right four positions of the center point of the enclosed area between each prediction box and the corresponding set parking space boundary box according to the relative positions of the coordinate points in the box and the corresponding linear functions;

[0178] An equation determination module, used to determine the corresponding first clamped area equation and second clamped area equation respectively based on the linear function inequality;

[0179] A first intersection determination module, used for determining a first intersection of the discrete point set and the first enclosed area equation, and a second intersection of the discrete point set and the second enclosed area equation, respectively, according to the discrete point set set in the image;

[0180] A second intersection determination module, used to determine the intersection of the first intersection and the second intersection;

[0181] The intersection-and-union ratio determination module is used to determine the ratio of the number of points in the intersection to the first intersection and the second intersection excluding the number of repeated points in the intersection, as the intersection-and-union ratio.

[0182] Optionally, in another embodiment, based on the above embodiment, the location prediction module includes:

[0183] A confidence acquisition module, used to obtain the confidence of the prediction box;

[0184] An average value determination module, used to determine an average value of the intersection-over-union ratio and the confidence of the prediction box, and use the average value as the final intersection-over-union ratio;

[0185] The position prediction submodule is used to predict the occupancy status and vehicle position of each parking space under different cameras based on the final intersection-to-union ratio.

[0186] Optionally, in another embodiment, based on the above embodiment, the device further includes: a training module, which is used to pre-train the parking space occupancy detection model in the following manner:

[0187] Acquire a training image, wherein the training image includes: a plurality of anchor points tiled on the image, and anchor frames of different sizes and proportions stacked around each anchor point;

[0188] A training module is used to input the training image into the parking space occupancy detection model for training, specifically including: the mobile neural network in the parking space occupancy detection model uses a deep separable convolutional network to generate feature maps of various scales, wherein the feature points on the feature map correspond one-to-one to each anchor point; based on the real bounding box on the training image and the corresponding anchor box, the intersection and union ratio is calculated; based on the size of the intersection and union ratio, a positive sample is determined, wherein the component responsible for regression of the positive sample is the conversion of the anchor box to its rotated parallelogram box format, and the conversion includes: long side, short side scaling, center offset, rotation angle and tilt angle; non-maximum suppression processing is performed on the positive sample to obtain a detection result with a prediction box for each camera; the detection result is matched with the real parking space bounding box under the corresponding camera to obtain the occupancy status of each parking space under different cameras.

[0189] Optionally, in another embodiment, based on the above embodiment, the display module specifically uses a graphical user interface (GUI) to display the occupancy status of parking spaces under each camera in the parking lot.

[0190] See also Figure 8 , is a block diagram of a parking space occupancy detection system provided in an embodiment of the present invention, the system comprising: a sensing terminal 801, a computing server 802 and a display 803, the sensing terminal is connected to the computing server and the display respectively, wherein,

[0191] The sensing terminal 801 is used to obtain detection images of each camera in the parking lot, wherein the detection images include camera identifications and location labels of each parking space under each camera; and send the detection images to the computing server;

[0192] The computing server 802 is used to perform prediction processing on the detection image using a parking space occupancy detection model to obtain the occupancy status of each parking space and the predicted frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on a deep separable convolutional network; and send the occupancy status of each parking space and the predicted frame of the vehicle under each camera to the display;

[0193] The display 803 is used to display the occupancy status of the parking spaces under each camera in the parking lot when receiving the occupancy status of each parking space under each camera and the predicted frame of the vehicle.

[0194] See also Fig. 9 , is an application example diagram of a parking space occupancy detection system provided by an embodiment of the present invention, in which: Fig. 9 As shown:

[0195] Hardware deployment: The hardware is mainly divided into cameras, perception computers, and computing computers. Regarding cameras, N elevated cameras cover the entire parking lot at a bird's-eye view. Since the height of the camera affects the field of view during deployment, the number of cameras required for monitoring the same parking lot using cameras at different heights is different. This solution can adapt to different scenarios and allows N cameras to be installed at different heights. The monitoring areas of n (n<=N) cameras can also overlap. After the cameras are installed, they must not be moved. Regarding the perception computer, it only needs to be connected to the camera and the computing computer, so a lightweight Raspberry Pi is used, and the computing computer is a cloud-based GPU server with a good network connection and good heat dissipation conditions with the perception computer.

[0196] Software deployment: The main contents of software deployment are: a. agreeing on the location of parking spaces in the images of each camera in the parking lot and numbering them as detection model parameters; b. setting up the algorithm running environment on the computing computer and writing the algorithm; c. debugging the communication program between the computing computer and the perception computer; d. running and debugging of the GUI on the perception computer.

[0197] After completing the above hardware and software deployment, the system starts to run. When the system is running, the installed cameras are transmitted to the perception computer (such as Raspberry Pi, etc.) through the line. The perception computer will use multiple cameras to capture real-time images and transmit the recorded corresponding videos (this embodiment takes video 1, video 2, video 3 and video 4 as examples) to the computing computer (GPU server). The GPU server uses the parking space occupancy detection model to output the image prediction results of whether each parking space is occupied and the prediction frame of the vehicle, and returns them to the perception computer. The perception computer displays the occupancy status of the parking spaces under each camera through the display and the running GUI interface.

[0198] In the embodiment of the present invention, the key to the operation of the system lies in the parking space occupancy detection model. The following is the entire model workflow of inputting an image and outputting the parking space occupancy status and the parking space occupancy model structure proposed in this article. The parking space occupancy model is detailed above and will not be repeated here.

[0199] After the images of N cameras are input into the image, many prediction boxes with camera IDs are output. The intersection and union ratios of these prediction boxes and the agreed parking space boundary boxes Box of each camera are calculated to obtain the occupancy status of each parking space and the vehicle position distributed under different lenses. The intersection and union ratio of the above inclined parallelogram is calculated.

[0200] In an embodiment of the present invention, the parking space occupancy detection model optimized for parking scenarios has a streamlined structure, a regression method that is more adaptable to vehicle affine deformation, can meet the inference speed requirements of multi-camera images, and outputs a bounding box that conforms to the overhead scene.

[0201] In the embodiment of the present invention, the intersection-and-union ratio of convex quadrilaterals can be calculated to serve the calculation of the intersection-and-union ratio of tilted quadrilateral tensors in the model reasoning process.

[0202] In the embodiment of the present invention, based on the combined detection of multiple camera scenes, the detection results are matched with the multi-camera parking space agreement to output the occupancy detection results. The multi-camera scene is in line with reality, taking into account the problem of overlapping monitoring areas. The parking space positions of each camera are agreed upon rather than obtained through detection because there are a large number of conventional irregular parking spaces in reality. In order to avoid ignoring these parking spaces, the parking spaces are defined in an agreed manner.

[0203] An embodiment of the present invention further provides an electronic device, including:

[0204] processor;

[0205] a memory for storing instructions executable by the processor;

[0206] The processor is configured to execute the instructions to implement the parking space occupancy detection method as described above.

[0207] An embodiment of the present invention further provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the parking space occupancy detection method as described above.

[0208] An embodiment of the present invention further provides a computer program product, including a computer program or instructions, which implements the parking space occupancy detection method as described above when executed by a processor of an electronic device.

[0209] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0210] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0211] Fig.10 1 is a block diagram of an electronic device 1000 provided in an embodiment of the present invention. For example, the electronic device 1000 may be a mobile terminal or a server, and the electronic device is a mobile terminal as an example for explanation in the embodiment of the present invention. For example, the electronic device 1000 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0212] Reference Fig.10 , the electronic device 1000 may include one or more of the following components: a processing component 1002 , a memory 1004 , a power component 1006 , a multimedia component 1008 , an audio component 1010 , an input / output (I / O) interface 1012 , a sensor component 1014 , and a communication component 1016 .

[0213] The processing component 1002 generally controls the overall operation of the electronic device 1000, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1002 may include one or more modules to facilitate the interaction between the processing component 1002 and other components. For example, the processing component 1002 may include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0214] The memory 1004 is configured to store various types of data to support operations on the device 1000. Examples of such data include instructions for any application or method operating on the electronic device 1000, contact data, phone book data, messages, pictures, videos, etc. The memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0215] The power supply component 1006 provides power to various components of the electronic device 1000. The power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 1000.

[0216] The multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. When the device 1000 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0217] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC), and when the electronic device 1000 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1004 or sent via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.

[0218] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0219] The sensor assembly 1014 includes one or more sensors for providing various aspects of status assessment for the electronic device 1000. For example, the sensor assembly 1014 can detect the open / closed state of the device 1000, the relative positioning of components, such as the display and keypad of the electronic device 1000, and the sensor assembly 1014 can also detect the position change of the electronic device 1000 or a component of the electronic device 1000, the presence or absence of user contact with the electronic device 1000, the orientation or acceleration / deceleration of the electronic device 1000, and the temperature change of the electronic device 1000. The sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0220] The communication component 1016 is configured to facilitate wired or wireless communication between the electronic device 1000 and other devices. The electronic device 1000 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0221] In an embodiment, the electronic device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the parking space occupancy detection method shown above.

[0222] In an embodiment, a computer-readable storage medium is also provided, and when the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device 1000 can execute the parking space occupancy detection method shown above. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0223] In an embodiment, a computer program product is also provided, including a computer program or instructions. When the computer program or instructions are executed by the processor 1020 of the electronic device 1000, the electronic device 1000 executes the above-mentioned parking space occupancy detection method.

[0224] Fig.11 1 is a block diagram of a device 1100 for parking space occupancy detection provided by an embodiment of the present invention. For example, the device 1100 may be provided as a server. Fig.11 , the apparatus 1100 includes a processing component 1122, which further includes one or more processors, and a memory resource represented by a memory 1132 for storing instructions, such as applications, that can be executed by the processing component 1122. The application stored in the memory 1132 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1122 is configured to execute instructions to perform the above method.

[0225] The device 1100 may also include a power supply component 1126 configured to perform power management of the device 1100, a wired or wireless network interface 1150 configured to connect the device 1100 to a network, and an input / output (I / O) interface 1158. The device 1100 may operate based on an operating system stored in the memory 1132, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.

[0226] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0227] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A parking space occupancy detection method, characterized in that: include: Acquire detection images of each camera in the parking lot, wherein the detection images include camera identifiers and location labels of each parking space under each camera; The detection image is predicted and processed by using a parking space occupancy detection model to obtain the occupancy status of each parking space and the prediction frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on a deep separable convolutional network; Display the occupancy status of parking spaces under each camera in the parking lot.

2. The parking space occupancy detection method according to claim 1, characterized in that: The method of using the parking space occupancy detection model to perform prediction processing on the detection image to obtain the occupancy status of each parking space and the predicted frame of the vehicle under each camera includes: Using a parking space occupancy detection model to perform feature classification processing on the detection image, to obtain a classification branch for predicting vehicles and a regression branch for cutting angles; Performing non-maximum suppression processing on the classification branch and the regression branch to obtain multiple prediction boxes with camera logos; The multiple prediction frames are matched with the set parking space boundary frames under the corresponding cameras to predict the occupancy status and vehicle positions of each parking space under different cameras.

3. The parking space occupancy detection method according to claim 2, characterized in that: The step of matching the plurality of prediction frames with the set parking space boundary frames corresponding to the cameras to predict the occupancy status and vehicle positions of the parking spaces under different cameras includes: Determine an intersection-over-union ratio between the multiple prediction boxes and the set parking space boundary boxes under each corresponding camera; Based on the intersection-and-union ratio, the occupancy status of each parking space and the vehicle position under different cameras are predicted.

4. The parking space occupancy detection method according to claim 3, characterized in that: The determining of the intersection-and-union ratios of the multiple prediction frames and the parking space boundary frames corresponding to the cameras includes: Determine four linear functions between four coordinate points of each prediction box, and determine four linear functions between four coordinate points of a set parking space boundary box under the corresponding camera; Based on the four coordinate points of each prediction box and the four coordinate points of the corresponding set parking space boundary box, respectively determine the coordinate points in the corresponding prediction box and the set parking space boundary box; According to the relative positions of the coordinates of the points in the frame and the corresponding linear functions, the linear function inequalities of the upper, lower, left and right sides of the center point of the enclosed area between each prediction frame and the corresponding set parking space boundary frame are determined; Determine the corresponding first clamped area equation and second clamped area equation respectively based on the linear function inequality; According to the discrete point set set in the image, respectively determine a first intersection of the discrete point set with the first enclosed area equation and a second intersection with the second enclosed area equation; Determine the intersection of the first intersection and the second intersection; The ratio of the number of points in the intersection to the number of duplicate points in the first intersection and the second intersection excluding the intersection is determined as the intersection-union ratio.

5. The parking space occupancy detection method according to claim 3, characterized in that: The method of predicting the occupancy of each parking space and the position of the vehicle under different cameras based on the intersection-to-union ratio includes: Obtaining the confidence of the prediction box; Determine an average value of the intersection-over-union ratio and the confidence of the prediction box, and use the average value as the final intersection-over-union ratio; Based on the final intersection-to-union ratio, the occupancy status of each parking space and the vehicle position under different cameras are predicted.

6. The parking space occupancy detection method according to any one of claims 1 to 5, characterized in that: The method further includes: pre-training the parking space occupancy detection model in the following manner: Acquire a training image, wherein the training image includes: a plurality of anchor points tiled on the image, and anchor frames of different sizes and proportions stacked around each anchor point; Inputting the training image into the parking space occupancy detection model for training specifically includes: The mobile neural network in the parking space occupancy detection model uses a deep separable convolutional network to generate feature maps of various scales, wherein the feature points on the feature map correspond one-to-one to each anchor point; the intersection-and-union ratio is calculated based on the real bounding box on the training image and the corresponding anchor box; the positive sample is determined based on the size of the intersection-and-union ratio, wherein the component responsible for regression of the positive sample is the conversion of the anchor box to its rotated parallelogram box format, and the conversion includes: long side, short side scaling, center offset, rotation angle and tilt angle; the positive sample is subjected to non-maximum suppression processing to obtain the detection result with the prediction box of each camera; the detection result is matched with the real parking space bounding box under the corresponding camera to obtain the occupancy status of each parking space under different cameras.

7. A parking space occupancy detection device, characterized in that: include: An acquisition module is used to acquire detection images of each camera in the parking lot, wherein the detection images include camera identifications and position labels of each parking space under each camera; A prediction module, used to perform prediction processing on the detection image using a parking space occupancy detection model to obtain the occupancy status of each parking space and the prediction frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on a deep separable convolutional network; The display module is used to display the occupancy status of parking spaces under each camera in the parking lot.

8. A parking space occupancy detection system, characterized in that: include: A perception terminal, a computing server and a display, wherein the perception terminal is connected to the computing server and the display respectively, wherein: The sensing terminal is used to obtain detection images of each camera in the parking lot, wherein the detection images include camera identifiers and position labels of each parking space under each camera; and send the detection images to the computing server; The computing server is used to perform prediction processing on the detection image using a parking space occupancy detection model to obtain the occupancy status of each parking space and the predicted frame of the vehicle under each camera, wherein the parking space occupancy detection model is a detection model based on a deep separable convolutional network; and send the occupancy status of each parking space and the predicted frame of the vehicle under each camera to the display; The display is used to display the occupancy status of the parking spaces under each camera in the parking lot when receiving the occupancy status of each parking space under each camera and the predicted frame of the vehicle.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the parking space occupancy detection method as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the parking space occupancy detection method as described in any one of claims 1 to 6.

Citation Information

Cited By

  • Electric vehicle charging station occupation detection method based on YOLO model platform

    CN120853070A