Target detection model training method, parking information processing method, device and equipment

Abstract

The invention provides a target detection model training method, a parking information processing method, devices and equipment, and relates to the technical field of intelligent parking. The target detection model training method comprises the following steps: acquiring a plurality of sample images of a target parking scene; obtaining a detection result of each sample image by adopting an initial detection model, wherein the detection result comprises a prediction detection frame of a vehicle grounding surface and a prediction detection frame of a license plate; and adjusting parameters of the initial feature extraction module, the initial integration module and the initial detection module according to the prediction detection frame and the label detection frame of the vehicle grounding surface and the prediction detection frame and the label detection frame of the license plate to obtain a target detection model with a target feature extraction module, a target integration module and a target detection module. By constructing the low-cost labeling data set, a single model is realized, information such as the vehicle grounding surface and the license plate angular point coordinates is accurately predicted, and finally, the accuracy and efficiency of judging whether the vehicle enters and exits the parking space in the intelligent parking lot scene are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent parking technology, and more specifically, to a target detection model training method, a parking information processing method, a device, and equipment. Background Technology

[0002] In the fields of computer vision and intelligent perception, 3D target detection technology, as a core means of acquiring target spatial location and shape information, has been widely applied in various scenarios such as autonomous driving, intelligent robots, and intelligent transportation.

[0003] In complex scenarios such as autonomous driving, simple 3D detection solutions are insufficient to meet the high accuracy and reliability requirements due to dynamic traffic flow, complex road conditions, and multi-target interactions. In such cases, it is necessary to simultaneously predict both 2D (e.g., bounding box coordinates within the image plane) and 3D information of targets like vehicles to provide comprehensive environmental perception data for the autonomous driving decision-making system, supporting critical operations such as path planning and emergency braking. Currently, in the field of autonomous driving, to ensure detection coverage and accuracy, more than two cameras or radar devices are required, and model training relies on massive amounts of labeled autonomous driving-specific datasets to adapt to complex road environments.

[0004] However, in the specific application area of ​​smart parking, existing 3D object detection technologies and solutions suffer from significant limitations in scenario adaptability and cost. The core requirements of smart parking scenarios are to determine the compliance of vehicle parking and identify vehicle identities (license plates, vehicle models). Its hardware deployment has distinct characteristics: to achieve comprehensive coverage and blind-spot-free monitoring of parking areas, cameras are typically mounted at a high position, approximately 5-8 meters above the ground. This high-positioning results in significant differences in the image samples collected by the cameras compared to open-source data collected in autonomous driving scenarios—autonomous driving data is mostly collected by onboard cameras or low-position devices, with vehicle information primarily from close-range perspectives such as side and front views, and backgrounds consisting mainly of roads, pedestrians, and traffic signs; while in high-position images from smart parking scenarios, vehicles are mostly presented from a top-down perspective, with relatively smaller target scales and backgrounds consisting mainly of parking lines and parking lot surface textures. This difference in sample distribution means that 3D object detection models trained on autonomous driving datasets cannot be directly transferred to smart parking scenarios, leading to a significant decrease in model detection accuracy and making it difficult to meet the business requirements for parking space identification and vehicle information recognition.

[0005] Meanwhile, if the training dataset for intelligent parking scenarios is constructed by referring to the annotation standards of autonomous driving datasets, complex hardware such as LiDAR and multi-camera synchronous acquisition equipment is required to accurately collect and annotate the 3D point cloud information of vehicles, camera intrinsic and extrinsic parameters, and target spatial coordinates. This not only results in high equipment procurement and maintenance costs and a cumbersome and time-consuming annotation process, but also significantly increases the deployment cost and promotion difficulty of intelligent parking systems, thus restricting the large-scale application of 3D target detection technology in the field of intelligent parking. Summary of the Invention

[0006] This application addresses the shortcomings of the prior art by providing a target detection model training method, a parking information processing method, an apparatus, and a device to solve the problems existing in the prior art.

[0007] The technical solution adopted in the embodiments of this application is as follows: In a first aspect, embodiments of this application provide a method for training an object detection model, the method comprising: Acquire multiple sample images of the target parking scene. Each sample image contains: a bounding box for the vehicle's ground contact and a bounding box for the license plate. The initial feature extraction module in the initial detection model is used to perform multi-scale feature extraction on each sample image to obtain feature maps of multiple scales corresponding to each sample image; The initial integration module in the initial detection model is used to fuse the feature maps at multiple scales to obtain the fused feature map corresponding to each sample image; The initial detection module in the initial detection model is used to detect the fused feature map to obtain the detection result of each sample image. The detection result includes: the predicted detection box of the vehicle ground surface and the predicted detection box of the license plate. Based on the predicted detection bounding box and labeled detection bounding box of the vehicle ground surface, and the predicted detection bounding box and labeled detection bounding box of the license plate, the parameters of the initial feature extraction module, the initial integration module and the initial detection module are adjusted to obtain a target detection model with a target feature extraction module, a target integration module and a target detection module.

[0008] In one embodiment, each sample image further includes: vehicle type annotation information, and the detection result further includes: vehicle type prediction information; The step involves adjusting the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the predicted detection boxes and labeled detection boxes of the vehicle ground surface and the predicted detection boxes and labeled detection boxes of the license plate, to obtain a target detection model with a target feature extraction module, a target integration module, and a target detection module, including: Based on the predicted and labeled detection boxes of the vehicle ground surface, the predicted and labeled detection boxes of the license plate, and the predicted and labeled information of the vehicle type, the parameters of the initial feature extraction module, the initial integration module, and the initial detection module are adjusted to obtain the target detection model.

[0009] In one embodiment, each sample image further includes: a labeled detection box for the vehicle; the detection result further includes: a predicted detection box for the vehicle; The step involves adjusting the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the predicted detection boxes and labeled detection boxes of the vehicle ground surface and the predicted detection boxes and labeled detection boxes of the license plate, to obtain a target detection model with a target feature extraction module, a target integration module, and a target detection module, including: Based on the predicted detection bounding box and the labeled detection bounding box of the vehicle, the center point of the predicted grid and the center point of the labeled grid are determined respectively; Based on the center point of the predicted grid, the vertex of the predicted detection box of the vehicle ground surface, and the predicted detection box of the license plate, calculate the first offset information of the vehicle ground surface and the second offset information of the license plate. Based on the center point of the labeled grid, the vertex of the labeled detection box of the vehicle ground surface, and the labeled detection box of the license plate, calculate the third offset information of the vehicle ground surface and the fourth offset information of the license plate. Based on the first offset information and the third offset information, calculate the first offset loss function value of the vehicle's ground contact surface; Calculate the second offset loss function value of the license plate based on the second offset information and the fourth offset information; Based on the first offset loss function value and the second offset loss function value, the parameters of the initial feature extraction module, the initial integration module and the initial detection module are tuned to obtain a target detection model with a target feature extraction module, a target integration module and a target detection module.

[0010] In one embodiment, calculating the first offset information of the vehicle ground plane and the second offset information of the license plate based on the center point of the predicted grid, the vertices of the predicted detection box of the vehicle ground plane, and the predicted detection box of the license plate includes: Based on the predicted grid center point, determine the first predicted center point of the predicted detection frame of the vehicle ground surface and the second predicted center point of the predicted detection frame of the license plate. The first offset information is calculated based on the first predicted center point and the vertex of the predicted detection box of the vehicle ground surface; The second offset information is calculated based on the second predicted center point and the vertex of the predicted detection box of the license plate.

[0011] In one embodiment, calculating the third offset information of the vehicle ground plane and the fourth offset information of the license plate based on the center point of the labeled grid, the vertices of the labeled detection boxes of the vehicle ground plane, and the labeled detection boxes of the license plate includes: Based on the center point of the marked grid, determine the first center point of the marked detection frame of the vehicle ground surface and the second center point of the marked detection frame of the license plate; The third offset information is calculated based on the first annotation center point and the vertex of the annotation detection box of the vehicle ground surface; The fourth offset information is calculated based on the second annotation center point and the vertex of the annotation detection box of the license plate.

[0012] Secondly, embodiments of this application provide a parking information processing method, the method comprising: Acquire the image to be detected in the target parking scene; A target detection model is used to perform target detection on the image to be detected, and the detection result of the image to be detected is obtained. The detection result includes: the detection box of the vehicle ground surface of the target vehicle and the detection box of the license plate in the image to be detected; the target detection model is the target detection model described in any of the above embodiments. The parking status of the target vehicle is detected based on the detection frames of the vehicle's ground contact area and the license plate.

[0013] In one embodiment, detecting the parking status of the target vehicle based on the detection frame of the vehicle's ground contact surface and the detection frame of the license plate includes: Based on the detection frame of the vehicle's ground contact area, determine whether the target vehicle is parked at the preset parking position; If the target vehicle is not parked at the preset parking location, the license plate number of the target vehicle is detected according to the license plate detection frame to obtain the license plate number of the target vehicle. Based on the license plate number, a notification message is sent to the owner of the target vehicle.

[0014] Thirdly, embodiments of this application provide a training apparatus for an object detection model, the apparatus comprising: The first acquisition module is used to acquire multiple sample images of the target parking scene. Each sample image contains: a bounding box for the vehicle's ground contact and a bounding box for the license plate. The extraction module is used to perform multi-scale feature extraction on each sample image using the initial feature extraction module in the initial detection model, so as to obtain feature maps of each sample image at multiple scales. The fusion module is used to fuse the feature maps of the multiple scales using the initial integration module in the initial detection model to obtain the fused feature map corresponding to each sample image; The feature map detection module is used to detect the fused feature map using the initial detection module in the initial detection model to obtain the detection result of each sample image. The detection result includes: the predicted detection box of the vehicle ground surface and the predicted detection box of the license plate. The parameter tuning module is used to tune the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the predicted detection box and labeled detection box of the vehicle ground surface and the predicted detection box and labeled detection box of the license plate, so as to obtain a target detection model with a target feature extraction module, a target integration module, and a target detection module.

[0015] Fourthly, embodiments of this application provide a parking information processing device, the device comprising: The second acquisition module is used to acquire the image to be detected in the target parking scene; The target detection module is used to perform target detection on the image to be detected using a target detection model, and obtain the detection result of the image to be detected. The detection result includes: the detection box of the vehicle ground plane of the target vehicle in the image to be detected and the detection box of the license plate; the target detection model is the target detection model described in any of the above embodiments. The parking detection module is used to detect the parking status of the target vehicle based on the detection frame of the vehicle's ground contact area and the detection frame of the license plate.

[0016] Fifthly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the method described in any of the above embodiments.

[0017] The beneficial effects of this application are as follows: This application provides a method for training an object detection model, aiming to solve the problems of high data annotation costs, low model efficiency, and insufficient detection accuracy of traditional detection schemes in intelligent parking scenarios. This method constructs a low-cost labeled dataset, enabling a single model to accurately predict information such as vehicle ground contact points and license plate corner coordinates simultaneously, ultimately improving the accuracy and efficiency of vehicle entry and exit judgment in intelligent parking scenarios. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 One of the flowcharts for the target detection model training method provided in the embodiments of this application; Figure 2 This is a schematic diagram of the sample image; Figure 3 The second schematic flowchart of the target detection model training method provided in the embodiments of this application; Figure 4 The third schematic diagram of the target detection model training method provided in the embodiments of this application; Figure 5 The fourth flowchart illustrates the target detection model training method provided in this application embodiment. Figure 6 One of the flowcharts of the parking information processing method provided in the embodiments of this application; Figure 7 A second schematic flowchart illustrating the parking information processing method provided in this application embodiment; Figure 8 A schematic diagram of the structure of the training device for the target detection model provided in the embodiments of this application; Figure 9 This is a schematic diagram of the structure of the parking information processing device provided in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0021] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] Furthermore, the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Additionally, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0023] It should be noted that, where there is no conflict, the features in the embodiments of this application can be combined with each other.

[0024] In the field of intelligent parking, the main application scenario for detection models is to determine whether a vehicle is parked legally in a parking space and to record the vehicle's license plate, model, and brand information. In this scenario, the most crucial element is obtaining the coordinates of the four ground-based corner points out of the vehicle's eight 3D coordinate points (these four coordinates are used to determine the vehicle's position). The coordinates of the four corner points on the vehicle's surface are not of practical use. Furthermore, this scenario eliminates the need for a camera coordinate system; prediction directly from the image is sufficient to meet business requirements, significantly reducing data annotation costs.

[0025] For license plate recognition, in smart parking scenarios, the license plate also has 4 corner points, which is the same as the number of 4 corner points of the vehicle's ground contact. This means that one model can be used to predict the 4 corner points of both the license plate and the vehicle.

[0026] Therefore, this application provides a method for training an object detection model and a method for processing parking information. The following examples, in conjunction with the accompanying drawings, provide specific illustrations of the object detection model training method provided in this application.

[0027] Figure 1 This is one of the flowcharts illustrating the target detection model training method provided in the embodiments of this application, such as... Figure 1 As shown, the object detection model training methods include: S101. Obtain multiple sample images of the target parking scene.

[0028] First, sample images were collected: In parking scenarios, elevated cameras with a height of 5-8 meters were installed to collect vehicle sample images at different times, under different lighting conditions, with different vehicle types (sedans, vans, trucks, SUVs, MPVs, etc.) and different license plate types (ordinary license plates, new energy vehicle license plates, police / military vehicle license plates, etc.). This ensured that the samples covered typical scenarios such as vehicles parked normally, partially obscured, and license plates that were skewed or distorted, and a total of no less than 10,000 valid sample images were obtained.

[0029] Then, sample image annotation is performed: without using expensive acquisition tools such as LiDAR and point cloud equipment, manual annotation is performed directly on the acquired 2D sample images according to preset rules. Figure 2 This is a schematic diagram of a sample image, such as... Figure 2 As shown, the annotations include: Vehicle-related data: Vehicle type (e.g., sedan, SUV), whether the vehicle is obscured, 2D vehicle detection bounding box (including the coordinates of the top left and bottom right corners of the box), and the ground contact area marker detection box (red box, including the 3D coordinates of the four corner points of the vehicle's ground contact area; the marking order must ensure that the corner points correspond to the front left, front right, rear right, and rear left positions of the vehicle's ground contact area, for example, ...). Figure 2 In the diagram, 0 indicates the front left position of the vehicle when it touches the ground, 1 indicates the front right position, 2 indicates the rear right position, and 3 indicates the rear left position.

[0030] License plate related: license plate type (such as ordinary license plate, new energy vehicle license plate), whether the license plate is obscured, the label detection box of the license plate, and the 3D coordinates of the four corner points of the license plate (the labeling order is consistent with the labeling logic of the corner points of the vehicle's ground surface to avoid confusion in subsequent model predictions).

[0031] Each final sample image contains a bounding box (red box) for the vehicle's ground contact area and a bounding box (blue box) for the license plate.

[0032] S102. Using the initial feature extraction module in the initial detection model, multi-scale feature extraction is performed on each sample image to obtain feature maps of multiple scales corresponding to each sample image.

[0033] ResNet-50 is used as the initial feature extraction module (backbone) of the initial detection model to perform multi-scale feature extraction on each sample image. Specifically, the sample image is resized to 640×640 resolution and then input into the backbone, passing through 5 convolutional stages (conv1-conv5) to output feature maps with strides of 2, 4, 8, 16, and 32, respectively, resulting in 5 feature maps at different scales (denoted as C1-C5). C5 is a feature map downsampled by 32 times, which is used for accurate prediction of corner coordinates in the subsequent process.

[0034] S103. Using the initial integration module in the initial detection model, feature maps at multiple scales are fused to obtain the fused feature map corresponding to each sample image.

[0035] Feature Pyramid Network (FPN) is used as the initial fusion module to fuse multi-scale feature maps. The C5 feature map is reduced in dimensionality by 1×1 convolution and then element-wise added to the C4 feature map, which is upsampled by 2, to obtain the fused feature map. Similarly, the dimensionality-reduced C5 feature map is fused with the C3 feature map, which is upsampled by 2. Finally, three fused feature maps are retained: one downsampled by 8 times, one downsampled by 16 times, and one downsampled by 32 times. The 32-times downsampled feature map is used as the core input feature map for the subsequent detection module.

[0036] S104. Using the initial detection module in the initial detection model, the fused feature map is detected to obtain the detection result for each sample image.

[0037] The initial detection module is designed based on an anchor-free detection framework. A detection head is built with a 32x downsampling. The detection head includes a classification branch, a regression branch, and a corner offset branch. The specific detection process is as follows: Classification branch: Through 3×3 convolution and sigmoid activation function, output the category probability of vehicle / license plate (e.g., the probability of vehicle type being SUV, the probability of license plate type being new energy vehicle license plate) and occlusion status (1 for occluded, 0 for not occluded).

[0038] Regression branch: Predict the width and height (WH-2d) of the vehicle / license plate 2D detection box and the offset (Center_Offset-2d) of the grid point from the center point of the 2D detection box. Combined with the feature map grid coordinates, the predicted coordinates of the vehicle / license plate 2D detection box are calculated.

[0039] Corner offset branch: predict the offset from the grid point to the center point of the four corner points of the vehicle's ground surface (Center_Offset-3d), and the offset from the center point of the corner point to the four corner points (Corner_Offset-3d). By accumulating the two offsets, the predicted coordinates of the four corner points of the vehicle's ground surface and the four corner points of the license plate are obtained.

[0040] By integrating the outputs of each branch of the initial detection module, the detection results for each sample image are obtained, including vehicle type prediction information, vehicle occlusion status, vehicle 2D prediction detection box, and predicted coordinates of the four corner points of the vehicle ground plane (forming the predicted detection box of the vehicle ground plane), as well as license plate type prediction information, license plate occlusion status, license plate 2D prediction detection box, and predicted coordinates of the four corner points of the license plate (forming the predicted detection box of the license plate).

[0041] S105. Based on the predicted detection boxes and labeled detection boxes of the vehicle ground surface and the predicted detection boxes and labeled detection boxes of the license plate, the parameters of the initial feature extraction module, the initial integration module and the initial detection module are adjusted to obtain a target detection model with a target feature extraction module, a target integration module and a target detection module.

[0042] In summary, this embodiment provides a method for training an object detection model, aiming to address the problems of high data annotation costs, low model efficiency, and insufficient detection accuracy in traditional detection schemes for intelligent parking scenarios. This method constructs a low-cost labeled dataset, enabling a single model to simultaneously and accurately predict information such as the coordinates of the four corner points of the vehicle's ground contact and the four corner points of the license plate, ultimately improving the accuracy and efficiency of vehicle entry and exit judgment in intelligent parking scenarios.

[0043] In one embodiment, each sample image also contains vehicle type labeling information, and the detection result also includes vehicle type prediction information.

[0044] S105 may include: adjusting the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the predicted detection boxes and labeled detection boxes of the vehicle ground surface, the predicted detection boxes and labeled detection boxes of the license plate, and the predicted and labeled information of the vehicle type, to obtain the target detection model.

[0045] In one embodiment, such as Figure 2 As shown, each sample image also contains a labeled detection box for the vehicle (green box), and the detection results also include the predicted detection box for the vehicle.

[0046] Figure 3 This is the second flowchart illustrating the target detection model training method provided in the embodiments of this application, as shown below. Figure 3 As shown, S105 may include: S201. Based on the vehicle's predicted detection frame and labeled detection frame, determine the center point of the predicted grid and the center point of the labeled grid, respectively.

[0047] Reference Figure 2 The center point of the grid is Figure 2 The green dot is denoted as point G.

[0048] S202. Based on the predicted grid center point, the vertices of the predicted detection box of the vehicle ground surface, and the predicted detection box of the license plate, calculate the first offset information of the vehicle ground surface and the second offset information of the license plate.

[0049] Specifically, Figure 4 This is the third flowchart illustrating the target detection model training method provided in the embodiments of this application, as shown below. Figure 4 As shown, S202 includes: S301. Based on the predicted grid center point, determine the first predicted center point of the predicted detection box for the vehicle ground contact and the second predicted center point of the predicted detection box for the license plate.

[0050] The first predicted center point of the vehicle's ground contact is calculated based on the predicted grid center point G' and the "offset from the grid point to the center point of the vehicle's ground contact corner" output by the initial model, and is denoted as O1'.

[0051] Similarly, the second predicted center point of the license plate is calculated based on the predicted grid center point G' and the "offset from the grid point to the center point of the license plate corner" output by the initial model, and is denoted as O2'.

[0052] S302. Calculate the first offset information based on the first predicted center point and the vertex of the predicted detection box of the vehicle ground surface.

[0053] The first offset information is the offset of the predicted coordinates of the four corner points of the vehicle's ground contact with the first predicted center point O1', with a total of four sets of offset values.

[0054] S303. Calculate the second offset information based on the second prediction center point and the vertices of the predicted detection box of the license plate.

[0055] The second offset information is the offset of the predicted coordinates of the four corner points of the license plate relative to the second predicted center point O2', with a total of four sets of offset values.

[0056] S203. Based on the center point of the labeled grid, the vertex of the labeled detection box of the vehicle ground surface, and the labeled detection box of the license plate, calculate the third offset information of the vehicle ground surface and the fourth offset information of the license plate.

[0057] Figure 5 This is the fourth flowchart illustrating the target detection model training method provided in the embodiments of this application. Figure 5 As shown, S203 includes S401~S403, and the principle is similar to that of S301~S303: S401. Based on the center point of the marked grid, determine the first center point of the marked detection frame of the vehicle ground surface and the second center point of the marked detection frame of the license plate.

[0058] S402. Calculate the third offset information based on the first annotation center point and the vertex of the annotation detection box of the vehicle ground surface.

[0059] S403. Calculate the fourth offset information based on the center point of the second annotation and the vertex of the license plate annotation detection box.

[0060] S204. Based on the first offset information and the third offset information, calculate the first offset loss function value of the vehicle's ground contact surface.

[0061] The first offset loss function value represents the average error of the corresponding corner offset in the first offset information and the third offset information, denoted as L1.

[0062] S205. Calculate the second offset loss function value of the license plate based on the second offset information and the fourth offset information.

[0063] The second offset loss function value represents the average error of the corresponding corner offsets in the second offset information and the fourth offset information, denoted as L2.

[0064] S206. Based on the first offset loss function value and the second offset loss function value, adjust the parameters of the initial feature extraction module, the initial integration module and the initial detection module to obtain a target detection model with a target feature extraction module, a target integration module and a target detection module.

[0065] To balance the accuracy of vehicle and license plate corner detection, the first offset loss function value L1 and the second offset loss function value L2 are fused according to weights to obtain the total offset loss of the model, denoted as L.

[0066] A stochastic gradient descent (SGD) optimizer was used, with an initial learning rate of 0.01, which was decayed to 1 / 10 of its original value every 30 iterations. The parameters of the three main modules of the initial detection model were iteratively updated with the objective of minimizing the total offset loss L. Initial feature extraction module: Adjusts convolutional layer weights to optimize multi-scale feature extraction accuracy; Initial integration module: Adjusts convolutional kernel parameters during feature fusion to enhance the transmission of corner-related features; Initial detection module: Adjusts the weights of fully connected layers in the offset prediction branch to improve offset prediction accuracy.

[0067] Training is stopped when the total offset loss L of the model no longer decreases after 5 consecutive iterations. The model obtained at this time is "a target detection model with a target feature extraction module, a target integration module, and a target detection module".

[0068] Optionally, since the four corner points of the vehicle's ground contact with the center point have a certain degree of symmetry, the accuracy of corner point prediction can be improved by using a symmetric loss function.

[0069] Figure 6 This is one of the flowcharts illustrating the parking information processing method provided in the embodiments of this application, such as... Figure 6 As shown, the parking information processing methods include: S501. Obtain the image to be detected in the target parking scene.

[0070] At the pre-set monitoring points in the target parking scene (such as indoor parking lots or open-air parking areas), a high-rise camera with a height of 5-8 meters is installed (the camera deployment height is consistent with that during the target detection model training phase to avoid a decrease in detection accuracy due to differences in viewing angle). The camera resolution is set to 1920×1080 and the frame rate is set to 15fps to ensure that the overall outline of the vehicle, the ground surface shape, and the license plate details can be clearly captured.

[0071] Configure the image acquisition trigger mechanism, including two modes: Timed trigger: Automatically acquires 1 frame of image to be detected every 30 seconds for real-time monitoring of parking space occupancy and parking status; Event trigger: When the infrared sensor at the entrance of the parking space detects a vehicle entering, or the pressure sensor under the parking space detects a pressure change, the camera is immediately triggered to acquire 3 consecutive frames of images to be detected (0.5 seconds apart) to avoid detection errors caused by blurry or occluded single frames.

[0072] After acquiring the images, the images to be detected are preprocessed, including: resolution normalization: the images are resized to 640×640 (consistent with the input size during the training of the object detection model); illumination compensation: a histogram equalization algorithm is used to adjust the brightness of images in backlight and low-light scenes to ensure that the grayscale contrast of the vehicle ground plane and the license plate area meets the detection requirements; noise removal: salt-and-pepper noise in the image is eliminated by Gaussian filtering (convolution kernel size 3×3) to avoid noise interfering with corner coordinate detection.

[0073] S502. Using a target detection model, target detection is performed on the image to be detected to obtain the detection result of the image to be detected.

[0074] The preprocessed image is input into the target detection model to perform target detection on the image to be detected, and the detection result is obtained. The detection result includes the detection box of the vehicle ground plane of the target vehicle in the image to be detected: determined by the 3D coordinates of the four corner points, forming a closed quadrilateral (coordinate format is (x,y) values ​​in the image pixel coordinate system); the license plate detection box: determined by the 3D coordinates of the four corner points, forming a closed quadrilateral; auxiliary information: vehicle type, vehicle occlusion status, license plate type, license plate occlusion status (if the occlusion status is 1, only the detection box information is retained, and it does not participate in the subsequent parking status judgment and license plate recognition, because the owner's information cannot be obtained).

[0075] S503. Based on the detection frames of the vehicle's ground contact area and the license plate, the parking status of the target vehicle is detected.

[0076] In the initialization phase of the target parking scene, images of empty parking spaces are acquired by an overhead camera. Using the same target detection model as the image to be detected, the ground boundary of the empty parking space (i.e., the "standard ground detection box" of the preset parking position) is detected, and the coordinates of its four corner points (denoted as S1(x1,y1), S2(x2,y2), S3(x3,y3), S4(x4,y4)) are recorded as the reference coordinates of the parking space.

[0077] For the ground plane detection box of the target vehicle in the image to be detected, extract the coordinates of its four corner points (denoted as C1(a1,b1), C2(a2,b2), C3(a3,b3), C4(a4,b4)). Based on the reference coordinates of the parking space and the coordinates of the four corner points of the ground plane detection box, perform parking status detection on the target vehicle to determine whether the target vehicle is parked in the preset parking position (if it is on the line, occupies an adjacent parking space, or is parked in the passage, it is not parked in the preset parking position).

[0078] The parking information processing method provided in this embodiment is based on the target detection model constructed by "a vehicle and license plate corner coordinate detection method based on multi-task learning". It addresses the core needs of intelligent parking scenarios such as vehicle parking status judgment and abnormal parking reminders. By performing target detection and data parsing on the image to be detected, it achieves accurate determination of parking location, extraction of license plate information and abnormal parking reminders. This solves the problems of ambiguous location judgment, low efficiency and high cost of manual intervention in traditional parking management, and improves the level of automated management in intelligent parking scenarios.

[0079] Figure 7 This is a second flowchart illustrating the parking information processing method provided in the embodiments of this application, as shown below. Figure 7 As shown, S503 includes: S601. Based on the detection frame of the vehicle's ground contact area, determine whether the target vehicle is parked at the preset parking position.

[0080] S602. If the target vehicle is not parked at the preset parking location, the license plate number of the target vehicle will be detected according to the license plate detection frame to obtain the license plate number of the target vehicle.

[0081] When it is determined that the target vehicle is not parked in the preset parking position, if the license plate occlusion status is 0 (unoccluded), a perspective transformation algorithm is used to correct the license plate area based on the coordinates of the four corner points of the license plate detection frame: using the four corner points of the license plate detection frame as the source coordinates, the target coordinates are set as a standard rectangle (e.g., 340 pixels wide and 100 pixels high, which conforms to the size ratio of domestic license plates); through homography matrix calculation, the skewed and distorted license plate image is converted into an upright rectangular image to eliminate the influence of perspective distortion on character recognition.

[0082] Adopt a CNN (Convolutional Neural Network) character recognition model to perform character segmentation on the corrected license plate image: segment the license plate image into 7 character regions through the vertical projection method (for example, "Jing A12345" contains 1 province abbreviation, 1 letter, and 5 numbers); Character recognition: input each character region into a pre-trained character classification model (the training set includes all province abbreviations in China, 26 English letters, and 10 numbers), output the recognition result of each character, and integrate to obtain the license plate number of the target vehicle (such as "Jing A12345").

[0083] S603. Send a prompt message to the owner user of the target vehicle according to the license plate number.

[0084] Match the recognized license plate number with the vehicle registration database in the parking lot management system to obtain the contact information of the owner user (such as mobile phone number, WeChat ID, which needs to comply with privacy protection regulations).

[0085] Generate a standardized prompt message, the content of which includes "

XX Parking Lot

[0086] Optionally, whether the target vehicle is parked normally or not, record the parking information and store it in the parking lot management system database. The recorded content includes: basic information: the acquisition time of the image to be detected, the acquisition camera number, the parking space number; vehicle information: vehicle type, license plate number (if recognized successfully), the coordinates of the vehicle ground contact detection frame; status information: the parking position determination result (normal / abnormal), the prompt message sending record (sending time, number of sends).

[0087] This record can be used for subsequent parking lot operation analysis (such as statistics of abnormal parking frequencies, analysis of the utilization rate of popular parking spaces), and can also be used as a basis for vehicle owners to query parking records.

[0088] Continue to explain the device, equipment, and storage medium for implementing the object detection model training method and parking information processing method provided in any of the above embodiments of the present application as follows. The specific implementation process and the resulting technical effects are the same as those in the corresponding method embodiments. For a brief description, for the parts not mentioned in the following embodiments, reference can be made to the corresponding content in the method embodiments.

[0089] Figure 8 It is a schematic structural diagram of the training device of the object detection model provided in an embodiment of the present application. As Figure 8 shown, the present application provides a training device for an object detection model, including: The first acquisition module 10 is used to acquire multiple sample images of the target parking scene. Each sample image contains: a bounding box for the vehicle's ground contact and a bounding box for the license plate.

[0090] The extraction module 20 is used to perform multi-scale feature extraction on each sample image using the initial feature extraction module in the initial detection model, so as to obtain feature maps of multiple scales corresponding to each sample image.

[0091] The fusion module 30 is used to fuse the feature maps of the multiple scales using the initial integration module in the initial detection model to obtain the fused feature map corresponding to each sample image.

[0092] The feature map detection module 40 is used to detect the fused feature map using the initial detection module in the initial detection model to obtain the detection result of each sample image. The detection result includes: the predicted detection box of the vehicle ground surface and the predicted detection box of the license plate.

[0093] The parameter tuning module 50 is used to tune the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the predicted detection box and labeled detection box of the vehicle ground surface and the predicted detection box and labeled detection box of the license plate, so as to obtain a target detection model with a target feature extraction module, a target integration module, and a target detection module.

[0094] Optionally, each sample image further includes: vehicle type annotation information, and the detection result further includes: vehicle type prediction information; the parameter tuning module 50 is also used to tune the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the predicted detection box and labeled detection box of the vehicle ground surface, the predicted detection box and labeled detection box of the license plate, and the predicted and labeled information of the vehicle type, so as to obtain the target detection model.

[0095] Optionally, each sample image further includes: a labeled detection box for the vehicle; the detection result further includes: a predicted detection box for the vehicle; the parameter tuning module 50 is further configured to: determine the center point of the predicted grid and the center point of the labeled grid based on the predicted detection box and the labeled detection box for the vehicle, respectively; calculate the first offset information of the vehicle's ground plane and the second offset information of the license plate based on the predicted grid center point, the vertex of the predicted detection box of the vehicle's ground plane, and the predicted detection box of the license plate; calculate the third offset information of the vehicle's ground plane and the fourth offset information of the license plate based on the labeled grid center point, the vertex of the labeled detection box of the vehicle's ground plane, and the labeled detection box of the license plate; calculate the first offset loss function value of the vehicle's ground plane based on the first offset information and the third offset information; calculate the second offset loss function value of the license plate based on the second offset information and the fourth offset information; and tune the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the first offset loss function value and the second offset loss function value to obtain a target detection model with a target feature extraction module, a target integration module, and a target detection module.

[0096] Optionally, the parameter tuning module 50 is further configured to determine, based on the center point of the predicted grid, a first predicted center point of the predicted detection frame of the vehicle ground surface and a second predicted center point of the predicted detection frame of the license plate; calculate the first offset information based on the first predicted center point and the vertex of the predicted detection frame of the vehicle ground surface; and calculate the second offset information based on the second predicted center point and the vertex of the predicted detection frame of the license plate.

[0097] Optionally, the parameter tuning module 50 is further configured to determine, based on the center point of the marked grid, the first center point of the marked detection frame of the vehicle ground surface and the second center point of the marked detection frame of the license plate; calculate the third offset information based on the first center point and the vertex of the marked detection frame of the vehicle ground surface; and calculate the fourth offset information based on the second center point and the vertex of the marked detection frame of the license plate.

[0098] Figure 9 This is a schematic diagram of the structure of the parking information processing device provided in the embodiments of this application, as shown below. Figure 9 As shown, this application provides a parking information processing device, including: The second acquisition module 60 is used to acquire the image to be detected in the target parking scene.

[0099] The target detection module 70 is used to perform target detection on the image to be detected using a target detection model, and obtain the detection result of the image to be detected. The detection result includes: the detection box of the vehicle ground surface of the target vehicle in the image to be detected and the detection box of the license plate. The target detection model is the target detection model described in any of the above embodiments.

[0100] The parking detection module 80 is used to detect the parking status of the target vehicle based on the detection frame of the vehicle's ground contact area and the detection frame of the license plate.

[0101] Optionally, the parking detection module 80 is further configured to determine whether the target vehicle is parked at a preset parking location based on the detection frame of the vehicle's ground contact point; if the target vehicle is not parked at the preset parking location, the module will detect the license plate number of the target vehicle based on the detection frame of the license plate to obtain the license plate number of the target vehicle; and send a prompt message to the owner of the target vehicle based on the license plate number.

[0102] The above-described device is used to execute the method provided in the foregoing embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0103] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more microprocessors, or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0104] Figure 10 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 10 As shown, this application also provides an electronic device, including a processor 100, a storage medium 200 and a bus 300. The storage medium stores program instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the method described in any of the above embodiments.

[0105] This application also provides a readable storage medium storing program instructions, which, when executed by a processor, implement the intelligent parking detection method described in any of the above embodiments.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0107] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0109] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training an object detection model, characterized in that, The method includes: Acquire multiple sample images of the target parking scene. Each sample image contains: a bounding box for the vehicle's ground contact and a bounding box for the license plate. The initial feature extraction module in the initial detection model is used to perform multi-scale feature extraction on each sample image to obtain feature maps of multiple scales corresponding to each sample image; The initial integration module in the initial detection model is used to fuse the feature maps at multiple scales to obtain the fused feature map corresponding to each sample image; The initial detection module in the initial detection model is used to detect the fused feature map to obtain the detection result of each sample image. The detection result includes: the predicted detection box of the vehicle ground surface and the predicted detection box of the license plate. Based on the predicted detection bounding box and labeled detection bounding box of the vehicle ground surface, and the predicted detection bounding box and labeled detection bounding box of the license plate, the parameters of the initial feature extraction module, the initial integration module and the initial detection module are adjusted to obtain a target detection model with a target feature extraction module, a target integration module and a target detection module.

2. The method according to claim 1, characterized in that, Each sample image also contains: vehicle type annotation information, and the detection result further includes: vehicle type prediction information; The step involves adjusting the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the predicted detection boxes and labeled detection boxes of the vehicle ground surface and the predicted detection boxes and labeled detection boxes of the license plate, to obtain a target detection model with a target feature extraction module, a target integration module, and a target detection module, including: Based on the predicted and labeled detection boxes of the vehicle ground surface, the predicted and labeled detection boxes of the license plate, and the predicted and labeled information of the vehicle type, the parameters of the initial feature extraction module, the initial integration module, and the initial detection module are adjusted to obtain the target detection model.

3. The method according to claim 1, characterized in that, Each sample image also includes: a labeled detection box for the vehicle; the detection result further includes: a predicted detection box for the vehicle; The step involves adjusting the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the predicted detection boxes and labeled detection boxes of the vehicle ground surface and the predicted detection boxes and labeled detection boxes of the license plate, to obtain a target detection model with a target feature extraction module, a target integration module, and a target detection module, including: Based on the predicted detection bounding box and the labeled detection bounding box of the vehicle, the center point of the predicted grid and the center point of the labeled grid are determined respectively; Based on the center point of the predicted grid, the vertex of the predicted detection box of the vehicle ground surface, and the predicted detection box of the license plate, calculate the first offset information of the vehicle ground surface and the second offset information of the license plate. Based on the center point of the labeled grid, the vertex of the labeled detection box of the vehicle ground surface, and the labeled detection box of the license plate, calculate the third offset information of the vehicle ground surface and the fourth offset information of the license plate. Based on the first offset information and the third offset information, calculate the first offset loss function value of the vehicle's ground contact surface; Calculate the second offset loss function value of the license plate based on the second offset information and the fourth offset information; Based on the first offset loss function value and the second offset loss function value, the parameters of the initial feature extraction module, the initial integration module and the initial detection module are tuned to obtain a target detection model with a target feature extraction module, a target integration module and a target detection module.

4. The method according to claim 3, characterized in that, The step of calculating the first offset information of the vehicle ground plane and the second offset information of the license plate based on the center point of the predicted grid, the vertices of the predicted detection box of the vehicle ground plane, and the predicted detection box of the license plate includes: Based on the predicted grid center point, determine the first predicted center point of the predicted detection frame of the vehicle ground surface and the second predicted center point of the predicted detection frame of the license plate. The first offset information is calculated based on the first predicted center point and the vertex of the predicted detection box of the vehicle ground surface; The second offset information is calculated based on the second predicted center point and the vertex of the predicted detection box of the license plate.

5. The method according to claim 3, characterized in that, The step of calculating the third offset information of the vehicle ground plane and the fourth offset information of the license plate based on the center point of the labeled grid, the vertex of the labeled detection box of the vehicle ground plane, and the labeled detection box of the license plate includes: Based on the center point of the marked grid, determine the first center point of the marked detection frame of the vehicle ground surface and the second center point of the marked detection frame of the license plate; The third offset information is calculated based on the first annotation center point and the vertex of the annotation detection box of the vehicle ground surface; The fourth offset information is calculated based on the second annotation center point and the vertex of the annotation detection box of the license plate.

6. A parking information processing method, characterized in that, The method includes: Acquire the image to be detected in the target parking scene; An object detection model is used to perform object detection on the image to be detected, and the detection result of the image to be detected is obtained. The detection result includes: the detection box of the vehicle ground surface of the target vehicle and the detection box of the license plate in the image to be detected; the object detection model is the object detection model according to any one of claims 1-5. The parking status of the target vehicle is detected based on the detection frames of the vehicle's ground contact area and the license plate.

7. The method according to claim 6, characterized in that, The step of detecting the parking status of the target vehicle based on the detection frame of the vehicle's ground contact surface and the detection frame of the license plate includes: Based on the detection frame of the vehicle's ground contact area, determine whether the target vehicle is parked at the preset parking position; If the target vehicle is not parked at the preset parking location, the license plate number of the target vehicle is detected according to the license plate detection frame to obtain the license plate number of the target vehicle. Based on the license plate number, a notification message is sent to the owner of the target vehicle.

8. A training device for an object detection model, characterized in that, The device includes: The first acquisition module is used to acquire multiple sample images of the target parking scene. Each sample image contains: a bounding box for the vehicle's ground contact and a bounding box for the license plate. The extraction module is used to perform multi-scale feature extraction on each sample image using the initial feature extraction module in the initial detection model, so as to obtain feature maps of each sample image at multiple scales. The fusion module is used to fuse the feature maps of the multiple scales using the initial integration module in the initial detection model to obtain the fused feature map corresponding to each sample image; The feature map detection module is used to detect the fused feature map using the initial detection module in the initial detection model to obtain the detection result of each sample image. The detection result includes: the predicted detection box of the vehicle ground surface and the predicted detection box of the license plate. The parameter tuning module is used to tune the parameters of the initial feature extraction module, the initial integration module, and the initial detection module based on the predicted detection box and labeled detection box of the vehicle ground surface and the predicted detection box and labeled detection box of the license plate, so as to obtain a target detection model with a target feature extraction module, a target integration module, and a target detection module.

9. A parking information processing device, characterized in that, The device includes: The second acquisition module is used to acquire the image to be detected in the target parking scene; The target detection module is used to perform target detection on the image to be detected using a target detection model, and obtain the detection result of the image to be detected. The detection result includes: the detection box of the vehicle ground plane of the target vehicle in the image to be detected and the detection box of the license plate; the target detection model is the target detection model according to any one of claims 1-5. The parking detection module is used to detect the parking status of the target vehicle based on the detection frame of the vehicle's ground contact area and the detection frame of the license plate.

10. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to implement the method according to any one of claims 1 to 7.

Images