Training method and use method of multi-class target detection model, medium and equipment

Abstract

The application provides a multi-class target detection model training method, a multi-class target detection model use method, a medium and equipment, and is applied to the technical field of computer vision.The application labels target object information through multi-scene image data, constructs a multi-class target detection model, and contains a feature extraction layer, a detection branch and an attribute recognition branch, and respectively identifies the features and state information of non-vehicles and vehicles.After iterative training, the model realizes accurate identification and output of multi-class targets in actual scene videos.Through multi-class target detection model training based on multi-scene labeled data, combined with the modular design and iterative optimization of the feature extraction layer and the classification and attribute recognition branch, higher accuracy and stability of multi-class targets in real-time detection are realized, thereby providing reliable support for automatic parking technology.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to training methods, usage methods, media, and devices for multi-category object detection models. Background Technology

[0002] With the development of autonomous driving technology, automated parking technology, as an important component, has received widespread attention and research. During automated parking, the vehicle needs to monitor its surroundings in real time and make corresponding decisions during the parking search, parking, and exit phases. This process requires perception algorithms to provide a 360° all-around view and to support the identification of various types of target objects, such as parking lot pillars, different types of cars, pedestrians, and wheel chocks. This information forms the basis for subsequent path planning and vehicle control.

[0003] The safety requirements of automated parking place extremely high demands on the accuracy and efficiency of perception algorithms. However, in practical applications, due to the complexity of the environment, the diversity of target types, the quality of camera equipment, and the variety of platform deployments, the optimization direction of existing surround-view perception algorithms is often unclear, making it difficult to effectively improve the overall performance of the system.

[0004] Therefore, how to improve the accuracy and stability of real-time detection of multiple target categories has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of the above problems, the present invention provides a training method, usage method, medium, and device for a multi-class object detection model that overcomes or at least partially solves the above problems. The technical solution is as follows:

[0006] A training method for a multi-class object detection model includes:

[0007] Obtain an image data training set, wherein the image data training set includes image data in multiple scenes, and the image data is labeled with at least one target object, non-vehicle grounding point label information when the target object is a non-vehicle object, vehicle status label information and vehicle grounding point label information when the target object is a vehicle object;

[0008] A multi-category target detection model is constructed, wherein the multi-category target detection model is based on a target detection network, the target detection network includes a feature extraction layer, a detection branch, a non-vehicle attribute recognition branch, and a vehicle attribute recognition branch. The feature extraction layer is used to extract basic image features, the detection branch is used to output object feature information of the target object based on the basic image features, the non-vehicle attribute recognition branch is used to output the first grounding point position of the non-vehicle object based on the identified object feature information, and the vehicle attribute recognition branch is used to output the vehicle status information and the second grounding point position of the vehicle object based on the identified object feature information.

[0009] Each image data in the image data training set is input into the multi-class object detection model for model training. The loss is calculated based on the difference between the output results of the detection branch, the non-vehicle attribute recognition branch and the vehicle attribute recognition branch and the corresponding label information of the target object. The multi-class object detection model is then iterated through the backpropagation algorithm to obtain the trained multi-class object detection model.

[0010] Optionally, the target detection network uses the YOLOv7 algorithm as its core, generates detection prior boxes based on clustering algorithms and target size, and adopts a PAFPN multi-scale structure that integrates high and low-level features to solve the problem of small target detection.

[0011] Optionally, the classifier and confidence loss function of the detection head corresponding to the detection branch adopt cross-entropy, and a specific IoU threshold is set for each type of target in the confidence loss. When the sample confidence IoU is greater than the corresponding IoU threshold, it is set to 1, otherwise it remains unchanged; in the bounding box loss calculation, Inner-CIoU is used as the loss function, a scale factor is introduced and an auxiliary bounding box is generated.

[0012] Optionally, the non-vehicle attribute recognition branch and the key point head classifier of the vehicle attribute recognition branch use cross-entropy as the loss function, the grounding point regression uses the Smooth L1 loss function, and / or, the vehicle state head classifier of the vehicle attribute recognition branch uses cross-entropy as the loss function.

[0013] Optionally, in the process of inputting each of the image data in the image data training set into the multi-class object detection model for model training, the method further includes:

[0014] The original label generated for the non-vehicle object as the pillar object is expanded by 3 pixels to the left and right and 5 pixels downward to form a new label for the pillar object, while the grounding point of the pillar remains unchanged.

[0015] Optionally, before inputting each of the image data from the image data training set into the multi-class object detection model for model training, the method further includes:

[0016] The MOSiAC and MIXUP algorithms are used to perform data augmentation on the image data training set to obtain the data-augmented image data training set.

[0017] Optionally, before inputting each of the image data from the image data training set into the multi-class object detection model for model training, the method further includes:

[0018] Filter the target objects in the image data whose area or length does not meet the corresponding target threshold.

[0019] A method for using a multi-class object detection model includes:

[0020] Obtain video frames captured by the vehicle in a real-world scenario;

[0021] The video frame is input into the trained multi-class target detection model obtained according to any of the above training methods to obtain the multi-class target recognition result output by the multi-class target detection model. The multi-class target recognition result includes the object feature information of each target object in the video frame, the first grounding point position of the non-vehicle object, the vehicle state information and the second grounding point position of the vehicle object.

[0022] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements a training method for the multi-class object detection model described in any of the preceding claims, and / or implements a method of using the multi-class object detection model.

[0023] An electronic device includes at least one processor, at least one memory connected to the processor, and a bus; wherein the processor and the memory communicate with each other via the bus; the processor is used to call program instructions in the memory to execute the training method of the multi-class object detection model described above, and / or the method of using the multi-class object detection model.

[0024] By utilizing the above technical solutions, the present invention provides a training method, usage method, medium, and device for a multi-category target detection model. This method obtains an image data training set, which includes image data from multiple scenes. The image data is labeled with at least one target object, non-vehicle grounding point label information when the target object is a non-vehicle object, vehicle status label information and vehicle grounding point label information when the target object is a vehicle object. A multi-category target detection model is constructed based on a target detection network. This network includes a feature extraction layer, a detection branch, a non-vehicle attribute recognition branch, and a vehicle attribute recognition branch. The feature extraction layer is used to extract basic image features and detect... The system employs three branches: a branch for outputting object feature information based on basic image features, a branch for non-vehicle attribute recognition for outputting the first grounding point position of non-vehicle objects based on the identified object feature information, and a branch for vehicle attribute recognition for outputting the vehicle state information and second grounding point position of vehicle objects based on the identified object feature information. Each image data point in the image training set is input into the multi-class object detection model for training. The loss is calculated based on the differences between the outputs of the detection branches, non-vehicle attribute recognition branches, and vehicle attribute recognition branches and the corresponding target object label information. The multi-class object detection model is then iterated using a backpropagation algorithm to obtain a well-trained model. Video frames captured by the vehicle in the actual scene are obtained. These video frames are input into the trained multi-class object detection model to obtain the multi-class object recognition results output by the model. These results include the object feature information of each target object in the video frame, the first grounding point position of non-vehicle objects, and the vehicle state information and second grounding point position of vehicle objects. This invention achieves higher accuracy and stability in real-time detection of multi-class targets by training a multi-class target detection model based on multi-scene labeled data, combined with modular design and iterative optimization of feature extraction layer and classification and attribute recognition branches, thus providing reliable support for automatic parking technology.

[0025] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0026] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0027] Figure 1 This diagram illustrates a flowchart of one implementation of the training method for a multi-class target detection model provided in this invention.

[0028] Figure 2 A schematic diagram illustrating the Inner-CIoU principle provided by an embodiment of the present invention is shown;

[0029] Figure 3 A schematic diagram illustrating the tag extension provided in an embodiment of the present invention is shown;

[0030] Figure 4 This diagram illustrates the network structure of the multi-category target detection model provided in an embodiment of the present invention.

[0031] Figure 5 This diagram illustrates the overall process of the training method for multi-category target detection provided in an embodiment of the present invention.

[0032] Figure 6 This diagram illustrates a flowchart of one implementation of the multi-category target detection model provided in this invention.

[0033] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention is shown. Detailed Implementation

[0034] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0035] With the rapid development of autonomous driving technology, automated parking, as one of its key technologies, has become a research hotspot. During automated parking, the vehicle needs to perceive environmental information in real time during the parking search, parking entry, and parking exit stages, and make decisions based on the perception results. This process requires the perception algorithm to have a 360° all-around view and be able to accurately identify various target objects, including parking lot pillars, different types of vehicles, pedestrians, wheel chocks, etc. The accurate detection and identification of these targets is the core foundation for path planning and safe driving control.

[0036] However, automated parking scenarios typically face complex real-world environments, such as varying lighting conditions, occlusion, and interference from multiple targets. Furthermore, the diversity of target types (e.g., vehicles and non-vehicle targets) and the specific attribute information of different targets (e.g., vehicle status, location, and grounding point) further increase the difficulty of perception algorithms. Meanwhile, limited by camera resolution, sensor performance, and computing platform hardware, existing technologies still face numerous challenges in terms of real-time performance, accuracy, and stability of target detection.

[0037] While current multi-object detection algorithms perform well in specific scenarios, their robustness and accuracy are insufficient when dealing with multiple object categories, especially in dynamic and complex environments. Furthermore, the consistency and accuracy of outputting attribute information for different object categories (such as the vehicle's contact point location or state, and the specific location information of non-vehicle objects) fall short of ideal levels. These issues lead to bottlenecks in the system's detection accuracy and stability, impacting the overall safety and reliability of automated parking systems. Therefore, a technology that can balance accuracy, efficiency, and robustness in multi-object detection is urgently needed to optimize the perception and decision-making capabilities during automated parking.

[0038] Based on this, this embodiment of the invention provides a training method and application method for a multi-category target detection model. First, an image data training set is acquired, containing image data from multiple scenes and annotating target objects and their corresponding grounding points or state information, such as non-vehicle grounding point information for non-vehicle objects and vehicle state and grounding point information for vehicle objects. Second, a multi-category target detection model is constructed. This model consists of a feature extraction layer, a detection branch, and non-vehicle and vehicle attribute recognition branches, effectively extracting basic image features and outputting detailed feature information of the target objects. It is evident that by training the model on the image data training set and inputting actual scene video frames, the model can output multi-category target recognition results, including object features, grounding point locations of non-vehicle objects, and state and grounding point locations of vehicle objects, thereby improving the accuracy and stability of multi-category target detection and providing reliable support for automatic parking technology.

[0039] like Figure 1 The diagram shows a flowchart of one implementation of the training method for a multi-class object detection model provided in this invention. The method may include:

[0040] S100. Obtain an image data training set, wherein the image data training set includes image data in multiple scenarios, and the image data is labeled with at least one target object, non-vehicle grounding point label information when the target object is a non-vehicle object, vehicle status label information and vehicle grounding point label information when the target object is a vehicle object.

[0041] The image training set is a collection of images used to train a multi-class object detection model. Each image contains annotation information to help the model learn to recognize specific target objects and their attributes. The images in the training set are typically collected from real-world scenes, covering various environments and conditions to improve the model's generalization ability.

[0042] In this context, the target object refers to the specific object that is labeled in the image data and needs to be recognized by the model. The target object represents what the model needs to detect and recognize in practical applications.

[0043] Optionally, in this embodiment of the invention, the target object can be marked in the image data in the form of a closed rectangle, and non-vehicle grounding point label information and vehicle grounding point label information can be marked in the form of solid dots. The vehicle grounding point label information includes the tire grounding point location information and grounding point location status information of the vehicle object. The grounding point location status information includes the visible status and the predicted status. The vehicle status label information includes the normal status, the truncated status and the occluded status.

[0044] Optionally, the target objects labeled in the image data training set provided in this embodiment of the invention include 28 categories of targets, such as cars, buses, trucks, construction vehicles, pedestrians, bicycles, cyclists, tricycles, standing ground locks, traffic cones, wheel chocks, water barriers, fire hydrants, shopping carts, lying ground locks, parking lot pillars, warning signs, trash cans, balance bikes, strollers, shopping carts, warning posts, spherical obstacles, scooters, flatbed carts, parking poles, speed bumps, and fire boxes. Each target object is labeled with its location, category, and grounding point label information.

[0045] Non-vehicle objects refer to all other objects in the image data that are not vehicles, such as pedestrians, pillars, and buildings. Labeling non-vehicle objects in the image data training set helps the model distinguish between vehicle and non-vehicle targets.

[0046] Vehicle objects refer to objects in the image data that belong to the vehicle category, including various types of vehicles such as cars, trucks, and motorcycles. Labeling vehicle objects in the image data training set helps the model specifically learn to recognize vehicles and their attributes.

[0047] The ground contact label information refers to the specific location information of the target object in contact with the ground. For non-vehicle and vehicle-type objects, labeling the ground contact information helps the model clearly identify the object's location in space. For non-vehicle target objects (such as pedestrians), the specific location information of their contact with the ground is labeled (ground contact label). For vehicle-type target objects, the ground contact status of the vehicle is further labeled, including: visible points (the ground contact point of the vehicle tire is clearly visible) and predicted points (the ground contact point is not visible due to the tire being obscured or other factors, and the tire's ground contact point needs to be predicted by the model).

[0048] Among them, vehicle state label information is used to describe the specific state of vehicle objects in image data, which helps the model learn how to correctly identify and judge vehicles under different occlusion and viewpoint conditions during training. For vehicle targets such as cars, trucks, buses and construction vehicles, their state information is further labeled, including: Normal (the vehicle is not occluded or only partially occluded), Truncated (the vehicle is located at the edge of the image and is partially truncated), and Crowded (the vehicle is occluded by other targets).

[0049] The image data training set provided in this embodiment of the invention provides the model with rich target object information, while covering a variety of scenes and complex conditions, which helps to improve the accuracy and robustness of the model in target recognition and attribute analysis.

[0050] S110. Construct a multi-category object detection model, wherein the multi-category object detection model is based on an object detection network, which includes a feature extraction layer, a detection branch, a non-vehicle attribute recognition branch, and a vehicle attribute recognition branch. The feature extraction layer is used to extract basic image features, the detection branch is used to output object feature information of the target object based on the basic image features, the non-vehicle attribute recognition branch is used to output the first grounding point position of the non-vehicle object based on the identified object feature information, and the vehicle attribute recognition branch is used to output the vehicle status information and the second grounding point position of the vehicle object based on the identified object feature information.

[0051] Among them, the multi-class object detection model is a deep learning-based computer vision model used to detect and identify multiple categories of target objects in images. This model can not only accurately identify the category of the target, but also locate its specific position in the image and extract relevant attribute information (such as vehicle status and grounding point location). Optionally, the multi-class object detection model can be built based on the YOLOv7 framework.

[0052] The object detection network is the core structure for building multi-class object detection models. It is a deep learning framework responsible for extracting features from input images, detecting target objects, and extracting relevant attribute information of the targets. The object detection network consists of multiple functional modules, each performing a different task.

[0053] The feature extraction layer is the foundation of the object detection network, and its main task is to extract basic image features from the input image. These features include information such as edges, textures, and shapes, providing high-quality image feature representations for subsequent detection and attribute recognition tasks. Common feature extraction layers are based on convolutional neural networks (CNNs), such as ResNet (Residual Network) and VGG (Visual Geometry Group).

[0054] Optionally, to adapt to real-world deployment environments, the model employs a lightweight backbone network for efficient feature extraction. Building upon this, a PAFPN (Path Aggregation Feature Pyramid Network) module is combined to fuse multi-scale feature information, ensuring good detection performance for targets of all sizes. The feature extraction layer is responsible for extracting the basic features of the image, providing support for subsequent tasks.

[0055] The detection branch is the module in the object detection network responsible for detecting target objects. Based on the basic image features output by the feature extraction layer, the detection branch generates object feature information of the target object, such as the object's category (e.g., vehicle or pedestrian) and location (e.g., bounding box coordinates).

[0056] The non-vehicle attribute recognition branch is specifically designed to process detected non-vehicle targets. Based on the object feature information provided by the detection branch, this branch further analyzes and outputs the attribute information of non-vehicle objects, particularly the location of their first contact point (i.e., the specific location where the non-vehicle target contacts the ground). To handle non-vehicle targets, the non-vehicle attribute recognition branch shares a feature extraction network with the detection branch. It also introduces a KeypointHead module by extracting region features corresponding to positive samples, focusing on predicting the target's first contact point location and providing a foundation for subsequent ranging tasks.

[0057] The vehicle attribute recognition branch processes vehicle-type target objects. Based on the object feature information provided by the detection branch, it further analyzes and outputs two core attribute information for vehicle-type objects: vehicle state information (e.g., whether the vehicle is occluded, Normal, Truncated, or Crowded) and the second ground contact point location (the specific location where the vehicle tire contacts the ground, which may include a visible ground contact point or a ground contact point that needs to be predicted). For vehicle-type targets, the vehicle attribute recognition branch also shares the feature extraction network, using the CarStateHead module to determine the vehicle state (e.g., vehicle direction, dynamic information, etc.) and output the vehicle's second ground contact point location. The design of the vehicle attribute recognition branch aims to solve the problem of vehicle state recognition under different viewpoints, ensuring the accuracy and rationality of viewpoint selection.

[0058] This invention relates to a multi-class target detection model based on YOLOv7, focusing on the optimization and improvement of surround-view perception tasks. By introducing various optimization strategies, this model aims to improve the detection accuracy and stability of targets at different distances and of various types within the vehicle's 360° surround-view range. Based on modular design and strategy optimization, the model can efficiently handle multi-class target detection tasks, achieving comprehensive judgment of target state and position while ensuring real-time performance and high accuracy. It is not only easy to deploy but also widely applicable to scenarios such as automatic parking, providing strong technical support for intelligent driving systems.

[0059] S120. Input each image data in the image data training set into the multi-class object detection model for model training. Calculate the loss based on the difference between the output results of the detection branch, the non-vehicle attribute recognition branch, and the vehicle attribute recognition branch and the corresponding target object label information. Iterate the multi-class object detection model through the backpropagation algorithm to obtain a well-trained multi-class object detection model.

[0060] During the model training phase, this embodiment of the invention inputs each image from the image data training set into the multi-class object detection model. The model extracts the basic features of the image through a feature extraction layer and outputs corresponding prediction results through detection branches, non-vehicle attribute recognition branches, and vehicle attribute recognition branches, respectively. The goal of training is to optimize the model parameters based on the differences between these output results and the true labels, so that the output results are as close to the true labels as possible, thereby enabling more accurate detection and classification of target objects.

[0061] During the model training phase, this embodiment of the invention quantifies the difference between the model's output and the true label using a loss function. The detection branch loss includes classification loss and bounding box regression loss. Classification loss measures the difference between the predicted target category and the true category, and can use methods such as Cross Entropy loss or Focal Loss. Bounding box regression loss compares the difference between the predicted bounding box and the true bounding box, and can use cross entropy loss, L1 loss, L2 loss, or IoU (Intersection over Union) or Inner-CIoU (Inner Complete Intersection over Union). The non-vehicle attribute recognition branch loss is mainly the grounding point location loss, which measures the difference between the model's predicted first grounding point coordinates and the true grounding point coordinates, and can be measured based on the Smooth L1 loss function. The vehicle attribute recognition branch loss includes vehicle state loss and second grounding point location loss. Vehicle state loss measures the difference between the predicted vehicle state and the true state, and can use cross entropy loss. The loss for the second grounding point location can also be calculated using the Smooth L1 loss function, comparing the coordinate differences between the predicted and actual grounding points. In this embodiment of the invention, the total loss can be obtained by weighted summation of the above-mentioned losses. It is understood that the weighting coefficients for each loss can be set according to actual needs.

[0062] After calculating the total loss, this invention utilizes the chain rule of backpropagation to calculate the partial derivatives (gradients) of the loss function with respect to the model parameters. The gradient describes the sensitivity of the loss function to changes in each parameter. Based on the calculated gradients, optimization algorithms (such as SGD (Stochastic Gradient Descent) and Adam (Adaptive Moment Estimation)) are used to update the model parameters to reduce the loss function.

[0063] In this embodiment of the invention, after updating the model parameters, the above steps can be repeated until preset training conditions are met. The preset training conditions may be reaching the maximum number of iterations or the loss value no longer decreasing significantly.

[0064] This invention provides a training method for a multi-class target detection model. It constructs a high-quality image data training set containing multi-scene and multi-class annotation information and designs a multi-class target detection model. This model integrates multiple branches, including feature extraction, target detection, non-vehicle attribute recognition, and vehicle attribute recognition, to achieve comprehensive detection and classification of target objects. By optimizing the model using the backpropagation algorithm, it can accurately identify the grounding points of non-vehicle objects and the states and grounding point information of vehicle objects. This technology effectively improves the accuracy and stability of multi-class target detection in real-time, providing accurate environmental perception capabilities for automatic parking scenarios. It ensures that vehicles can safely and efficiently complete parking tasks in complex parking environments, thereby significantly improving the reliability and adaptability of automatic parking systems.

[0065] Optionally, in the above Figure 1 Based on one or more corresponding embodiments, in another optional embodiment provided by the present invention, the target detection network uses the YOLOV7 algorithm as its core, generates detection prior boxes based on clustering algorithms and target size, and adopts a PAFPN multi-scale structure that integrates high and low layer features to solve the problem of small target detection.

[0066] Based on the efficiency and real-time performance of the YOLOv7 algorithm, this invention optimizes the network architecture of the target detection network, which can significantly improve detection accuracy while ensuring detection speed.

[0067] In this invention, the detection prior boxes are a set of bounding boxes initially set by the object detection network, serving as an important reference for matching the real object location during model training. This embodiment of the invention can use clustering algorithms (such as K-Means or K-Means++) to generate prior boxes based on the target object size distribution, generating a more realistic frame configuration according to the target object size distribution in the image data training set. In this way, the model can effectively adapt to targets of different scales, thereby improving detection accuracy, especially performing better in small object detection.

[0068] PAFPN (Path Aggregation Feature Pyramid Network) is a multi-scale feature fusion network structure for object detection. This invention, by introducing the PAFPN multi-scale structure, can better preserve the detailed features of small targets while enhancing global contextual information, thus effectively solving the problem of feature loss in small target detection.

[0069] This invention improves the initial box matching ability for small targets by generating prior boxes through clustering. Combined with multi-scale feature fusion of the PAFPN network structure, it can significantly improve the detection accuracy and stability of the model for small targets.

[0070] Optionally, in the embodiments of the present invention, the loss function of the classifier and confidence of the detection head corresponding to the detection branch adopts cross-entropy, and a specific IoU threshold is set for each type of target in the confidence loss. When the sample confidence IoU is greater than the corresponding IoU threshold, it is set to 1, otherwise it remains unchanged; in the bounding box loss calculation, Inner-CIoU is used as the loss function, a scale factor is introduced and an auxiliary bounding box is generated.

[0071] This invention allows for the setting of specific IoU thresholds for different target categories, enabling the model to better adapt to the characteristics of different targets and improve its generalization ability across multiple target categories. Through Inner-CIoU and auxiliary bounding boxes, the model's optimization of small and complex targets becomes more accurate, reducing focus on unimportant details and thus enhancing its adaptability to various scenarios. Furthermore, due to the efficient alignment mechanism of Inner-CIoU, the matching error between the predicted and ground truth bounding boxes can be measured more accurately, allowing the model to converge to a reasonable matching box more quickly. The auxiliary bounding box provides additional optimization objectives, helping the bounding box regression task to more rapidly approximate the ground truth bounding box.

[0072] Figure 2 The diagram shown is an illustration of the Inner-CIoU principle provided in an embodiment of the present invention. The formula for calculating the internal complete crossover and union ratio loss is as follows:

[0073]

[0074]

[0075]

[0076] in, For internal complete intersection and union ratio loss; For complete intersection and union, the loss is compared; Intersection over Union (IoU) is used to measure the degree of overlap between the predicted bounding box and the ground truth bounding box. The internal intersection-union ratio measures the ratio of the area of ​​the red region (the internal part of the intersection between the predicted box and the ground truth box) to the area of ​​the yellow region (the entire union of the predicted box and the ground truth box). Center point of the prediction box Center point of the real frame Examples of Euclidean squares between them; The diagonal length of the smallest closure box containing both the predicted and ground truth boxes is squared. This is a weighting factor used to balance the loss of shape consistency; The aspect ratio difference between the predicted bounding box and the ground truth bounding box.

[0077] By providing these optimization strategies in the detection branch, the embodiments of the present invention enable multi-class object detection models to accelerate the regression speed of bounding boxes while improving generalization ability, thereby demonstrating more efficient and accurate object detection capabilities in practical application scenarios.

[0078] Optionally, in the above Figure 1 Based on one or more corresponding embodiments, in another optional embodiment provided by the present invention, the key point classifiers of the non-vehicle attribute recognition branch and the vehicle attribute recognition branch adopt cross-entropy as the loss function, the grounding point regression adopts the Smooth L1 loss function, and / or, the vehicle state head classifier of the vehicle attribute recognition branch adopts cross-entropy as the loss function.

[0079] The formula for calculating the Smooth L1 loss function is as follows:

[0080]

[0081] in, This represents the value of the SmoothL1 loss, used to measure the difference between the predicted and actual values; This represents the error between the predicted value and the actual value.

[0082] In this embodiment of the invention, the detection task of the key point head classifier and the recognition task of the vehicle state head classifier are optimized simultaneously within the same training cycle. The design of the loss function allows the two tasks to be collaboratively optimized by sharing weights and gradients. By jointly optimizing the loss functions of each task, the model can improve both detection accuracy and attribute recognition performance in a single training cycle, avoiding the repetitive steps of separate training and accelerating the gradient calculation and weight optimization process.

[0083] By rationally selecting the loss function and jointly optimizing the detection and recognition tasks, the embodiments of the present invention significantly improve the training speed of the model while ensuring the efficiency and accuracy of the detection and recognition tasks.

[0084] Optionally, in the above Figure 1 Based on one or more corresponding embodiments, in another optional embodiment provided by the present invention, during the process of inputting each image data in the image data training set into the multi-class object detection model for model training, the method may further include:

[0085] The original label generated for the pillar object from a non-vehicle object is expanded 3 pixels to the left and right and 5 pixels downward to form a new label for the pillar object, while the grounding point of the pillar remains unchanged.

[0086] In order to optimize the problem of the detection box being biased upwards when detecting column-like targets at close range, the embodiments of the present invention provide the following solutions: Figure 3As shown, when generating the label of a column object, the original label ( Figure 3 Extend the left and right edges of the red box (in the image) by 3 pixels each, and extend the bottom edge downwards by 5 pixels to obtain the new label for the pillar object. Figure 3 (in the yellow box), while maintaining the grounding point of the column ( Figure 3 The position of the red dot in the image remains unchanged. This design is mainly based on empirical analysis and test results of the characteristics of column-type targets. Considering that columns are long, vertical targets, their detection boxes often shift upwards at close range. Therefore, the labels are appropriately adjusted downwards during the training phase to improve the model's detection accuracy at the bottom of the columns. In order to balance the overall integrity of the labels, the left and right boundaries are also expanded in a balanced manner.

[0087] It is important to note that adjustments to the labels (such as the specific number of pixels to expand downwards) need to be adaptively adjusted according to the characteristics of the pillar targets to ensure that the model's detection performance is maximized in different application scenarios.

[0088] This invention expands the labels of column-like objects by 3 pixels horizontally and 5 pixels downwards, while keeping the ground point unchanged. This helps optimize the model's localization accuracy and the rationality of the detection box for column targets. The expanded labels better match the characteristics of elongated column targets, guiding the model to focus on the bottom region of the target and reducing the problem of detection boxes being biased upwards in close-range scenes. Simultaneously, the horizontal expansion enhances the model's ability to learn edge features, improving the completeness and discriminative power of target feature representation. This label optimization tailored to specific target characteristics not only significantly improves the model's accuracy in detecting columns but also helps the model more stably handle targets of different shapes and characteristics in multi-class target detection tasks, reducing false positives and false negatives, thereby comprehensively improving the overall accuracy and stability of real-time detection.

[0089] Optionally, in the above Figure 1 Based on one or more corresponding embodiments, in another optional embodiment provided by the present invention, before inputting each of the image data in the image data training set into the multi-class object detection model for model training, the method may further include:

[0090] The MOSiAC and MIXUP algorithms are used to perform data augmentation on the image data training set to obtain the data-augmented image data training set.

[0091] Among them, the MOSiAC algorithm (Mosaic Augmentation) is used to generate diverse image training samples by randomly cropping and scaling four images and stitching them into a new image, combined with adjustments to the target bounding box. The MOSiAC algorithm generates image data of highly complex scenes in the image data training set for model training, thereby improving the model's adaptability to complex environments with multiple targets.

[0092] The MIXUP algorithm (MixUp Data Augmentation) generates image training samples by linearly weighting and mixing two images and their corresponding labels. Through linear mixing of images and labels, the MIXUP algorithm generates transitional samples, further enriching the distribution of image data used for model training within the image training set.

[0093] The embodiments of this invention improve the robustness and generalization of the model through image spatial stitching and feature linear mixing, respectively. Using these two algorithms on the training set can effectively generate more diverse samples, helping the model better adapt to complex environments, thereby improving the accuracy and stability of multi-class object detection.

[0094] Optionally, in the above Figure 1 Based on one or more corresponding embodiments, in another optional embodiment provided by the present invention, before inputting each image data in the image data training set into the multi-class object detection model for model training, the method may further include:

[0095] Filter out target objects in image data whose area or length does not meet the corresponding target threshold.

[0096] In this embodiment of the invention, when filtering target objects in image data, different area or length thresholds are set for each type of target. Target objects that do not meet the threshold requirements (such as those with too small an area or too short a length) are processed. Specifically, this involves filling these target regions (e.g., using black or white fill) to occlude these non-compliant target regions, thereby reducing their interference with the image feature extraction process. Simultaneously, by deleting the corresponding target labels, filtering of extremely small targets can be effectively achieved.

[0097] By filtering out extremely small target objects, this invention helps the model focus on more meaningful target objects, thereby improving the model's training efficiency and detection performance.

[0098] Figure 4The diagram shows the network structure of the multi-class target detection model provided in this embodiment of the invention. The input to the multi-class target detection model is a single-frame image. Preprocessing operations are performed on the input image before the feature extraction layer, including adjusting the size of the pillars, filtering small targets, and data augmentation. The general feature extraction layer uses Inner-CIoU to further improve the accuracy of bounding box regression, enabling the model to more accurately predict the target's location and extract features from the input image to generate visual feature representations of multi-class targets. Then, the PAFPN feature pyramid network is used to fuse feature layer information at different scales to detect targets of different sizes, thereby improving the model's ability to detect multi-class targets. After feature extraction and fusion, the network completes target category recognition, ground point coordinate prediction, and vehicle state classification tasks through a series of fully connected layers. A Softmax classifier is used to predict the category of features in each region, outputting the category of the detected target. The SmoothL1 loss function is used to regress and predict the ground point coordinates. A Softmax classifier is used to classify the vehicle state, outputting the vehicle's motion state. Through the above process, the model ultimately generates three outputs: target category recognition, ground point coordinates, and vehicle state. The network structure of the multi-class target detection model provided in this invention, based on powerful feature extraction capabilities (such as PAFPN) and optimized loss functions (such as Inner-CIoU, SmoothL1, etc.), can achieve higher accuracy and stability when detecting multi-class targets. By comprehensively recognizing target category, grounding point location, and vehicle state, this model provides comprehensive and reliable technical support for application scenarios such as intelligent driving, automatic parking, and traffic monitoring.

[0099] Figure 5The diagram illustrates the overall flow of the multi-class object detection training method provided in this embodiment of the invention. This embodiment first acquires raw data through data collection, including images and their corresponding annotations, such as bounding box annotations, grounding point annotations, and vehicle status annotations. In the data augmentation stage, various data augmentation operations are performed on the acquired images and annotation information to improve the robustness and generalization ability of the model. Augmentation operations include: resize: adjusting the image size to fit the image size input to the model; MOSAiC: using stitching augmentation technology to stitch multiple images into one, increasing the diversity of training samples; Mixup: linearly mixing two images and their labels to generate new training samples, smoothing the model's decision boundaries; random flip: randomly flipping the image (horizontally or vertically) to expand the data samples; random Affine transformation: performing geometric transformations on the image through affine transformations (such as rotation, translation, scaling, etc.); HSVrandomAug: randomly adjusting the image's hue, saturation, and brightness to enhance the model's adaptability to changes in lighting. These data augmentation operations diversify the training data, improving the model's adaptability to different scenes and complex environments. The data-enhanced images and annotation information are input into the target detection and classification module to complete the target detection, grounding point detection, and vehicle status classification tasks. After target detection and classification, the system outputs structured information including target information, vehicle status, grounding point, and vehicle grounding point category. This invention, through various data enhancement methods and target detection technologies, effectively improves the accuracy and stability of real-time detection of multiple target categories, providing precise environmental perception capabilities for automatic parking scenarios.

[0100] like Figure 6 The diagram illustrates a flowchart of one implementation of the multi-category target detection model provided in this invention. The method may include:

[0101] S600: Obtain video frames captured by the vehicle in a real-world scenario.

[0102] In this context, "real-world scenario" can refer to naturally occurring environments or application scenarios. Examples include: city roads, highways, industrial sites, above-ground parking lots, underground parking garages, and parking towers.

[0103] Specifically, in this embodiment of the invention, the vehicle's onboard camera can collect video of the vehicle's surrounding environment in a real-world scenario, and then extract real-time video frames.

[0104] S610. Input the video frame into the trained multi-class target detection model obtained by the training method provided in the embodiment of the present invention, and obtain the multi-class target recognition result output by the multi-class target detection model. The multi-class target recognition result includes the object feature information of each target object in the video frame, the first grounding point position of non-vehicle objects, and the vehicle status information and second grounding point position of vehicle objects.

[0105] This invention, through the acquisition of video frames in real-world scenarios and inputting them into an optimized, trained multi-class target detection model, achieves accurate multi-class target recognition results. The output includes detailed information such as the target object's category, location, and grounding point, as well as vehicle status information. This achieves high accuracy and stability in target detection, effectively identifying vehicles, obstacles, and the surrounding environment in parking lots. It provides accurate environmental perception and reliable decision support for automated parking technology, contributing to improved safety and efficiency.

[0106] Although the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous.

[0107] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0108] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements a training method for the multi-class object detection model and / or a method for using the multi-class object detection model.

[0109] This invention provides a processor for running a program, wherein the program executes a training method for the multi-class object detection model and / or a method for using the multi-class object detection model.

[0110] like Figure 7As shown, this embodiment of the invention provides an electronic device 1000, which includes at least one processor 1001, at least one memory 1002 connected to the processor 1001, and a bus 1003. The processor 1001 and the memory 1002 communicate with each other via the bus 1003. The processor 1001 is used to call program instructions in the memory 1002 to execute the training method of the multi-class object detection model described above, and / or the method of using the multi-class object detection model. The electronic device in this document can be a server, PC, PAD, mobile phone, etc.

[0111] The present invention also provides a computer program product, which, when executed on an electronic device, is adapted to perform the steps of a training method for initializing the multi-class object detection model and / or a method for using the multi-class object detection model.

[0112] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, electronic devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0113] In a typical configuration, an electronic device includes one or more processors (CPUs), memory, and a bus. The electronic device may also include input / output interfaces, network interfaces, etc.

[0114] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM, and memory includes at least one memory chip. Memory is an example of computer-readable media.

[0115] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0116] In the description of this invention, it should be understood that if the terms "upper", "lower", "front", "rear", "left" and "right" are used to indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the position or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0117] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0118] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0119] The above are merely embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the present invention.

Claims

1. A training method for a multi-class object detection model, characterized in that, The method comprises the following steps: obtain an image data training set, wherein the image data training set comprises image data under multiple scenes, and the image data is labeled with at least one target object, non-vehicle grounding point label information when the target object is a non-vehicle object, and vehicle state label information and vehicle grounding point label information when the target object is a vehicle object; build a multi-class target detection model based on a YoloV7 framework, wherein the multi-class target detection model is composed of a target detection network, the target detection network comprises a feature extraction layer, a detection branch, a non-vehicle attribute recognition branch, and a vehicle attribute recognition branch, the feature extraction layer is used to extract image basic features, the detection branch is used to output object feature information of the target object based on the image basic features, the non-vehicle attribute recognition branch is used to output a first grounding point position of the non-vehicle object according to the identified object feature information, and the vehicle attribute recognition branch is used to output vehicle state information and a second grounding point position of the vehicle object according to the identified object feature information; input each image data in the image data training set into the multi-class target detection model for model training, calculate a classification loss and a bounding box regression loss according to the difference between the output result of the detection branch and the label information of the corresponding target object, calculate a grounding point position loss according to the difference between the output result of the non-vehicle attribute recognition branch and the label information of the corresponding target object, calculate a vehicle state loss and a second grounding point position loss according to the difference between the output result of the vehicle attribute recognition branch and the label information of the corresponding target object, obtain a total loss according to the classification loss, the bounding box regression loss, the grounding point position loss, the vehicle state loss, and the second grounding point position loss, and iteratively train the multi-class target detection model through a back propagation algorithm to obtain a trained multi-class target detection model.

2. The training method of claim 1, wherein, The target detection network takes YoloV7 algorithm as the core, generates detection prior boxes based on a clustering algorithm and target size, and adopts a PAFPN multi-scale structure for fusing high and low layer features to solve the small target detection problem.

3. The training method of claim 1, wherein, The loss function of the classifier and confidence of the detection head corresponding to the detection branch adopts cross entropy, and a specific IoU threshold is set for each target in the confidence loss, when the sample confidence IoU is greater than the corresponding IoU threshold, it is set to 1, otherwise it remains unchanged; in the bounding box loss calculation, Inner-CIoU is used as the loss function, a scale factor is introduced, and an auxiliary bounding box is generated.

4. The training method of claim 1, wherein, The key point head classifier of the non-vehicle attribute recognition branch and the vehicle attribute recognition branch adopts cross entropy as the loss function, the grounding point regression adopts Smooth L1 loss function, and / or the vehicle state head classifier of the vehicle attribute recognition branch adopts cross entropy as the loss function.

5. The training method of claim 1, wherein, In the process of inputting each image data in the image data training set into the multi-category target detection model for model training, the method further comprises: The original label generated by the non-vehicle object as a column object is extended by 3 pixels to the left and right and 5 pixels downward to form a new label of the column object, and the column grounding point remains unchanged.

6. The training method of claim 1, wherein, Before inputting each image data in the image data training set into the multi-category target detection model for model training, the method further comprises: The image data training set is processed by using the MOSiAC algorithm and the MIXUP algorithm for data enhancement to obtain an image data training set after data enhancement.

7. The training method according to any one of claims 1 to 6, characterized in that, Before inputting each image data in the image data training set into the multi-category target detection model for model training, the method further comprises: Filtering the target objects in the image data whose areas or lengths do not meet the corresponding target threshold.

8. A method for using a multi-class target detection model, characterized in that, Comprise: Obtaining a video frame collected in an actual scene by a vehicle; Inputting the video frame into the trained multi-category target detection model obtained by the training method according to any one of claims 1 to 7 to obtain a multi-category target recognition result output by the multi-category target detection model, wherein the multi-category target recognition result comprises object feature information of each target object in the video frame, a first grounding point position of the non-vehicle object, vehicle state information of the vehicle object, and a second grounding point position.

9. A computer-readable storage medium having stored thereon a program, characterized in that, The program is executed by a processor to implement the training method of the multi-category target detection model according to any one of claims 1 to 7, and / or implement the use method of the multi-category target detection model according to claim 8.

10. An electronic device, comprising: The electronic device comprises at least one processor, at least one memory connected with the processor, and a bus; wherein the processor, the memory and the bus complete mutual communication through the bus; the processor is used to call program instructions in the memory to execute the training method of the multi-category target detection model according to any one of claims 1 to 7, and / or the use method of the multi-category target detection model according to claim 8. The electronic device comprises at least one processor, at least one memory connected with the processor, and a bus; wherein the processor, the memory and the bus complete mutual communication through the bus; the processor is used to call program instructions in the memory to execute the training method of the multi-category target detection model according to any one of claims 1 to 7, and / or the use method of the multi-category target detection model according to claim 8.

Images