Pedestrian detection model training method, pedestrian detection method, training device and equipment
By adding center point prediction and loss function training to the pedestrian detection model, the accuracy problem of pedestrian detection in complex environments is solved, and high-precision pedestrian detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2026-04-03
AI Technical Summary
Existing pedestrian detection algorithms struggle to accurately detect pedestrians due to factors such as lighting, viewing angle, posture, and clothing, leading to false alarms and missed detections in ADAS systems and impacting safety.
By adding the prediction of pedestrian center points to the pedestrian detection model, center point labels are generated using pedestrian bounding box labels, and a loss function containing pedestrian prediction boxes, prediction categories, and center point loss functions is constructed for model training to improve detection accuracy.
Without increasing annotation costs and computational load, the robustness and detection capability of the pedestrian detection model were significantly improved, the distinction between pedestrians and background was enhanced, and the detection accuracy was increased.
Smart Images
Figure CN116758510B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and in particular to a pedestrian detection model training method, pedestrian detection method, training device and equipment. Background Technology
[0002] External obstacle detection is a crucial component of Advanced Driver Assistance Systems (ADAS) and is essential for the overall safety of the system. External obstacles include passenger cars, vans, buses, pedestrians, animals, and traffic cones.
[0003] Pedestrian detection among obstacles is extremely important. Pedestrian detection is a technology that uses computer vision algorithms to determine the presence of pedestrians in images or video sequences and to accurately locate their position and size within the image. Pedestrian detection technology can be combined with pedestrian tracking, pedestrian re-identification, and other technologies for application in fields such as artificial intelligence systems, autonomous driving, and intelligent video surveillance. However, the appearance of pedestrians is easily affected by factors such as lighting, viewing angle, posture, clothing, scale, and occlusion. For example, pedestrians often cannot be distinguished from the background under different lighting conditions or with different clothing, and existing pedestrian detection algorithms struggle to accurately detect pedestrians in such scenarios.
[0004] Due to the complexity of pedestrian size and the variability of the surrounding environment, obstacle detection models often perform poorly when detecting pedestrians, leading to false alarms and missed detections in ADAS systems, which can easily cause safety issues. Therefore, there is an urgent need in this field for a method to improve pedestrian detection capabilities. Summary of the Invention
[0005] In view of the problems existing in the prior art, the pedestrian detection model training method, pedestrian detection method, training device and equipment provided by the present invention greatly improve the model's ability to detect pedestrians by adding the prediction of the pedestrian center point.
[0006] This invention provides a method for training a pedestrian detection model, comprising:
[0007] Obtain pedestrian image data and corresponding annotation data, wherein the annotation data includes: pedestrian annotation box labels, pedestrian center point labels and category labels, and the pedestrian center point labels are obtained based on the pedestrian annotation box labels;
[0008] Based on the pedestrian image data, the pedestrian detection model outputs the pedestrian detection training results, which include: pedestrian prediction boxes, the prediction categories corresponding to the pedestrian prediction boxes, and the pedestrian center prediction points.
[0009] Based on the pedestrian detection training results and the labeled data, a loss value is generated through a loss function, and the pedestrian detection model is updated until convergence to obtain the trained pedestrian detection model.
[0010] According to a pedestrian detection model training method provided by the present invention, the pedestrian center point label is obtained based on the pedestrian bounding box label, including:
[0011] Obtain the coordinates of the four corner points corresponding to the pedestrian annotation box label;
[0012] Based on the coordinates of the four corner points, the coordinates of the center point are generated and used as the center point label of the pedestrian.
[0013] According to a pedestrian detection model training method provided by the present invention, the loss function includes: a pedestrian prediction box loss function, a prediction category loss function, a center point loss function, and a center point comparison loss function. Correspondingly, the loss value includes: the prediction box loss value of the pedestrian prediction box loss function, the cross-entropy loss value of the prediction category loss function, the center point loss value of the center point loss function, and the center point comparison loss value of the center point comparison loss function.
[0014] According to a pedestrian detection model training method provided by the present invention, generating the loss value includes:
[0015] The corresponding center point is obtained based on the predicted pedestrian prediction box and used as the first center point;
[0016] The center point comparison loss value is calculated based on the first center point and the predicted pedestrian center point.
[0017] According to a pedestrian detection model training method provided by the present invention, generating the loss value includes:
[0018] Calculate the intersection-union ratio (IUU) between the predicted pedestrian bounding box and the pedestrian labeled bounding box, using it as the predicted box loss value, and...
[0019] The cross-entropy loss value is calculated based on the predicted category corresponding to the pedestrian prediction box and the category label.
[0020] According to a pedestrian detection model training method provided by the present invention, generating the loss value includes:
[0021] The center point loss value is calculated based on the predicted pedestrian center point and the pedestrian center point label.
[0022] The present invention also provides a pedestrian detection method, comprising:
[0023] Acquire the image to be detected that contains pedestrians;
[0024] The image to be detected is input into a pre-trained pedestrian detection model to obtain the pedestrian detection result output by the pedestrian detection model. The pedestrian detection result includes a pedestrian detection box and a pedestrian center point. The pre-trained pedestrian detection model is trained by any of the pedestrian detection model training methods described above.
[0025] The present invention also provides a pedestrian detection model training device, comprising:
[0026] The training data acquisition module is used to acquire pedestrian image data and corresponding annotation data. The annotation data includes: pedestrian annotation box labels, pedestrian center point labels and category labels. The pedestrian center point labels are obtained based on the pedestrian annotation box labels.
[0027] The training result acquisition module is used to output pedestrian detection training results based on the pedestrian image data and through the pedestrian detection model. The pedestrian detection training results include: pedestrian prediction boxes, prediction categories corresponding to the pedestrian prediction boxes, and pedestrian center prediction points.
[0028] The pedestrian detection model training module is used to update the pedestrian detection model until convergence by generating a loss value through a loss function based on the pedestrian detection training results and the labeled data, so as to obtain the trained pedestrian detection model.
[0029] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the pedestrian detection model training method as described above, or implements the pedestrian detection method as described above.
[0030] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the pedestrian detection model training method as described above, or implements the pedestrian detection method as described above.
[0031] The present invention provides a pedestrian detection model training method, pedestrian detection method, training device, and equipment. It acquires pedestrian image data and corresponding annotation data, wherein the annotation data includes: pedestrian bounding box labels, pedestrian center point labels, and category labels, with the pedestrian center point labels obtained based on the pedestrian bounding box labels. Based on the pedestrian image data, the pedestrian detection model outputs pedestrian detection training results, including: pedestrian prediction boxes, the predicted categories corresponding to the prediction boxes, and the predicted pedestrian center point. Based on the pedestrian detection training results and annotation data, a loss value is generated using a loss function, and the pedestrian detection model is updated until convergence to obtain a trained pedestrian detection model. This invention improves the robustness of the pedestrian detection model by adding center point prediction, without increasing any annotation costs or the model's computational load. Furthermore, the detection results help distinguish pedestrians from the background environment, greatly improving the performance of the pedestrian detection model for pedestrian detection. Attached Figure Description
[0032] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0033] Figure 1 A flowchart illustrating a pedestrian detection model training method provided by the present invention;
[0034] Figure 2 This is a schematic diagram of the network structure of the pedestrian detection model provided in an embodiment of the present invention;
[0035] Figure 3 A flowchart illustrating the specific steps of a pedestrian detection method provided by this invention;
[0036] Figure 4 This is a schematic diagram of the structure of a pedestrian detection model training device provided by the present invention;
[0037] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention.
[0038] Figure label:
[0039] 21: Training data acquisition module; 22: Training result acquisition module; 23: Pedestrian detection model training module. Detailed Implementation
[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described clearly and completely below with reference to specific embodiments and accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0041] It should be noted that those skilled in the art will understand, explicitly and implicitly, that the embodiments described in this invention can be combined with other embodiments without conflict. Unless otherwise defined, the technical or scientific terms used in this invention should be understood in their ordinary sense by those skilled in the art. The terms "a," "an," "an," "the," etc., used in this invention do not indicate quantity limitation and can represent singular or plural. The terms "comprising," "including," "having," and any variations thereof used in this invention are intended to cover non-exclusive inclusion; the terms "first," "second," "third," etc., used in this invention are merely to distinguish similar objects and do not represent a specific ordering of objects.
[0042] In pedestrian detection scenarios under autonomous driving conditions, the number of image acquisition devices can be one or more. These devices include, but are not limited to, at least one of grayscale cameras and RGB cameras, or at least one of monocular cameras, binocular cameras, and depth cameras. The image acquisition devices are used to capture images of the environment surrounding the vehicle. If the pedestrians in the captured images are exposed to different lighting conditions or wear different clothing, resulting in low distinction between them and the background, they are often indistinguishable from the background, leading to poor pedestrian detection performance by the model.
[0043] To address the problems existing in the prior art, the present invention has made improvements through the following embodiments, thereby improving the performance of the model in pedestrian detection without increasing the model's computational load or requiring any additional data annotation.
[0044] Example 1
[0045] Reference Figure 1 As shown, this embodiment provides a method for training a pedestrian detection model, including:
[0046] Step S1: Obtain pedestrian image data and corresponding annotation data. The annotation data includes: pedestrian annotation box labels, pedestrian center point labels and category labels. The pedestrian center point labels are obtained based on the pedestrian annotation box labels.
[0047] In general, there are multiple pedestrian bounding boxes in a pedestrian detection scenario. The bounding boxes are drawn close to the target object. The pedestrian bounding box is a rectangle, the pedestrian's height is the height of the pedestrian bounding box, and the pedestrian's shoulder width is the width of the pedestrian bounding box.
[0048] In step S1, the pedestrian center point label is obtained based on the pedestrian bounding box label, specifically including:
[0049] Obtain the coordinates of the four corner points corresponding to the pedestrian annotation box label;
[0050] Based on the coordinates of the four corner points, the coordinates of the center point are generated and used as the pedestrian center point label.
[0051] Based on the pedestrian annotation box being a rectangle enclosed by the coordinates of its four corner points, the center point is determined by the intersection of the intersecting lines formed by the coordinates of the four corner points. Alternatively, the center point coordinates can be calculated using a formula.
[0052] X=(x1+x2+x3+x4) / 4, Y=(y1+y2+y3+y4) / 4;
[0053] Where X and Y are the x and y coordinates of the center point, respectively; x1, x2, x3, and x4 are the x coordinates of the four corner points, respectively; and y1, y2, y3, and y4 are the y coordinates of the four corner points, respectively.
[0054] Add a center point (center_point) inside the person's bounding box as the center point label for that pedestrian. Therefore, no additional annotation data is needed, and no annotation cost is increased.
[0055] Step S2: Based on the pedestrian image data, the pedestrian detection model outputs the pedestrian detection training results, which include: pedestrian prediction boxes, the prediction category corresponding to the pedestrian prediction boxes, and the pedestrian center prediction point.
[0056] Step S3: Construct a pedestrian detection model and its corresponding loss function. The pedestrian detection model is used to output pedestrian detection training results based on pedestrian image data. The pedestrian detection training results include: pedestrian prediction boxes, the prediction category corresponding to the pedestrian prediction boxes, and the center prediction point.
[0057] In this embodiment, the pedestrian detection model is built based on CenterNet. It should be noted that in other embodiments of this invention, the target detection model can also be an improved version of CenterNet, where the improved model retains the center point-related loss calculation during the loss calculation process. The loss function of this pedestrian detection model is obtained by weighting the pedestrian prediction bounding boxes, the prediction categories corresponding to the pedestrian prediction bounding boxes, and the sub-loss function corresponding to the pedestrian center prediction point included in the pedestrian detection training results. By adding pedestrian center point prediction, the pedestrian detection results of this model do not increase the computational load of the model due to additional annotations, and it also helps to distinguish pedestrians from the image background environment.
[0058] The pedestrian detection model training method provided in this embodiment utilizes existing training data (pedestrian image data) with corresponding labeled data (pedestrian bounding boxes). Based on this existing data, a center point (`center_point`) is added within each pedestrian bounding box. The pedestrian image data with the added center point is then used to construct a training dataset. This training dataset is used to train the pedestrian detection model, yielding training results. In addition to predicting the pedestrian prediction box (`pred_bbox`) and the corresponding prediction category (`pred_cls`), the model also outputs the pedestrian center prediction point (`pred_center_point`). By adding the prediction of the pedestrian center point, it ensures that the predicted pedestrian center prediction point is within the pedestrian prediction box. This prevents poor pedestrian detection results caused by the center prediction point not being within the prediction box due to low differentiation between the pedestrian and the background. Therefore, the pedestrian detection model trained using this method improves the robustness of model recognition and significantly enhances the pedestrian detection model's ability to detect pedestrians without increasing model computation or data labeling costs.
[0059] Correspondingly, the network structure of the existing pedestrian detection model is adjusted to address the aforementioned improvements. In this new network structure, only the corresponding convolutional and output layers need to be added to the neck module, such as... Figure 2 As shown, the dashed box represents the added branch structure to correspond to the pedestrian center point prediction results.
[0060] In this embodiment, the loss function Loss1 includes: pedestrian prediction box loss function Loss11, prediction class loss function Loss12, center point loss function Loss13, and center point comparison loss function Loss14. Correspondingly, the loss values include: the prediction box loss value of pedestrian prediction box loss function Loss11, the cross-entropy loss value of prediction class loss function Loss12, the center point loss value of center point loss function Loss13, and the center point comparison loss value of center point comparison loss function Loss14.
[0061] In this embodiment, the center point alignment loss value of the Loss14 center point alignment loss function is generated, including:
[0062] The center point of the predicted pedestrian bounding box is obtained as the first center point pred_bbox_points;
[0063] Based on the first center point and the predicted center points pred_center_points, calculate the center point alignment loss value of the center point alignment loss function Loss14.
[0064] Specifically, obtaining the pedestrian prediction box position allows us to further determine the center point position. We then incorporate a center point comparison loss function, Loss14, into the loss function. This Loss14 loss function calculates the center point comparison loss value to better distinguish pedestrians from the background. The center point comparison loss function can be constructed using the smooth L2 regression loss function. The specific functional expression of loss14 is: loss14 = L2_loss(pred_bbox_points, pred_center_points), where pred_bbox_point is the corresponding center point obtained from the pedestrian prediction box, and pred_center_points is the predicted center point.
[0065] In this embodiment, the prediction box loss value of the pedestrian prediction box loss function Loss11 is generated, including:
[0066] The cross-union ratio (CUI) between the predicted pedestrian bounding box and the labeled pedestrian bounding box is calculated and used as the prediction box loss value.
[0067] Specifically, the corner distance loss function Loss11 can be constructed using the regression loss function IOU loss, i.e., loss11 = IOU_loss(pred_bbox, bbox), where pred_bbox is the predicted pedestrian bounding box and bbox is the pedestrian label box.
[0068] In this embodiment, generating the cross-entropy loss value of the prediction category loss function Loss12 includes:
[0069] Calculate the cross-entropy loss value based on the predicted category and category label corresponding to the pedestrian prediction bounding box.
[0070] Specifically, the cross-entropy loss function is used to construct the prediction class loss function Loss12, i.e., loss12 = CrossEntropy(pred_cls, gt_cls), where pred_cls is the prediction class and gt_cls is the class label.
[0071] In this embodiment, generating the center point loss value of the center point loss function Loss13 includes:
[0072] Calculate the center point loss value based on the predicted pedestrian center point and the pedestrian center point label.
[0073] Specifically, the corner distance loss function Loss13 can be constructed using the regression loss function smooth L2 loss, where Loss13 = L2_loss(pred_center_points, gt_center_points), where pred_center_points are the predicted pedestrian center points and gt_center_points are the pedestrian center point labels.
[0074] The final loss function, Loss1, is:
[0075] Loss1=w1·loss11+w2·loss12+w3·loss13+w4·loss14;
[0076] When training the pedestrian detection model, the total loss function value of Loss1 is used as the loss function value of the predicted bounding box (loss11), the cross-entropy loss value of the predicted category (loss12), the center point loss value of the center point loss function (loss13), and the center point comparison loss value of the center point comparison loss function (loss14). Backpropagation training is then performed, updating the weight parameters (w1, w2, w3, w4) corresponding to each loss function. When each of the above loss values approaches its corresponding target value, and the loss value of Loss1 is less than a preset threshold, the loss function is considered to have converged, training is complete, and the pedestrian detection model is finally obtained. In this embodiment, the weight parameters of the loss function of the finally trained pedestrian detection model are w1 = 1, w2 = 2, w3 = 1, and w4 = 2.
[0077] The pedestrian detection model with added branches includes a backbone module, a neck module, and a head module. Pedestrian image data is input into the pedestrian detection model, and multi-dimensional feature information is extracted through the backbone module in the network structure. The multi-dimensional feature information is then input into the neck and head modules in the network structure. The multitask head module outputs the pedestrian prediction bounding box, the prediction category corresponding to the pedestrian prediction bounding box, and the center prediction point information. The neck and head modules can be flexibly selected with appropriate networks as needed. For example, the neck module can be a Feature Pyramid Network (FPN), and the head module can be a centerpoint network.
[0078] Example 2
[0079] Reference Figure 3 As shown, this embodiment provides a pedestrian detection method, including:
[0080] Step A1: Obtain the image to be detected that contains pedestrians;
[0081] Step A2: Input the image to be detected into the pre-trained pedestrian detection model to obtain the pedestrian detection result output by the pedestrian detection model. The pedestrian detection result includes the pedestrian detection box and the pedestrian center point. The pre-trained pedestrian detection model is trained by the pedestrian detection model training method described in Example 1.
[0082] In this embodiment, by inputting the image to be detected containing pedestrians into a trained pedestrian detection model, pedestrian detection results are obtained, which include pedestrian detection boxes and pedestrian center points. For recognition scenarios where the distinction between pedestrians and the image background is low, the prediction of the pedestrian center point is added to ensure that the predicted center point is within the pedestrian prediction box, greatly improving the pedestrian detection model's ability to detect pedestrians and increasing detection accuracy.
[0083] Example 3
[0084] Based on the same inventive concept as the method described in Embodiment 1, and referring to... Figure 4 As shown, this embodiment provides a pedestrian detection model training device, including:
[0085] The training data acquisition module 21 is used to acquire pedestrian image data and corresponding annotation data. The annotation data includes: pedestrian annotation box labels, pedestrian center point labels and category labels. The pedestrian center point labels are obtained based on the pedestrian annotation box labels.
[0086] The training result acquisition module 22 is used to output the pedestrian detection training result based on the pedestrian image data and the pedestrian detection model. The pedestrian detection training result includes: pedestrian prediction box, the prediction category corresponding to the pedestrian prediction box, and the pedestrian center prediction point.
[0087] The pedestrian detection model training module 23 is used to generate loss values based on the pedestrian detection training results and labeled data through a loss function, and update the pedestrian detection model until convergence, so as to obtain a trained pedestrian detection model.
[0088] The training data acquisition module 21 specifically includes: a corner coordinate acquisition unit, used to acquire the coordinates of the four corner points corresponding to the pedestrian annotation box label; and a pedestrian center point label generation unit, used to generate the center point coordinates based on the four corner point coordinates, as the pedestrian center point label.
[0089] The pedestrian detection model training module 23 specifically includes: a first loss value generation unit, used to generate the prediction box loss value of the pedestrian prediction box loss function; a second loss value generation unit, used to generate the cross-entropy loss value of the prediction category loss function; a third loss value generation unit, used to generate the center point loss value of the center point loss function; and a fourth loss value generation unit, used to generate the center point comparison loss value of the center point comparison loss function.
[0090] The fourth loss value generation unit specifically includes: a first center point acquisition subunit, used to acquire the corresponding center point based on the predicted pedestrian prediction box, as the first center point; and a center point comparison loss value calculation subunit, used to calculate the center point comparison loss value based on the first center point and the predicted pedestrian center prediction point.
[0091] The first loss value generation unit is specifically used to calculate the intersection-union ratio between the pedestrian prediction box and the pedestrian annotation box label, which is used as the prediction box loss value;
[0092] The second loss value generation unit is specifically used to calculate the cross-entropy loss value based on the prediction category and category label corresponding to the pedestrian prediction box;
[0093] The third loss value generation unit is specifically used to calculate the center point loss value based on the pedestrian center prediction point and the pedestrian center point label;
[0094] The pedestrian detection model training module 23 is specifically used to train the pedestrian detection model. It uses the prediction box loss value of the pedestrian prediction box loss function, the cross-entropy loss value of the prediction category loss function, the center point loss value of the center point loss function, and the center point comparison loss value of the center point comparison loss function as the total loss function value of the loss function. It performs backpropagation training, updates the weight parameters corresponding to each loss function, and finally obtains the pedestrian detection model.
[0095] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method. Therefore, relevant parts can be referred to in the description of the method embodiment, and will not be repeated here.
[0096] The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division. In actual implementation, there may be other division methods. In the embodiments, each functional module can be integrated into a processor, or each module can be a separate device, or two or more modules can be integrated into a device. Each functional module in each embodiment can be implemented in hardware or in the form of hardware plus software functional units.
[0097] Example 4
[0098] Reference Figure 5 As shown, this embodiment provides an electronic device, which includes: a processor 310, a communication interface 320, a memory 330, and a communication bus 340. The processor 310, communication interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330. The processor 310 executes the pedestrian detection model training method as described in Embodiment 1, which includes:
[0099] Obtain pedestrian image data and corresponding annotation data. The annotation data includes: pedestrian bounding box labels, pedestrian center point labels, and category labels. The pedestrian center point labels are obtained based on the pedestrian bounding box labels.
[0100] Based on pedestrian image data, the pedestrian detection model outputs pedestrian detection training results, which include: pedestrian prediction bounding boxes, the prediction category corresponding to the pedestrian prediction bounding boxes, and the pedestrian center prediction point.
[0101] A pedestrian detection model and its corresponding loss function are constructed. The pedestrian detection model is used to output pedestrian detection training results based on pedestrian image data. The pedestrian detection training results include: pedestrian prediction boxes, the prediction category corresponding to the pedestrian prediction boxes, and the center prediction point.
[0102] Alternatively, the pedestrian detection method as described in Example 2 can be performed, the method comprising:
[0103] Acquire the image to be detected that contains pedestrians;
[0104] The image to be detected is input into a pre-trained pedestrian detection model to obtain the pedestrian detection result output by the pedestrian detection model. The pedestrian detection result includes the pedestrian detection box and the pedestrian center point. The pre-trained pedestrian detection model is trained by the pedestrian detection model training method described in Example 1.
[0105] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0106] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the pedestrian detection model training method as described in Embodiment 1, the method comprising:
[0107] Obtain pedestrian image data and corresponding annotation data. The annotation data includes: pedestrian bounding box labels, pedestrian center point labels, and category labels. The pedestrian center point labels are obtained based on the pedestrian bounding box labels.
[0108] Based on pedestrian image data, the pedestrian detection model outputs pedestrian detection training results, which include: pedestrian prediction bounding boxes, the prediction category corresponding to the pedestrian prediction bounding boxes, and the pedestrian center prediction point.
[0109] A pedestrian detection model and its corresponding loss function are constructed. The pedestrian detection model is used to output pedestrian detection training results based on pedestrian image data. The pedestrian detection training results include: pedestrian prediction boxes, the prediction category corresponding to the pedestrian prediction boxes, and the center prediction point.
[0110] Alternatively, the pedestrian detection method as described in Example 2 can be performed, the method comprising:
[0111] Acquire the image to be detected that contains pedestrians;
[0112] The image to be detected is input into a pre-trained pedestrian detection model to obtain the pedestrian detection result output by the pedestrian detection model. The pedestrian detection result includes the pedestrian detection box and the pedestrian center point. The pre-trained pedestrian detection model is trained by the pedestrian detection model training method described in Example 1.
[0113] Example 5
[0114] This embodiment provides a non-transitory computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the pedestrian detection model training method as described in Embodiment 1. The method includes:
[0115] Obtain pedestrian image data and corresponding annotation data. The annotation data includes: pedestrian bounding box labels, pedestrian center point labels, and category labels. The pedestrian center point labels are obtained based on the pedestrian bounding box labels.
[0116] Based on pedestrian image data, the pedestrian detection model outputs pedestrian detection training results, which include: pedestrian prediction bounding boxes, the prediction category corresponding to the pedestrian prediction bounding boxes, and the pedestrian center prediction point.
[0117] A pedestrian detection model and its corresponding loss function are constructed. The pedestrian detection model is used to output pedestrian detection training results based on pedestrian image data. The pedestrian detection training results include: pedestrian prediction boxes, the prediction category corresponding to the pedestrian prediction boxes, and the center prediction point.
[0118] Alternatively, the pedestrian detection method as described in Example 2 can be performed, the method comprising:
[0119] Acquire the image to be detected that contains pedestrians;
[0120] The image to be detected is input into a pre-trained pedestrian detection model to obtain the pedestrian detection result output by the pedestrian detection model. The pedestrian detection result includes the pedestrian detection box and the pedestrian center point. The pre-trained pedestrian detection model is trained by the pedestrian detection model training method described in Example 1.
[0121] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are relatively simple in description because they are fundamentally similar to the method embodiments; relevant parts can be referred to the descriptions in the method embodiments.
[0122] The devices, media, and methods provided in the embodiments of the present invention are one-to-one correspondences. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.
[0123] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process method or product that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process method or product. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process method or product that includes that element.
[0124] The above are merely embodiments of the present invention and are not intended to limit the invention. Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.
Claims
1. A method for training a pedestrian detection model, characterized in that, include: Obtain pedestrian image data and corresponding annotation data, wherein the annotation data includes: pedestrian annotation box labels, pedestrian center point labels and category labels, and the pedestrian center point labels are obtained based on the pedestrian annotation box labels; Based on the pedestrian image data, the pedestrian detection model outputs the pedestrian detection training results, which include: pedestrian prediction boxes, the prediction categories corresponding to the pedestrian prediction boxes, and the pedestrian center prediction points. Based on the pedestrian detection training results and the labeled data, a loss value is generated through a loss function, and the pedestrian detection model is updated until convergence to obtain the trained pedestrian detection model. The pedestrian center point label is obtained based on the pedestrian annotation box label, including: Obtain the coordinates of the four corner points corresponding to the pedestrian annotation box label; Based on the coordinates of the four corner points, the coordinates of the center point are generated and used as the center point label of the pedestrian. The loss function includes a center point comparison loss function, and the generation of the loss value includes: The center point is obtained from the predicted pedestrian bounding box as the first center point; the center point comparison loss value is calculated based on the first center point and the predicted pedestrian center point; specifically, the center point comparison loss value is calculated using the loss function Loss14 to better distinguish pedestrians from the background. The center point comparison loss function is constructed using the regression loss function smooth L2 loss. The specific function expression of loss14 is: loss14 = L2_loss(pred_bbox_points,pred_center_points), where pred_bbox_point is the center point obtained from the predicted pedestrian bounding box, and pred_center_points is the predicted center point.
2. The pedestrian detection model training method according to claim 1, characterized in that, The loss function further includes: a pedestrian prediction box loss function, a prediction category loss function, and a center point loss function. Correspondingly, the loss value includes: the prediction box loss value of the pedestrian prediction box loss function, the cross-entropy loss value of the prediction category loss function, and the center point loss value of the center point loss function.
3. The pedestrian detection model training method according to claim 1, characterized in that, Generating the loss value includes: Calculate the intersection-union ratio (IUU) between the predicted pedestrian bounding box and the pedestrian labeled bounding box, using it as the predicted box loss value, and... The cross-entropy loss value is calculated based on the predicted category corresponding to the pedestrian prediction box and the category label.
4. The pedestrian detection model training method according to claim 1, characterized in that, Generating the loss value includes: The center point loss value is calculated based on the predicted pedestrian center point and the pedestrian center point label.
5. A pedestrian detection method, characterized in that, include: Acquire the image to be detected that contains pedestrians; The image to be detected is input into a pre-trained pedestrian detection model to obtain the pedestrian detection result output by the pedestrian detection model. The pedestrian detection result includes a pedestrian detection box and a pedestrian center point. The pre-trained pedestrian detection model is trained by the pedestrian detection model training method as described in any one of claims 1-4.
6. A pedestrian detection model training device, characterized in that, include: The training data acquisition module is used to acquire pedestrian image data and corresponding annotation data. The annotation data includes: pedestrian bounding box labels, pedestrian center point labels, and category labels. The pedestrian center point labels are obtained based on the pedestrian bounding box labels. Obtaining the pedestrian center point labels includes: acquiring the coordinates of the four corner points corresponding to the pedestrian bounding box labels; and generating center point coordinates based on the four corner point coordinates as the pedestrian center point labels. The training result acquisition module is used to output pedestrian detection training results based on the pedestrian image data and through the pedestrian detection model. The pedestrian detection training results include: pedestrian prediction boxes, prediction categories corresponding to the pedestrian prediction boxes, and pedestrian center prediction points. The pedestrian detection model training module is used to generate a loss value based on the pedestrian detection training results and the labeled data through a loss function, and update the pedestrian detection model until convergence, so as to obtain the trained pedestrian detection model. The loss function includes a center point comparison loss function, and the generation of the loss value includes: The center point is obtained from the predicted pedestrian bounding box as the first center point; the center point comparison loss value is calculated based on the first center point and the predicted pedestrian center point; specifically, the center point comparison loss value is calculated using the loss function Loss14 to better distinguish pedestrians from the background. The center point comparison loss function is constructed using the regression loss function smooth L2 loss. The specific function expression of loss14 is: loss14 = L2_loss(pred_bbox_points,pred_center_points), where pred_bbox_point is the center point obtained from the predicted pedestrian bounding box, and pred_center_points is the predicted center point.
7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the pedestrian detection model training method as described in any one of claims 1 to 4, or the pedestrian detection method as described in claim 5.
8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the pedestrian detection model training method as described in any one of claims 1 to 4, or the pedestrian detection method as described in claim 5.
Citation Information
Patent Citations
Sidewalk detection method and device based on artificial intelligence, equipment and medium
CN112132037A