Target detection method, target detection model training method, device, equipment, medium and product

Abstract

The invention discloses a target detection method, a target detection model training method, devices, equipment, a medium and a product. The method comprises the following steps: acquiring a to-be-processed road image; inputting the to-be-processed road image into a target detection model to obtain a target detection frame coordinate, target category information, a confidence coefficient of a plurality of target grounding point abscissa offsets and a confidence coefficient of a plurality of target grounding point ordinate offsets; and determining the coordinates of the target grounding point according to the coordinates of the target detection frame, the confidence coefficient of the offset of the abscissa of the plurality of target grounding points and the confidence coefficient of the offset of the ordinate of the plurality of target grounding points, and through the technical scheme of the invention, the accuracy of target detection can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of target detection technology, and in particular to a target detection method, a target detection model training method, an apparatus, a device, a medium and a product. Background Art

[0002] Autonomous driving technology is one of the rapidly developing high-tech fields in recent years. It combines multiple research results such as artificial intelligence, machine learning, sensor technology, big data processing and control strategies, aiming to achieve autonomous driving of cars, reduce traffic accidents, and improve road safety and traffic efficiency.

[0003] Object detection technology is the foundation of autonomous driving technology. Object detection technology can identify and track various targets on the road, such as vehicles, pedestrians, cyclists and obstacles in real time. By accurately detecting these targets, the vehicle can make more accurate decisions based on the detection results to ensure the safety of passengers and other road users. The target spatial coordinate information provided by object detection technology helps autonomous vehicles understand the location, size and motion state of the target. This information is crucial for vehicle path planning and control in complex environments.

[0004] With the development of Transformer technology, the accuracy of 3D object detection based on Transformer is relatively high. However, when the computing power on the vehicle side is limited, due to the high computational complexity of Transformer and its dependence on a large amount of computing resources, it is often impossible to actually deploy Transformer on the vehicle side. Therefore, the traditional convolutional small model solution is usually adopted. The convolutional small model can run on the processor or embedded system installed on the vehicle and can meet the requirements of real-time and practicality.

[0005] Traditional target detection technology based on convolutional small models mainly detects the 2D rectangular box of the target, then takes the midpoint of the bottom edge of the target box to approximate the target ground point, and then converts it into coordinates in 3D space through coordinate transformation to obtain the position of the target in 3D space. The 3D space position is beneficial to the subsequent decision-making and planning of the vehicle. However, when the target is relatively large, the midpoint of its bottom edge cannot represent the actual position of the target, and the accuracy of target detection is low. Summary of the invention

[0006] Embodiments of the present invention provide a target detection method, a target detection model training method, an apparatus, a device, a medium and a product to improve the accuracy of target detection.

[0007] According to one aspect of the present invention, there is provided a target detection method, comprising:

[0008] Acquire a road image to be processed;

[0009] Inputting the road image to be processed into a target detection model to obtain target detection frame coordinates, target category information, confidences of a plurality of target ground point horizontal coordinate offsets, and confidences of a plurality of target ground point vertical coordinate offsets;

[0010] The target grounding point coordinates are determined according to the target detection frame coordinates, the confidence levels of the horizontal coordinate offsets of the multiple target grounding points, and the confidence levels of the vertical coordinate offsets of the multiple target grounding points.

[0011] According to another aspect of the present invention, a target detection model training method is provided, and the target detection model training method comprises:

[0012] Acquire a training sample set, wherein the training sample set includes: a road image sample, the road image sample carries target information, and the target information includes: target detection frame coordinates, target category information, target ground point coordinates, and target ground point confidence;

[0013] Inputting the road image samples in the training sample set into the initial detection model, obtaining a plurality of predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of a plurality of predicted ground point horizontal coordinates, confidence of a plurality of predicted ground point vertical coordinates, and confidence of the predicted ground point;

[0014] Determine a target category loss value, a target rectangular box regression loss value, a ground point coordinate prediction loss value, and a ground point confidence loss value according to the multiple predicted detection box coordinates, the category information of each predicted detection box, the confidence of each category information, the confidence of the multiple predicted ground point horizontal coordinates, the confidence of the multiple predicted ground point vertical coordinates, the confidence of the predicted ground point, and the target information carried by the road image sample;

[0015] According to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value and the ground point confidence loss value, the parameters of the initial detection model are trained to obtain a target detection model.

[0016] According to another aspect of the present invention, there is provided a target detection device, the target detection device comprising:

[0017] A road image acquisition module to be processed, used to acquire the road image to be processed;

[0018] A first detection module is used to input the road image to be processed into a target detection model to obtain target detection frame coordinates, target category information, confidences of abscissa offsets of multiple target ground points, and confidences of ordinate offsets of multiple target ground points;

[0019] The second detection module is used to determine the coordinates of the target grounding point according to the target detection frame coordinates, the confidence levels of the horizontal coordinate offsets of the multiple target grounding points, and the confidence levels of the vertical coordinate offsets of the multiple target grounding points.

[0020] According to another aspect of the present invention, a target detection model training device is provided, the target detection model training device comprising:

[0021] A training sample set acquisition module is used to acquire a training sample set, wherein the training sample set includes: a road image sample, the road image sample carries target information, and the target information includes: target detection frame coordinates, target category information, target ground point coordinates, and target ground point confidence;

[0022] A prediction module, used for inputting the road image samples in the training sample set into the initial detection model to obtain a plurality of predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of a plurality of predicted ground point horizontal coordinates, confidence of a plurality of predicted ground point vertical coordinates and confidence of the predicted ground point;

[0023] a loss value determination module, configured to determine a target category loss value, a target rectangular box regression loss value, a ground point coordinate prediction loss value, and a ground point confidence loss value according to the multiple predicted detection box coordinates, the category information of each predicted detection box, the confidence of each category information, the multiple horizontal coordinates and multiple vertical coordinates of the predicted ground point, the confidence of each horizontal coordinate, the confidence of each vertical coordinate, the confidence of the predicted ground point, and the target information carried by the road image sample;

[0024] A training module is used to train the parameters of the initial detection model according to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value and the ground point confidence loss value to obtain a target detection model.

[0025] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0026] at least one processor; and

[0027] a memory communicatively connected to the at least one processor; wherein,

[0028] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the method described in any embodiment of the present invention.

[0029] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method described in any embodiment of the present invention when executed.

[0030] According to another aspect of the present invention, a computer program product is provided. When the computer program is executed by a processor, the computer program implements any method as described in the embodiments of the present invention.

[0031] The embodiment of the present invention first inputs the road image to be processed into the target detection model to obtain the target detection frame coordinates, target category information, the confidences of the horizontal coordinate offsets of multiple target ground points, and the confidences of the vertical coordinate offsets of multiple target ground points. Then, according to the target detection frame coordinates, the confidences of the horizontal coordinate offsets of multiple target ground points, and the confidences of the vertical coordinate offsets of multiple target ground points, the target ground point coordinates are determined, which can more accurately determine the target ground point coordinates, thereby improving the accuracy of target detection.

[0032] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0034] Figure 1 is a flow chart of a target detection method in an embodiment of the present invention;

[0035] Figure 2 is a flow chart of a target detection model training method in an embodiment of the present invention;

[0036] Figure 3 is a schematic structural diagram of a target detection device in an embodiment of the present invention;

[0037] Figure 4 is a structural schematic diagram of a target detection model training device in an embodiment of the present invention;

[0038] Figure 5 It is a structural schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0042] Embodiment 1

[0043] Figure 1 This is a flow chart of a target detection method provided by an embodiment of the present invention. This embodiment is applicable to the case of target detection. The method can be executed by a target detection device in an embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 1 As shown, the method specifically comprises the following steps:

[0044] S110, obtaining a road image to be processed.

[0045] In this embodiment, the road image to be processed can be acquired by an image acquisition device arranged on the vehicle body.

[0046] S120, inputting the road image to be processed into a target detection model to obtain target detection frame coordinates, target category information, confidences of a plurality of target ground point horizontal coordinate offsets, and confidences of a plurality of target ground point vertical coordinate offsets.

[0047] In this embodiment, the target category information may include: pedestrians, vehicles, two-wheeled vehicles, etc. The target touchdown point may be a pedestrian touchdown point, a vehicle touchdown point, or a two-wheeled vehicle touchdown point.

[0048] In this embodiment, the road image to be processed is input into the target detection model to obtain the target detection frame coordinates, target category information, the confidence of a preset number of target ground point horizontal coordinate offsets, and the confidence of a preset number of target ground point vertical coordinate offsets. It should be noted that the number of target ground point horizontal coordinate offsets is the same as the number of target ground point vertical coordinate offsets.

[0049] In this embodiment, the confidences of the horizontal coordinate offsets of the multiple target grounding points and the confidences of the vertical coordinate offsets of the multiple target grounding points may include: the confidences of the horizontal coordinate offsets of the multiple target grounding points and the confidences of the vertical coordinate offsets of the multiple target grounding points. For example, the road image to be processed is input into the target detection model to obtain the target detection frame coordinates, target category information, the confidences of the horizontal coordinate offsets of the first target grounding point and the confidences of the vertical coordinate offsets, the confidences of the horizontal coordinate offsets of the second target grounding point and the confidences of the vertical coordinate offsets, the confidences of the horizontal coordinate offsets of the third target grounding point and the confidences of the horizontal coordinate offsets of the fourth target grounding point.

[0050] In this embodiment, the target detection model can be trained in the following manner: obtaining a training sample set, wherein the training sample set includes: road image samples, the road image samples carry target information, and the target information includes: target detection frame coordinates, target category information, target ground point coordinates, and target ground point confidence; inputting the road image samples in the training sample set into the initial detection model to obtain multiple predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of multiple predicted ground point horizontal coordinates, confidence of multiple predicted ground point vertical coordinates, and confidence of the predicted ground point. degree; according to the multiple predicted detection frame coordinates, the category information of each predicted detection frame, the confidence of each category information, the confidence of multiple predicted grounding point horizontal coordinates, the confidence of multiple predicted grounding point vertical coordinates, the confidence of the predicted grounding point and the target information carried by the road image sample, determine the target category loss value, the target rectangular frame regression loss value, the grounding point coordinate prediction loss value and the grounding point confidence loss value; according to the target category loss value, the target rectangular frame regression loss value, the grounding point coordinate prediction loss value and the grounding point confidence loss value, train the parameters of the initial detection model to obtain the target detection model. It should be noted that during the training process, the number of horizontal coordinates of the predicted grounding point is the same as the number of horizontal coordinates of the target grounding point, and the number of vertical coordinates of the predicted grounding point is the same as the number of vertical coordinates of the target grounding point. For example, if the number of horizontal coordinate offsets of the predicted grounding point is 8, the number of horizontal coordinate offsets of the target grounding point is 8, the number of vertical coordinate offsets of the predicted grounding point is 8, and the number of vertical coordinate offsets of the target grounding point is 8.

[0051] S130, determining the target grounding point coordinates according to the target detection frame coordinates, the confidences of the horizontal coordinate offsets of the multiple target grounding points, and the confidences of the vertical coordinate offsets of the multiple target grounding points.

[0052] In this embodiment, according to the target detection frame coordinates, the confidences of the horizontal coordinate offsets of the multiple target grounding points, and the confidences of the vertical coordinate offsets of the multiple target grounding points, the method for determining the target grounding point coordinates can be: obtaining a grounding point integral sequence, and determining the target grounding point coordinates according to the target detection frame coordinates, the confidences of the horizontal coordinate offsets of the multiple target grounding points, and the confidences of the vertical coordinate offsets of the multiple target grounding points. It should be noted that the grounding point integral sequence can be a pre-set sequence, or a grounding point integral sequence determined according to the number of the vertical coordinate offsets of the target grounding points and / or the horizontal coordinate offsets of the target grounding points, or a grounding point integral sequence determined according to a pre-set threshold.

[0053] It should be noted that the target detection model provided by the embodiment of the present invention can simultaneously detect the target detection frame and the target ground point, and accurately obtain the coordinates of the target in 3D space through the target ground point, which is beneficial to the subsequent decision-making and planning of the vehicle.

[0054] In the embodiment of the present invention, the road image to be processed is first input into the target detection model to obtain the target detection frame coordinates, target category information, confidences of multiple target ground point horizontal coordinate offsets, and confidences of multiple target ground point vertical coordinate offsets. Then, according to the target detection frame coordinates, the confidences of multiple target ground point horizontal coordinate offsets, and the confidences of multiple target ground point vertical coordinate offsets, the target ground point coordinates are determined, so that the target ground point coordinates can be determined more accurately, thereby improving the accuracy of target detection.

[0055] The embodiment of the present invention uses one target detection model to simultaneously detect the target detection frame and the target grounding point, and can be deployed on a low computing power platform, saving computing resources.

[0056] Optionally, determining the target ground point coordinates according to the target detection frame coordinates, the confidences of the horizontal coordinate offsets of the multiple target ground points, and the confidences of the vertical coordinate offsets of the multiple target ground points includes:

[0057] Get the ground point integration sequence.

[0058] In this embodiment, the method for obtaining the grounding point integral sequence may be: presetting the grounding point integral sequence. It may also be a grounding point integral sequence determined according to the target grounding point ordinate offset and / or the target grounding point abscissa offset. For example, if the target grounding point abscissa offsets are 0, 1, 2, 3, 4, 5, 6, 7, respectively, and the target grounding point ordinate offsets are 0, 1, 2, 3, 4, 5, 6, 7, respectively, then the grounding point integral sequence is determined to be 0, 1, 2, 3, 4, 5, 6, 7. Or, the maximum value of the grounding point integral sequence may be pre-set, and then the grounding point integral sequence may be determined according to the pre-set maximum value. For example, if the maximum value of the grounding point integral sequence is pre-set to 8, then the grounding point integral sequence is determined to be: 0, 1, 2, 3, 4, 5, 6, 7.

[0059] The sum of the products of each value in the touchdown point integral sequence and the confidence level of the horizontal coordinate offset of each target touchdown point is determined as the horizontal coordinate offset.

[0060] In this embodiment, the method of determining the sum of the products of the values ​​in the grounding point integral sequence and the confidence of the horizontal coordinate offset of each target grounding point as the horizontal coordinate offset may be: if there are multiple target grounding points, the sum of the products of the values ​​in the grounding point integral sequence and the confidence of each horizontal coordinate offset of each corresponding target grounding point is determined as the horizontal coordinate offset of each target grounding point. For example, if there are four target grounding points, namely the first target grounding point, the second target grounding point, the third target grounding point and the fourth target grounding point, the sum of the products of the values ​​in the grounding point integral sequence and the confidence of each horizontal coordinate offset of the corresponding first target grounding point is determined as the horizontal coordinate offset of the first target grounding point. The calculation method of the horizontal coordinate offset of the second target grounding point, the third target grounding point and the fourth target grounding point is the same as that of the first target grounding point, which will not be repeated here.

[0061] The sum of the products of each numerical value in the touchdown point integral sequence and the confidence level of the ordinate offset of each target touchdown point is determined as the ordinate offset.

[0062] In this embodiment, the method of determining the sum of the products of each numerical value in the grounding point integral sequence and the confidence of each target grounding point ordinate offset as the ordinate offset may be: if there are multiple target grounding points, the sum of the products of each numerical value in the grounding point integral sequence and the confidence of each ordinate offset of each corresponding target grounding point is determined as the ordinate offset of each target grounding point. For example, if there are four target grounding points, namely the first target grounding point, the second target grounding point, the third target grounding point and the fourth target grounding point, the sum of the products of each numerical value in the grounding point integral sequence and the confidence of each ordinate offset of the corresponding first target grounding point is determined as the ordinate offset of the first target grounding point. The calculation method of the ordinate offset of the second target grounding point, the third target grounding point and the fourth target grounding point is the same as that of the first target grounding point, which will not be repeated here. The coordinates of the target grounding point are determined according to the target detection frame coordinates, the horizontal coordinate offset and the vertical coordinate offset.

[0063] In this embodiment, the coordinates of the target grounding point may be determined according to the target detection frame coordinates, the horizontal coordinate offset, and the vertical coordinate offset: if there are multiple target grounding points, the sum of the horizontal coordinate of the target detection frame and the horizontal coordinate offset of each target grounding point is determined as the horizontal coordinate of each target grounding point, and the sum of the vertical coordinate of the target detection frame and the vertical coordinate offset of each target grounding point is determined as the vertical coordinate of each target grounding point. If there is only one target grounding point, the sum of the horizontal coordinate of the target detection frame and the horizontal coordinate offset is determined as the horizontal coordinate of the target grounding point, and the sum of the vertical coordinate of the target detection frame and the vertical coordinate offset is determined as the vertical coordinate of the target grounding point.

[0064] In this embodiment, if the target detection frame coordinates are (10, 10), the ground point integral sequence is 0, 1, 2, 3, 4, 5, 6, 7, the confidences of the 8 horizontal coordinate offsets and the 8 vertical coordinate offsets of the first target ground point are shown in Table 1:

[0065] Table 1

[0066] 0 1 2 3 4 5 6 7 <![CDATA[x1]]> 0 0.03 0.07 0.8 0.1 0 0 0 <![CDATA[y1]]> 0 0 0 0 0.05 0.7 0.2 0.05

[0067] According to Table 1, the horizontal coordinate of the first target grounding point is x1=10+1×0.03+2×0.07+3×0.8+4×0.1=12.9; y1=10+4×0.05+5×0.7+6×0.2+7×0.05=15.25. If there are 4 target grounding points, the coordinates of the remaining three target grounding points are determined in the same way as the coordinates of the first target grounding point, which will not be repeated here.

[0068] In an embodiment of the present invention, the horizontal coordinate offset and the vertical coordinate offset of the target grounding point are determined according to the grounding point integral sequence, the confidence of the horizontal coordinate offsets of multiple target grounding points, and the confidence of the vertical coordinate offsets of multiple target grounding points. Then, the coordinates of the target grounding point are determined according to the target detection frame coordinates, the horizontal coordinate offset and the vertical coordinate offset. This can solve the problem of inaccurate grounding point coordinates and improve the accuracy of the target grounding point coordinates.

[0069] Optionally, determining the target ground point coordinates according to the target detection frame coordinates, the horizontal coordinate offset, and the vertical coordinate offset includes:

[0070] The horizontal coordinate of the target grounding point is determined according to the horizontal coordinate of the target detection frame and the horizontal coordinate offset.

[0071] In this embodiment, the sum of the target detection frame horizontal coordinate and the horizontal coordinate offset is determined as the target grounding point horizontal coordinate.

[0072] The ordinate of the target grounding point is determined according to the ordinate of the target detection frame and the ordinate offset.

[0073] In this embodiment, the sum of the ordinate of the target detection frame and the ordinate offset is determined as the ordinate of the target grounding point.

[0074] The target grounding point coordinates are determined according to the target grounding point abscissa and the target grounding point ordinate.

[0075] The technical solution provided by the embodiment of the present invention can improve the accuracy of the target ground point coordinates obtained by target detection by determining the target ground point coordinates based on the target detection frame, the horizontal coordinate offset of the target ground point and the vertical coordinate offset of the target ground point.

[0076] Embodiment 2

[0077] Figure 2 A flowchart of a target detection model training method provided by an embodiment of the present invention. This embodiment is applicable to the case of target detection model training. The method can be executed by a target detection model training device in an embodiment of the present invention. The device can be implemented in software and / or hardware. Figure 2 As shown, the method specifically comprises the following steps:

[0078] S210, obtaining a training sample set.

[0079] In this embodiment, the training sample set includes: road image samples, the road image samples carry target information, and the target information includes: target detection frame coordinates, target category information, target ground point coordinates, and target ground point confidence.

[0080] In this embodiment, when the target is a vehicle or a two-wheeled vehicle, the target grounding point coordinates may be multiple, for example, four; when the target is a person, the target grounding point coordinates may be one or multiple.

[0081] In this embodiment, a historical road image is obtained and used as a road image sample. The target detection frame coordinates may be the detection frame coordinates containing the target, and the target category information may include: pedestrians, vehicles, two-wheeled vehicles, etc. If the target is a pedestrian, the confidence of the pedestrian is 1, the confidence of the vehicle is 0, and the confidence of the two-wheeled vehicle is 0. If the target is a vehicle, the confidence of the vehicle is 1, the confidence of the pedestrian is 0, and the confidence of the two-wheeled vehicle is 0. In this embodiment, if the target grounding point coordinates are marked in the road image sample, the target grounding point confidence is 1, and if the target grounding point coordinates are not marked in the road image sample, the target grounding point confidence is 0.

[0082] Optionally, obtain a training sample set, including:

[0083] Acquire a historical road image set, wherein the historical road image set includes: historical road images, the historical road images carrying target information;

[0084] The historical road image is partitioned according to the target information carried by the historical road image to obtain a first area and a second area.

[0085] In this embodiment, the first area is an area within the target detection frame, and the second area is an area outside the target detection frame.

[0086] In this embodiment, the area in the target detection frame in the historical road image is determined as the first area, and the area outside the target detection frame in the historical road image is determined as the second area.

[0087] Determine the first area in the historical road image as a road image sample in a positive sample set;

[0088] Determine the second area in the historical road image as a road image sample in a negative sample set;

[0089] A training sample set is determined according to the positive sample set and the negative sample set.

[0090] In an embodiment of the present invention, by partitioning the historical road image according to the target information carried by the historical road image to obtain a first area and a second area, a positive sample set is generated according to the first area, and a negative sample set is generated according to the second area, the number of training samples can be increased, and the target ground point detection performance of the model can be further improved.

[0091] Optionally, determining a training sample set according to the positive sample set and the negative sample set includes:

[0092] The positive sample set is screened according to target information carried by the road image samples in the positive sample set to obtain a screened positive sample set.

[0093] In this embodiment, the positive sample set is screened according to the target information carried by the road image samples in the positive sample set, and the method for obtaining the screened positive sample set can be: obtaining road image samples whose target category information is not a vehicle and road image samples whose target category information is a vehicle in the positive sample set, deleting road image samples whose target category information is not a vehicle in the positive sample set, and deleting road image samples whose target category information is a vehicle and whose grounding point does not exist in the road image samples whose target category information is a vehicle.

[0094] A training sample set is determined according to the screened positive sample set and the negative sample set.

[0095] In an embodiment of the present invention, by screening positive samples and training an initial detection model based on the screened positive and negative samples, a target detection model is obtained, which can delete samples that have a negative impact on the target detection results, thereby improving the accuracy of target detection.

[0096] Optionally, the positive sample set is screened according to target information carried by the road image samples in the positive sample set to obtain a screened positive sample set, including:

[0097] The road image samples whose target category information is not a vehicle and the road image samples whose target category information is a vehicle and whose target touchdown point confidence is a first value are deleted from the positive sample set to obtain a filtered positive sample set.

[0098] In this embodiment, the road image sample whose target grounding point confidence is a first value, that is, the road image sample is not labeled with the grounding point coordinates. The first value can be 0 or other values ​​used to represent the unlabeled grounding point coordinates, which is not limited in the embodiment of the present invention.

[0099] It should be noted that since the target detection category is not limited to vehicles, but also includes pedestrians and two-wheeled vehicles, only vehicles need to perform grounding point detection. Therefore, it is necessary to filter the positive samples before grounding point detection (the original positive samples contain targets of different categories). The filtering method is: only vehicle targets with grounding point annotation data. It can be understood that if the vehicle target does not have a grounding point marked, it will affect the accuracy of the grounding point confidence, that is, the vehicle target actually contains a grounding point, but because it is not marked, it may cause the model to be considered to have no grounding point during training, resulting in deviation. In an embodiment of the present invention, by deleting the road image samples whose target category information is not a vehicle in the positive sample set, and the road image samples whose target category information is a vehicle and whose target grounding point confidence is a first value, the initial detection model is trained with the obtained filtered positive sample set and negative sample set to obtain a target detection model. The accuracy of target detection can be improved.

[0100] S220, input the road image samples in the training sample set into the initial detection model to obtain multiple predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of multiple predicted ground point horizontal coordinates, confidence of multiple predicted ground point vertical coordinates and confidence of the predicted ground point.

[0101] In this embodiment, the initial detection model can be divided into three parts in terms of model structure. The first part is the backbone network, the second part is the feature fusion module, and the third part is the detection head module. The function of the backbone network module is to extract the features of the image and output feature maps of different scales. Objects of different sizes correspond to different receptive fields. The feature map of large size represents the features of small targets, and the feature map of small size represents the features of large targets. The function of the feature fusion module is to fuse features of different scales, and to improve the expression ability of each size feature map by continuously upsampling the small-scale features and fusing them with the original scale, and then continuing to perform secondary fusion of features through downsampling after fusion. The detection head module further extracts features according to different detection targets on features of different scales and performs corresponding detection tasks. Taking the input image size of 640x640 as an example, the embodiment of the present invention takes the output of features of 3 different scales as an example, and the final sizes of the 3 scales are 80×80, 40×40, and 20×20 respectively. Multiple grids are set on each feature according to corresponding pixels, such as a total of 400 grids on a 20×20 feature map. The grid is the basis for subsequent positive and negative sample judgment, calculation of target detection box coordinates, and calculation of ground point coordinates. Unlike the traditional method of detecting the vehicle target rectangular box through the first network, and then extracting the image of the vehicle target rectangular box area and sending it to the second network for vehicle ground point detection, the embodiment of the present invention directly uses one network to detect the target rectangular box and the target ground point at the same time. This method is conducive to deployment on low computing power platforms and saves computing resources. The output of the target detection model in the embodiment of the present invention includes: target detection box, target category information and target ground point. The target refers to the common targets that affect driving in autonomous driving perception, such as: vehicles, pedestrians, two-wheeled vehicles, etc.

[0102] In this embodiment, the road image samples in the training sample set are input into the initial detection model to obtain multiple predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of multiple predicted ground point horizontal coordinates, confidence of multiple predicted ground point vertical coordinates, and confidence of the predicted ground point in a manner that: the area within the target detection frame in the road image sample is taken as a positive sample, the area outside the target detection frame in the road image sample is taken as a negative sample, and the target category is predicted based on the positive sample and the negative sample to obtain the category information of each predicted detection frame and the confidence of each category information. The target detection frame prediction and the target ground point prediction are performed based on the positive sample to obtain multiple predicted detection frame coordinates, confidence of multiple predicted ground point horizontal coordinates, confidence of multiple predicted ground point vertical coordinates, and confidence of the predicted ground point.

[0103] S230, determining a target category loss value, a target rectangular box regression loss value, a ground point coordinate prediction loss value, and a ground point confidence loss value based on the multiple predicted detection box coordinates, the category information of each predicted detection box, the confidence of each category information, the confidence of the multiple predicted ground point horizontal coordinates, the confidence of the multiple predicted ground point vertical coordinates, the confidence of the predicted ground point, and the target information carried by the road image sample.

[0104] In this embodiment, the target category loss value, the target rectangular frame regression loss value, the ground point coordinate prediction loss value, and the ground point confidence loss value are determined according to the multiple predicted detection frame coordinates, the category information of each predicted detection frame, the confidence of each category information, the confidence of multiple predicted ground point horizontal coordinates, the confidence of multiple predicted ground point vertical coordinates, the confidence of the predicted ground point, and the target information carried by the road image sample. The method can be: determining the target category loss value according to the difference between the category information of the predicted detection frame and the target category information carried by the road image sample; determining the target rectangular frame regression loss value according to the difference between the predicted detection frame coordinates and the target detection frame coordinates carried by the road image sample; determining the ground point coordinate prediction loss value according to the predicted detection frame coordinates, the category information of each predicted detection frame, the confidence of each category information, the confidence of multiple predicted ground point horizontal coordinates, the confidence of multiple predicted ground point vertical coordinates, and the target ground point coordinates carried by the road image sample; determining the ground point confidence loss value according to the difference between the predicted ground point confidence and the target ground point confidence carried by the road image sample.

[0105] Optionally, according to the multiple predicted detection box coordinates, the category information of each predicted detection box, the confidence of each category information, the confidence of the multiple predicted ground point horizontal coordinates, the confidence of the multiple predicted ground point vertical coordinates, the confidence of the predicted ground point, and the target information carried by the road image sample, the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value, and the ground point confidence loss value are determined, including:

[0106] A target category loss value is determined according to a difference between category information of the predicted detection box and target category information carried by the road image sample.

[0107] In this embodiment, based on the binary cross entropy loss function, the difference between the category information of the predicted detection box and the target category information carried by the road image sample is determined, and the target category loss value is determined according to the difference between the category information of the predicted detection box and the target category information carried by the road image sample. The binary cross entropy loss function (BCELoss) is a loss function for binary classification problems. It measures the difference between the probability distribution predicted by the model and the true label, and optimizes the model parameters by minimizing this difference, thereby improving the classification performance.

[0108] A target rectangular box regression loss value is determined according to a difference between the predicted detection box coordinates and the target detection box coordinates carried by the road image sample.

[0109] In this embodiment, based on the intersection over union loss function, the difference between the predicted detection frame coordinates and the target detection frame coordinates carried by the road image sample is determined, and the target rectangular frame regression loss value is determined according to the difference between the predicted detection frame coordinates and the target detection frame coordinates carried by the road image sample. The intersection over union (IoU) loss function is an algorithm that calculates the overlap ratio of different images and is often used in target detection tasks in the field of deep learning.

[0110] The predicted grounding point coordinates are determined according to the predicted detection frame coordinates, the category information of each predicted detection frame, the confidence of each category information, the confidence of the horizontal coordinates of multiple predicted grounding points, and the confidence of the vertical coordinates of multiple predicted grounding points.

[0111] In this embodiment, according to the predicted detection frame coordinates, the category information of each predicted detection frame, the confidence of each category information, the confidence of multiple predicted grounding point horizontal coordinates and the confidence of multiple predicted grounding point vertical coordinates, the method of determining the predicted grounding point coordinates can be: obtaining a grounding point integral sequence, selecting a target predicted detection frame coordinate from multiple predicted detection frame coordinates according to the category information of each predicted detection frame and the confidence of each category information, and determining the predicted grounding point coordinates according to the target predicted detection frame coordinates, the grounding point integral sequence, the confidence of multiple predicted grounding point horizontal coordinates and the confidence of multiple predicted grounding point vertical coordinates. It should be noted that the method of obtaining the grounding point integral sequence can be: presetting the grounding point integral sequence. It can also be a grounding point integral sequence determined according to the predicted grounding point vertical coordinate offset and / or the predicted grounding point horizontal coordinate offset. For example, if the predicted horizontal coordinate offsets of the grounding points are 0, 1, 2, 3, 4, 5, 6, 7, and the predicted vertical coordinate offsets of the grounding points are 0, 1, 2, 3, 4, 5, 6, 7, then the grounding point integral sequence is determined to be 0, 1, 2, 3, 4, 5, 6, 7. Alternatively, the maximum value of the grounding point integral sequence may be preset, and then the grounding point integral sequence may be determined based on the preset maximum value. For example, if the maximum value of the grounding point integral sequence is preset to 8, then the grounding point integral sequence may be determined to be: 0, 1, 2, 3, 4, 5, 6, 7.

[0112] In this embodiment, according to the target prediction detection frame coordinates, the ground point integral sequence, the confidences of the multiple predicted ground point horizontal coordinates, and the confidences of the multiple predicted ground point vertical coordinates, the method of determining the predicted ground point coordinates can be: determining the predicted ground point horizontal coordinate offset according to the ground point integral sequence and the confidences of the multiple predicted ground point horizontal coordinates, determining the predicted ground point vertical coordinate offset according to the ground point integral sequence and the confidences of the multiple predicted ground point vertical coordinates, determining the predicted ground point horizontal coordinate according to the target prediction detection frame horizontal coordinate and the predicted ground point horizontal coordinate offset, and determining the predicted ground point vertical coordinate according to the target prediction detection frame vertical coordinate and the predicted ground point vertical coordinate offset. Determine the predicted ground point coordinates according to the predicted ground point horizontal coordinate and the predicted ground point vertical coordinate.

[0113] It should be noted that if there are multiple predicted grounding points, the sum of the products of the values ​​in the grounding point integral sequence and the confidences of the corresponding horizontal coordinate offsets of the predicted grounding points is determined as the horizontal coordinate offsets of the predicted grounding points. For example, if there are four predicted grounding points, namely the first predicted grounding point, the second predicted grounding point, the third predicted grounding point and the fourth predicted grounding point, the sum of the products of the values ​​in the grounding point integral sequence and the confidences of the corresponding horizontal coordinate offsets of the first predicted grounding point is determined as the horizontal coordinate offset of the first predicted grounding point. The calculation method of the horizontal coordinate offsets of the second predicted grounding point, the third predicted grounding point and the fourth predicted grounding point is the same as that of the first predicted grounding point, which will not be repeated here. If there are multiple predicted grounding points, the sum of the products of the values ​​in the grounding point integral sequence and the confidences of the corresponding vertical coordinate offsets of the predicted grounding points is determined as the vertical coordinate offset of the predicted grounding points. For example, if there are four predicted grounding points, namely the first predicted grounding point, the second predicted grounding point, the third predicted grounding point and the fourth predicted grounding point, the sum of the products of each numerical value in the grounding point integral sequence and the confidence of each vertical coordinate offset of the corresponding first predicted grounding point is determined as the vertical coordinate offset of the first predicted grounding point. The calculation method of the vertical coordinate offsets of the second predicted grounding point, the third predicted grounding point and the fourth predicted grounding point is the same as that of the first predicted grounding point, and will not be repeated here.

[0114] A ground point coordinate prediction loss value is determined according to a difference between the predicted ground point coordinates and the target ground point coordinates carried by the road image sample.

[0115] In this embodiment, based on the DFL loss function, the difference between the predicted ground point coordinates and the target ground point coordinates carried by the road image sample is determined, and the ground point coordinate prediction loss value is determined according to the difference between the predicted ground point coordinates and the target ground point coordinates carried by the road image sample. DFL stands for Distribution Focal Loss, which is a loss function used for target detection.

[0116] A touchdown point confidence loss value is determined according to a difference between the predicted touchdown point confidence and the target touchdown point confidence carried by the road image sample.

[0117] In this embodiment, based on the binary cross entropy loss function, the difference between the predicted ground point confidence and the target ground point confidence carried by the road image sample is determined, and the ground point confidence loss value is determined according to the difference between the predicted ground point confidence and the target ground point confidence carried by the road image sample.

[0118] The technical solution provided by the embodiment of the present invention determines the target category loss value according to the difference between the category information of the predicted detection box and the target category information carried by the road image sample; determines the target rectangular frame regression loss value according to the difference between the predicted detection box coordinates and the target detection box coordinates carried by the road image sample; determines the predicted grounding point coordinates according to the predicted detection box coordinates, the category information of each predicted detection box, the confidence of each category information, the confidence of multiple predicted grounding point horizontal coordinates and the confidence of multiple predicted grounding point vertical coordinates; determines the grounding point coordinate prediction loss value according to the difference between the predicted grounding point coordinates and the target grounding point coordinates carried by the road image sample; determines the grounding point confidence loss value according to the difference between the predicted grounding point confidence and the target grounding point confidence carried by the road image sample, and then trains the model based on the above loss values, which can improve the accuracy of target detection.

[0119] S240, training the parameters of the initial detection model according to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value and the ground point confidence loss value to obtain a target detection model.

[0120] In this embodiment, the parameters of the initial detection model are trained according to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value, and the ground point confidence loss value, and the target detection model can be obtained by: obtaining the position constraint loss value, and training the parameters of the initial detection model according to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value, the ground point confidence loss value, and the position constraint loss value to obtain the target detection model.

[0121] Optionally, training the parameters of the initial detection model according to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value, and the ground point confidence loss value to obtain the target detection model includes:

[0122] A position constraint loss value is determined according to a difference between the predicted ground point coordinates beyond the predicted detection frame and the target ground point coordinates carried by the road image sample.

[0123] In this embodiment, according to the difference between the predicted ground point coordinates beyond the predicted detection frame and the target ground point coordinates carried by the road image sample, the method of determining the position constraint loss value can be: based on the position constraint loss function, determine the difference between the predicted ground point coordinates beyond the predicted detection frame and the target ground point coordinates carried by the road image sample, and determine the position constraint loss value according to the difference between the predicted ground point coordinates beyond the predicted detection frame and the target ground point coordinates carried by the road image sample. The position constraint loss function is as follows:

[0124]

[0125] Among them, L point_out is the position constraint loss value, that is, the mean square error between the coordinates of the predicted ground point beyond the predicted detection box and the coordinates of the target ground point carried by the road image sample, f(x) is the coordinates of the predicted ground point beyond the predicted detection box, y is the coordinates of the target ground point carried by the road image sample, and n is the total number of predicted ground points beyond the predicted detection box.

[0126] In this embodiment, the method for determining the position constraint loss value according to the difference between the predicted ground point coordinates beyond the predicted detection frame and the target ground point coordinates carried by the road image sample can be: obtaining a ground point integral sequence, selecting a target predicted detection frame coordinate from multiple predicted detection frame coordinates according to the category information of each predicted detection frame and the confidence of each category information, determining the predicted ground point coordinates according to the target predicted detection frame coordinates, the ground point integral sequence, the confidence of the horizontal coordinates of the multiple predicted ground points, and the confidence of the vertical coordinates of the multiple predicted ground points, determining the predicted ground point coordinates beyond the predicted detection frame according to the predicted ground point coordinates and the multiple predicted detection frame coordinates, and determining the position constraint loss value according to the difference between the predicted ground point coordinates beyond the predicted detection frame and the target ground point coordinates carried by the road image sample.

[0127] According to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value, the ground point confidence loss value and the position constraint loss value, the parameters of the initial detection model are trained to obtain a target detection model.

[0128] It should be noted that during model training, there may be errors in the ground point prediction results, that is, the predicted ground point coordinates exceed the predicted detection box. The technical solution provided by the embodiment of the present invention determines the position constraint loss value based on the difference between the predicted ground point coordinates that exceed the predicted detection box and the target ground point coordinates carried by the road image sample, and then trains the parameters of the initial detection model based on the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value, the ground point confidence loss value and the position constraint loss value to obtain the target detection model, which can avoid the occurrence of incorrect ground point prediction results and further improve the ground point detection performance.

[0129] In a specific example, the object detection model training process includes:

[0130] Step 1: Load historical road images and corresponding labels. The labels include: target detection frame coordinates (X1, Y1, X2, Y2), which represent the upper left corner coordinates and lower right corner coordinates of the detection frame respectively; target categories, such as pedestrians, vehicles, two-wheeled vehicles, etc.; when the target is a vehicle, there are 4 target grounding points, representing 4 wheels, and the coordinates of each wheel are (x, y, s), which represent the grounding point coordinates and visibility respectively, where s=0 means that the point is invisible and blocked, and s=1 means that the point is visible.

[0131] Step 2: Input the road image samples into the initial detection model, perform forward inference on the model, and obtain the corresponding output tensors on different detection branches at different scales.

[0132] Step 3: Parse the tensor output by the model. The output tensor sizes of different detection branches are described as follows (for the convenience of description, only the "20×20" scale is used as an example in the following, and the other scales are the same): The tensor size of the detection frame detection branch is "1×4×20×20", which is the number of batches, the number of channels, the height, and the width in order. The number of channels here is 4, representing the 4 coordinate points (X1, Y1, X2, Y2) of the output target detection frame. The tensor size of the category detection branch is "1×3×20×20", where the number of channels is 3, representing the identification of 3 categories of targets: pedestrians, vehicles, and two-wheeled vehicles. The regular tensor size of the ground point detection branch is "1×12×20×20", where the number of channels is 12, representing a total of 4 ground points, each of which has three attributes (x, y, s), where S represents the confidence of the target ground point. In the embodiment of the present invention, the tensor size of the grounding point detection branch is "1×68×20×20", where the number of channels is 68, which means that there are 4 grounding points in total, and each target grounding point has 17 values ​​in total. For example, each target grounding point includes: reg x1 ,reg x2 ,…reg x8 ,reg y1 ,reg y2 ,…reg y8 ,score. Among them, score indicates whether it is visible (a value between 0-1).

[0133] The contact point between the vehicle wheel and the ground is often not a point, but a plane. There is a certain deviation when marking the wheel contact point, so there may be errors when using a point to accurately describe a contact surface. Based on this, the embodiment of the present invention uses the form of integration to represent the wheel contact point, that is, the contact point prediction is represented as a distribution prediction, which is specifically implemented as follows:

[0134] First, determine the number of wheel point integral templates "reg_max", here take 8 as an example. Then generate the following integral templates: 0, 1, 2, 3, 4, 5, 6, 7, which respectively represent the offset values ​​of the corresponding grids of the actual grounding point distance.

[0135] Step 4: Select positive and negative samples. Positive samples are: grids that contain the target. Negative samples are: grids that do not contain the target.

[0136] Step 5: Predict the target category based on positive and negative samples, calculate the target category loss, and use the BCELoss function for the target category loss.

[0137] Step 6: Predict the target detection box based on the positive sample and calculate the target detection box regression loss. The IOULoss function is used for the target detection box regression loss.

[0138] Step 7: Predict the grounding point based on the positive sample and calculate the target grounding point loss. The target grounding point loss includes the grounding point coordinate prediction loss and the grounding point confidence prediction loss. The grounding point coordinate prediction loss uses the Distribution Focal Loss function, and the grounding point confidence prediction loss uses the BCELoss function.

[0139] In the above process, since the target detection categories are not limited to vehicles, but also include pedestrians and two-wheeled vehicles, only vehicles need to perform grounding point detection. Therefore, it is necessary to filter the positive samples before grounding point detection (the original positive samples contain targets of different categories). The filtering method is: only vehicle targets with vehicle grounding point annotation data. If the vehicle target does not have a grounding point marked, it will affect the accuracy of the grounding point confidence, that is, the vehicle target actually contains a grounding point, but because it is not marked, it may cause the model to be considered to have no grounding point during training, resulting in deviation. Therefore, in the above process, a positive sample filtering module is added, and the module processing flow is as follows:

[0140] First, determine the category of the corresponding label in the positive sample. If it is a vehicle, keep it, otherwise discard it. Then determine whether the vehicle label contains a grounding point. If so, keep it, otherwise discard it.

[0141] During model training, there may be cases where the vehicle touchdown point prediction result is incorrect and exceeds the vehicle target box.

[0142] To address this problem, an embodiment of the present invention proposes a position constraint loss function, which is used to constrain the grounding points that exceed the target frame to avoid this situation. The position constraint loss function is as follows:

[0143]

[0144] Among them, L point_outis the position constraint loss value, that is, the mean square error between the coordinates of the predicted ground point beyond the predicted detection box and the coordinates of the target ground point carried by the road image sample, f(x) is the coordinates of the predicted ground point beyond the predicted detection box, y is the coordinates of the target ground point carried by the road image sample, and n is the total number of predicted ground points beyond the predicted detection box.

[0145] The final loss function is defined as follows:

[0146] L=αL box +βL cls +γ all (γ1L point_pose +γ2L point_score +γ3L point_out )

[0147] Among them, α, β, γ all ,γ1,γ2,γ3 are weight parameters between different losses, L box is the target rectangle regression loss value, L cls is the target category loss value, L point_pose is the predicted loss value of the grounding point coordinates, L point_score is the confidence loss value of the grounding point, L point_out is the position constraint loss value.

[0148] The technical solution of this embodiment is to first input the road image samples in the training sample set into the initial detection model to obtain multiple predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of multiple predicted ground point horizontal coordinates, confidence of multiple predicted ground point vertical coordinates, and confidence of predicted ground point. Then, according to the multiple predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of multiple predicted ground point horizontal coordinates, confidence of multiple predicted ground point vertical coordinates, confidence of predicted ground point, and target information carried by the road image samples, determine the target category loss value, target rectangular frame regression loss value, ground point coordinate prediction loss value, and ground point confidence loss value. Finally, according to the target category loss value, target rectangular frame regression loss value, ground point coordinate prediction loss value, and ground point confidence loss value, train the parameters of the initial detection model to obtain the target detection model. The target detection model trained in the above manner is used for target detection, which can improve the accuracy of target detection. In addition, using one target detection model can simultaneously detect the target detection box and the target ground point, which also saves computing resources.

[0149] Embodiment 3

[0150] Figure 3This is a schematic diagram of the structure of a target detection device provided by an embodiment of the present invention. This embodiment is applicable to the case of target detection. The device can be implemented in software and / or hardware. The device can be integrated in any device that provides target detection function, such as Figure 3 As shown, the target detection device specifically includes: a road image acquisition module 310 to be processed, a first detection module 320 and a second detection module 330.

[0151] Wherein, the road image to be processed acquisition module is used to acquire the road image to be processed;

[0152] A first detection module is used to input the road image to be processed into a target detection model to obtain target detection frame coordinates, target category information, confidences of abscissa offsets of multiple target ground points, and confidences of ordinate offsets of multiple target ground points;

[0153] The second detection module is used to determine the coordinates of the target grounding point according to the target detection frame coordinates, the confidence levels of the horizontal coordinate offsets of the multiple target grounding points, and the confidence levels of the vertical coordinate offsets of the multiple target grounding points.

[0154] The above-mentioned product can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0155] The technical solution of this embodiment first inputs the road image to be processed into the target detection model to obtain the target detection frame coordinates, target category information, confidences of multiple target ground point horizontal coordinate offsets, and confidences of multiple target ground point vertical coordinate offsets. Then, according to the target detection frame coordinates, the confidences of multiple target ground point horizontal coordinate offsets, and the confidences of multiple target ground point vertical coordinate offsets, the target ground point coordinates are determined, which can more accurately determine the target ground point coordinates, thereby improving the accuracy of target detection.

[0156] Embodiment 4

[0157] Figure 4 This is a schematic diagram of the structure of a target detection model training device provided by an embodiment of the present invention. This embodiment is applicable to the case of target detection model training. The device can be implemented in software and / or hardware. The device can be integrated in any device that provides target detection model training function, such as Figure 4 As shown, the target detection model training device specifically includes: a training sample set acquisition module 410, a prediction module 420, a loss value determination module 430 and a training module 440.

[0158] The training sample set acquisition module is used to acquire a training sample set, wherein the training sample set includes: a road image sample, the road image sample carries target information, and the target information includes: target detection frame coordinates, target category information, target ground point coordinates, and target ground point confidence;

[0159] A prediction module, used for inputting the road image samples in the training sample set into the initial detection model to obtain a plurality of predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of a plurality of predicted ground point horizontal coordinates, confidence of a plurality of predicted ground point vertical coordinates and confidence of the predicted ground point;

[0160] a loss value determination module, configured to determine a target category loss value, a target rectangular box regression loss value, a ground point coordinate prediction loss value, and a ground point confidence loss value according to the multiple predicted detection box coordinates, the category information of each predicted detection box, the confidence of each category information, the multiple horizontal coordinates and multiple vertical coordinates of the predicted ground point, the confidence of each horizontal coordinate, the confidence of each vertical coordinate, the confidence of the predicted ground point, and the target information carried by the road image sample;

[0161] A training module is used to train the parameters of the initial detection model according to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value and the ground point confidence loss value to obtain a target detection model.

[0162] The above-mentioned product can execute the method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0163] The technical solution of this embodiment is to first input the road image samples in the training sample set into the initial detection model to obtain multiple predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of multiple predicted ground point horizontal coordinates, confidence of multiple predicted ground point vertical coordinates, and confidence of predicted ground point. Then, according to the multiple predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of multiple predicted ground point horizontal coordinates, confidence of multiple predicted ground point vertical coordinates, confidence of predicted ground point, and target information carried by the road image samples, determine the target category loss value, target rectangular frame regression loss value, ground point coordinate prediction loss value, and ground point confidence loss value. Finally, according to the target category loss value, target rectangular frame regression loss value, ground point coordinate prediction loss value, and ground point confidence loss value, train the parameters of the initial detection model to obtain the target detection model. The target detection model trained in the above manner is used for target detection, which can improve the accuracy of target detection. In addition, using one target detection model can simultaneously detect the target detection box and the target ground point, which also saves computing resources.

[0164] Embodiment 5

[0165] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0166] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0167] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0168] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 performs the various methods and processes described above, such as a target detection method, or a target detection model training method.

[0169] In some embodiments, the target detection method, or the target detection model training method, may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the target detection method, or the target detection model training method described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute the target detection method, or the target detection model training method, in any other appropriate manner (e.g., by means of firmware).

[0170] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0171] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0172] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0173] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0174] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0175] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0176] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0177] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the target detection method according to any embodiment of the present invention, or the target detection model training method.

[0178] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0179] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A target detection method, characterized in that: include: Acquire a road image to be processed; Inputting the road image to be processed into a target detection model to obtain target detection frame coordinates, target category information, confidences of a plurality of target ground point horizontal coordinate offsets, and confidences of a plurality of target ground point vertical coordinate offsets; The target grounding point coordinates are determined according to the target detection frame coordinates, the confidence levels of the horizontal coordinate offsets of the multiple target grounding points, and the confidence levels of the vertical coordinate offsets of the multiple target grounding points.

2. The method according to claim 1, characterized in that Determining the target grounding point coordinates according to the target detection frame coordinates, the confidences of the horizontal coordinate offsets of the multiple target grounding points, and the confidences of the vertical coordinate offsets of the multiple target grounding points includes: Get the grounding point integral sequence; The sum of the products of each value in the touchdown point integral sequence and the confidence level of the horizontal coordinate offset of each target touchdown point is determined as the horizontal coordinate offset; The sum of the products of each value in the touchdown point integral sequence and the confidence level of the ordinate offset of each target touchdown point is determined as the ordinate offset; The target grounding point coordinates are determined according to the target detection frame coordinates, the horizontal coordinate offset and the vertical coordinate offset.

3. The method according to claim 2, characterized in that Determining the target ground point coordinates according to the target detection frame coordinates, the horizontal coordinate offset and the vertical coordinate offset includes: Determine the horizontal coordinate of the target grounding point according to the horizontal coordinate of the target detection frame and the horizontal coordinate offset; Determine the target grounding point ordinate according to the target detection frame ordinate and the ordinate offset; The target grounding point coordinates are determined according to the target grounding point abscissa and the target grounding point ordinate.

4. A target detection model training method, characterized in that: include: Acquire a training sample set, wherein the training sample set includes: a road image sample, the road image sample carries target information, and the target information includes: target detection frame coordinates, target category information, target ground point coordinates, and target ground point confidence; Inputting the road image samples in the training sample set into the initial detection model, obtaining a plurality of predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of a plurality of predicted ground point horizontal coordinates, confidence of a plurality of predicted ground point vertical coordinates, and confidence of the predicted ground point; Determine a target category loss value, a target rectangular box regression loss value, a ground point coordinate prediction loss value, and a ground point confidence loss value according to the multiple predicted detection box coordinates, the category information of each predicted detection box, the confidence of each category information, the confidence of the multiple predicted ground point horizontal coordinates, the confidence of the multiple predicted ground point vertical coordinates, the confidence of the predicted ground point, and the target information carried by the road image sample; According to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value and the ground point confidence loss value, the parameters of the initial detection model are trained to obtain a target detection model.

5. The method according to claim 4, characterized in that According to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value, and the ground point confidence loss value, the parameters of the initial detection model are trained to obtain a target detection model, including: Determining a position constraint loss value according to a difference between the predicted ground point coordinates beyond the predicted detection box and the target ground point coordinates carried by the road image sample; According to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value, the ground point confidence loss value and the position constraint loss value, the parameters of the initial detection model are trained to obtain a target detection model.

6. The method according to claim 4, characterized in that According to the multiple predicted detection frame coordinates, the category information of each predicted detection frame, the confidence of each category information, the confidence of the multiple predicted ground point horizontal coordinates, the confidence of the multiple predicted ground point vertical coordinates, the confidence of the predicted ground point, and the target information carried by the road image sample, the target category loss value, the target rectangular frame regression loss value, the ground point coordinate prediction loss value, and the ground point confidence loss value are determined, including: Determining a target category loss value according to a difference between category information of the predicted detection box and target category information carried by the road image sample; Determining a target rectangular frame regression loss value according to a difference between the predicted detection frame coordinates and the target detection frame coordinates carried by the road image sample; Determining the predicted grounding point coordinates according to the predicted detection frame coordinates, the category information of each predicted detection frame, the confidence of each category information, the confidence of the horizontal coordinates of the multiple predicted grounding points, and the confidence of the vertical coordinates of the multiple predicted grounding points; Determining a grounding point coordinate prediction loss value according to a difference between the predicted grounding point coordinates and the target grounding point coordinates carried by the road image sample; A touchdown point confidence loss value is determined according to a difference between the predicted touchdown point confidence and the target touchdown point confidence carried by the road image sample.

7. The method according to claim 4, characterized in that Get the training sample set, including: Acquire a historical road image set, wherein the historical road image set includes: historical road images, the historical road images carrying target information; Partitioning the historical road image according to the target information carried by the historical road image to obtain a first area and a second area, wherein the first area is an area within the target detection frame, and the second area is an area outside the target detection frame; Determine the first area in the historical road image as a road image sample in a positive sample set; Determine the second area in the historical road image as a road image sample in a negative sample set; A training sample set is determined according to the positive sample set and the negative sample set.

8. The method according to claim 7, characterized in that Determining a training sample set according to the positive sample set and the negative sample set includes: According to target information carried by road image samples in the positive sample set, the positive sample set is screened to obtain a screened positive sample set; A training sample set is determined according to the screened positive sample set and the negative sample set.

9. The method according to claim 8, characterized in that According to the target information carried by the road image samples in the positive sample set, the positive sample set is screened to obtain a screened positive sample set, including: The road image samples whose target category information is not a vehicle and the road image samples whose target category information is a vehicle and whose target touchdown point confidence is a first value are deleted from the positive sample set to obtain a filtered positive sample set.

10. A target detection device, characterized in that: include: A road image acquisition module to be processed, used to acquire the road image to be processed; A first detection module is used to input the road image to be processed into a target detection model to obtain target detection frame coordinates, target category information, confidences of abscissa offsets of multiple target ground points, and confidences of ordinate offsets of multiple target ground points; The second detection module is used to determine the coordinates of the target grounding point according to the target detection frame coordinates, the confidence levels of the horizontal coordinate offsets of the multiple target grounding points, and the confidence levels of the vertical coordinate offsets of the multiple target grounding points.

11. A target detection model training device, characterized in that: include: A training sample set acquisition module is used to acquire a training sample set, wherein the training sample set includes: a road image sample, the road image sample carries target information, and the target information includes: target detection frame coordinates, target category information, target ground point coordinates, and target ground point confidence; A prediction module, used for inputting the road image samples in the training sample set into the initial detection model to obtain a plurality of predicted detection frame coordinates, category information of each predicted detection frame, confidence of each category information, confidence of a plurality of predicted ground point horizontal coordinates, confidence of a plurality of predicted ground point vertical coordinates and confidence of the predicted ground point; a loss value determination module, configured to determine a target category loss value, a target rectangular box regression loss value, a ground point coordinate prediction loss value, and a ground point confidence loss value according to the multiple predicted detection box coordinates, the category information of each predicted detection box, the confidence of each category information, the multiple horizontal coordinates and multiple vertical coordinates of the predicted ground point, the confidence of each horizontal coordinate, the confidence of each vertical coordinate, the confidence of the predicted ground point, and the target information carried by the road image sample; A training module is used to train the parameters of the initial detection model according to the target category loss value, the target rectangular box regression loss value, the ground point coordinate prediction loss value and the ground point confidence loss value to obtain a target detection model.

12. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so as to enable the at least one processor to perform the method according to any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method according to any one of claims 1 to 9 when executed.

14. A computer program product, characterized in that The computer program product comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images