Target detection method and related device

Abstract

The embodiment of the invention provides a target detection method and a related device. The method comprises the following steps: inputting a road image into a trained detection model, and enabling the trained detection model to output marking information corresponding to detection objects respectively. The detection model comprises a trunk neural network and a plurality of neural network heads, and image features of the road image are extracted through the trunk neural network. And then, respectively determining annotation information of the detection object according to the image features through the plurality of neural network heads. According to the embodiment of the invention, the plurality of neural network heads are trained according to different detection objects, so that the plurality of neural network heads can detect the corresponding detection objects in the road image. The vehicle can realize the detection task of a plurality of detection objects through one detection model, and the speed and efficiency of target detection are improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a target detection method and related devices. Background Art

[0002] Object detection is an important branch of computer vision, which involves the task of identifying and locating specific objects from images or videos. The purpose of object detection is to identify specific objects in an image and give the precise location and size of these objects. Object detection is usually used to solve classification problems and localization problems. Classification problems are to determine whether a specific object exists in an image. Localization problems are to determine the exact location of an object in an image.

[0003] With the development of vehicle assisted driving and autonomous driving, vehicles are increasingly relying on target detection technology. For example, vehicles use target detection technology to identify traffic lights, traffic signs, etc. on the road to support the driving decisions of users and / or vehicles. This means that vehicles need chips with stronger computing power to provide operators for a large number of target detection tasks. If the vehicle cannot provide enough computing power for a large number of target detection tasks, it may take a long time to infer the corresponding results. The timeliness of target detection for vehicle assisted driving or autonomous driving cannot be met, affecting driving safety. Summary of the invention

[0004] The embodiment of the present application provides a target detection method and related device, wherein a vehicle detects different types of detection objects respectively through multiple neural network heads in a detection model, thereby improving detection efficiency and accelerating detection speed.

[0005] In a first aspect, an embodiment of the present application provides a target detection method, the method comprising:

[0006] Acquire a road image to be detected, wherein the road image includes a detection object;

[0007] The road image is input into a trained detection model so that the trained detection model outputs the labeling information of the detection object, wherein the detection model includes a backbone neural network and multiple neural network heads, the backbone neural network is used to extract image features from the road image, and the labeling information of the detection object includes results output by the multiple neural network heads based on the image features.

[0008] In the above method, the image features are extracted through the backbone neural network, and different detection objects are identified respectively through multiple neural network heads. The vehicle can detect and identify multiple detection objects based on one neural network model. Compared with the existing solution that detects the corresponding detection objects through the corresponding neural network models, the present application uses the same backbone neural network to extract image features, and then identifies based on different neural network heads, which can achieve multiple detection tasks in a short time. At the same time, the present application uses one neural network model to detect multiple detection objects, instead of using multiple neural network models to detect them separately. It reduces the computing pressure of the vehicle and saves the storage space of the vehicle.

[0009] In an optional solution of the first aspect, inputting the road image into a trained detection model so that the trained detection model outputs the labeling information of the detection object includes:

[0010] Inputting the road image into the backbone neural network of the trained detection model so that the backbone neural network outputs image features of the road image;

[0011] The image features are respectively input into the multiple neural network heads to obtain the labeling information of the detection object.

[0012] In the above method, since the backbone neural network is used to extract image features from the road image, and multiple neural network heads are used to determine the labeling information of the detection object based on the image features. Therefore, the vehicle first inputs the road image to be detected into the backbone neural network, and extracts features of the road image through the backbone neural network to obtain image features. Then, the vehicle determines the labeling information of the detection object based on the above image features through multiple neural network heads. Based on the characteristics of the backbone neural network for feature extraction and the characteristics of the neural network head for detection, feature extraction is performed through the same backbone neural network instead of using multiple backbone neural networks to extract features from multiple images respectively, which simplifies the number of neural network models. In addition, by detecting target objects separately through multiple neural network heads, detection and recognition of multiple detection objects can be achieved on the basis of one neural network model.

[0013] In an optional solution of the first aspect, inputting the image features into the multiple neural network heads respectively to obtain the labeling information of the detection object includes:

[0014] Inputting the image features into the multiple neural network heads respectively, so that the multiple neural network heads respectively output detection results, wherein the multiple detection results include the labeling information of the detection object, the labeling information of the detection object is output by the target neural network head, and the target neural network head is trained according to the image features of the detection object;

[0015] Determine labeling information of the detection object according to the multiple detection results.

[0016] In the above method, the image features are input into multiple neural network heads to obtain multiple detection results. Since the target neural network head is trained by a specific detection object, the target neural network head can determine whether a specific detection object exists based on the image features. If it exists, the detection result output by the target neural network head includes the labeling information of the specific detection object. If it does not exist, the detection result output by the target neural network head is irrelevant to the specific detection object.

[0017] In an optional solution of the first aspect, the image features include image features of a first detection object and image features of a second detection object, and inputting the road image into a trained detection model so that the trained detection model outputs labeling information of the detection object includes:

[0018] Inputting the image features into the multiple neural network heads respectively, and outputting a first detection result based on a first neural network head, wherein the first detection result includes labeling information of the first detection object, and the first neural network head is trained according to the image features of the first detection object;

[0019] outputting a second detection result based on a second neural network head, wherein the second detection result includes labeling information of the second detection object, and the second neural network head is trained according to image features of the second detection object;

[0020] A third detection result is output based on a third neural network head, wherein the third detection result is independent of the first detection object and the second detection object, and the third neural network head is trained according to image features of the third detection object.

[0021] In the above method, since the target neural network head is trained by a specific detection object, the target neural network head can determine whether there is a specific detection object based on the image features. If it exists, the detection result output by the target neural network head includes the labeling information of the specific detection object. If it does not exist, the detection result output by the target neural network head has nothing to do with the specific detection object.

[0022] In an optional solution of the first aspect, labeling information of multiple detection objects is reflected on the road image.

[0023] In the above method, the labeling information of the detection objects output by multiple neural network heads can be reflected on the same image, which is convenient for observing the position and type of the detection objects on the image.

[0024] In an optional solution of the first aspect, the backbone neural network in the trained detection model is trained based on multiple training data sets, and each training data set in the multiple training data sets corresponds to one of the detection objects;

[0025] The multiple neural network heads in the trained detection model are trained based on the output results respectively, and the output results are the results output by the backbone neural network.

[0026] In the above method, the backbone neural network is trained through multiple training data sets, and then different neural network heads are trained separately according to the output of the backbone neural network. Since the training of multiple neural network heads is separate, each neural network head has a strong ability to detect specific objects, ensuring the accuracy of the inference results.

[0027] In an optional scheme of the first aspect, the first neural network head in the trained detection model is trained based on the first output result, the first neural network head is any one of the multiple neural network heads, and the first output result is the result output during the training of the backbone neural network based on the training data set corresponding to the first detection object.

[0028] In the above method, the first neural network head is trained by the first output result. The first output result corresponds to the output result of the backbone neural network for the data set corresponding to the first detection object, so the first neural network head trained by the first output result can be used to detect the first detection object. Since the training of multiple neural network heads is separate, each neural network head has a strong ability to detect a specific object, ensuring the accuracy of the reasoning result.

[0029] In an optional solution of the first aspect, during the training process of the detection model, the loss value of the detection model includes loss values ​​of a plurality of the detection objects;

[0030] If the difference between the loss values ​​of the multiple detection objects is greater than a preset threshold, the loss values ​​of the multiple detection objects are adjusted so that the difference between the loss values ​​of the detection objects after adjustment is less than or equal to the preset threshold.

[0031] In the above method, the electronic device can dynamically adjust the weight of the loss value according to the difference between multiple loss values, that is, adjust the weight a and weight b mentioned above. This method of dynamically adjusting the weight can change with the change of each loss value, so that the neural network can better balance the relationship between different losses during the training process. This method can effectively improve the training efficiency and performance of the model.

[0032] In an optional solution of the first aspect, during the training process of the detection model, when the loss value of the first detection object is greater than the loss value of the second detection object, the loss value of the second detection object is increased.

[0033] In a second aspect, an embodiment of the present application provides a target detection device, which includes a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store a computer program, and the processor is used to call and run the computer program so that the target detection device performs a method as described in any one of the first aspects above.

[0034] In a third aspect, an embodiment of the present application provides a computing device, comprising a processor and a memory; the processor is coupled to the memory, the memory is used to store a computer program, and the processor is used to call and run the computer program so that the computing device executes a method as described in any one of the first aspects above.

[0035] Optionally, the computing device further comprises a communication interface, wherein the communication interface is used to receive and / or send data, and / or the communication interface is used to provide input and / or output for the processor.

[0036] It should be noted that the above embodiment is described by taking a processor (or general-purpose processor) that executes the method by calling a computer specification as an example. In the specific implementation process, the processor can also be a dedicated processor, in which case the computer instructions have been pre-loaded in the processor. Optionally, the processor can also include both a dedicated processor and a general-purpose processor.

[0037] Optionally, the processor and the memory may also be integrated into one device, that is, the processor and the memory may also be integrated together.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer or a processor, the method described in any one of the first aspects above is implemented.

[0039] In a fifth aspect, an embodiment of the present application provides a vehicle, comprising the target detection device of the second aspect, so that the vehicle implements the method described in any one of the first aspect.

[0040] The beneficial effects of the technical solutions provided in the second to fifth aspects of the present application can refer to the beneficial effects of the technical solution of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The following is a brief introduction to the drawings required for describing the embodiments.

[0042] Figure 1 is a schematic diagram of the architecture of a vehicle provided in an embodiment of the present application;

[0043] Figure 2 It is a flowchart of a target detection method provided in an embodiment of the present application;

[0044] Figure 3 is a schematic diagram of a detection model provided in an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of a road image provided in an embodiment of the present application;

[0046] Figure 5 It is a flowchart of a target detection provided by an embodiment of the present application;

[0047] Figure 6 It is a flow chart of a training detection model provided in an embodiment of the present application;

[0048] Figure 7 It is a block diagram of the functional units of a target detection device provided in an embodiment of the present application;

[0049] Figure 8 It is a structural diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0051] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0052] In order to facilitate the understanding of the embodiments of the present application, the technical problems that the present application specifically aims to solve are analyzed and proposed below.

[0053] Object detection is a basic application in the field of computer vision and is widely used in scenarios such as robot navigation, autonomous driving, and defect detection. At present, the development of various detection models used in object detection is relatively mature, such as the Fast-Rcnn series, the YOLO series, the Transformer and its derivative detection model series are relatively mature. The above detection models mainly use convolutional neural networks (CNN) or attention mechanisms. By inputting a large amount of data into the above detection models for training, a trained detection model is obtained. Finally, the trained detection model can be used to achieve the detection task of the detection object.

[0054] With the development of target detection technology, more and more neural network models are used to identify designated detection objects. For example, in the field of vehicles, vehicles can identify designated detection objects such as traffic lights, traffic signs, obstacles, pedestrians, etc. on the road by deploying neural network models. However, most of the current neural network models can only complete one detection task, that is, a detection model trained by a neural network model can mostly only be used to detect one detection object. If multiple detection objects are to be detected, different neural network models need to be deployed on the vehicle to detect different detection objects respectively. However, deploying multiple neural network models on the vehicle requires the on-board chip to provide sufficient computing power. In addition, if multiple neural network models are running on the vehicle at the same time, the computing pressure of the on-board chip will be increased. Therefore, the vehicle may not be able to achieve multiple detection tasks in a short period of time, thereby failing to meet the timeliness of the vehicle's target detection, and even affecting driving safety.

[0055] If the vehicle uses a chip with stronger computing power, although the chip with stronger computing power can provide more operators for the operation of multiple neural network models, the price of the chip will also be more expensive, increasing the cost of the vehicle.

[0056] If multiple detection tasks are implemented using the same detection model, then the same neural network model needs to be trained based on multiple detection objects. This will lead to serious model stacking problems in the trained detection model, poor feature extraction effect for the images to be detected, and low detection accuracy for the detection objects.

[0057] Therefore, in existing target detection schemes, it is difficult to achieve efficient detection and recognition of multiple detection objects in a short time.

[0058] In summary, an embodiment of the present application provides a method for target detection, using a detection model including a backbone neural network and multiple neural network heads to detect and identify multiple detection objects. For example, the backbone neural network is used to extract features of a road image to be detected to obtain image features. The labeling information of the detection object is determined based on the image features by multiple neural network heads respectively. Since different neural network heads are trained based on different detection objects, the multiple neural network heads can respectively detect the corresponding detection objects in the road image.

[0059] In order to more clearly describe the solution of the present application, some of the terms involved in the present application are explained below.

[0060] 1. Convolutional Neural Networks: Convolutional Neural Networks (CNN) are a type of feedforward neural network that includes convolution calculations and has a deep structure. It is one of the representative algorithms of deep learning. Convolutional neural networks have representation learning capabilities and can perform shift-invariant classification on input information according to their hierarchical structure. Therefore, they are also called shift-invariant artificial neural networks (SIANN).

[0061] 2. Backbone neural network: The backbone neural network is used to obtain image features by extracting features from images. As part of the network, it is generally used to extract image information at the front end of the neural network model and generate a feature map. Then it is used by the network after the backbone neural network. The VGG model (Visual Geometry Group Network, VGG) or the residual network (Residual Neural Network, ResNet) is usually used as the backbone neural network because these networks have strong feature extraction capabilities. And the official of the above network provides the model parameters of the above network trained on large data sets (such as Pascal, Imagenet). Since the data volume of the above training set is large enough, the above network has been trained in large quantities through the above training set, which can meet the requirements of the target detection task. Therefore, if the above network is used as the backbone neural network, and then multiple neural network heads are connected after the above network, then on the basis of ensuring the accuracy of the detection model reasoning results, it can also save training time and improve training efficiency. Of course, the choice of the backbone neural network in the embodiment of the present application may include but is not limited to the above VGG or ResNet network.

[0062] 3. Convolutional neural network head: The convolutional neural network head is a network that obtains the network output content, uses the image features extracted by the backbone neural network to make predictions on the detection object.

[0063] The following is an introduction to the system architecture of the embodiment of the present application. It should be noted that the system architecture and business scenarios described in this application are intended to more clearly illustrate the technical solution of this application, and do not constitute a limitation on the technical solution provided by this application. It is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided by this application is also applicable to similar technical problems.

[0064] See also Figure 1 , Figure 1 Schematic diagram of a vehicle structure provided by an embodiment of the present application. Figure 1 As shown, vehicle 10 includes one or more cameras.

[0065] The vehicle 10 may be a vehicle driven by electric energy, a vehicle driven by fuel, or a vehicle driven by new energy hybrid power. Exemplarily, when the vehicle 10 is a vehicle driven by electric energy, it may be a new energy vehicle, such as a pure electric vehicle, an extended-range electric vehicle, a hybrid vehicle, a fuel cell electric vehicle, etc. When the vehicle 10 is a vehicle driven by fuel, it may be a car, an agricultural transport vehicle, a tractor, or a trailer, etc. When the vehicle 10 is a car, it may be a sedan, an off-road vehicle, a truck, a bus, or a van, etc.

[0066] One or more cameras are used to acquire images of the road to be detected. Figure 1 1014, a front view camera 1011, a left view camera 1012, a right view camera 1013, and a rear view camera 1014 are exemplarily shown. It can be understood that the cameras at different positions are used to collect road images in front of, on the left side, on the right side, and / or on the rear of the vehicle 10.

[0067] Exemplarily, the vehicle 10 collects road images through one or more cameras, for example, through the front-view camera 1011 to collect the road image in front of the vehicle 10. Then, the vehicle 10 inputs the collected road image into the trained detection model. The image features of the road image are extracted through the backbone neural network in the detection model, and the labeling information of the detection object is determined according to the image features through the multiple neural network heads in the detection model. Finally, the vehicle 10 outputs the labeling information of the detection object through the detection model.

[0068] Exemplarily, the detection model may be one that is trained by other electronic devices and then sent to the vehicle 10. The detection model may be obtained by training a neural network model with other electronic devices, and the neural network model includes a backbone neural network and multiple neural network heads. Since multiple neural network heads can respectively detect different types of detection objects, during the training process, the electronic device can train the neural network heads separately through multiple training data sets. For example, each of the multiple training data sets corresponds to a detection object. For example, the electronic device trains the first neural network head according to the first training data set, wherein the first training data set is a training data set corresponding to the first detection object. The electronic device first trains the backbone neural network of the neural network model according to the first training data set, and then trains the first neural network head of the neural network model according to the first output result output by the backbone neural network. By training all the neural network heads using the above training method, a trained detection model can be obtained.

[0069] See also Figure 2 , Figure 2 is a flow chart of a target detection method provided in an embodiment of the present application, and the method is applied to Figure 1The vehicle shown. Figure 2 As shown, the method includes but is not limited to the following steps:

[0070] Step S201, obtaining a road image to be detected.

[0071] The road image to be detected may be collected by the vehicle or obtained by the vehicle from other devices. The embodiment of the present application does not impose any restriction on the source of the road image.

[0072] In a possible implementation, the vehicle includes a camera device, which can capture images and / or video streams. Therefore, the vehicle can use images captured by the camera device as road images, or use image frames in the video stream captured by the camera device as road images.

[0073] For example, the camera device may be one or more cameras installed on the vehicle, such as a front view camera, a rear view camera, or a side view camera installed on the vehicle. The vehicle may obtain road images in front of, behind, or around the vehicle in real time through one or more cameras.

[0074] In a possible implementation, in addition to the road images collected by the vehicle itself, the vehicle may also receive road images from other devices. For example, the vehicle may receive road images from other vehicles, perform target detection on the road images from other vehicles in step S202, and then send the target detection results to other vehicles.

[0075] Step S202: input the road image into the trained detection model, so that the trained detection model outputs the labeling information of the detection object.

[0076] Specifically, the vehicle inputs the acquired road image to be detected into the trained detection model, and the detection model can detect and identify the detection object in the road image, thereby outputting the labeling information corresponding to one or more detection objects.

[0077] Wherein, the detection model includes a backbone neural network and multiple neural network heads. The backbone neural network is used to extract image features from road images, and multiple neural network heads are used to determine the labeling information of the detection object according to the image features. The detection model used in the embodiment of the present application can detect and identify various types of detection objects. For example, the detection model can detect and identify detection objects including but not limited to traffic lights, traffic signs, etc. Therefore, the detection model is provided with multiple neural network heads in order to detect and identify different categories of detection objects respectively through multiple neural network heads.

[0078] In one possible implementation, the vehicle inputs a road image into a backbone neural network of a trained detection model, so that the backbone neural network outputs image features of the road image, and then the vehicle inputs the image features into multiple neural network heads respectively to obtain labeling information of the detection object.

[0079] Specifically, since the backbone neural network is used to extract image features from the road image, and the multiple neural network heads are used to determine the labeling information of the detection object according to the image features, the vehicle first inputs the road image to be detected into the backbone neural network, and the backbone neural network extracts features of the road image to obtain image features. Then, the vehicle determines the labeling information of the detection object according to the above image features through multiple neural network heads.

[0080] In a possible implementation, the vehicle inputs the image features into multiple neural network heads respectively, so that the multiple neural network heads respectively output detection results. Then, the labeling information of the detection object is determined according to the multiple detection results.

[0081] Among them, multiple detection results include labeling information of the detection object, and the labeling information of the detection object is output by the target neural network head, and the target neural network head is trained according to the image features of the detection object.

[0082] Specifically, since multiple neural network heads are used to detect and identify different detection objects respectively, when multiple neural network heads detect image features respectively, multiple neural network heads can determine whether there is a corresponding detection object based on the image features. If there is a corresponding detection object, the neural network head will output the detection result corresponding to the detection object. For example, if there are objects such as pedestrians, vehicles, and traffic lights in the road image. And, the traffic light is the detection object, then the neural network head used to detect the traffic light can output the detection result corresponding to the traffic light based on the image features.

[0083] Since multiple neural network heads will detect the image features corresponding to the road image, multiple neural network heads will output corresponding detection results. Among them, multiple detection results include the labeling information of the detection object, and may not include the labeling information of the detection object. For example, if the detection object is a traffic light. Then, the detection result output by the neural network head used to detect the traffic light includes the labeling information of the detection object. The detection result output by the neural network head used to detect traffic signs does not include the labeling information of the detection object. Therefore, the vehicle determines the labeling information of the detection object by integrating multiple detection results output by multiple neural network heads.

[0084] In a possible implementation, labeling information of multiple detected objects may be reflected on the road image.

[0085] Specifically, since there may be more than one category and / or more than one number of detection objects in the road image, multiple neural network heads may output multiple detection results. In order to better display the above multiple detection results, the vehicle can output the labeling information for the detection object according to all the detection results through the detection model. For example, if the vehicle determines that there are 2 first detection objects and 1 second detection object in the road image according to the detection results through the detection model. Then, the vehicle can output the labeling information and quantity of the first detection object, as well as the labeling information and quantity of the second detection object through the detection model.

[0086] In a possible implementation, when the image features include image features of a first detection object and image features of a second detection object. The vehicle inputs the image features into a plurality of neural network heads respectively, and outputs a first detection result based on the first neural network head, wherein the first detection result includes the annotation information of the first detection object, and the first neural network head is trained based on the image features of the first detection object. The vehicle outputs a second detection result based on the second neural network head, wherein the second detection result includes the annotation information of the second detection object, and the second neural network head is trained based on the image features of the second detection object. The vehicle outputs a third detection result based on the third neural network head, wherein the third detection result is independent of the first detection object and the second detection object, and the third neural network head is trained based on the image features of the third detection object.

[0087] For example, see Figure 3 , Figure 3 Schematic diagram of a detection model provided in an embodiment of the present application. Figure 3 As shown, the detection model includes a backbone neural network and multiple neural network heads. Figure 3 Taking two neural network heads as an example, it can be understood that the neural network head in the detection model is not limited to Figure 3 Neural network head 1 and neural network head 2 are shown.

[0088] Among them, neuron a is a neuron of the backbone neural network, and the number of layers of the backbone neural network is N. The number of layers of the backbone neural network is determined according to the actual detection task when training the detection model. The backbone neural network is mainly used to extract image features from road images, and then input the image features into neural network head 1 and neural network head 2.

[0089] Neuron b is a neuron of neural network head 1, and neuron c is a neuron of neural network head 2. Neural network head 1 and neural network head 2 detect different detection objects respectively. For example, the vehicle neural network head 1 detects traffic lights, and the neural network head 2 detects traffic signs. Specifically, the vehicle detects whether there is a traffic light through neural network head 1 based on the image features from the backbone neural network. Similarly, the vehicle detects whether there is a traffic sign through neural network head 2 based on the image features from the backbone neural network.

[0090] like Figure 3 As shown in FIG. 1 , if the vehicle determines that there is a corresponding detection object in the image feature through the neural network head, the neural network head can output the labeling information of the detection object. The labeling information includes but is not limited to the detection box coordinates, category information, target number, confidence level, etc. For example Figure 3 In the example, the vehicle determines that there are traffic lights 1 and traffic lights 2 in the road image through the neural network head 1, and the vehicle can output the labeling information corresponding to traffic lights 1 and the labeling information corresponding to traffic lights 2 through the neural network head 1. For example, the labeling information includes but is not limited to detection box coordinates, category information, target number, confidence, and other information. The vehicle determines that there is traffic sign 1 in the road image through the neural network head 2, and the vehicle can output the labeling information corresponding to traffic sign 1 through the neural network head 2, and the labeling information includes but is not limited to detection box coordinates, category information, target number, confidence, and other information.

[0091] The detection frame coordinates are used to determine the position of the detection object. For example, the vehicle uses a detection model to represent the outline of the detection object in the form of a rectangular frame. At this time, the coordinates of the rectangular frame in the image are the detection frame coordinates of the detection object.

[0092] The category information is used to characterize the category of the detection object. For example, if the detection object is a traffic light, the category information may be the detailed category of the traffic light, including but not limited to red light, yellow light, left turn arrow, straight arrow, etc.

[0093] The confidence level is used to characterize the probability that the detected object belongs to the category corresponding to the confidence level. The higher the confidence level, the higher the probability that the detected object corresponding to the labeled information belongs to the category corresponding to the confidence level. For example, if the confidence level of the traffic light 1 output by the vehicle through the neural network head 1 is greater than the preset value, then it can be determined that the category of the traffic light 1 is the category information in the labeled information.

[0094] The above is a method for a vehicle to detect a target based on a trained detection model provided by an embodiment of the present application. The following will specifically introduce the training process of the detection model. Among them, the training of the detection model can be performed on an electronic device. After the electronic device has trained the detection model, it can send the trained detection model to the vehicle. The vehicle performs target detection based on the trained detection model.

[0095] In a possible implementation, the backbone neural network in the trained detection model is trained based on multiple training data sets, each of the multiple training data sets corresponds to a detection object. The multiple neural network heads in the trained detection model are trained based on output results, respectively, and the output results are the output results of the backbone neural network.

[0096] Specifically, the trained detection model can be obtained by training a neural network model by the electronic device based on a plurality of training data sets, wherein each of the plurality of training data sets corresponds to a detection object.

[0097] Specifically, before training the detection model, the electronic device needs to first obtain a training data set for training. According to the detection object that the detection model needs to check, the training data set obtained by the electronic device needs to include at least one of multiple detection object categories. For example, if the detection objects that the detection model needs to detect and identify include traffic lights and traffic signs, then the training data set obtained by the electronic device needs to include one of traffic lights and traffic signs.

[0098] In a possible implementation, in order to ensure the training effect of the neural network model, the road image in the training data set acquired by the electronic device should not be an image with obvious distortion. The type of road image collected by the electronic device can be determined according to the category of the detection object. For example, if the detection object includes a traffic light and a traffic sign, the road image collected by the electronic device can be an image including a traffic light, an image including a traffic sign, and an image including both a traffic light and a traffic sign.

[0099] For example, see Figure 4 , Figure 4 is a schematic diagram of a road image provided by an embodiment of the present application. Figure 4 As shown, Figure 4 (a) is an image with obvious distortion. It can be seen that Figure 4 In (a), the sidewalk is curved and the size ratio is inconsistent. If the neural network model is trained with images with obvious distortion, the accuracy of the neural network model's reasoning will be affected.

[0100] Figure 4(b) is a road image including traffic lights. It can be seen that Figure 4 (b) is an image of the road ahead captured by the vehicle, which includes elements such as traffic lights and vehicles.

[0101] Figure 4 (c) is a road image including traffic signs. It can be seen that Figure 4 The traffic sign in (b) is used to indicate the type of lane and the speed limit of the lane, for example, a lane with a speed limit of 60km / h to 80km / h, a lane with a speed limit of 40km / h to 80km / h, an emergency lane, etc.

[0102] In a possible implementation, the training data set further includes annotation information, where the annotation information is obtained by annotating the detection object in the road image. The annotation information may be specific information of a detection box in the road image for representing the contour of the detection object.

[0103] For example, the format of the yolo model is used as an example to illustrate that the annotation information is the specific coordinates of the annotation box after normalizing the annotation object on the road image. For example, the coordinates of the center point of the annotation box, the width of the annotation box, and the height of the annotation box. The following is the specific form of the annotation box:

[0104]

[0105] Among them, x is the horizontal coordinate of the center point of the annotation box, y is the vertical coordinate of the center point of the annotation box, X is the horizontal coordinate of the upper left corner of the annotation box, and Y is the horizontal coordinate of the upper left corner of the annotation box. W is the width of the annotation box, H is the height of the annotation box, Img_W is the actual width of the road image, and Img_H is the actual height of the road image. [x, y, w, h] is the label format of the annotated image.

[0106] In a possible implementation, since the electronic device trains the neural network model through multiple detection objects, it is also necessary to classify the road images and their annotation information obtained above according to the categories of the detection objects to obtain multiple training data sets.

[0107] For example, if the categories of the detected objects include traffic lights and traffic signs, then the road image acquired by the electronic device needs to be divided into two categories according to traffic lights and traffic signs. In the case where both traffic lights and traffic signs exist in the road image, one of the traffic lights and traffic signs is selected as the category of the road image.

[0108] Please refer to Table 1, which is a training data set provided by an embodiment of the present application. As shown in Table 1, the road images and annotation information acquired by the electronic device are classified according to the category of the detection object. For example, taking the detection object including traffic lights and traffic signs as an example, the road image including the traffic lights is the data set 1 image, and the annotation information including the traffic lights is the data set 1 label. The road image including the traffic sign is the data set 2 image, and the annotation information including the traffic sign is the data set 2 label. For the convenience of training, the road image and the annotation information can also be divided into corresponding training sets, validation sets and test sets. Among them, the training set is used to train the neural network model, and the validation set is used to evaluate the performance of the neural network model and adjust the hyperparameters of the model during the training process of the neural network model. The test set is used to finally evaluate the performance of the neural network model, that is, the data in the test set are samples that have not yet been learned on the neural network model.

[0109] Table 1

[0110] Number of training sets Number of validation sets Number of test sets Dataset 1 images 4971 1420 1420 Dataset 1 Label 11486 3281 3281 Dataset 2 images 11588 4491 4491 Dataset 2 Labels 14524 5629 5629

[0111] In addition, since the sizes of the road images collected by the electronic device may be inconsistent, the road images may be cropped before training the neural network model to make the sizes of the road images uniform. For example, the road images may be cropped so that the length L is 640 and the width H is 384.

[0112] In one possible implementation, the first neural network head in the trained detection model is trained based on the first output result, the first neural network head is any one of a plurality of neural network heads, and the first output result is the result output during the training of the trunk neural network based on the training data set corresponding to the first detection object.

[0113] Specifically, since the backbone neural network needs to extract features from all road images, the electronic device can train the backbone neural network through multiple training data sets. Taking the first neural network head as an example, the electronic device trains the first neural network head according to the first output result output by the backbone neural network. The trained first neural network head can be used to detect the first detection object. The electronic device can obtain multiple trained neural network heads by training multiple neural network heads respectively through the above training method, thereby obtaining a trained detection model.

[0114] Since the electronic device uses the acquired multiple training data sets to train the backbone neural network, the backbone neural network can have a stronger image feature extraction capability. Then the result output by the backbone neural network is input into the corresponding neural network head for training. The trained neural network head can be used to identify a specific detection object. When there are multiple neural network heads in the neural network model, since the electronic device trains the multiple neural network heads separately, each neural network head has a strong ability to detect specific objects.

[0115] In a possible implementation, the structure of the neural network model used for training can be found in Figure 3 It is understood that the number of neural network heads in the neural network model can be multiple, not limited to Figure 3 Neural network head 1 and neural network head 2 in the figure. Training with electronic devices Figure 3 Taking the neural network model shown as an example, the type of the neural network head can be determined according to the category of the detection object, and the number of layers of the backbone neural network can also be determined according to the actual detection task. The electronic device inputs the road image corresponding to the traffic light and the road image corresponding to the traffic sign into the neural network model at the same time, and trains through the backbone neural network. Then the output result of the traffic light obtained by the backbone neural network is input to the neural network head 1, and the training continues through the neural network head 1. The output result of the traffic sign obtained by the backbone neural network is input to the neural network head 2, and the training continues through the neural network head 2. By training the neural network head 1 and the neural network head 2 respectively, the electronic device can realize the training of the entire neural network model, thereby obtaining a trained detection model.

[0116] In a possible implementation, since the amount of data used to train the neural network model is usually large, the training data can be divided into multiple times and input into the neural network model by setting a data loader.

[0117] For example, please refer to Table 2, which is a data table of a data loader provided in an embodiment of the present application. As shown in Table 2, the electronic device loads different data sets by creating multiple data loaders, thereby ensuring that in each round of training, the amount of data input into the neural network model by multiple data sets is the same. Among them, the number of data loaders is consistent with the number of neural network heads, and the number of data sets corresponds to the number of neural network heads.

[0118] Taking the example of two neural network heads in the neural network model, there are two different data sets in Table 2, corresponding to different detection objects. Therefore, the electronic device can create two data loaders to input the data in the two data sets respectively. Taking the example of inputting 64 data into the neural network model in each round, data set 1 needs to be input for a total of 182 rounds, and data set 2 needs to be input for a total of 77 rounds. Since the training of data sets 1 and 2 is carried out simultaneously, when data set 2 is imported 77 times, all the data in data set 2 is imported into the neural network model. However, at this time, the data in data set 1 has not been fully imported into the neural network model, so the electronic device needs to re-import the data in data set 2 according to the number of data imported in each round for training.

[0119] Table 2

[0120] Number of training set images Number of imports per round Required import rounds Dataset 1 11588 64 182 Dataset 2 4971 64 77

[0121] In a possible implementation, during the training process of the detection model, the electronic device may also optimize the neural network model by adjusting the loss value.

[0122] The loss value of the neural network model includes the loss values ​​of multiple detection objects. During the training process, if the difference between the loss values ​​of the multiple detection objects is greater than a preset threshold, the loss values ​​of the multiple detection objects are adjusted so that the difference between the loss values ​​of the detection objects after adjustment is less than or equal to the preset threshold.

[0123] In the process of training the neural network model, we hope that the output of the neural network model is as close to the true value as possible. Therefore, we can compare the predicted value and the true value output by the current neural network model, and then update the weight vector of each layer of the neural network according to the difference between the two. Taking the loss value as an example, the higher the loss value, the greater the difference, so the training of the neural network model becomes a process of minimizing the loss value as much as possible. Generally speaking, the smaller the loss value, the higher the training quality of the neural network model, and the larger the loss value, the lower the training quality of the neural network model.

[0124] Specifically, the electronic device first determines a plurality of loss values ​​of the neural network model, and then adjusts the loss values ​​of the plurality of detection objects according to the relationship between the difference between the loss values ​​of the plurality of detection objects and a preset threshold.

[0125] In a possible implementation, multiple loss values ​​correspond to different detection objects. For example, the loss value corresponding to the detection object is the difference between the true value of the detection object and the predicted value of the detection object. The true value corresponding to the detection object is the true value determined by the corresponding annotated image, and the predicted value corresponding to the detection object is the predicted value obtained by the detection model.

[0126] For example, the loss value (Loss) includes one or more of the following categories: classification loss (class), confidence loss (object) and positioning loss (location). The specific process of determining the loss value can be seen in the following expression:

[0127] Loss fianl =αLoss class +βLoss object +γLoss loaction

[0128] Loss class =ClassLoss true -ClassLoss predict

[0129] Loss object =ObjLoss true -ObjLoss predict

[0130] Loss location =LocationLoss true -LocationLoss predict

[0131] Among them, Loss fianl Loss is one of the multiple loss values ​​of the neural network model. class is the loss value of the class category, Loss object is the loss value of the object category, Loss loacation is the loss value of the location category. α, β and γ are the corresponding weight values. true is the true value of the class category, ClassLoss predict ObjLoss is the predicted value of the class category. true is the true value of the object category, ObjLoss predict It is the predicted value of the object category. LocationLoss true is the true value of the location category, LocationLoss predict is the predicted value of the location category.

[0132] The electronic device can determine multiple loss values ​​through the above expression, and then adjust the size of the loss value according to the difference between the multiple loss values.

[0133] Exemplarily, during the training process, when the loss value of the first detection object is greater than the loss value of the second detection object, the loss value of the second detection object is increased.

[0134] For example, taking the detection objects including traffic lights and traffic signs as an example, the loss value corresponding to the traffic light is Losslight fianl , the loss value corresponding to the traffic sign is Losssign fianl Then, electronic devices can be fianl and Losssign fianl to adjust the size.

[0135] For example, electronic devices adjust the Losslight fianl and Losssign fianl The weight of is used to adjust the proportion of a single loss value among multiple loss values. For details, please refer to the following expression:

[0136] Loss final =aLosssign fianl +bLosslight fianl

[0137] Among them, Loss final is the final loss value determined by the neural network model, and a and b correspond to weights to determine the contribution of each loss to the overall loss.

[0138] The electronic device can dynamically adjust the weight of the loss value according to the difference between multiple loss values, that is, adjust the weight a and weight b mentioned above. This can balance the relationship between different losses. For example, in Losssign fianl The value is greater than Losslight fianl In the case of a value of fianl The corresponding weight b can better balance the relationship between different losses in the subsequent training of the neural network model.

[0139] In one possible implementation, the neural network model can use the error backpropagation (BP) algorithm to correct the network parameters in the initial model during the training process, so that the error loss of the model becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the error loss information is backpropagated to converge the error loss. The backpropagation algorithm is a backpropagation movement dominated by error loss, which aims to obtain the optimal parameters of the super-resolution model, so backpropagation can be performed based on the adjusted loss value to achieve the purpose of optimizing the neural network model.

[0140] For example, see Figure 5 , Figure 51 is a flow chart of a target detection provided by an embodiment of the present application. In which, the detection model includes a neural network head 1 and a neural network head 2 as an example for explanation. It can be understood that the detection model includes multiple neural network heads, not limited to Figure 5 Neural network head 1 and neural network head 2 in. Figure 5 As shown, it includes one or more steps from step S501 to step S505, and each step is specifically as follows:

[0141] S501, the vehicle acquires a road image to be detected. The road image to be detected may be collected by the vehicle or acquired by the vehicle from other devices. The embodiment of the present application does not impose any restriction on the source of the road image.

[0142] In a possible implementation, the vehicle includes a camera device, which can capture images and / or video streams. Therefore, the vehicle can use images captured by the camera device as road images, or use image frames in the video stream captured by the camera device as road images.

[0143] S502, the vehicle extracts image features of the road image through the backbone neural network. After the backbone neural network in the detection model extracts features of the road image, corresponding image features are obtained.

[0144] S503, the image features are detected by neural network head 1 and the image features are detected by neural network head 2. The vehicle inputs the image features obtained in S502 into multiple neural network heads, such as Figure 5 In the neural network head 1 and the neural network head 2 shown, the image features are detected simultaneously by the neural network head 1 and the neural network head 2.

[0145] S504, outputting corresponding detection results. Neural network head 1 and neural network head 2 can respectively output corresponding detection results, for example, neural network head 1 outputs information such as the detection frame, execution degree, and quantity of the corresponding traffic light. Neural network head 2 outputs information such as the detection frame, execution degree, and quantity of the corresponding traffic sign.

[0146] S505: Output the annotation information corresponding to one or more detection objects. The vehicle detection model draws the annotation information corresponding to the detection object on the image according to all the detection results determined in S504, and outputs an image containing the annotation information corresponding to one or more detection objects.

[0147] For example, see Figure 6 , Figure 6 Schematic diagram of a flow chart of a training detection model provided in an embodiment of the present application. Figure 6As shown, the neural network model includes a neural network head 1 and a neural network head 2. The process includes one or more steps in steps S601 to S607, and each step is specifically as follows:

[0148] S601, obtain a training set. Before training a detection model, the electronic device needs to first obtain road images and corresponding annotation information for training to construct a data set. According to the detection object of the detection model, the road image obtained by the electronic device also needs to include at least one of multiple detection object categories. For example, if the detection object that the detection model needs to detect and identify includes traffic lights and traffic signs, then the road image obtained by the electronic device needs to include one of the traffic lights and traffic signs. Figure 6 The traffic light dataset and traffic sign dataset shown.

[0149] S602, create data loader 1, create data loader 2. The electronic device creates multiple data loaders to load different data sets, thereby ensuring that in each round of training, the amount of data input into the neural network model by multiple data sets is the same. The number of data loaders is consistent with the number of neural network heads, and the number of data sets corresponds to the number of neural network heads. For example, Figure 6 The data loader 1 corresponding to the traffic light and the data loader 2 corresponding to the traffic sign are shown in FIG.

[0150] S603, training the backbone neural network. The electronic device simultaneously inputs the road image corresponding to the traffic light and the road image corresponding to the traffic sign into the neural network model to train the backbone neural network.

[0151] S604, training neural network head 1, training neural network head 2. The electronic device inputs the data about traffic lights obtained through the backbone neural network into neural network head 1, and continues training through neural network head 1. The data about traffic signs obtained through the backbone neural network is input into neural network head 2, and continues training through neural network head 2.

[0152] S605, determine loss value 1, determine loss value 2. For example Figure 6 In the example, the electronic device determines a loss value 1 and a loss value 2. The multiple loss values ​​correspond to different detection objects, for example, the loss value corresponding to the detection object is the difference between the true value of the detection object and the predicted value of the detection object. The true value corresponding to the detection object is the true value determined by the corresponding annotated image, and the predicted value corresponding to the detection object is the predicted value obtained by the detection model.

[0153] S606, adjusting the loss value. The electronic device may adjust the loss value according to the difference between the multiple loss values.

[0154] S607, back propagation optimization of the neural network model. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the super-resolution model, so back propagation can be performed based on the adjusted loss value to achieve the purpose of optimizing the neural network model.

[0155] The method of the embodiment of the present application is described in detail above, and the device of the embodiment of the present application is provided below.

[0156] Figure 7 This is a functional unit block diagram of a target detection device provided in an embodiment of the present application. The target detection device is a device with processing capabilities, where the device can be a physical device, such as a server (such as a rack server), a host, etc., or a virtual device, such as a virtual machine, a container, etc.

[0157] like Figure 7 As shown, the target detection device 70 includes: a processor 701, a memory 702, and one or more programs, and may include a communication interface 703. It should be understood that the present application does not limit the number of processors and memories in the target detection device 70.

[0158] The processor 701 is a module for performing calculations, and may include a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), a microcontroller unit (MCU), or one or more integrated circuits for controlling the execution of the above program.

[0159] The memory 702 is used to provide a storage space, and the storage space can optionally store application data, user data, operating system, computer programs, etc. The memory 702 may include a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, and may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0160] The memory 702 may exist independently and be connected to the processor 701 via a bus. The memory 702 may also be integrated with the processor 701.

[0161] The communication interface 703 is used to provide information input or output for the at least one processor. And / or, the communication interface 703 can be used to receive data sent externally and / or send data to the outside. The communication interface 703 can be a wired link interface including an Ethernet cable, etc., or a wireless link (Wi-Fi, Bluetooth, general wireless transmission and other wireless communication technologies, etc.) interface. Optionally, the communication interface 703 can also include a transmitter (such as a radio frequency transmitter, an antenna, etc.) coupled to the interface, or a receiver, etc.

[0162] In the embodiment of the present application, the one or more programs are stored in the memory 702 in the form of program codes and are configured to be executed by the processor 701. The programs include instructions for implementing the steps in the aforementioned target detection method. Figure 2 That is, the memory 702 stores executable instructions, and the processor 701 executes the executable instructions to implement the aforementioned target detection method, for example Figure 2 That is, the memory 702 stores instructions for executing the target detection method.

[0163] See also Figure 8 , Figure 8 Schematic diagram of a computing device provided in an embodiment of the present application. Figure 8As shown, the computing device 80 may include: one or more processors 801, one or more memories 802, and one or more communication interfaces 803. These components may be connected via a bus 804 or other means. Figure 8 Take the connection via bus 804 as an example.

[0164] The communication interface 803 may be used for the computing device 80 to communicate with other communication devices, such as other computing devices. Specifically, the communication interface 803 may be a wired interface.

[0165] The memory 802 can be coupled to the processor 801 through the bus 804 or the input and output ports, and the memory 802 can also be integrated with the processor 801. The memory 802 is used to store various software programs and / or multiple sets of instructions or data. Specifically, the memory 802 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage devices, flash memory devices or other non-volatile solid-state storage devices. The memory 802 may store an operating system (hereinafter referred to as system), such as an embedded operating system such as uCOS, VxWorks, RTLinux, etc. The memory 802 may also store a network communication program, which may be used to communicate with one or more additional devices, one or more user devices, or one or more terminals. The memory 802 may be independent and connected to the processor 801 via a bus 804. The memory 802 may also be integrated with the processor 801.

[0166] The memory 802 is used to store application code for executing the above solution, and the execution is controlled by the processor 801. The processor 801 is used to execute the application code stored in the memory 802.

[0167] The processor 801 may be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. The processor 801 may also be a combination that implements a certain function, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like.

[0168] It should be noted that in the embodiments of the present application, the specific implementation and technical effects of each unit can also refer to Figure 2 The corresponding description of the method embodiment shown in .

[0169] The present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on at least one processor, the aforementioned target detection method is implemented, for example Figure 2 method.

[0170] The embodiment of the present application further provides a vehicle, the vehicle comprising the aforementioned target detection device 70, the vehicle being used to implement the aforementioned target detection method, for example Figure 2 The target detection method in the embodiment.

[0171] The present application also provides a computer program product, which includes computer instructions, and when executed by a computing device, implements the aforementioned target detection method, such as Figure 2 method.

[0172] In the embodiments of the present application, the words "for example" or "such as" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "such as" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the words "for example" or "such as" is intended to present related concepts in a specific way.

[0173] The "at least one" mentioned in the embodiments of the present application refers to one or more, and "plurality" refers to two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, c can be single or multiple. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can be represented by: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0174] Furthermore, unless otherwise specified, the ordinal numbers such as "first" and "second" used in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. For example, the first device and the second device are only for the convenience of description, and do not represent the difference in structure, importance, etc. between the first device and the second device. In some embodiments, the first device and the second device can also be the same device.

[0175] In the above embodiments, the term "when..." can be interpreted as meaning "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the concept and principle of the present application shall be included in the protection scope of the present application.

[0176] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0177] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A target detection method, characterized in that: The method comprises: Acquire a road image to be detected, wherein the road image includes a detection object; The road image is input into a trained detection model so that the trained detection model outputs the labeling information of the detection object, wherein the detection model includes a backbone neural network and multiple neural network heads, the backbone neural network is used to extract image features from the road image, and the labeling information of the detection object includes results output by the multiple neural network heads based on the image features.

2. The method according to claim 1, characterized in that The step of inputting the road image into a trained detection model so that the trained detection model outputs the labeling information of the detection object includes: Inputting the road image into the backbone neural network of the trained detection model so that the backbone neural network outputs image features of the road image; The image features are respectively input into the multiple neural network heads to obtain the labeling information of the detection object.

3. The method according to claim 2, characterized in that The step of inputting the image features into the plurality of neural network heads respectively to obtain the labeling information of the detection object comprises: Inputting the image features into the multiple neural network heads respectively, so that the multiple neural network heads respectively output detection results, wherein the multiple detection results include the labeling information of the detection object, the labeling information of the detection object is output by the target neural network head, and the target neural network head is trained according to the image features of the detection object; Determine labeling information of the detection object according to the multiple detection results.

4. The method according to any one of claims 1 to 3, characterized in that: The image features include image features of a first detection object and image features of a second detection object, and the step of inputting the road image into a trained detection model so that the trained detection model outputs labeling information of the detection object includes: Inputting the image features into the multiple neural network heads respectively, and outputting a first detection result based on a first neural network head, wherein the first detection result includes labeling information of the first detection object, and the first neural network head is trained according to the image features of the first detection object; outputting a second detection result based on a second neural network head, wherein the second detection result includes labeling information of the second detection object, and the second neural network head is trained according to image features of the second detection object; A third detection result is output based on a third neural network head, wherein the third detection result is independent of the first detection object and the second detection object, and the third neural network head is trained according to image features of the third detection object.

5. The method according to any one of claims 1 to 4, characterized in that: The labeling information of the plurality of detected objects is reflected on the road image.

6. The method according to any one of claims 1 to 5, characterized in that: The backbone neural network in the trained detection model is obtained by training based on multiple training data sets, and each training data set in the multiple training data sets corresponds to one of the detection objects; The multiple neural network heads in the trained detection model are trained based on the output results respectively, and the output results are the results output by the backbone neural network.

7. The method according to claim 6, characterized in that The first neural network head in the trained detection model is trained based on the first output result, the first neural network head is any one of the multiple neural network heads, and the first output result is the result output during the training of the backbone neural network based on the training data set corresponding to the first detection object.

8. The method according to any one of claims 1 to 7, characterized in that: During the training process of the detection model, the loss value of the detection model includes the loss values ​​of the plurality of detection objects; If the difference between the loss values ​​of the multiple detection objects is greater than a preset threshold, the loss values ​​of the multiple detection objects are adjusted so that the difference between the loss values ​​of the detection objects after adjustment is less than or equal to the preset threshold.

9. The method according to claim 8, characterized in that During the training process of the detection model, when the loss value of the first detection object is greater than the loss value of the second detection object, the loss value of the second detection object is increased.

10. A target detection device, characterized in that: The device comprises a processor and a memory, wherein the processor is coupled to the memory, the memory is used to store a computer program, and the processor is used to call and run the computer program, so that the target detection device executes the method according to any one of claims 1 to 9.

11. A computing device, characterized in that: The method comprises a processor coupled to a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program, so that the computing device executes the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes instructions for executing the method according to any one of claims 1 to 9.

13. A vehicle, characterized in that: The vehicle comprises the device according to claim 10, so that the vehicle performs the method according to any one of claims 1-9.

Images