A target detection method, device, apparatus and storage medium

Abstract

A target detection method, device and equipment and storage medium are disclosed. The method comprises: acquiring a to-be-processed image; inputting the to-be-processed image into a predetermined target recognition model, the target recognition model comprising at least two recognition task branches, and a loss function of the target recognition model being determined according to task labels of training samples in at least one training data set; and determining a target detection result according to at least two recognition results output by the target recognition model. The method solves the problem of low efficiency when multiple single tasks are simultaneously run. The to-be-processed image is processed by the target recognition model, the target recognition model comprises at least two recognition task branches, and at least two recognition results can be obtained as the target detection result. The target recognition model in the application comprises at least two recognition task branches, the loss function is determined according to the task labels of the training samples in the training process, the model accuracy is ensured on the basis of an incomplete data set, and the target recognition accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a target detection method, apparatus, device, and storage medium. Background Technology

[0002] A perception system is a system that enables autonomous vehicles to accurately perceive their surrounding environment. Its output includes information on the location, shape (2D or 3D), category, and speed of obstacles, as well as semantic understanding of certain scenes (such as lane line types, drivable areas, construction zones, traffic lights, and road signs).

[0003] Currently, vision-based perception research typically involves single-task deep learning networks, which greatly limits their application. If multi-task requirements exist, multiple single-task networks usually need to run simultaneously. However, in practical applications, running multiple single-task networks on a single hardware device inevitably leads to decreased hardware resource utilization, resulting in significantly reduced operating efficiency. Summary of the Invention

[0004] This invention provides a target detection method, apparatus, device, and storage medium to solve the problem of low efficiency when multiple single tasks are run simultaneously.

[0005] According to one aspect of the present invention, a target detection method is provided, comprising:

[0006] Obtain the image to be processed;

[0007] The image to be processed is input into a predetermined target recognition model, which includes at least two recognition task branches, and the loss function of the target recognition model is determined based on the task labels of training samples in at least one training dataset.

[0008] The target detection result is determined based on at least two recognition results output by the target recognition model.

[0009] According to another aspect of the present invention, a target detection device is provided, comprising:

[0010] The image acquisition module is used to acquire the image to be processed.

[0011] An image recognition module is used to input the image to be processed into a predetermined target recognition model, the target recognition model including at least two recognition task branches, and the loss function of the target recognition model is determined based on the task labels of training samples in at least one training dataset;

[0012] The detection result determination module is used to determine the target detection result based on at least two recognition results output by the target recognition model.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] Image acquisition device, used to acquire images to be processed;

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the target detection method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the target detection method according to any embodiment of the present invention.

[0019] The technical solution of this invention involves acquiring an image to be processed; inputting the image to be processed into a pre-determined target recognition model, wherein the target recognition model includes at least two recognition task branches, and the loss function of the target recognition model is determined based on the task labels of training samples in at least one training dataset; and determining the target detection result based on at least two recognition results output by the target recognition model. This solves the problem of low efficiency when multiple single tasks run simultaneously. By processing the image to be processed through the target recognition model, and since the target recognition model includes at least two recognition task branches, at least two recognition results can be obtained as target detection results. The target recognition model in this application includes at least two recognition task branches, and the loss function is determined based on the task labels of training samples during training, ensuring model accuracy and improving target recognition accuracy even with incomplete datasets.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1This is a flowchart of a target detection method provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a flowchart of a target detection method provided according to Embodiment 2 of the present invention;

[0024] Figure 3 This is an example diagram illustrating the implementation of determining target detection results according to Embodiment 2 of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of a target detection device according to Embodiment 3 of the present invention;

[0026] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the target detection method of this invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1 This is a flowchart of a target detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to detecting different types of targets in an image. The method can be executed by a target detection device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0031] S101. Obtain the image to be processed.

[0032] In this embodiment, the image to be processed can be specifically understood as an image requiring detection. The image to be processed can be acquired by an image acquisition device, such as a camera or video camera. Taking the identification of targets in the environment surrounding a vehicle as an example, the image acquisition device can be installed on the vehicle to acquire images of the surrounding environment while the vehicle is in motion. The image to be processed may include traffic signs (e.g., traffic signs), road markings (e.g., arrows, pedestrian lines, speed bumps, lane dividers, crosswalks, etc.), obstacles (e.g., motor vehicles, non-motor vehicles, pedestrians, etc.), lane lines, etc. The image to be processed can be acquired at a certain frequency and can be processed in real time or in batches.

[0033] S102. Input the image to be processed into a pre-determined target recognition model. The target recognition model includes at least two recognition task branches. The loss function of the target recognition model is determined based on the task labels of training samples in at least one training dataset.

[0034] In this embodiment, the target recognition model can be understood as being pre-trained on a large number of images. During training, the model parameters are continuously adjusted based on the convergence of the loss function until a target recognition model that meets the requirements is obtained, thus completing the training. The training dataset can be understood as the dataset used during training. The labeled images in the training dataset serve as training samples for model training, resulting in the target recognition model. The task label can be understood as the classification label of the task corresponding to each training sample, such as obstacle, lane line, etc. The classification label of a training sample indicates that this training sample is used to train the recognition of this type of task. For example, a training sample with the classification label of obstacle is used to train the obstacle recognition task branch.

[0035] A trained target recognition model can be directly input into an image and obtain prediction results based on its learning experience. The target recognition model in this application includes at least two recognition task branches, each recognizing different types of targets, such as traffic signs, road markings, obstacles, lane lines, etc. During training, each training sample in the training dataset has a corresponding task label. The loss function for each recognition task branch is determined based on the task label, and then the loss function of the target recognition model is determined based on the loss function of each recognition task branch.

[0036] For multi-task learning, a comprehensive dataset is needed to meet the requirements of model training and accuracy. However, existing technologies do not comprehensively cover all training types of datasets, and a single dataset usually cannot simultaneously meet the requirements of all learning tasks, thus affecting the model training results and model accuracy. This application uses training samples from multiple training datasets during model training. Training samples from different datasets are used for training different categories of object detection. Therefore, training samples are labeled using task labels, and the loss function is determined based on the task labels to complete model training. This solves the problem of incomplete datasets preventing multi-task recognition. By fusing multiple training datasets, it effectively utilizes existing datasets to achieve fusion training for all perceptual tasks, eliminating the need for manual comprehensive dataset labeling and saving resources and time.

[0037] S103. Determine the target detection result based on at least two recognition results output by the target recognition model.

[0038] In this embodiment, the target detection result can be specifically understood as the result obtained by detecting targets in the image to be processed. The target detection result can be a rectangle obtained by selecting obstacles, road markings, traffic signs, etc. in the image to be processed, or it can be a line that marks lane lines, etc. The target detection result can be directly displayed in the image to be processed.

[0039] Specifically, each recognition task branch of the target recognition model outputs a corresponding recognition result. Since different recognition task branches recognize different task types, the recognition results output by each branch are also different. Recognition results can include 3D bounding boxes, 2D bounding boxes, mask codes, etc. Based on the recognition results output by each recognition task branch of the target recognition model, the target detection result is determined. The recognition results output by the recognition task branch can be directly used as the target detection result; for example, the detection box can be directly used as the target recognition result. Alternatively, the recognition results output by the recognition task branch can be processed, and the resulting data can be used as the target detection result. For example, in lane line recognition, the location of each lane line can be determined based on the mask code, and the curve equation of the lane line can be fitted as the target detection result. When outputting lane lines, if a curve function needs to be output, contour detection and other processing can also be performed. When there are multiple lane lines, different lane lines are identified. For example, the lane lines from left to right are labeled as 1, 2, 3... and their corresponding mask codes are 1, 2, 3... respectively. That is, the mask code corresponding to lane line 1 is 1, and lane line 1 is fitted based on the points with a pixel value of 1.

[0040] This application provides a target detection method, which involves acquiring an image to be processed; inputting the image to be processed into a pre-determined target recognition model, the target recognition model including at least two recognition task branches, the loss function of the target recognition model being determined based on the task labels of training samples in at least one training dataset; and determining the target detection result based on at least two recognition results output by the target recognition model. This method solves the problem of low efficiency when multiple single tasks are run simultaneously. By processing the image to be processed through the target recognition model, and since the target recognition model includes at least two recognition task branches, at least two recognition results can be obtained as the target detection result. The target recognition model in this application includes at least two recognition task branches, and the loss function is determined based on the task labels of the training samples during training, ensuring model accuracy and improving target recognition accuracy even with incomplete datasets.

[0041] Example 2

[0042] Figure 2 This is a flowchart of a target detection method provided in Embodiment 2 of the present invention. This embodiment is a refinement based on the above embodiments. Figure 2 As shown, the method includes:

[0043] S201. Obtain at least one training dataset and perform label classification. Determine the task label for each training dataset. The training dataset includes at least one training sample and the corresponding standard result.

[0044] In this embodiment, the standard result can be specifically understood as the labeled data of the training samples, which serves as the ground truth for training. Different task labels of the training samples correspond to different types of standard results. Standard results are usually labeled manually or through other methods, and need to be determined in advance before training.

[0045] Obtain at least one training dataset, each training dataset including at least one training sample and the corresponding standard result. Classify the training datasets according to the types of data they can recognize or train, determining the task label for each training dataset. For example, if the training dataset consists of training samples for recognizing obstacle detection boxes, the corresponding task label could be "obstacle recognition task" or a string representing the task label. Once the task labels for the training datasets are determined, the task labels for the training samples in the training datasets are correspondingly determined; that is, the task labels for all training samples in the training datasets are the same as the task labels for the training datasets.

[0046] S202. Input the training samples corresponding to the current iteration into the current recognition model to be trained, and obtain at least two training results output by the recognition model to be trained. The recognition model to be trained includes at least two recognition task branches.

[0047] In this embodiment, the recognition model to be trained can be specifically understood as an untrained, deep learning-based neural network model. The training result can be specifically understood as the result obtained by performing object detection on the training samples during the model training process. In the current iteration, the training samples are input into the recognition model to be trained, and each recognition task branch in the model outputs the corresponding training result.

[0048] Optionally, the recognition model to be trained includes an encoding layer, a fusion layer, and at least two recognition task branches.

[0049] As an optional embodiment of this example, this optional embodiment further optimizes the input of the training samples corresponding to the current iteration into the current recognition model to be trained, obtaining at least two training results output by the recognition model to be trained, including:

[0050] A1. Input the training samples into the coding layer of the current recognition model to be trained for feature extraction, and obtain at least one image feature. Each image feature has a different scale.

[0051] In this embodiment, image features can be specifically understood as feature data representing an image. Training samples are input into the encoding layer of the current recognition model to be trained. Feature extraction is performed through the encoding layer, which can be one or more layers. Therefore, during feature extraction, one or more image features can be obtained, each with a different scale. This embodiment can use EfficientNet as the encoding layer.

[0052] A2. Input each image feature into the fusion layer of the current recognition model to be trained to perform feature fusion and obtain fused features.

[0053] In this embodiment, the fused features can be specifically understood as feature data obtained after fusing image features at different scales. The image features are input into the fusion layer of the current recognition model to be trained, and feature fusion is performed through the fusion layer to obtain the fused features. This embodiment can use BiFPN as the fusion layer to fuse various image features to obtain the fused features.

[0054] A3. Input the fused features into each recognition task branch in the current recognition model to be trained, and obtain the training results output by each recognition task branch.

[0055] The fused features are input into each recognition task branch, and each recognition task branch decodes the fused features to obtain the training result. In this application, the overall recognition task branches select different types of branches according to different tasks. If the output training result is a semantic output, semantic segmentation is used to decode it to obtain the corresponding training result; if the output training result is a bounding box, an approximate YOLO decoder is used to decode it to obtain the corresponding training result.

[0056] This application preferably uses an encoding layer and a fusion layer with undetermined parameters, and adjusts the parameters of the encoding layer, the fusion layer, and each recognition task branch simultaneously during training.

[0057] S203. For each recognition task branch, based on the loss function expression corresponding to the recognition task branch, combined with the task label of the training samples, the training results and the standard results, the corresponding loss function is obtained, and a fitting loss function is formed by fusing the loss functions.

[0058] In this embodiment, the fitting loss function can be understood as a loss function obtained by fitting multiple loss functions together. When performing backpropagation on the recognition model to be trained, a loss function is required. Different recognition task branches in this application have corresponding loss functions. Therefore, when multiple loss functions exist, they need to be fitted together, and then backpropagation is performed based on the fitted loss function. The loss function can be a GAN loss function, L1 loss function, focal loss function, VGG perceptual loss function, cross-entropy loss function, etc.

[0059] The loss function expression corresponding to each recognition task branch is determined. For each recognition task branch, the task label of the training samples is determined. The training results and standard results obtained through the recognition task branch are used to determine whether the training results affect the loss function, i.e., whether they can be used to determine the loss function. If so, the corresponding loss function is calculated using the training results and standard results. Multiple loss functions are fitted to obtain the fitted loss function. The method of fusing the various loss functions to form the fitted loss function can be by setting different weights for each loss function, taking the average value, etc., and this embodiment of the invention does not specifically limit this.

[0060] As an optional embodiment of this example, the optional embodiment is further optimized by: determining the loss function expression for identifying task branches based on the task type.

[0061] In this embodiment, the task type can be semantic detection, bounding box detection, etc., and the recognition task branches are classified according to the detection method; it can also be obstacle detection, lane line detection, etc., and the recognition task branches are classified according to the type of the detected target. Different loss functions are applicable to different task types. For example, when recognizing and outputting bounding boxes, the focal loss function is used to improve the detection rate of small target objects; semantic output such as drivable areas and lane lines uses the cross-entropy loss function.

[0062] As an optional embodiment of this example, this optional embodiment further obtains the corresponding loss function based on the loss function expression corresponding to the recognition task branch, combined with the task label of the training sample, the training result, and the standard result. The optimization is as follows: if the task label of the training sample matches the recognition task branch, the loss function is calculated based on the loss function expression of the recognition task branch, combined with the corresponding training result and the standard result.

[0063] The system determines whether the task label of the training sample matches the recognition task branch. If they match, the training sample is determined to be usable for training the recognition task branch. The training results and standard results are then substituted into the loss function expression to calculate the loss function. If they do not match, the loss function for this recognition task branch is not updated based on the training results and standard results of this training sample.

[0064] Taking two training datasets as an example, the task label for training dataset 1 is "obstacles," and the standard result corresponding to the training samples in training dataset 1 is the obstacle detection box (or bounding box). The standard result corresponding to the training samples in training dataset 2 is the lane line mask. Recognition task branch 1 is used to detect obstacles, and recognition task branch 2 is used to detect lane lines. The task labels of the training samples in training dataset 1 match those of recognition task branch 1, and the loss function of recognition task branch 1 is calculated based on the training results and standard results of the training samples. The task labels of the training samples in training dataset 2 do not match those of recognition task branch 1, and the training samples are not used to calculate the loss function of recognition task branch 1. Similarly, the loss function of recognition task branch 2 is calculated based on the training results and standard results of the training samples in training dataset 2, and the training samples in training dataset 1 are not used to calculate the loss function of recognition task branch 2.

[0065] The training method described in this application can use multiple training datasets of different types to train the recognition model. Task labels are used to determine whether the training samples are effective for training the model's recognition task branch. This solves the problem of incomplete datasets preventing multi-task recognition training and allows for the integration of various datasets from existing technologies to achieve multi-task recognition. Furthermore, the encoding layer and fusion layer serve as a common data processing layer. The processed data is then input to the recognition task branches for recognition. Only one encoding and fusion layer is needed for data processing and sharing, saving hardware resources and improving operational efficiency compared to deploying multiple single-task recognition methods. Due to the mutual integration of multi-task learning, the accuracy of each recognition task branch can reach or even exceed the accuracy of a single-task network.

[0066] As an optional embodiment of this example, this optional embodiment further optimizes the fitting loss function formed by fusing the various loss functions as follows:

[0067] B1. Determine the weight of each loss function among all loss functions.

[0068] The sum of all loss functions is calculated, and the proportion of each loss function in the sum is calculated as its corresponding weight. In this embodiment, the weights of the loss functions are dynamically adjusted.

[0069] B2. Determine the fitting loss function based on the weights of each loss function.

[0070] Based on the weights of each loss function, a weighted sum is performed on each loss function to obtain the fitting loss function.

[0071] The coding and fusion layers used in this application can also be coding and fusion layers with predefined parameters. During training, no parameter adjustment is required; only the parameters of the recognition task branch need to be adjusted.

[0072] S204. Backpropagation is performed on the training recognition model based on the fitting loss function to obtain the training recognition model for the next iteration, until the iteration convergence condition is met, and the target recognition model is obtained.

[0073] During the training of the neural network model, the model is continuously updated and adjusted using backpropagation until the model's output closely matches the target. After determining the fitting loss function, backpropagation is performed on the training recognition model using this fitting loss function to obtain the target recognition model. This embodiment of the invention does not limit the specific backpropagation process and can be set according to specific circumstances. After the model training is complete, target detection can be performed on the image to be processed using the target recognition model.

[0074] Optionally, the iterative convergence conditions include: the reduction in the fitted loss function is less than a preset threshold, or the detection accuracy is higher than a preset accuracy when the training recognition model under the current iteration is detected by the validation set.

[0075] In this embodiment, the preset threshold and preset accuracy can be set in advance according to the model's accuracy requirements. The validation set can be specifically understood as a dataset used to validate the model, including the images used for recognition and the corresponding ground truth values.

[0076] During model training, after each iteration, the current fitted loss function is compared with the previous fitted loss function. When the fitted loss function decreases, and the decrease is less than a preset threshold, the iteration convergence condition is satisfied. Alternatively, a pre-formed validation set is obtained, and each image in the validation set is input into the recognition model to be trained in the current iteration. The recognition result is determined, and the result is compared with the ground truth to determine whether the recognition is correct. The number of correctly recognized images and the total number of images used for recognition are determined, and the proportion of correctly recognized images is calculated. This proportion is used as the detection accuracy. The detection accuracy is compared with the preset accuracy. When the detection accuracy is higher than the preset accuracy, the iteration convergence condition is satisfied.

[0077] S205. Obtain the image to be processed.

[0078] S206. Perform image processing on the image to be processed, including at least one of the following: cropping, edge trimming, and scaling.

[0079] When processing images, target recognition models may have certain requirements regarding the size of the input images. Therefore, if the size of the image to be processed does not meet the size requirements of the target recognition model, the image to be processed must be processed before inputting it into the target recognition model. This can be done by cropping, edge-fitting, shrinking, or enlarging one or more methods to adjust it to the required size.

[0080] S207. Input the processed image to be processed into the predetermined target recognition model.

[0081] S208. Determine the target detection result based on at least two recognition results output by the target recognition model.

[0082] For example, Figure 3An example diagram illustrating the implementation of determining target detection results is provided. The image to be processed 31 is input to the preprocessing module 32 for preprocessing, such as scaling. The processed image is then input to the encoding layer 33 to obtain at least one image feature. These features have different scales. The obtained image features are then input to the fusion layer 34 to obtain fused features. These fused features are then input to different recognition task branches 35 to detect different types of targets and obtain corresponding recognition results. Recognition task branches 35 can perform obstacle recognition, road marking recognition, lane line recognition, and drivable area recognition. Obstacle recognition can be performed using bounding box recognition or obstacle category recognition; road marking recognition can be performed using bounding box recognition or road marking category recognition; lane line recognition and drivable area recognition can be processed through fully connected layers. Bounding box or category recognition can use YOLO decoding, while lane lines and drivable areas can be decoded using semantic segmentation.

[0083] This application provides an object detection method that addresses the low efficiency of running multiple single tasks simultaneously. The object recognition model includes at least two recognition task branches. When processing an image, the model yields at least two recognition results as the object detection result. Task labels are used to determine whether training samples are effective for training the model's recognition task branches, solving the problem of incomplete datasets hindering multi-task recognition training. This method can integrate various datasets from existing technologies to achieve multi-task recognition. Furthermore, the encoding and fusion layers function as a common data processing layer, saving hardware resources and improving operational efficiency.

[0084] Example 3

[0085] Figure 4 This is a schematic diagram of a target detection device provided in Embodiment 3 of the present invention. Figure 4 As shown, the device includes: an image acquisition module 41, an image recognition module 42, and a detection result determination module 43.

[0086] The image acquisition module 41 is used to acquire the image to be processed.

[0087] Image recognition module 42 is used to input the image to be processed into a predetermined target recognition model, the target recognition model including at least two recognition task branches, and the loss function of the target recognition model is determined based on the task labels of training samples in at least one training dataset;

[0088] The detection result determination module 43 is used to determine the target detection result based on at least two recognition results output by the target recognition model.

[0089] This application provides a target detection device that solves the problem of low efficiency when multiple single tasks are run simultaneously. The target recognition model processes the image to be processed. Since the target recognition model includes at least two recognition task branches, at least two recognition results can be obtained as target detection results. The target recognition model in this application includes at least two recognition task branches. During the training process, the loss function is determined according to the task labels of the training samples, which ensures the accuracy of the model and improves the target recognition accuracy even when the dataset is incomplete.

[0090] Optionally, the device may also include:

[0091] An image processing module is used to perform image processing on the image to be processed before inputting it into a predetermined target recognition model. The image processing includes at least one of the following: cropping, edge pasting, and scaling.

[0092] Optionally, the device may also include:

[0093] The training data acquisition module is used to acquire at least one training dataset and perform label classification, and determine the task label for each training dataset. The training dataset includes at least one training sample and the corresponding standard result.

[0094] The training result acquisition module is used to input the training samples corresponding to the current iteration into the current recognition model to be trained, and obtain at least two training results output by the recognition model to be trained, wherein the recognition model to be trained includes at least two recognition task branches;

[0095] The loss function determination module is used to obtain the corresponding loss function for each recognition task branch based on the loss function expression corresponding to the recognition task branch, combined with the task label of the training sample, the training result and the standard result, and form a fitting loss function by fusing the loss functions.

[0096] The target model determination module is used to backpropagate the recognition model to be trained based on the fitting loss function to obtain the recognition model to be trained for the next iteration, until the iteration convergence condition is met to obtain the target recognition model.

[0097] Optional, the training result acquisition module includes:

[0098] The feature extraction unit is used to input training samples into the coding layer of the current recognition model to be trained for feature extraction, and to obtain at least one image feature, wherein each image feature has a different scale;

[0099] The feature fusion unit is used to input the image features into the fusion layer of the current recognition model to be trained to perform feature fusion and obtain fused features.

[0100] The training result determination unit is used to input the fused features into each recognition task branch in the current recognition model to be trained, and obtain the training result output by each recognition task branch.

[0101] Optionally, the loss function determination module is specifically used to: if the task label of the training sample matches the recognition task branch, calculate the loss function based on the loss function expression of the recognition task branch combined with the corresponding training results and standard results.

[0102] Optional, the loss function determination module includes:

[0103] The weight determination unit is used to determine the weight of each loss function among all loss functions;

[0104] The loss function fitting unit is used to determine the fitting loss function based on the weights of each of the loss functions.

[0105] Optionally, the iterative convergence condition includes: the reduction in the fitting loss function is less than a preset threshold, or the detection accuracy is higher than a preset accuracy when the training recognition model under the current iteration is detected by the verification set.

[0106] Optionally, the loss function expression for the identification task branch is determined according to the task type.

[0107] The target detection device provided in the embodiments of the present invention can execute the target detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0108] Example 4

[0109] Figure 5 A schematic diagram of an electronic device that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0110] like Figure 5As shown, the electronic device includes an image acquisition device 50, at least one processor 51, and a memory, such as a read-only memory (ROM) 52 or a random access memory (RAM) 53, communicatively connected to the at least one processor 51. The image acquisition device 50 is used to acquire images to be processed, and the number of image acquisition devices 50 can be one or more. Figure 5 For example, the memory stores computer programs that can be executed by at least one processor. The processor 51 can perform various appropriate actions and processes based on the computer program stored in the read-only memory (ROM) 52 or loaded from the storage unit 58 into the random access memory (RAM) 53. The RAM 53 can also store various programs and data required for the operation of the electronic device. The processor 51, ROM 52, and RAM 53 are interconnected via a bus 54. An input / output (I / O) interface 55 is also connected to the bus 54.

[0111] Multiple components in the electronic device are connected to the I / O interface 55, including: an input unit 56, such as a keyboard, mouse, etc.; an output unit 57, such as various types of displays, speakers, etc.; a storage unit 58, such as a disk, optical disk, etc.; and a communication unit 59, such as a network card, modem, wireless transceiver, etc. The communication unit 59 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0112] Processor 51 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 51 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 51 performs the various methods and processes described above, such as object detection methods.

[0113] In some embodiments, the target detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 58. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 52 and / or communication unit 59. When the computer program is loaded into RAM 53 and executed by processor 51, one or more steps of the target detection method described above may be performed. Alternatively, in other embodiments, processor 51 may be configured to perform the target detection method by any other suitable means (e.g., by means of firmware).

[0114] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0116] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0118] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0119] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0120] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0121] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A target detection method, characterized in that, include: Acquire the image to be processed; wherein the image to be processed is an image that requires detection. The image to be processed is input into a predetermined target recognition model, which includes at least two recognition task branches, and the loss function of the target recognition model is determined based on the task labels of training samples in at least one training dataset. The target detection result is determined based on at least two recognition results output by the target recognition model; The training steps of the target recognition model include: Obtain at least one training dataset and perform label classification to determine the task label for each training dataset. The training dataset includes at least one training sample and the corresponding standard result. The training samples are input into the coding layer of the current recognition model to be trained for feature extraction, resulting in at least one image feature, each of which has a different scale. Each of the aforementioned image features is input into the fusion layer of the current recognition model to be trained for feature fusion, resulting in fused features; The fused features are input into each recognition task branch in the current recognition model to be trained, and the training result output by each recognition task branch is obtained. The recognition model to be trained includes at least two recognition task branches. For each recognition task branch, based on the loss function expression corresponding to the recognition task branch, combined with the task label of the training samples, the training results and the standard results, the corresponding loss function is obtained, and a fitting loss function is formed by fusing the loss functions. Backpropagation is performed on the target recognition model based on the fitting loss function to obtain the target recognition model for the next iteration, until the iteration convergence condition is met, and the target recognition model is obtained.

2. The method according to claim 1, characterized in that, Before inputting the image to be processed into the predetermined target recognition model, the process further includes: The image to be processed is subjected to image processing, which includes at least one of the following: cropping, edge pasting, and scaling.

3. The method according to claim 1, characterized in that, The step of obtaining the corresponding loss function based on the loss function expression corresponding to the recognition task branch, combined with the task labels of the training samples, the training results, and the standard results, includes: If the task label of the training sample matches the recognition task branch, the loss function is calculated based on the loss function expression of the recognition task branch, combined with the corresponding training results and standard results.

4. The method according to claim 1, characterized in that, The process of fusing the various loss functions to form a fitting loss function includes: Determine the weight of each loss function among all loss functions; The fitting loss function is determined based on the weights of each loss function.

5. The method according to claim 4, characterized in that, The iterative convergence conditions include: the reduction in the fitting loss function is less than a preset threshold, or the detection accuracy is higher than a preset accuracy when the training recognition model under the current iteration is detected by the verification set.

6. The method according to any one of claims 1-5, characterized in that, The loss function expression for the identification task branch is determined according to the task type.

7. A target detection device, characterized in that, include: An image acquisition module is used to acquire an image to be processed; wherein the image to be processed is an image that requires detection. An image recognition module is used to input the image to be processed into a predetermined target recognition model, the target recognition model including at least two recognition task branches, and the loss function of the target recognition model is determined based on the task labels of training samples in at least one training dataset; The detection result determination module is used to determine the target detection result based on at least two recognition results output by the target recognition model; The device further includes: The training data acquisition module is used to acquire at least one training dataset and perform label classification, and determine the task label for each training dataset. The training dataset includes at least one training sample and the corresponding standard result. The training result acquisition module is used to input the training samples corresponding to the current iteration into the current recognition model to be trained, and obtain at least two training results output by the recognition model to be trained, wherein the recognition model to be trained includes at least two recognition task branches; The loss function determination module is used to obtain the corresponding loss function for each recognition task branch based on the loss function expression corresponding to the recognition task branch, combined with the task label of the training sample, the training result and the standard result, and form a fitting loss function by fusing the loss functions. The target model determination module is used to backpropagate the recognition model to be trained based on the fitting loss function to obtain the recognition model to be trained for the next iteration, until the iteration convergence condition is met to obtain the target recognition model. The training result acquisition module includes: The feature extraction unit is used to input training samples into the coding layer of the current recognition model to be trained for feature extraction, and to obtain at least one image feature, wherein each image feature has a different scale; The feature fusion unit is used to input the image features into the fusion layer of the current recognition model to be trained to perform feature fusion and obtain fused features. The training result determination unit is used to input the fused features into each recognition task branch in the current recognition model to be trained, and obtain the training result output by each recognition task branch.

8. An electronic device, characterized in that, The electronic device includes: Image acquisition device, used to acquire images to be processed; At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the target detection method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the target detection method according to any one of claims 1-6.

Images