Multi-attribute target detection method, electronic equipment and readable storage medium

Abstract

The invention discloses a multi-attribute target detection method, electronic equipment and a readable storage medium, and belongs to the field of machine learning and computer vision. The multi-attribute target detection method provided by the invention comprises the following steps: acquiring a target image; inputting the target image into a target multi-attribute detection model for detection processing to obtain a prediction result; wherein the target multi-attribute detection model is a multi-task detection model used for detecting a target object and attributes of the target object, and the prediction result comprises a detection result of the target object and a multi-attribute classification result of the target object.

Description

Technical Field

[0001] The present application belongs to the field of machine learning and computer vision, and specifically relates to a multi-attribute target detection method, an electronic device, and a readable storage medium. Background Art

[0002] The target detection method based on deep learning is widely used in industrial scenarios such as smart parks and smart supervision. In AI-driven smart scenarios, there is usually a need for multi-attribute detection. For example, in a smart kitchen, the chef's uniform, mask, chef hat, etc. are intelligently analyzed; in smart transportation, the type of vehicle, the driver's driving status, whether there is a license plate, the color of the vehicle, and other issues are intelligently analyzed. In the field of agricultural production, the growth and illness of plants are intelligently analyzed.

[0003] In the process of using target detection methods for multi-attribute detection, related technologies generally use multiple models cascaded (such as target detection model + target classification model) to perform multi-task reasoning. For example, in the first stage, the target detection model is used to detect the target object, and in the second stage, the target classification model is used to classify the target object.

[0004] However, this detection method in the related art has the problem of long detection time due to the use of multiple models for detection. For example, the number of target objects detected by the target detection model in the first stage is n, and the model detection time is t1. The detection time of the target classification model in the second stage for one target object is t2, and the model detection time is n*t2. The total detection time of this detection method is t1+n*t2, which is long. Summary of the invention

[0005] The embodiments of the present application provide a multi-attribute target detection method, an electronic device, and a readable storage medium to solve the problem that multi-attribute detection of target objects takes a long time in the related art.

[0006] In a first aspect, an embodiment of the present application provides a multi-attribute target detection method, the method comprising:

[0007] Get the target image;

[0008] Inputting the target image into a target multi-attribute detection model for detection processing to obtain a prediction result;

[0009] The target multi-attribute detection model is a multi-task detection model for detecting a target object and the attributes of the target object, and the prediction result includes a detection result of the target object and a multi-attribute classification result of the target object.

[0010] In a second aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0011] In a third aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0012] In an embodiment of the present application, a target image is obtained; the target image is input into a target multi-attribute detection model for detection processing to obtain a prediction result; wherein the target multi-attribute detection model is a multi-task detection model for detecting a target object and the attributes of the target object, and the prediction result includes the detection result of the target object, and the multi-attribute classification result of the target object. In this way, since the target multi-attribute detection model can detect and obtain the detection result of the target object and the multi-attribute classification result of the target object at the same time, compared with the related art using a plurality of model cascades (such as a target detection model + a target classification model) to perform multi-task detection (such as a target object detection task and a target object multi-attribute classification task), the embodiment of the present application performs the detection task of the target object and the multi-attribute classification task of the target object through a single target multi-attribute detection model, thereby improving the efficiency of the multi-attribute detection of the target object and solving the problem of a long time for the multi-attribute detection of the target object in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic flow chart of a multi-attribute target detection method provided in an embodiment of the present application;

[0014] Figure 2-1 A schematic structural diagram of a target multi-attribute detection model provided in an embodiment of the present application;

[0015] Figure 2-2 A schematic flow chart of another multi-attribute target detection method provided in an embodiment of the present application;

[0016] Figure 3-1 A schematic flow chart of another multi-attribute target detection method provided in an embodiment of the present application;

[0017] Figure 3-2 A schematic flow chart of another multi-attribute target detection method provided in an embodiment of the present application;

[0018] Figure 3-3 A schematic flow chart of another multi-attribute target detection method provided in an embodiment of the present application;

[0019] Figure 3-4 A schematic flow chart of another multi-attribute target detection method provided in an embodiment of the present application;

[0020] Figure 3-5 A schematic flow chart of another multi-attribute target detection method provided in an embodiment of the present application;

[0021] Figure 3-6 A schematic flow chart of another multi-attribute target detection method provided in an embodiment of the present application;

[0022] Figure 3-7 A schematic flow chart of another multi-attribute target detection method provided in an embodiment of the present application;

[0023] Figure 4 A schematic flowchart of a training process of a multi-attribute detection model provided in an embodiment of the present application;

[0024] Figure 5-1 A schematic flowchart of a multi-attribute target detection method performed by a cloud server in a smart scenario according to an embodiment of the present application;

[0025] Figure 5-2 A schematic flow chart of a multi-attribute target detection method performed by a multi-attribute target detection system in a smart scenario provided by an embodiment of the present application;

[0026] Figure 6 A schematic structural diagram of a multi-attribute target detection device provided in an embodiment of the present application;

[0027] Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0030] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0031] The target detection method based on deep learning has been widely used in industrial scenarios such as smart parks and smart supervision. In AI-driven smart scenarios, there is usually a need for multi-attribute detection. For example, in a smart kitchen, multiple attributes such as the chef's chef uniform, mask, and chef hat are intelligently analyzed; in smart transportation, multiple attributes of vehicles and safety compliance issues are intelligently analyzed. In the field of agricultural production, multiple attributes of plants such as growth and illness are intelligently analyzed. In the process of using target detection methods for multi-attribute detection, related technologies generally use a cascade of multiple models (such as target detection model + target classification model) to perform multi-stage task reasoning. For example, the detection task is divided into two stages. In the first stage, the target detection model is used to detect the target object, and in the second stage, the target classification model is used to classify multiple attributes of the target object.

[0032] However, the applicant noted that this detection method in the related art has the problem of long detection time due to the use of multiple models for detection. For example, the number of target objects detected by the target detection model in the first stage is n, and the model detection time is t1. The detection time of each target object by the target classification model in the second stage is t2, and the model detection time is n*t2. The total detection time of this detection method is t1+n*t2, which is long.

[0033] Based on this, the embodiment of the present application provides a multi-attribute target detection method, by acquiring a target image; inputting the target image into a target multi-attribute detection model for detection processing, and obtaining a prediction result; wherein the target multi-attribute detection model is a multi-task detection model for detecting a target object and the attributes of the target object, and the prediction result includes the detection result of the target object, and the multi-attribute classification result of the target object. In this way, compared with the method of using multiple model cascades (such as target detection model + target classification model) in the related art to perform multi-task detection (such as the detection task of the target object and the multi-attribute classification task of the target object), since the target multi-attribute detection model of the present application can detect the detection result of the target object and the multi-attribute classification result of the target object at the same time, the detection task of the target object and the multi-attribute classification task of the target object are performed by a single target multi-attribute detection model (such as the reasoning time of a single target multi-attribute detection model is t1, which is significantly less than t1+n*t2), which improves the efficiency of the multi-attribute detection of the target object, and solves the problem of the long time of the multi-attribute detection of the target object in the related art.

[0034] The multi-attribute target detection method provided in the embodiment of the present application can be applied to various AI-driven smart scenes such as smart kitchens, smart parks, smart supervision, smart factories, and smart agricultural production. For example, in the smart kitchen scene, multiple attributes such as the chef's chef uniform, mask, and chef hat are detected; in the smart transportation scene, multiple attributes such as the type of vehicle, the driver's driving status, whether there is a license plate, and the specific color of the vehicle are detected; in the smart agricultural production scene, multiple attributes such as the growth of specific plants, illness, and maturity of the available parts are detected. Of course, the multi-attribute target detection method provided in the embodiment of the present application can also be applied to other smart scenes, and this application does not limit this.

[0035] The multi-attribute target detection method provided by the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0036] Figure 1 A schematic flowchart of a multi-attribute target detection method provided in an embodiment of the present application.

[0037] like Figure 1 As shown, the multi-attribute target detection method provided in the embodiment of the present application may include:

[0038] Step 110: Acquire a target image;

[0039] Step 120: input the target image into the target multi-attribute detection model for detection processing to obtain a prediction result;

[0040] The target multi-attribute detection model is a multi-task detection model for detecting a target object and its attributes, and the prediction result includes the detection result of the target object and the multi-attribute classification result of the target object.

[0041] In step 110, the target image can be an image of an industrial scene or an image disclosed on the Internet. The present application does not impose any specific restrictions on the source of the target image. For example, the present application can transmit a video stream from a surveillance camera in real time via the Real Time Messaging Protocol (RTMP), and extract the target image from the real-time transmitted video stream. For another example, the present application can determine the target image from images disclosed on the Internet via the Hyper Text Transfer Protocol (HTTP).

[0042] In step 120, the target multi-attribute detection model may be a pre-trained multi-task detection model for detecting the target object and multiple attribute categories of the target object, and ultimately the detection result of the target object and the multi-attribute classification result of the target object may be obtained together.

[0043] Among them, the target object can be the detection object in various smart scenes driven by AI, such as chefs in smart kitchen scenes, vehicles in smart transportation scenes, specific plants in smart agricultural production scenes, etc., and this application does not make specific restrictions.

[0044] Among them, the attributes of the target object may include multiple attribute categories of the target object. For example, in the smart kitchen scenario, the chef's chef uniform wearing, mask wearing, chef hat wearing and other different attribute categories; in the smart traffic scenario, the type of vehicle, the driver's driving status, whether there is a license plate, the specific color of the vehicle and other different attribute categories; in the smart agricultural production scenario, the growth of specific plants, disease conditions, maturity of the available parts and other different attribute categories; etc., this application does not make specific restrictions.

[0045] Among them, the multi-task detection model can be understood as a multi-task detection model that can implement multiple different types of detection tasks, for example, the detection task of the target object and the detection task of multiple attribute categories of the target object.

[0046] It can be understood that in the embodiments of the present application, compared with the related art that uses multiple model cascades (for example, target detection model + target classification model) to perform multi-task detection (for example, target object detection task and target object multi-attribute classification task), the embodiments of the present application use a single target multi-attribute detection model to perform target object detection task and target object multi-attribute classification task, thereby improving the efficiency of multi-attribute detection of target objects and solving the problem of long time for multi-attribute detection of target objects in the related art.

[0047] For example, the detection task of the prior art can be divided into two stages. In the first stage, the number of target objects detected by the target detection model is n, and the model detection time is t1; in the second stage, the target classification model is used to classify the multiple attributes of each target object, and the detection time for each target object is t2, and the model detection time is n*t2. The total detection time of the two stages of this detection method is t1+n*t2. In the embodiment of the present application, the reasoning time of the single target multi-attribute detection model can be t1, which is significantly less than the total detection time t1+n*t2 in the prior art, which improves the efficiency of multi-attribute detection of target objects and solves the problem of long multi-attribute detection time of target objects in related technologies.

[0048] According to the multi-attribute target detection method provided by the embodiment of the present application, by acquiring a target image; inputting the target image into a target multi-attribute detection model for detection processing, a prediction result is obtained; wherein the target multi-attribute detection model is a multi-task detection model for detecting a target object and the attributes of the target object, and the prediction result includes the detection result of the target object, and the multi-attribute classification result of the target object. In this way, since the target multi-attribute detection model can detect the detection result of the target object and the multi-attribute classification result of the target object at the same time, compared with the multi-task detection (such as the detection task of the target object and the multi-attribute classification task of the target object) performed by a plurality of model cascades (such as a target detection model + a target classification model) in the related art, the embodiment of the present application performs the detection task of the target object and the multi-attribute classification task of the target object through a single target multi-attribute detection model, thereby improving the efficiency of the multi-attribute detection of the target object and solving the problem of the long time of the multi-attribute detection of the target object in the related art.

[0049] In a specific embodiment, the detection result of the target object may include the position of the detection box and the first prediction probability that the target object exists in the detection box; the multi-attribute classification result of the target object includes N second prediction probabilities, and the N second prediction probabilities correspond one-to-one to the N attribute categories of the target object, where N is an integer greater than 1.

[0050] The position of the detection frame may reflect the position of the target object in the target image, and the first predicted probability that the target object exists in the detection frame is used to reflect the probability that the target object is located at the position of the detection frame in the target image.

[0051] Among them, the detection result of the target object corresponds to the multi-attribute classification result of the target object, that is, the N second prediction probabilities are used to reflect the one-to-one corresponding probabilities of the N attribute categories of the target object located at the position of the detection frame.

[0052] It is understandable that in the prediction result, the number of target object detection results is the same as the number of target object multi-attribute classification results. For example, the prediction result includes K target object detection results and K multi-attribute classification results corresponding to the K target objects, where K is a positive integer. Wherein, when K is a positive integer greater than 1, the position of the detection box of each target object detection result in the K target object detection results is different.

[0053] In this way, the embodiment of the present application can obtain the detection result of the target object and the multi-attribute classification result of the target object at the same time through the target multi-attribute detection model, and obtain the position of the detection box, the first prediction probability that the target object exists in the detection box, and N second prediction probabilities corresponding to the N attribute categories of the target object.

[0054] In a specific embodiment, Figure 2-1 As shown, the target multi-attribute detection model may include: a backbone network, a feature pyramid network, a region proposal network, and a detection head network connected in sequence;

[0055] Among them, the backbone network can be used to extract multiple feature maps of different scales from the target image;

[0056] The feature pyramid network can be used to fuse multiple feature maps of different scales to obtain a target feature map. The size of the target feature map remains unchanged relative to feature maps of different scales, and the number of channels is normalized to a fixed value.

[0057] The region proposal network can be used to predict the target feature map and obtain the location of the candidate detection box area;

[0058] The multiple prediction layers of the detection head network can be used to perform prediction processing on the part of the target feature map that is mapped to the candidate detection box area, to obtain the position of the detection box where the target object exists in the candidate detection box area, the first prediction probability that the target object exists in the detection box, and N second prediction probabilities.

[0059] like Figure 2-2 As shown, the multi-attribute target detection method provided in the embodiment of the present application may include:

[0060] Step 210: Acquire a target image;

[0061] Step 220: extracting multiple feature maps of different scales from the target image through the backbone network;

[0062] Step 230: A plurality of feature maps of different scales are fused through a feature pyramid network to obtain a target feature map, the size of the target feature map remains unchanged relative to feature maps of different scales, and the number of channels is normalized to a fixed value;

[0063] Step 240: Predict the target feature map through the region proposal network to obtain the position of the candidate detection box area;

[0064] Step 250: By detecting multiple prediction layers of the head network, prediction processing is performed on the portion of the target feature map that is mapped to the candidate detection frame area, and the position of the detection frame where the target object exists in the candidate detection frame area is obtained, the first prediction probability that the target object exists in the detection frame, and N second prediction probabilities.

[0065] Among them, step 210 may refer to the specific content of step 110, and steps 220 to 250 may be sub-steps of step 120.

[0066] In step 220, the backbone network may be a network for extracting features, such as resnet, VGG, etc., which is not specifically limited in this application. Multiple feature maps of different scales are extracted from the target image through the backbone network, such as feature maps of multiple scales of 1 / 2 scale, 1 / 4 scale, 1 / 8 scale, 1 / 16 scale, 1 / 32 scale, etc., which are not specifically limited in this application.

[0067] In step 230, a plurality of feature maps of different scales are fused through a feature pyramid network to obtain a target feature map, the size of the target feature map remains unchanged relative to the feature maps of different scales, and the number of channels is normalized to a fixed value. The fixed value of the number of channels can be set according to actual needs.

[0068] In step 240, the target feature map is predicted by a region proposal network to obtain the location of a candidate detection box region, where the target object may exist in the candidate detection box region.

[0069] In step 250, multiple prediction layers of the detection head network are detected, and each prediction layer is used for prediction in different aspects. For example, the multiple prediction layers may include a first prediction layer and a second prediction layer. The first prediction layer is used to predict the position of the detection box where the target object exists in the candidate detection box area, and a first prediction probability that the target object exists in the detection box; the second prediction layer is used to predict N second prediction probabilities corresponding to N attribute categories of the target object in the detection box.

[0070] In step 250, the position of the detection box where the target object exists, the first prediction probability that the target object exists in the detection box can be used as the detection result of the target object, and N second prediction probabilities corresponding one-to-one to the N attribute categories of the target object in the detection box can be used as the multi-attribute classification result of the target object.

[0071] In this way, the embodiment of the present application can extract multiple feature maps of different scales from the target image through the backbone network of the target multi-attribute detection model; fuse the multiple feature maps of different scales through the feature pyramid network to obtain the target feature map; predict the target feature map through the region proposal network to obtain the position of the candidate detection frame area; predict the part of the target feature map mapped to the candidate detection frame area through multiple prediction layers of the detection head network to obtain the position of the detection frame where the target object exists in the candidate detection frame area, the first prediction probability of the target object existing in the detection frame, and N second prediction probabilities, so as to achieve the detection result of the target object and the multi-attribute classification result of the target object predicted by a single model.

[0072] In a specific embodiment, the target object may have M main attributes, each main attribute has multiple sub-attributes; each main attribute is used to reflect a characteristic of the target object; each sub-attribute is used to reflect a state of the characteristic; a main attribute and a sub-attribute are combined to form an attribute category.

[0073] Furthermore, each of the M main attributes is combined with a plurality of sub-attributes under the main attribute to form a plurality of attribute categories, thereby obtaining N attribute categories.

[0074] in, N is the number of attribute categories of the target object, M is the number of main attributes of the target object, and M j Indicates the number of sub-attributes of the i-th main attribute, i is a positive integer less than or equal to M, j is a positive integer less than or equal to M j A positive integer.

[0075] For example, if the target object is a chef in a smart kitchen scenario, the target object can have three main attributes: chef uniform wearing, mask wearing, and chef hat wearing; each main attribute has three sub-attributes: wearing (W), not wearing (N), and uncertain whether to wear (U). Furthermore, each of the three main attributes is combined with the three sub-attributes under the main attribute to form 9 attribute categories, and the target object is classified into 9 attribute categories: wearing chef uniform (W_shirt), not wearing chef uniform (N_shirt), uncertain whether to wear chef uniform (U_shirt), wearing mask (W_mask), not wearing mask (N_mask), uncertain whether to wear mask (U_mask), wearing chef hat (W_hat), not wearing chef hat (N_hat), and uncertain whether to wear chef hat (U_hat).

[0076] For another example, if the target object is a vehicle in a smart traffic scenario, the target object can have four main attributes: the driver's seat belt wearing condition, whether there is a license plate, the specific color of the vehicle, and the specific type of the vehicle. Among them, the driver's seat belt wearing condition has three sub-attributes: wearing (W), not wearing (N), and not sure whether to wear (U). Whether there is a license plate has three sub-attributes: yes (Y), no (N), and uncertain (U). The specific color of the vehicle has five sub-attributes: white (WH), black (BK), yellow (YE), red (RD), and blue (BU). The specific type of the vehicle has four sub-attributes: truck (MT), car (S), bus (B), and truck (T). Furthermore, each of the four main attributes is combined with multiple sub-attributes under the main attribute to form 15 attribute categories, and the target objects are classified into 15 attribute categories: wearing seat belt (W_belt), not wearing seat belt (N_belt), uncertain whether to wear seat belt (U_belt), with license plate (W_plate), without license plate (N_plate), uncertain whether to have license plate (U_plate), white vehicle (WH_color), black vehicle (BK_color), yellow vehicle (YE_color), red vehicle (RD_color), blue vehicle (BU_color), truck (MT_car), sedan (S_car), bus (B_car), and van (T_car).

[0077] Of course, in the embodiment of the present application, the number of main attributes of the target object and the number of sub-attributes of each main attribute can be set according to actual needs, and the present application does not impose any restrictions on this.

[0078] In the embodiment of the present application, after the target multi-attribute detection model outputs the prediction results, the prediction results can be screened based on the number of main attributes of the target object to obtain the target prediction results, so as to facilitate the use of the target prediction results to perform intelligent analysis on the target object in the smart scene.

[0079] like Figure 3-1 As shown, the multi-attribute target detection method provided in the embodiment of the present application may include:

[0080] Step 310: Acquire a target image;

[0081] Step 320: input the target image into the target multi-attribute detection model for detection processing to obtain a prediction result; wherein the target multi-attribute detection model is a multi-task detection model for detecting the target object and the attributes of the target object, and the prediction result includes the detection result of the target object and the multi-attribute classification result of the target object;

[0082] Step 330: Filter the prediction results according to a preset filtering strategy to obtain a target prediction result; the target prediction result includes the position of the detection box, the first prediction probability that the target object exists in the detection box, and M second prediction probabilities out of N second prediction probabilities, where M is an integer less than N.

[0083] Among them, step 310 may refer to the specific content of step 110, and step 320 may refer to the specific content of step 120; step 330 is executed after step 320.

[0084] Among them, screening the prediction results may include screening the multi-attribute classification results of the target object. For example, in the prediction results output by the target multi-attribute detection model, the multi-attribute classification results of the target object include N second prediction probabilities corresponding to N attribute categories of the target object. In the embodiment of the present application, the multi-attribute classification results of the target object may be screened through step 330 to screen out M second prediction probabilities from the N second prediction probabilities, where M is an integer less than N. Among them, M is the number of main attributes of the target object.

[0085] For example, Figure 3-2 As shown, in the above step 330, the prediction results are screened according to the preset screening strategy, which may specifically include:

[0086] Step 3301: according to the number of main attributes of the target object, determine M second prediction probabilities from the N second prediction probabilities of the multi-attribute classification result of the target object; wherein each of the M second prediction probabilities corresponds to the maximum value of the second prediction probability under one main attribute;

[0087] Step 3302: Determine whether the M second prediction probabilities are all greater than the first confidence threshold;

[0088] Step 3303: When the M second prediction probabilities are all greater than the first confidence threshold, the M second prediction probabilities are retained and used as part of the target prediction result.

[0089] Among them, step 3301 is the initial screening of the prediction values ​​within the main attributes. For each of the M main attributes: determine the maximum prediction value of the sub-attribute under each main attribute, obtain the attribute type of its combination, and obtain the target prediction result of "the position of the detection box + the first prediction probability that the target object exists in the detection box + M second prediction probabilities".

[0090] Among them, step 3302 and step 3303 are for the confidence threshold screening of the second prediction probability. The first confidence threshold is set in advance, and M second prediction probabilities are screened. When the M second prediction probabilities are all greater than the first confidence threshold, the detection result is valid and can be retained; if there is a second prediction probability less than the first confidence threshold, the detection result is invalid and can be deleted. This step can ensure that each main attribute of the target object output by the model has a valid second prediction probability.

[0091] In addition, the embodiment of the present application can also perform confidence threshold screening on the detection results of the target object to ensure that the first prediction probability of the target object output by the model is valid. Figure 3-3 As shown, in the above step 330, the prediction results are screened according to the preset screening strategy, which may specifically include:

[0092] Step 3304: Determine whether the first prediction probability that the target object exists in the detection box in the prediction result is greater than a second confidence threshold;

[0093] Step 3305: When the first prediction probability that the target object exists in the detection frame is greater than the second confidence threshold, retain the first prediction probability that the target object exists in the detection frame, and use the position of the detection frame and the first prediction probability that the target object exists in the detection frame as part of the target prediction result.

[0094] Among them, step 3304 and step 3305 are for screening the confidence threshold of the first prediction probability, presetting the second confidence threshold, screening the first prediction probability of the target object in the detection box in the detection result of the target object, when the first prediction probability is greater than the second confidence threshold, the detection result is valid and can be retained; if the first prediction probability is less than the second confidence threshold, the detection result is invalid and can be deleted. This step can ensure that the first prediction probability of the target object output by the model is valid.

[0095] In addition to the above screening processing, the embodiment of the present application can also filter out duplicate detection frames from the detection results of the target object. Figure 3-4 As shown, in the above step 320, the target image is input into the target multi-attribute detection model for detection processing to obtain a prediction result, including:

[0096] Step 3201: input the target image into the target multi-attribute detection model for detection processing to obtain multiple candidate results; wherein the detection box position included in each candidate result is different;

[0097] Each candidate result includes a detection result of the target object and a multi-attribute classification result of the target object. However, in each candidate result, the position of the detection box included in the detection result of the target object is different.

[0098] Step 3202: Determine a first candidate result from multiple candidate results, the first candidate result being a candidate result with the highest first prediction probability that a target object exists in the detection frame, and use the first candidate result as the prediction result.

[0099] Among them, step 3201 and step 3202 are used to filter out repeated detection frames. When there is one target object in the target image, if the target multi-attribute detection model outputs multiple candidate results, it means that these multiple candidate results are all detection results for this one target object, and the multiple candidate results are all repeated detection frames. Through steps 3201 and 3202, the candidate result with the largest first prediction probability is selected as the final prediction result, and other repeated candidate results can be filtered out.

[0100] In addition, when there are multiple target objects in the target image, such as Figure 3-5 As shown, in the above step 320, the target image is input into the target multi-attribute detection model for detection processing to obtain a prediction result, which may also include:

[0101] Step 3203: determining whether there is a second candidate result among the candidate results other than the first candidate result, wherein an intersection-over-union ratio of a detection box corresponding to the second candidate result and a detection box corresponding to the first candidate result is less than a non-maximum suppression threshold;

[0102] Step 3204: When there is a second candidate result, both the first candidate result and the second candidate result are used as prediction results.

[0103] It can be understood that the intersection-over-union ratio of the detection box corresponding to the second candidate result and the detection box corresponding to the first candidate result is less than the non-maximum suppression (NMS) threshold, indicating that the detection box corresponding to the second candidate result and the detection box corresponding to the first candidate result are not duplicate detection boxes, but are respectively prediction results of different target objects located at different positions in the target image. In this way, both the first candidate result and the second candidate result are used as prediction results, indicating that there are multiple target objects in the target image, and multiple prediction results corresponding to multiple target objects are output together through the target multi-attribute detection model.

[0104] In addition, after the prediction results are screened and processed in step 330 to obtain the target prediction results, the target prediction results can be used for detection result visualization, model accuracy evaluation, etc.

[0105] For example, the embodiment of the present application can also reconstruct the format of the target detection result of the target object, and perform intelligent alarm in the intelligent scene based on the reconstructed result. Figure 3-6 As shown, after obtaining the target prediction result through step 330, the multi-attribute target detection method provided by the embodiment of the present application may further include:

[0106] Step 340: reconstruct the target prediction result to obtain M reconstructed results;

[0107] Step 350: Based on the M reconstruction results, determine whether the target object matches a preset violation identification rule;

[0108] Step 360: When the target object matches the preset violation judgment rule, a prompt message is reported, where the prompt message is used to prompt that the target object in the target image violates the rule.

[0109] In step 340, the target prediction result is formatted and reconstructed, which may be to reconstruct one target prediction result "detection box position + first prediction probability of target object in detection box + M second prediction probabilities" into M reconstructed results "detection box position + first prediction probability of target object in detection box + 1 second prediction probability". In this way, the M second prediction probabilities contained in one target prediction result are split into M independent second prediction probabilities, so as to facilitate one-to-one matching of the M second prediction probabilities of target object in step 350.

[0110] In step 350, the prompt information is used to prompt that the target object in the target image violates the rules or that the target object in the target image passes the rules. The preset violation judgment rule includes violation requirements for M main attributes of the target object.

[0111] Taking the smart kitchen scenario as an example, the preset violation judgment rule can be that if all three main attributes are in the non-wearing state (N), it is considered a violation state, and the wearing state (W) and the uncertain wearing state (U) are considered qualified states.

[0112] In step 360, still taking the smart kitchen scenario as an example, if the chef wears the clothes continuously for a qualified time or continuously wears the clothes unqualified time for more than 5 seconds, then the non-standard wearing prompt / standard wearing prompt is output, otherwise no prompt information is reported. Of course, the violation judgment rules and the conditions for reporting prompt information can be set according to actual needs, and this application does not limit this.

[0113] In this way, the target prediction result is formatted and M reconstruction results are obtained; based on the M reconstruction results, it is determined whether the target object matches the preset violation judgment rule; when the target object matches the preset violation judgment rule, a prompt information is reported, and the prompt information is used to prompt that the target object in the target image is in violation. In this way, intelligent prompts for intelligent scenes are performed through the prediction results of multi-attribute detection.

[0114] In addition, after the prediction results are screened and processed through step 330 to obtain the target prediction results, the embodiment of the present application can also obtain incremental training samples based on the target prediction results when the accuracy of the target prediction results does not meet the requirements, and use the incremental training samples to update the target multi-attribute detection model to improve the accuracy of the target multi-attribute detection model.

[0115] like Figure 3-7 As shown, after obtaining the target prediction result through step 330, the multi-attribute target detection method provided by the embodiment of the present application may further include:

[0116] Step 370: Determine a performance index of the target multi-attribute detection model based on the target prediction result, where the performance index includes at least one of average accuracy, precision, and recall;

[0117] Step 380: When the performance indicator is less than a preset threshold, an incremental training sample is obtained, and the target multi-attribute detection model is updated using the incremental training sample.

[0118] In step 370, the present application can calculate the performance indicators such as the mean average precision (MAP), precision, and recall of the target multi-attribute detection model for each attribute category based on the target prediction results. The specific calculation process of the performance indicators is a mature technology in this field and will not be repeated here.

[0119] In step 380, when the performance indicator is less than a preset threshold, incremental training samples are obtained, and the target multi-attribute detection model is updated using the incremental training samples.

[0120] In this way, when the performance indicators of the target multi-attribute detection model are insufficient, the network parameters of the target multi-attribute detection model are fine-tuned and updated through incremental training samples, thereby improving the accuracy of the target multi-attribute detection model.

[0121] In step 380, the incremental training sample may include: a target image and annotation information corresponding to the target image; wherein the annotation information corresponding to the target image is obtained based on the target prediction result.

[0122] In an embodiment of the present application, the target image and the prediction results of the target image output by the target multi-attribute detection model can be converted into incremental training samples. For example, a pre-labeled label is automatically generated based on the target prediction result. When the target prediction result is a false detection / missed detection result, the labeling personnel correct the pre-labeled label to obtain the labeling information corresponding to the target image, so as to collect a sufficient amount of incremental training samples, and then use a sufficient amount of incremental training samples to update the target multi-attribute detection model. In this way, the model prediction results are converted into labeling labels, and the labeling personnel correct the test samples / newly added samples, and the model accuracy is improved through iteration.

[0123] In addition, before step 110, the multi-attribute target detection method provided by the embodiment of the present application may also include a training process of a target multi-attribute detection model. The target multi-attribute detection model trained by learning various features of the specified object can more accurately predict the detection result of the target object and the multi-attribute classification result of the target object. The training process of the target multi-attribute detection model is described in detail below.

[0124] Figure 4 A schematic flowchart of a training process of a multi-attribute detection model provided in an embodiment of the present application.

[0125] like Figure 4 As shown, before step 110, the multi-attribute target detection method provided in the embodiment of the present application may further include a target multi-attribute detection model training process, and the target multi-attribute detection model training process may perform the following steps:

[0126] Step 410: obtaining a multi-attribute detection model to be trained and a target training sample, where the target training sample includes a sample image and annotation information corresponding to the sample image;

[0127] The annotation information includes the position of the multi-attribute detection box where the specified object is located in the sample image, and the attribute category of the specified object;

[0128] Step 420: iteratively train the multi-attribute detection model to be trained using target training samples to obtain a target multi-attribute detection model.

[0129] In step 410, the present application may collect image samples disclosed on the Internet, or extract valid image samples from surveillance videos / public videos by video frame extraction as target training samples.

[0130] In step 410, the annotation information corresponding to the sample image includes the position of the multi-attribute detection box where the specified object is located in the sample image, and multiple attribute categories of the specified object.

[0131] For example, the attribute category is represented as a combination of a primary attribute and a sub-attribute. The primary attribute is the characteristic of the specified object, such as the chef's shirt and the color of the car; the sub-attribute is the state of the characteristic, such as the chef's shirt is worn and the car is red. Since a single specified object contains multiple primary attributes, a single detection box corresponds to multiple attribute categories during the annotation process, where each attribute category is separated by an identifier and can be represented as "sub-attribute_primary attribute", such as the chef's shirt is worn as "W_shirt" and the red car is "RD_color".

[0132] In practical applications, the present application can use target detection and annotation tools to annotate sample images, including the location of the multi-attribute detection box of the specified object in the sample image (for example, the upper left and lower right coordinates) and multi-attribute category information (for example, M attribute categories), and the results are stored in the format of an XML file.

[0133] Moreover, in practical applications, the present application can also convert the format of the annotation information. The annotation format of 1 {1 multi-attribute detection box position + M attribute categories} is converted into the annotation format of M {1 multi-attribute detection box position + 1 attribute category}, so as to facilitate the subsequent use of the annotation information for model training.

[0134] In step 410, the multi-attribute detection model to be trained can be constructed by the following steps:

[0135] (1) Build a basic network. Call APIs such as torch.nn.Conv2d(), torch.nn.BatchNorm2d(), and torch.nn.ReLU() to build modules such as the backbone network, feature pyramid network, and region proposal network, and use them to build a basic network.

[0136] (2) Construct a detection head network. For each scale feature, there is a detection head that generates prediction results. Multiple prediction layers of the detection head network can be constructed by torch.nn.Conv2d(), which can be used to predict the prediction results of "detection box position + detection box score + total number of attribute categories N second prediction probabilities".

[0137] (3) Reconstruct the true value label format. After the annotation format of the annotation information is converted, the true value label of the specified object is saved and loaded into the network in the format of M {1 multi-attribute detection box position + 1 attribute category}, which is different from the format of the prediction result of the detection head network. The duplicate detection boxes can be filtered out according to the number of main attributes M and the true value label can be reconstructed so that the format of the reconstructed true value label matches the format of the prediction result of the detection head network.

[0138] In step 420, the multi-attribute detection model to be trained is iteratively trained using the target training samples to obtain a target multi-attribute detection model.

[0139] In this way, by learning the various features of the specified object and training the target multi-attribute detection model, the target multi-attribute detection model can more accurately predict the detection result of the target object and the multi-attribute classification result of the target object.

[0140] Moreover, in the process of iteratively training the multi-attribute detection model to be trained through target training samples, the multi-attribute loss strategy of attribute calculation used in this application controls the loss of target detection within a single main attribute, and the category imbalance phenomenon in the model training process is suppressed.

[0141] For example, in the above step 420, iterative training of the multi-attribute detection model to be trained using the target training sample includes:

[0142] The target training samples are used to iteratively train the multi-attribute detection model to be trained using the following loss function:

[0143]

[0144] Among them, loss is the total loss of multi-attribute classification of the specified object in the multi-attribute detection model, loss i Indicates the loss value corresponding to the i-th primary attribute of the specified object, where i is a positive integer less than or equal to M; M is the number of primary attributes of the specified object.

[0145] In this way, compared with the pure target detection method in the related art, the multi-attribute loss strategy of attribute calculation used in this application controls the loss of target detection within a single main attribute, and the category imbalance phenomenon in the model training process is suppressed.

[0146] Among them, the loss value corresponding to the i-th main attribute of the specified object is loss i You can choose different loss functions according to actual needs, such as cross entropy loss, focal loss, etc.

[0147] For example, taking the cross entropy loss function as an example,

[0148] Among them, M j Indicates the number of sub-attributes of the i-th main attribute;

[0149] in,

[0150] Among them, P ij Indicates the second predicted probability of the jth sub-attribute corresponding to the i-th main attribute, where j is less than or equal to M j A positive integer of ;

[0151] Among them, T ij Indicates P ij The corresponding truth value.

[0152] In this way, by first solving the loss within the main attribute and then summing up the loss values ​​corresponding to each main attribute to obtain the total loss of multi-attribute classification, the loss of target detection is controlled within a single main attribute, and the category imbalance phenomenon in the model training process can be suppressed.

[0153] In practical applications, such as Figure 5-1 As shown, the multi-attribute target detection method provided by the embodiment of the present invention can be executed by a cloud server, and the target multi-attribute detection model is deployed in the cloud server. The cloud server can be an independent server or a server cluster composed of multiple servers. The multi-attribute target detection method can specifically include the following steps:

[0154] Step 510: receiving a multi-attribute detection request from a side server, where the multi-attribute detection request carries a target image;

[0155] Step 520: extracting a target image from the multi-attribute detection request;

[0156] Step 530: input the target image into the target multi-attribute detection model for detection processing to obtain a prediction result;

[0157] The target multi-attribute detection model is a multi-task detection model for detecting a target object and its attributes, and the prediction result includes the detection result of the target object and the multi-attribute classification result of the target object.

[0158] Among them, step 510 and step 520 can be sub-steps of step 110, and step 530 can refer to the specific content of step 120.

[0159] Among them, the target multi-attribute detection model in the cloud server deployment can include the following three steps: (1) Export the model in the intermediate format. Extract the weights of the training model and define the model input and output names. For example, call the torch.onnx.export API to convert the model into an intermediate format. (2) Model acceleration. Call the trtexec interface of TensorRT to convert the intermediate format model into a TensorRT model and generate a configuration file containing information such as the model name and calculation type. (3) Deploy the model to the cloud server. Store the model in the cloud warehouse, configure the multi-attribute detection service on the cloud server, and allocate ports for the side server to call.

[0160] In the embodiments of the present application, Figure 5-2 As shown, the execution steps (step 510-step 530) of the above-mentioned multi-attribute detection method can be encapsulated as an API service of a cloud server, and based on the API service of the cloud server and the multi-attribute target detection system, a real-time multi-attribute detection service is provided for the smart scene.

[0161] The multi-attribute target detection system includes end-side cameras, edge servers, and cloud servers. The end-side cameras are located in the monitoring area of ​​the smart scene, using cameras from mainstream manufacturers, and selecting camera models with low distortion such as gun cameras and hemispherical cameras, and try to expand the camera's field of view and reduce monitoring blind spots.

[0162] Among them, the multi-attribute target detection system supports the reasoning of multiple video sources, and realizes real-time, efficient and accurate prediction through cloud-edge collaboration, thereby improving resource utilization. The multi-attribute detection model can be saved in the model warehouse of the cloud server through model training, model deployment and other steps. The cloud server responds to the multi-attribute detection requests of multiple edge servers by configuring specific ports, and generates multi-attribute detection results at the same time.

[0163] Among them, the embodiment of the present application can use the RTMP protocol to transmit the video stream sent by the end-side camera through the cloud-edge-end collaborative process, and pre-process it on the edge server to provide continuous and stable input video for the cloud server. The cloud server executes the multi-attribute detection task and outputs the multi-attribute detection results, thereby realizing the multi-attribute detection task in the smart scene.

[0164] For example, in actual applications, the cloud-edge-device collaboration process can include the following four steps:

[0165] (1) Installation and calibration of edge cameras in smart scenarios. Determine the model, installation location, elevation angle and other parameters of the edge camera. If necessary, set up light sources in the monitoring area to ensure that the targets within the monitoring range are clearly visible in the picture. If the entire area is required to be monitored, multiple cameras can be set up to prevent blind spots.

[0166] (2) Video stream acquisition and image preprocessing. The camera video stream is transmitted via the RTMP protocol and preprocessed on the side server. First, the OPENCV interface cv2.resize() can be called to scale the input image in the same proportion, and the short side is filled with padding to a fixed size. Then, the numpy interface numpy.transpose() is called to perform transposition and channel transformation, and finally the image is normalized.

[0167] (3) Cloud streaming: Call the service port of the cloud server and send a multi-attribute detection request (which can be a multi-channel video stream request). The cloud server performs the multi-attribute detection task (i.e., executes the above S210 to S230) and returns the cloud inference result.

[0168] (4) Set violation identification rules. Set the system's alarm rules according to the needs of specific tasks. If the state cannot be identified due to occlusion, blur, etc., it is considered an uncertain state. This state and the compliance state are not considered an alarm state. For example, in the wearing compliance, if the target's attribute state switches between compliance and uncertainty within the set 5-second time threshold, the algorithm will not report an alarm.

[0169] In actual applications, when the accuracy of the cloud-based reasoning results does not meet the requirements, the present application can determine the performance indicators of the target multi-attribute detection model based on the cloud-based reasoning results, and the performance indicators include at least one of the average accuracy, precision rate and recall rate; when the performance indicator is less than the preset threshold, incremental training samples are obtained, and the incremental training samples are used to update the target multi-attribute detection model to improve the model accuracy.

[0170] For example, the present application can convert the detection results of the model into pre-labeled labels. For the pre-labeled labels converted from the erroneous detection results, the labeling personnel can use labeling software such as Labelimg to correct them, and add the corrected erroneous sample labels to the original data set for incremental training, thus achieving a closed loop of model training, model deployment, erroneous sample label correction, and model incremental training. Figure 5-2 As shown in Figure 1, the incremental training of the target multi-attribute detection model can include the following three steps:

[0171] (1) Generate pre-labeled tags. Obtain the false detection / missed detection results of the target multi-attribute detection model, and save the detection results as an XML file in the format of the Pascal VOC dataset according to the dictionary index of the multi-attribute category.

[0172] (2) Correct the pre-labeled labels. The labelers correct the label content and collect sufficient incremental training samples.

[0173] (3) Incremental training samples are used to incrementally train the multi-attribute detection model, and the updated model is deployed on the cloud server.

[0174] The multi-attribute target detection method provided in the embodiments of the present application has the following advantages over the prior art:

[0175] The embodiment of the present application adopts a target multi-attribute automatic detection technology based on deep learning, which can improve the management efficiency of managers compared with traditional methods;

[0176] The embodiment of the present application uses a single model to implement the multi-attribute classification task of target detection, which can solve the problem of the existing model that the model reasoning speed increases linearly with the number of targets in the image, while reducing the huge labeling workload of the cascade model and the difficulty of logical judgment.

[0177] Compared with the existing target detection methods, the multi-attribute loss strategy of attribute-by-attribute calculation used in the embodiment of the present application controls the loss of target detection within a single main attribute, and the category imbalance phenomenon in the model training process is suppressed.

[0178] The screening and processing method after multi-attribute target detection in the embodiment of the present application adopts the methods of initial screening of prediction values ​​within the main attributes, confidence threshold screening, repeated detection box screening, etc., so that detection and classification are integrated in post-processing, thereby improving the efficiency of multi-attribute target detection.

[0179] The attribute category labeling method of the embodiment of the present application adopts a labeling method of a combination of main attributes and sub-attributes, which integrates data labeling, model iteration, pre-labeling generation, model conversion and deployment into an end-to-end whole, thereby facilitating data processing by labelers. Even inexperienced labelers can effectively cope with the task.

[0180] Taking the smart kitchen as an example, the application of the multi-attribute target detection method provided in the embodiment of the present application in the smart transportation scenario is explained below.

[0181] In the requirement of a bright kitchen in a certain kitchen, it is necessary to monitor the chef's uniform, chef's hat and mask in the kitchen, and prevent irrelevant personnel from entering the kitchen. In response to the intelligent and automated needs of smart kitchens, the developed method is required to automatically monitor and warn chefs of irregular dressing, so that when the chef is dressed correctly, no alarm is reported, and the management personnel can arrange other matters by themselves. When the chef is not dressed in compliance, an alarm is reported and wait for processing. The kitchen contains only one camera, and the light is darker on the side away from the camera, so the chef in the far area of ​​the kitchen is not detected. The camera uses a top-down bird's-eye view to monitor the chefs in the scene. It is recommended that there are no more than 15 chefs in the kitchen to prevent the model detection performance from being reduced due to occlusion.

[0182] In the smart kitchen scenario of this example, the embodiment of the present application can be performed according to the following seven steps:

[0183] 1. Dataset construction. This step includes 5 steps: (1) Obtain kitchen samples. Obtain the kitchen history video recorded by MOC (Change Management System), and call the OPENCV interface cv2.videoCapture() interface and cv2.imwrite() interface to store the video image. In addition to the normal state, try to select states such as night, personnel occlusion, and abnormal wearing to enrich the sample types of the dataset. (2) Obtain public samples. Most of the kitchen samples are samples with standard wearing. It is necessary to add public samples such as human samples without chef hats in the CrowdHuman dataset to maintain the category balance of the dataset. (3) Obtain data annotation rules. Wearing (W) and not wearing (N) are regarded as stable states. If a chef's attribute cannot be judged due to occlusion / blurred picture, it is marked as uncertain whether it is worn (U). Therefore, the multi-attribute categories include W_hat, U_hat, N_hat, W_shirt, U_shirt, N_shirt, W_mask, U_mask, N_mask, a total of 9 categories. (4) Data annotation of image samples. The labelers use labeling software such as Labelimg to label the chef in the image. Since the attribute belongs to the chef, when the chef wears chef uniform, chef hat, and mask in compliance with the regulations, the category label should be "W_hat, W_shirt, W_mask". (5) Convert the label format. The task contains three main attributes, and each multi-attribute category is assigned a detection box. After the format conversion, the three labels are W_hat index + detection box, W_shirt index + detection box, and W_mask index + detection box.

[0184] 2. Build a multi-attribute network. This step includes 4 steps: (1) Build a basic network. Call the torch.nn interface to build the basic module of the network, and use the YAML file to record the network's hyperparameter information, such as the structure of the multi-attribute network, anchor settings, category information, etc. (2) Build a detection head network. Construct the number of category predictions for the detection head based on the total number of attribute categories N. (3) Reconstruct the true value label format. The number of main attributes is 3. Filter out the three-fold repeated true value detection frames loaded into the network, and reconstruct the format of the true value label to match the format of the prediction results output by the detection head network. (4) Model training and loss solution. Load the pre-trained weights of the model, set the training image size and batch size, select Adam as the optimizer, and iterate the model for 300 epochs. The sum of the losses of the three main attributes is used as the multi-attribute detection loss.

[0185] 3. Deploy the model on the cloud server. This step includes 3 steps: (1) Intermediate model conversion. Extract the weights of the training model, define the model input and output names, call the torch.onnx.export API, and convert the model into an intermediate format (2) TensorRT model conversion. Convert the model to plan format through the trtexec interface of TensorRT and generate the corresponding configuration file. (3) Cloud server deployment. Deploy the model in the cloud warehouse, configure the multi-attribute detection service on the cloud server, and allocate ports for the side server to call.

[0186] 4. Video stream reasoning. This step includes two steps: (1) Video stream preprocessing. Get the video stream of the RTMP protocol, call the OPENCV interface cv2.resize() to scale the input image in equal proportions, fill the short side with padding to (640,640), then call the numpy interface numpy.transpose() to perform transposition and channel transformation, and finally divide the image by 255 for normalization. (2) Cloud streaming. Call the service port of the cloud server, send a request for chef multi-attribute detection (which can be a multi-channel video stream request), and return the cloud inference result.

[0187] 5. Screening after obtaining the cloud-based inference results. Set the confidence threshold to 0.25 and the NMS threshold to 0.5, complete the initial screening of the predicted values ​​within the main attributes, the confidence threshold screening, the repeated detection box screening, and reconstruct the format of the cloud-based inference results to obtain the target prediction results.

[0188] 6. Set violation judgment rules. Uncertain wearing status and wearing status are considered qualified status, and not wearing status is considered illegal status. If the continuous wearing qualified or continuous wearing unqualified time exceeds 5 seconds, the output is not wearing standard / wearing standard prompt, otherwise no prompt is reported.

[0189] 7. Model incremental training / optimization. Select verification videos / images for model inference, and have the annotators verify and correct the inference results. After collecting enough data samples, a new data set can be constructed to perform incremental training on the model to improve the model accuracy.

[0190] In addition, taking smart transportation as an example, the application of the multi-attribute target detection method provided in the embodiment of the present application in the smart transportation scenario is explained below.

[0191] In the needs of a certain government department to monitor vehicles at intersections, it is necessary to monitor vehicles passing through the intersection, especially security vehicles and muck trucks, and count information such as vehicle type, driver's driving status, whether it has a license plate, vehicle color, etc. Alerts are issued for information such as the driver not wearing a seat belt, the vehicle is a muck truck, and there is no license plate and vehicle color. Other attribute information that does not involve safety regulations is only counted.

[0192] In the smart transportation scenario of this example, this application can be carried out according to the following steps:

[0193] 1. Dataset construction. This step includes 5 steps: (1) Obtain road monitoring samples. Detect vehicles in the video through an open source vehicle detection model, and obtain video frames with vehicles in the form of interval frames. (2) Obtain public samples. Collect vehicle samples from public datasets (such as MS COCO) to maintain the diversity of samples in the dataset and increase the total amount of sample data. (3) Determine the data annotation strategy. The annotators use Labelimg annotation software to annotate the vehicle samples in the image. The annotations that can stably judge the vehicle status are marked as the corresponding attributes. If a certain attribute of the vehicle cannot be judged due to occlusion / blurred image, it is marked as uncertain (such as U). Therefore, the total number of categories in the dataset is W_belt, U_belt, N_belt, W_plate, U_plate, N_plate, YE_color, WH_color, BK_color, RD_color, BU_color, MT_car, S_car, B_car, T_car, a total of 15 categories. (4) Data annotation of image samples. The annotators use Labelimg and other annotation software to annotate the vehicle attributes in the image. For example, a yellow car with a license plate and a driver wearing a seat belt is labeled as W_belt, W_plate, YE_color, S_car. (5) Convert label format. The task contains four main attributes, and each multi-attribute category is assigned a detection box. After the format conversion, the labels are W_belt index + detection box, W_plate index + detection box, YE_color index + detection box, and S_car index + detection box.

[0194] 2. Build a multi-attribute network. This step includes 4 steps: (1) Build a basic network. Call the torch.nn interface to build the basic module of the network, and use the yaml file to record the network's hyperparameter information, such as the structure of the multi-attribute network, anchor settings, category information, etc. (2) Build a detection head network. Construct the number of category predictions for the detection head based on the total number of attribute categories N. (3) Reconstruct the true value label format. The number of main attributes is 4. Filter out the 4-fold repeated true value detection frames loaded into the network, and reconstruct the format of the true value label to match the format of the prediction results output by the detection head network. (4) Model training and loss solution. Load the pre-trained weights of the model, set the training image size and batch size, select Adam as the optimizer, and iterate the model for 300 epochs. The sum of the losses of the three main attributes is used as the multi-attribute detection loss.

[0195] 3. Deploy the model on the cloud server. This step includes 3 steps: (1) Intermediate model conversion. Extract the weights of the training model, define the model input and output names, call the torch.onnx.export API, and convert the model into an intermediate format (2) TensorRT model conversion. Convert the model to plan format through the trtexec interface of TensorRT and generate the corresponding configuration file. (3) Cloud server deployment. Deploy the model in the cloud warehouse, configure the multi-attribute detection service on the cloud server, and allocate ports for the side server to call.

[0196] 4. Video stream reasoning. This step includes two steps: (1) Video stream preprocessing. Get the video stream of the RTMP protocol, call the OPENCV interface cv2.resize() to scale the input image, fill the short side with padding to (640,640), then call the numpy interface numpy.transpose() to perform transposition and channel transformation, and finally divide the image by 255 for normalization. (2) Cloud streaming. Call the service port of the cloud server, send a request for vehicle multi-attribute detection (which can be a request for multiple video streams), and return the cloud inference result.

[0197] 5. Screening after obtaining the cloud-based inference results. Set the confidence threshold to 0.25 and the NMS threshold to 0.5, complete the initial screening of the predicted values ​​within the main attributes, the confidence threshold screening, the repeated detection box screening, and reconstruct the format of the cloud-based inference results to obtain the target prediction results.

[0198] 6. Set violation judgment rules. Uncertain vehicle attributes such as U_belt, U_plate, etc. are considered qualified. Set 5 consecutive frames as stable judgment conditions. When the judgment result is that the driver is not wearing a seat belt, the vehicle is a muck truck, there is no license plate, etc., a report prompt will be issued. Otherwise, no report prompt will be issued and only vehicle information will be recorded.

[0199] 7. Model incremental training / optimization. Select verification videos / images for model inference, and have the annotators verify and correct the inference results. After collecting enough data samples, a new data set can be constructed to perform incremental training on the model to improve the model accuracy.

[0200] The multi-attribute target detection method provided in the embodiment of the present application can be executed by a multi-attribute target detection device. In the embodiment of the present application, the multi-attribute target detection device executing the multi-attribute target detection method is taken as an example to illustrate the multi-attribute target detection device provided in the embodiment of the present application.

[0201] like Figure 6 As shown, the multi-attribute target detection device 600 provided in the embodiment of the present application may include:

[0202] An acquisition module 601 is used to acquire a target image;

[0203] The detection module 602 is used to input the target image into the target multi-attribute detection model for detection processing to obtain a prediction result;

[0204] The target multi-attribute detection model is a multi-task detection model for detecting a target object and its attributes, and the prediction result includes the detection result of the target object and the multi-attribute classification result of the target object.

[0205] According to the multi-attribute target detection device provided by the embodiment of the present application, it includes an acquisition module for acquiring a target image; a detection module for inputting the target image into a target multi-attribute detection model for detection processing to obtain a prediction result; wherein the target multi-attribute detection model is a multi-task detection model for detecting a target object and the attributes of the target object, and the prediction result includes the detection result of the target object, and the multi-attribute classification result of the target object. In this way, since the target multi-attribute detection model can detect and obtain the detection result of the target object and the multi-attribute classification result of the target object at the same time, compared with the related art using a plurality of model cascades (such as a target detection model + a target classification model) to perform multi-task detection (such as a target object detection task and a target object multi-attribute classification task), the embodiment of the present application performs the detection task of the target object and the multi-attribute classification task of the target object through a single target multi-attribute detection model, thereby improving the efficiency of the multi-attribute detection of the target object and solving the problem of the long time of the multi-attribute detection of the target object in the related art.

[0206] Optionally, in the multi-attribute target detection device provided in an embodiment of the present application, the detection result of the target object includes the position of the detection box and the first prediction probability that the target object exists in the detection box; the multi-attribute classification result of the target object includes N second prediction probabilities, and the N second prediction probabilities correspond one-to-one to the N attribute categories of the target object, where N is an integer greater than 1.

[0207] In this way, the embodiment of the present application can obtain the detection result of the target object and the multi-attribute classification result of the target object at the same time through the target multi-attribute detection model, and obtain the position of the detection box, the first prediction probability that the target object exists in the detection box, and N second prediction probabilities corresponding to the N attribute categories of the target object.

[0208] Optionally, in the multi-attribute target detection device provided in the embodiment of the present application, the target multi-attribute detection model includes: a backbone network, a feature pyramid network, a region proposal network, and a detection head network connected in sequence;

[0209] The detection module is specifically used for:

[0210] Extract multiple feature maps of different scales from the target image through the backbone network;

[0211] The feature maps of multiple different scales are fused through the feature pyramid network to obtain the target feature map. The size of the target feature map remains unchanged relative to the feature maps of different scales, and the number of channels is normalized to a fixed value.

[0212] The target feature map is predicted through the region proposal network to obtain the location of the candidate detection box area;

[0213] By detecting multiple prediction layers of the head network, prediction processing is performed on the part of the target feature map mapped to the candidate detection frame area to obtain the position of the detection frame where the target object exists in the candidate detection frame area, the first prediction probability that the target object exists in the detection frame, and N second prediction probabilities.

[0214] In this way, the embodiment of the present application can extract multiple feature maps of different scales from the target image through the backbone network of the target multi-attribute detection model; fuse the multiple feature maps of different scales through the feature pyramid network to obtain the target feature map; predict the target feature map through the region proposal network to obtain the position of the candidate detection frame area; predict the part of the target feature map mapped to the candidate detection frame area through multiple prediction layers of the detection head network to obtain the position of the detection frame where the target object exists in the candidate detection frame area, the first prediction probability of the target object existing in the detection frame, and N second prediction probabilities, so as to achieve the detection result of the target object and the multi-attribute classification result of the target object predicted by a single model.

[0215] Optionally, in the multi-attribute target detection device provided in the embodiment of the present application, the target multi-attribute detection model is deployed in a cloud server;

[0216] Get the module, specifically for:

[0217] Receive a multi-attribute detection request from a side server, where the multi-attribute detection request carries a target image;

[0218] Extract target images from multi-attribute detection requests.

[0219] In this way, the execution steps of the multi-attribute detection method are encapsulated as an API service of the cloud server, and based on the API service of the cloud server, a real-time multi-attribute detection service is provided for the smart scene.

[0220] Optionally, in the multi-attribute target detection device provided in the embodiment of the present application, the target object has M main attributes, each main attribute has multiple sub-attributes; each main attribute is used to reflect a characteristic of the target object; each sub-attribute is used to reflect a state of the characteristic; a main attribute and a sub-attribute are combined to form an attribute category;

[0221] The multi-attribute target detection device provided in the embodiment of the present application further includes a screening module, which is used to:

[0222] After the target image is input into the target multi-attribute detection model for detection processing and the prediction result is obtained, the prediction result is screened according to the preset screening strategy to obtain the target prediction result; the target prediction result includes the position of the detection box, the first prediction probability that the target object exists in the detection box, and M second prediction probabilities out of N second prediction probabilities, where M is an integer less than N.

[0223] In this way, the target prediction result is obtained by screening the prediction results, so that detection and classification are integrated in the screening process, which improves the efficiency of multi-attribute target detection.

[0224] Optionally, in the multi-attribute target detection device provided in the embodiment of the present application, the screening module is specifically used to:

[0225] According to the number of main attributes of the target object, M second prediction probabilities are determined from the N second prediction probabilities of the multi-attribute classification result of the target object; wherein each of the M second prediction probabilities corresponds to the maximum value of the second prediction probability under one main attribute;

[0226] Determine whether the M second prediction probabilities are all greater than the first confidence threshold;

[0227] When the M second prediction probabilities are all greater than the first confidence threshold, the M second prediction probabilities are retained and used as part of the target prediction result.

[0228] In this way, it is ensured that each main attribute of the target object output by the model has a valid second prediction probability.

[0229] Optionally, in the multi-attribute target detection device provided in the embodiment of the present application, the screening module is specifically used to:

[0230] Determine whether a first prediction probability that a target object exists in the detection box in the prediction result is greater than a second confidence threshold;

[0231] When the first prediction probability that the target object exists in the detection frame is greater than the second confidence threshold, the first prediction probability that the target object exists in the detection frame is retained, and the position of the detection frame and the first prediction probability that the target object exists in the detection frame are used as part of the target prediction result.

[0232] In this way, it is ensured that the first predicted probability of the target object output by the model is valid.

[0233] Optionally, in the multi-attribute target detection device provided in the embodiment of the present application, the detection module is specifically used to:

[0234] Input the target image into the target multi-attribute detection model for detection processing to obtain multiple candidate results; wherein the detection box position included in each candidate result is different;

[0235] A first candidate result is determined from the multiple candidate results, where the first candidate result is a candidate result with the highest first prediction probability that the target object exists in the detection frame, and the first candidate result is used as the prediction result.

[0236] In this way, the candidate result with the largest first prediction probability is selected as the final prediction result, and other repeated candidate results can be filtered out.

[0237] Optionally, the multi-attribute target detection device provided in the embodiment of the present application further includes a determination module, and the determination module is used to:

[0238] Determine whether there is a second candidate result in the candidate results other than the first candidate result, wherein an intersection-over-union ratio of a detection box corresponding to the second candidate result and a detection box corresponding to the first candidate result is less than a non-maximum suppression (NMS) threshold;

[0239] When there is a second candidate result, both the first candidate result and the second candidate result are used as prediction results.

[0240] In this way, both the first candidate result and the second candidate result are used as prediction results, indicating that there are multiple target objects in the target image, and multiple prediction results corresponding to the multiple target objects are output together through the target multi-attribute detection model.

[0241] Optionally, the multi-attribute target detection device provided in the embodiment of the present application further includes an updating module, and the updating module is used to:

[0242] After obtaining the target prediction result, the performance index of the target multi-attribute detection model is determined based on the target prediction result, and the performance index includes at least one of average accuracy, precision rate and recall rate; when the performance index is less than a preset threshold, an incremental training sample is obtained, and the incremental training sample is used to update the target multi-attribute detection model.

[0243] In this way, when the performance indicators of the target multi-attribute detection model are insufficient, the network parameters of the target multi-attribute detection model are fine-tuned and updated through incremental training samples, thereby improving the accuracy of the target multi-attribute detection model.

[0244] Optionally, in the multi-attribute target detection device provided in the embodiment of the present application, the incremental training sample includes: a target image and annotation information corresponding to the target image;

[0245] Among them, the annotation information corresponding to the target image is obtained based on the target prediction result.

[0246] In this way, the model prediction results are converted into annotation labels, and the labelers correct the test samples / new samples in an iterative cycle to improve the model accuracy.

[0247] Optionally, the multi-attribute target detection device provided in the embodiment of the present application further includes:

[0248] A format reconstruction module is used to: after obtaining the target prediction result, reconstruct the format of the target prediction result to obtain M reconstruction results;

[0249] A matching module is used to: determine whether the target object matches a preset violation discrimination rule based on the M reconstruction results;

[0250] The reporting module is used to: when the target object matches the preset violation judgment rule, report prompt information, where the prompt information is used to prompt that the target object in the target image violates the rule.

[0251] In this way, the target prediction result is formatted and M reconstruction results are obtained; based on the M reconstruction results, it is determined whether the target object matches the preset violation judgment rule; when the target object matches the preset violation judgment rule, a prompt information is reported, and the prompt information is used to prompt that the target object in the target image is in violation. In this way, intelligent prompts for intelligent scenes are performed through the prediction results of multi-attribute detection.

[0252] Optionally, the multi-attribute target detection device provided in the embodiment of the present application further includes:

[0253] The training module of the target multi-attribute detection model is used to:

[0254] Obtain a multi-attribute detection model to be trained and a target training sample, where the target training sample includes a sample image and annotation information corresponding to the sample image;

[0255] Iteratively train the multi-attribute detection model to be trained through the target training samples to obtain the target multi-attribute detection model;

[0256] The annotation information includes the position of the multi-attribute detection box where the specified object is located in the sample image, and the attribute category of the specified object.

[0257] In this way, by learning the various features of the specified object and training the target multi-attribute detection model, the target multi-attribute detection model can more accurately predict the detection result of the target object and the multi-attribute classification result of the target object.

[0258] Optionally, in the multi-attribute target detection device provided in the embodiment of the present application, in the process of iteratively training the multi-attribute detection model to be trained using target training samples, the training module is specifically used to:

[0259] The target training samples are used to iteratively train the multi-attribute detection model to be trained using the following loss function:

[0260]

[0261] Among them, loss is the total loss of multi-attribute classification of the specified object in the multi-attribute detection model, loss i Indicates the loss value corresponding to the i-th primary attribute of the specified object, where i is a positive integer less than or equal to M; M is the number of primary attributes of the specified object.

[0262] In this way, the multi-attribute loss strategy of attribute-by-attribute calculation can control the loss of target detection within a single main attribute, and the category imbalance phenomenon in the model training process can be suppressed.

[0263] Optionally, in the multi-attribute target detection device provided in the embodiment of the present application,

[0264]

[0265] Among them, Mj represents the number of sub-attributes of the i-th main attribute;

[0266] in,

[0267] Among them, P ij represents the second predicted probability of the jth sub-attribute corresponding to the i-th main attribute, where j is a positive integer less than or equal to Mj;

[0268] Among them, T ij Indicates P ij The corresponding truth value.

[0269] In this way, by first solving the loss within the main attribute and then summing up the loss values ​​corresponding to each main attribute to obtain the total loss of multi-attribute classification, the loss of target detection is controlled within a single main attribute, and the category imbalance phenomenon in the model training process can be suppressed.

[0270] The multi-attribute target detection device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), etc., which is not specifically limited in the embodiment of the present application.

[0271] The multi-attribute target detection device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0272] The multi-attribute target detection device provided in the embodiment of the present application can achieve Figure 1To avoid repetition, the various processes implemented in the method embodiment of FIG. 5 are not described again here.

[0273] Alternatively, if Figure 7 As shown, an embodiment of the present application also provides an electronic device 700, including a processor 701 and a memory 702, wherein the memory 702 stores programs or instructions that can be executed on the processor 701, and when the program or instructions are executed by the processor 701, the various steps of the above-mentioned method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0274] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0275] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0276] The electronic device 800 includes but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810 and other components.

[0277] Those skilled in the art will appreciate that the electronic device 800 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 810 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0278] The input unit 804 is used to obtain a target image;

[0279] The processor 810 is used to input the target image into the target multi-attribute detection model for detection processing to obtain a prediction result;

[0280] The target multi-attribute detection model is a multi-task detection model for detecting a target object and its attributes, and the prediction result includes the detection result of the target object and the multi-attribute classification result of the target object.

[0281] According to the electronic device provided by the embodiment of the present application, the input unit is used to obtain the target image; the processor is used to input the target image into the target multi-attribute detection model for detection processing to obtain the prediction result; wherein the target multi-attribute detection model is a multi-task detection model for detecting the target object and the attributes of the target object, and the prediction result includes the detection result of the target object, and the multi-attribute classification result of the target object. In this way, since the target multi-attribute detection model can detect the detection result of the target object and the multi-attribute classification result of the target object together, compared with the multi-task detection (such as the detection task of the target object and the multi-attribute classification task of the target object) in the related art using multiple model cascades (such as target detection model + target classification model), the embodiment of the present application performs the detection task of the target object and the multi-attribute classification task of the target object through a single target multi-attribute detection model, which improves the efficiency of the multi-attribute detection of the target object and solves the problem of the long time of the multi-attribute detection of the target object in the related art.

[0282] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and to avoid repetition, it will not be described again here.

[0283] It should be understood that in the embodiment of the present application, the input unit 804 may include a graphics processor (Graphics Processing Unit, GPU) 8041 and a microphone 8042, and the graphics processor 8041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 807 includes a touch panel 8071 and at least one of other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0284] The memory 809 can be used to store software programs and various data. The memory 809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 809 may include a volatile memory or a non-volatile memory, or the memory 809 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 809 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0285] The processor 810 may include one or more processing units; optionally, the processor 810 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 810.

[0286] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0287] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0288] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0289] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0290] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0291] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0292] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0293] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A multi-attribute target detection method, characterized in that: include: Get the target image; Inputting the target image into a target multi-attribute detection model for detection processing to obtain a prediction result; The target multi-attribute detection model is a multi-task detection model for detecting a target object and the attributes of the target object, and the prediction result includes a detection result of the target object and a multi-attribute classification result of the target object.

2. The method according to claim 1, characterized in that The detection result of the target object includes the position of the detection box and the first prediction probability that the target object exists in the detection box; the multi-attribute classification result of the target object includes N second prediction probabilities, and the N second prediction probabilities correspond one-to-one to the N attribute categories of the target object, where N is an integer greater than 1.

3. The method according to claim 2, characterized in that The target multi-attribute detection model includes: a backbone network, a feature pyramid network, a region proposal network, and a detection head network connected in sequence; The inputting the target image into the target multi-attribute detection model for detection processing to obtain the prediction result includes: Extracting a plurality of feature maps of different scales from the target image through the backbone network; The feature maps of the multiple different scales are fused through the feature pyramid network to obtain a target feature map, wherein the size of the target feature map remains unchanged relative to the feature maps of the different scales, and the number of channels is normalized to a fixed value; The target feature map is predicted by the region proposal network to obtain the position of the candidate detection frame area; Through the multiple prediction layers of the detection head network, the part of the target feature map mapped to the candidate detection frame area is predicted to obtain the position of the detection frame where the target object exists in the candidate detection frame area, the first prediction probability that the target object exists in the detection frame, and N second prediction probabilities.

4. The method according to claim 1, characterized in that: The target multi-attribute detection model is deployed in a cloud server; The acquiring of the target image comprises: Receiving a multi-attribute detection request from a side server, wherein the multi-attribute detection request carries a target image; The target image is extracted from the multi-attribute detection request.

5. The method according to claim 1, characterized in that The target object has M main attributes, each of which has multiple sub-attributes; each main attribute is used to reflect a characteristic of the target object; each sub-attribute is used to reflect a state of the characteristic; a main attribute and a sub-attribute are combined to form an attribute category; After inputting the target image into the target multi-attribute detection model for detection processing and obtaining a prediction result, the method further includes: According to a preset screening strategy, the prediction results are screened to obtain target prediction results; The target prediction result includes the position of the detection frame, the first prediction probability that the target object exists in the detection frame, and M second prediction probabilities among N second prediction probabilities, where M is an integer less than N.

6. The method according to claim 5, characterized in that The screening process of the prediction results according to the preset screening strategy includes: According to the number of main attributes of the target object, M second prediction probabilities are determined from the N second prediction probabilities of the multi-attribute classification result of the target object; wherein each of the M second prediction probabilities corresponds to the maximum value of the second prediction probability under one main attribute; Determine whether the M second prediction probabilities are all greater than a first confidence threshold; When the M second prediction probabilities are all greater than the first confidence threshold, the M second prediction probabilities are retained and used as part of the target prediction result.

7. The method according to claim 5, characterized in that The screening process of the prediction results according to the preset screening strategy includes: Determine whether a first prediction probability that a target object exists in the detection frame in the prediction result is greater than a second confidence threshold; When the first prediction probability that the target object exists in the detection frame is greater than the second confidence threshold, the first prediction probability that the target object exists in the detection frame is retained, and the position of the detection frame and the first prediction probability that the target object exists in the detection frame are used as part of the target prediction result.

8. The method according to claim 1 or 5, characterized in that: The step of inputting the target image into the target multi-attribute detection model for detection processing to obtain a prediction result includes: Inputting the target image into a target multi-attribute detection model for detection processing to obtain multiple candidate results; wherein the detection frame position included in each candidate result is different; A first candidate result is determined from the multiple candidate results, where the first candidate result is a candidate result with the highest first prediction probability that a target object exists in the detection frame, and the first candidate result is used as a prediction result.

9. The method according to claim 8, characterized in that The step of inputting the target image into the target multi-attribute detection model for detection processing to obtain a prediction result further includes: Determine whether there is a second candidate result among the candidate results other than the first candidate result, wherein an intersection-over-union ratio of a detection box corresponding to the second candidate result and a detection box corresponding to the first candidate result is less than a non-maximum suppression (NMS) threshold; When there is a second candidate result, both the first candidate result and the second candidate result are used as prediction results.

10. The method according to claim 5, characterized in that After obtaining the target prediction result, the method further includes: Based on the target prediction result, determining a performance indicator of the target multi-attribute detection model, the performance indicator comprising at least one of average accuracy, precision and recall; When the performance indicator is less than a preset threshold, an incremental training sample is obtained, and the target multi-attribute detection model is updated using the incremental training sample.

11. The method according to claim 10, characterized in that The incremental training sample includes: the target image and annotation information corresponding to the target image; The annotation information corresponding to the target image is obtained based on the target prediction result.

12. The method according to claim 5, characterized in that After obtaining the target prediction result, the method further includes: Reconstructing the format of the target prediction result to obtain M reconstruction results; Based on the M reconstruction results, determining whether the target object matches a preset violation discrimination rule; In the case where the target object matches the preset violation determination rule, prompt information is reported, where the prompt information is used to prompt that the target object in the target image violates the rule.

13. The method according to claim 1, characterized in that The training process of the target multi-attribute detection model includes: Acquire a multi-attribute detection model to be trained and a target training sample, wherein the target training sample includes a sample image and annotation information corresponding to the sample image; Iteratively training the multi-attribute detection model to be trained using the target training samples to obtain the target multi-attribute detection model; The annotation information includes the position of the multi-attribute detection box where the specified object is located in the sample image, and the attribute category of the specified object.

14. The method according to claim 13, characterized in that The iterative training of the multi-attribute detection model to be trained by using the target training sample includes: The multi-attribute detection model to be trained is iteratively trained using the target training sample using the following loss function: Among them, loss is the total loss of multi-attribute classification of the specified object in the multi-attribute detection model, loss i Indicates the loss value corresponding to the i-th primary attribute of the specified object, where i is a positive integer less than or equal to M; M is the number of primary attributes of the specified object.

15. The method according to claim 14, characterized in that Among them, Mj represents the number of sub-attributes of the i-th main attribute; in, Among them, P ij represents the second predicted probability of the jth sub-attribute corresponding to the i-th main attribute, where j is a positive integer less than or equal to Mj; Among them, T ij Indicates P ij The corresponding truth value.

16. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 15 are implemented.

17. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 15 are implemented.

Images