Attribute prediction model training method and device, early warning method and device, electronic equipment and storage medium

Abstract

The invention provides a training method and device of an attribute prediction model, an early warning method and device, electronic equipment and a storage medium. The method comprises the steps of predicting a plurality of attribute prediction results of a target object included in a target sample image based on a neural network model; determining corresponding attribute loss based on each attribute prediction result and a corresponding attribute label included in the target object; each attribute loss is weighted to obtain the loss of the neural network model, and the weight of each attribute loss is associated with the positive and negative sample balance degree of the corresponding attribute tag; if the loss of the neural network model does not meet the convergence condition, updating model parameters of the neural network model; otherwise, determining the neural network model as an attribute prediction model. The method provided by the invention not only can eliminate the influence of the imbalance of the positive and negative samples on the loss of the neural network model by using the balance degree of the positive and negative samples of the attribute tag, improves the learning efficiency of the neural network model, but also can improve the attribute prediction precision of the attribute prediction model in the reasoning stage.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a training method, an early warning method, a device, an electronic device and a storage medium for an attribute prediction model. Background Art

[0002] With the rapid development of computer vision, artificial intelligence and other fields, attribute recognition technology has been widely used in security inspections, dangerous behavior warnings, traffic violation monitoring, industrial security and other fields.

[0003] In related technologies, pre-trained neural network models can be used in safety inspection scenarios to simultaneously identify multiple attributes of personnel in factories, power plants and other operating parks to detect whether there are any illegal clothing and dangerous actions. However, during the training phase of the neural network model, when collecting training samples, there may be an imbalance between positive and negative samples, resulting in low prediction accuracy of multiple attributes for the final trained neural network model. Summary of the invention

[0004] According to one aspect of the present disclosure, a method for training an attribute prediction model is provided, comprising:

[0005] Predicting multiple attribute prediction results of a target object included in a target sample image based on a neural network model;

[0006] Determine the corresponding attribute loss based on the prediction results of each attribute and the corresponding attribute label included in the same target object;

[0007] The loss of each attribute is weighted to obtain the loss of the neural network model. The weight of each attribute loss is associated with the balance of positive and negative samples of the corresponding attribute label.

[0008] If the loss of the neural network model does not meet the convergence conditions, the model parameters of the neural network model are updated based on the attribute prediction results and the corresponding attribute labels included in the target object; otherwise, the neural network model is determined to be an attribute prediction model.

[0009] According to another aspect of the present disclosure, there is provided an early warning method, comprising:

[0010] Inputting an image to be detected containing an object to be detected into an attribute prediction model to obtain multiple attribute detection results of the object to be detected, wherein the attribute prediction model is an attribute prediction model of an exemplary embodiment of the present disclosure;

[0011] Determine a detection result of the object to be detected based on multiple attribute detection results;

[0012] If the detection result of the object to be detected meets the warning condition, a warning is triggered for the object to be detected.

[0013] According to another aspect of the present disclosure, there is provided a training device for an attribute prediction model, comprising:

[0014] A prediction module, used for predicting multiple attribute prediction results of a target object included in the target sample image based on a neural network model;

[0015] A determination module, used to determine corresponding attribute losses based on various attribute prediction results and corresponding attribute labels included in the same target object;

[0016] The determination module is also used to weight the losses of each attribute to obtain the loss of the neural network model. The weight of each attribute loss is associated with the balance of positive and negative samples of the corresponding attribute label;

[0017] The training module is used to update the model parameters of the neural network model based on the respective attribute prediction results and the corresponding attribute labels included in the target object if the loss of the neural network model does not meet the convergence conditions; otherwise, the neural network model is determined to be an attribute prediction model.

[0018] According to another aspect of the present disclosure, there is provided an early warning device, comprising:

[0019] A detection module, used for inputting an image to be detected containing an object to be detected into an attribute prediction model to obtain multiple attribute detection results of the object to be detected, wherein the attribute prediction model is the attribute prediction model described in the exemplary embodiment of the present disclosure;

[0020] A determination module, used to determine the detection result of the object to be detected based on multiple attribute detection results;

[0021] The determination module is also used to trigger an early warning for the object to be detected if the detection result of the object to be detected meets the early warning condition.

[0022] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0023] processor; and,

[0024] A memory for storing programs;

[0025] The program includes instructions, and when the instructions are executed by a processor, the processor executes the method according to the exemplary embodiment of the present disclosure.

[0026] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause a computer to execute the method according to the exemplary embodiments of the present disclosure.

[0027] One or more technical solutions provided in the exemplary embodiments of the present disclosure can predict multiple attribute prediction results of the target object included in the target sample image based on the neural network model, determine the corresponding attribute loss based on each attribute prediction result and the corresponding attribute label included in the target object, and then weight each attribute loss to obtain the loss of the neural network model, and the weight of each attribute loss is associated with the positive and negative sample balance of the corresponding attribute label, so that when determining the loss of the neural network model, the influence of the positive and negative sample balance of the attribute label corresponding to each attribute prediction result on the corresponding attribute loss can be considered, reducing the loss difference caused by the imbalance of positive and negative samples of the attribute label, improving the learning efficiency of the neural network model, and thus increasing the attribute prediction accuracy of the attribute prediction model finally obtained in the reasoning stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Further details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0029] Figure 1 A flowchart showing a method for training an attribute prediction model according to an exemplary embodiment of the present disclosure is shown;

[0030] Figure 2 A schematic diagram showing the architecture of a neural network model of an exemplary embodiment of the present disclosure is shown;

[0031] Figure 3 A characteristic curve diagram of the first positive-negative sample balance and the positive-negative sample attribute labels of an exemplary embodiment of the present disclosure is shown;

[0032] Figure 4 A characteristic curve diagram of a second positive-negative sample balance and positive-negative sample attribute labels of an exemplary embodiment of the present disclosure is shown;

[0033] Figure 5 A third characteristic curve diagram of the positive and negative sample balance and the positive and negative sample attribute labels of an exemplary embodiment of the present disclosure is shown;

[0034] Figure 6 A fourth characteristic curve diagram of the positive-negative sample balance and the positive-negative sample attribute labels of an exemplary embodiment of the present disclosure is shown;

[0035] Figure 7 A flowchart showing an early warning method according to an exemplary embodiment of the present disclosure is shown;

[0036] Figure 8 A schematic block diagram of modules of a training device for an attribute prediction model according to an exemplary embodiment of the present disclosure is shown;

[0037] Fig. 9 A schematic block diagram of modules of an early warning device according to an exemplary embodiment of the present disclosure is shown;

[0038] Fig.10 A schematic block diagram of a chip according to an exemplary embodiment of the present disclosure is shown;

[0039] Fig.11 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0040] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0041] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0042] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0043] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0044] In related technologies, pre-trained neural network models are often used in safety inspection scenarios to simultaneously identify multiple attributes of personnel in factories, power plants and other working parks to detect whether there are any illegal clothing and dangerous actions, thereby ensuring the safety of personnel in the working park. However, during the training stage of the neural network model, when collecting training samples, there may be an imbalance between positive and negative samples, resulting in low prediction accuracy of multiple attributes for the final trained neural network model.

[0045] In response to the above problems, the exemplary embodiments of the present disclosure provide a training method and an early warning method for an attribute prediction model, which can utilize the balance of positive and negative samples of attribute labels when training the attribute prediction model, eliminate the influence of the corresponding attribute loss determined based on the attribute label on the loss of the neural network model, reduce the loss difference caused by the imbalance of positive and negative samples of the attribute label, and improve the learning efficiency of the neural network model, thereby increasing the attribute prediction accuracy of the attribute prediction model finally obtained in the reasoning stage. At the same time, the trained attribute prediction model can also be used to improve the early warning accuracy and reduce false alarms.

[0046] It should be understood that the method of the exemplary embodiment of the present disclosure can be executed by an electronic device or by a chip applied to the electronic device. For example, the electronic device of the exemplary embodiment of the present disclosure can be executed by an electronic device with a display function, and the electronic device can be a mobile phone, a tablet computer, a wearable device, an in-vehicle device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and other terminals.

[0047] Figure 1 FIG. 4 is a flow chart showing a method for training an attribute prediction model according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the training method of the attribute prediction model of the exemplary embodiment of the present disclosure may include:

[0048] Step 101: Predict multiple attribute prediction results of a target object included in a target sample image based on a neural network model.

[0049] The target sample image can be a video image of the monitored area captured by the image acquisition device. The target objects included therein can include various biological entities such as people and animals, non-biological entities such as static objects and movable objects, and various virtual characters, which are not specifically limited here.

[0050] In practical applications, the video images in the monitoring area can be collected by an image acquisition device, and then the video images are subjected to frame extraction processing. The video images obtained by frame extraction are then input into a neural network model to predict multiple attribute prediction results of the target object.

[0051] Step 102: Determine the corresponding attribute loss based on the attribute prediction results and corresponding attribute labels included in the same target object.

[0052] The prediction results of multiple attributes of the same target object are independent of each other and do not affect each other. Therefore, when determining the loss of the neural network model, it is necessary to first determine the attribute loss corresponding to each attribute.

[0053] The above-mentioned attribute labels may be attribute labels pre-labeled for the target object. After predicting multiple attribute prediction results of the target object based on the neural network model, the corresponding attribute loss may be determined based on each attribute prediction result and the corresponding attribute label.

[0054] In the related art, attribute labels can be divided into positive sample attribute labels and negative sample attribute labels. The positive sample attribute label can be regarded as the attribute label when the attribute of the attribute label exists or is correct, while the negative sample attribute label can be regarded as the attribute label when the attribute of the attribute label does not exist or is wrong.

[0055] If a certain attribute label of the target object is mostly a positive sample attribute label, then the probability of the attribute prediction result predicted by the neural network model belonging to the positive sample is relatively high, while the probability of predicting a negative sample is relatively low, which is not conducive to the learning of the neural network model for negative samples. If a certain attribute label of the target object is mostly a negative sample attribute label, then the probability of the attribute prediction result predicted by the neural network model belonging to the negative sample is relatively high, while the probability of predicting a positive sample is relatively low, which is not conducive to the learning of the neural network model for positive samples. Based on this, the method of the exemplary embodiment of the present disclosure may also include:

[0056] Step 103: Weight the losses of each attribute to obtain the loss of the neural network model. The weight of each attribute loss is associated with the balance of positive and negative samples of the corresponding attribute label.

[0057] For the positive and negative sample balance of each attribute label, the positive and negative sample balance of the attribute label is positively correlated with the probability of occurrence of the positive sample attribute label in the corresponding attribute label in different target objects. In other words, the higher the positive and negative sample balance here, the higher the probability of positive samples of the attribute label appearing in the training set, and vice versa, the higher the probability of negative samples of the attribute label appearing in the training set.

[0058] The weight of the attribute loss is used to indicate the degree of attention paid by the neural network model to the corresponding attributes of the target object during the training process. The greater the weight of the attribute loss, the more attention the neural network model pays to the corresponding attributes of the target object. Based on this, the weight of the attribute loss can be associated with the positive and negative sample balance of the corresponding attribute label, thereby introducing the influence of the positive and negative sample balance on the prediction results into the loss calculation to improve the attention of the neural network model to negative samples.

[0059] It can be seen that the exemplary embodiment of the present disclosure balances the learning difference of the neural network model for positive and negative samples caused by the imbalance of positive and negative samples through the weight balance of attribute loss. On this basis, each attribute loss can be weighted in a weighted manner to obtain the loss of the neural network model. At the same time, when the number of target objects is multiple, multiple attribute losses of the same target object can be weighted first to obtain the total attribute loss of the target object, and then the total attribute losses of each target object are summed to obtain the loss of the neural network model.

[0060] Step 104: Determine whether the loss of the neural network model meets the convergence condition.

[0061] Here, the above-mentioned convergence condition may include that the loss of the neural network model is less than or equal to a preset threshold, or it may refer to that the loss of the neural network model is stable, which is determined according to the actual application scenario and is not limited here.

[0062] If the loss of the neural network model does not meet the convergence conditions, it means that the accuracy of the attribute prediction results of the currently trained attribute prediction model is relatively poor, and it cannot be accurately used to predict the multiple attribute information of the target object included in the target sample image. Further training is required to meet the accuracy of attribute prediction. At this time, step 105 can be executed; if the loss of the neural network model meets the convergence conditions, it means that the accuracy of the attribute prediction results of the currently trained attribute prediction model is relatively high, and it can be accurately used to predict the multiple attribute information of the target object included in the target sample image. At this time, step 106 can be executed.

[0063] Step 105: Update the model parameters of the neural network model based on the attribute prediction results and the corresponding attribute labels of the target object. Here, the model parameters may include weights and / or offset values.

[0064] Step 106: Determine that the neural network model is an attribute prediction model. At this point, the training process of the attribute prediction model has been completed. The trained attribute prediction model is used to predict multiple attributes of the target object in the inference stage. When the detection result determined based on multiple attribute prediction information can trigger an early warning, therefore, when the attribute prediction accuracy of the attribute prediction model is improved, the early warning accuracy will also be improved, thereby reducing the possibility of false alarms.

[0065] It can be seen that the method of the exemplary embodiment of the present disclosure can not only utilize the positive and negative sample balance of the attribute label to balance the impact of the corresponding attribute loss determined based on the attribute label on the loss of the neural network model, thereby improving the learning efficiency of the neural network model, but also improve the attribute prediction accuracy of the attribute prediction model in the reasoning stage.

[0066] In an alternative approach, Figure 2The schematic diagram of the architecture of the neural network model of the exemplary embodiment of the present disclosure is shown as follows: Figure 2 As shown, the neural network model 200 of the exemplary embodiment of the present disclosure may include: a target detection unit 210 and an attribute recognition unit 220. At this time, the neural network model in the exemplary embodiment of the present disclosure predicts multiple attribute prediction results of the target object included in the target sample image, which may include: detecting the target sample image based on the target detection unit to obtain the target object; and predicting multiple attribute prediction results of the target object based on the attribute recognition unit.

[0067] An exemplary embodiment of the present disclosure may utilize the target detection unit 210 to obtain position information of one or more target objects from a target sample image, and then crop the target sample image to a corresponding target object image based on the position information of each target object, wherein the target object image is an image of the target object included in the target sample image.

[0068] Considering the size characteristics of the target object, the size of the cropped target object image can be determined according to the type of the target object. Taking the human body as an example, the crop size of the target object can be set to a target object image with an aspect ratio of 2:1. Exemplarily, the network model of the target detection unit can include one of the YOLOv5s network model and the mmdet series custom network model.

[0069] When predicting multiple attribute prediction results of a target object based on the attribute recognition unit 220, one or more target object images can be input into the attribute recognition unit 220, so as to predict multiple attribute prediction results of each target object, thereby reducing the impact of non-target objects on the multiple attribute prediction results of the target object. Here, multiple target object images can be simultaneously input into the attribute recognition unit as a batch of training data for attribute prediction, so as to achieve the purpose of batch prediction, thereby improving the training speed.

[0070] In practical applications, the cropped human images can be preprocessed by data enhancement, size adjustment, etc. to obtain a 256*128*3 target sample image, and then X human images are integrated into a batch image package to form an X*256*128*3 human image package, X=batch_size, that is, the number of human images X included in the batch image package is equal to the size batch_size of the processed image package.

[0071] The attribute recognition unit 220 of the exemplary embodiment of the present disclosure may include a backbone network 221, a pooling layer 222, a channel weighting layer 223, and a classification layer 224. Here, the backbone network 221 is responsible for extracting features from the target object image to obtain an image feature map; the pooling layer 222 performs a dimensionality reduction operation on the image feature map to obtain a multi-channel pooling feature map; the channel weighting layer 223 weights the different channel features included in the multi-channel pooling feature map to obtain a multi-channel weighted feature map, thereby enhancing the focus on important channels and improving the expressiveness of the attribute recognition unit; the classification layer 224 classifies the multi-channel weighted feature map to obtain multiple attribute prediction results.

[0072] Exemplarily, the architecture of the backbone network 221 can select the architecture of different classification models such as mobilev2, resnet-50, etc. for feature extraction. Taking resnet50 as an example, a human image package of [batch_size, 256, 128, 3] can be input into resnet50 to obtain a high-dimensional matrix feature of [batch_size, 8, 4, 2048].

[0073] The above-mentioned pooling layer 222 can be an average pooling layer, a maximum pooling layer or a random pooling layer. When the pooling layer is an average pooling layer, the high-dimensional matrix features of [batch_size, 8, 4, 2048] are processed by the average pooling layer to obtain a feature vector of [batch_size, 2048].

[0074] The channel weighting layer 223 may select a channel attention model (Squeeze-and-Excitation Networks, SENet) architecture. The channel attention model may be used to weight the feature vector of [batch_size, 2048] to obtain a weighted feature vector of [batch_size, 2048], thereby enhancing each channel feature included in the feature vector.

[0075] The classification layer 224 may include one or more fully connected layers, through which the weighted feature vectors of [batch_size, 2048] are fused to obtain multiple attribute prediction results of the target object. It can be seen that the classification layer can realize the process of converting (or mapping) the feature vector dimension to the final attribute category number.

[0076] In practical applications, depending on whether the target object has the attribute, the attribute prediction result output by the attribute prediction model of the exemplary embodiment of the present disclosure can be expressed as not having the attribute or having the attribute. When calculating the loss, the attribute loss of each attribute prediction result and the corresponding attribute label in the same target object can be evaluated through a binary cross entropy loss function, and then the attribute losses of all target objects are summed up in a weighted manner to obtain the loss of the neural network model.

[0077] Assuming that the number of target objects is N and the number of attributes of each target object is M, the loss of the neural network model can be described by the loss function expression shown in Formula 1.

[0078]

[0079] Among them, Loss represents the loss of the neural network model, x ij represents the j-th attribute prediction result of the i-th target object, y ij represents the jth attribute label of the i-th target object, w ij Represents the weight of the j-th attribute loss of the i-th target object, where i is an integer greater than or equal to 1 and less than or equal to N, and j is an integer greater than or equal to 1 and less than or equal to M.

[0080] In an optional manner, the exemplary embodiment of the present disclosure may determine the weight of the corresponding attribute loss based on the positive and negative sample balance of each attribute prediction result and the corresponding attribute label.

[0081] Here, when a certain attribute label of the target object is mostly a positive sample attribute label, it means that the neural network model is more likely to predict the attribute prediction result that matches the positive sample. Therefore, it can be judged whether the attribute prediction result matches the positive sample or the negative sample of the attribute to determine the correlation between the weight of the attribute loss and the balance of the positive and negative samples of the attribute label, thereby reducing the possibility of the neural network model learning the positive sample prediction result.

[0082] If the attribute prediction result matches the positive sample of the corresponding attribute, the impact of the attribute loss on the neural network model loss can be weakened. Therefore, the weight of the attribute loss can be set to be negatively correlated with the balance of positive and negative samples of the attribute label. Since the higher the balance of positive and negative samples, the more positive samples of the corresponding attribute label relative to the number of negative samples, therefore, when the attribute prediction result matches the positive sample of the corresponding attribute, the correlation between the set attribute loss weight and the balance of positive and negative samples of the attribute label can reduce the possibility of the neural network model learning the positive sample prediction result.

[0083] If the attribute prediction result matches the negative sample of the corresponding attribute, the impact of the attribute loss on the neural network model loss can be enhanced, and the possibility of the neural network model learning the negative sample prediction result of the attribute label can be increased. Based on this, the weight of the attribute loss can be set to be positively correlated with the balance of positive and negative samples of the attribute label. Since the smaller the balance of positive and negative samples, the smaller the number of negative samples of the corresponding attribute label relative to the number of positive samples, therefore, when the attribute prediction result matches the positive sample of the corresponding attribute, the correlation between the set attribute loss weight and the balance of positive and negative samples of the attribute label can enhance the possibility of the neural network model learning the positive sample prediction result.

[0084] It can be seen that the correlation between the weight of the attribute loss and the positive and negative sample balance of the attribute label in the exemplary embodiment of the present disclosure is affected by the matching of the attribute prediction result with the positive and negative samples of the attribute. Based on this, two different expressions can be introduced to define the correlation between the weight of the attribute loss and the positive and negative sample balance of the attribute label. For example, the exemplary embodiment of the present disclosure can define the correlation between the weight of the attribute loss and the positive and negative sample balance of the attribute label through the piecewise function expression shown in Formula 2.

[0085]

[0086] Among them, w ij represents the weight of the jth attribute loss of the i-th target object, r j represents the balance of positive and negative samples of the jth attribute label of the i-th target object, y ij Represents the j-th attribute label of the i-th target object.

[0087] When the jth attribute label of the i-th target object is a positive sample attribute label, y ij =1, the corresponding attribute loss weight and the expression of the balance between positive and negative samples of the attribute label are: At this time, r j The larger the w ij The smaller the j The smaller the w ij The bigger.

[0088] When the jth attribute label of the i-th target object is a negative sample attribute label, y ij = 0, the corresponding attribute loss weight and the expression of the balance between positive and negative samples of the attribute label are: At this time, r j The larger the w ij The larger the value, the smaller the j The smaller the w ij The smaller.

[0089] In an optional manner, the positive and negative sample balance of the exemplary embodiment of the present disclosure may be positively correlated with the probability of occurrence of the positive sample attribute labels in the corresponding attribute labels in different target objects, and the degree of imbalance between the positive and negative samples expressed by the positive and negative sample balance may be evaluated by a given preset balance.

[0090] In practical applications, for the same attribute label, when the probability of occurrence of the positive sample attribute label in different target objects is greater than the probability of occurrence of the negative sample attribute label in different target objects, it means that the different target objects are more inclined to the positive sample attribute label. At this time, the positive and negative sample balance is greater than the preset balance. For the same attribute label, when the probability of occurrence of the positive sample attribute label in different target objects is less than the probability of occurrence of the negative sample attribute label in different target objects, it means that the different target objects are more inclined to the negative sample attribute label. At this time, the positive and negative sample balance is less than the preset balance.

[0091] Exemplarily, the exemplary embodiments of the present disclosure can collect a large number of target object images, and annotate multiple attribute labels of the target objects in the target object images, and then for each attribute, count the number of positive sample attribute labels and negative sample attribute labels appearing in all target object images, calculate the probability of occurrence of positive sample attribute labels and negative sample attribute labels in a large number of target object images, and determine the degree of imbalance between positive and negative samples based on the probability of occurrence of positive and negative sample attribute labels in a large number of target object images.

[0092] When the probability of occurrence of the positive sample attribute label in a large number of target object images is equal to the probability of occurrence of the negative sample attribute label in a large number of target object images, the positive and negative samples of the large number of target object images are evenly distributed, and the positive and negative sample balance of the attribute label is equal to the preset balance. In the method of the exemplary embodiment of the present disclosure, the value of the preset balance is 0.5.

[0093] When the probability of occurrence of a positive sample attribute label in a large number of target object images is greater than the probability of occurrence of a negative sample attribute label in a large number of target object images, a large number of target object images are more inclined to the positive sample attribute label, and the positive and negative sample balance of the attribute label is less than the preset balance, and its value is greater than 0.5 and less than 1, and the more positive sample attribute labels there are, the closer the positive and negative sample balance of the attribute label is to 1.

[0094] When the probability of occurrence of a positive sample attribute label in a large number of target object images is less than the probability of occurrence of a negative sample attribute label in a large number of target object images, a large number of target object images are more inclined to the negative sample attribute label, and the positive and negative sample balance of the attribute label is less than the preset balance, and its value is greater than 0 and less than 0.5. The more negative sample attribute labels there are, the closer the positive and negative sample balance of the attribute label is to 0.

[0095] It can be seen that the method of the exemplary embodiment of the present disclosure can determine the positive and negative sample balance of the attribute label corresponding to the attribute by the probability of occurrence of the positive sample attribute label and the negative sample attribute label included in the same attribute in different target objects, and then use the relationship between the positive and negative sample balance and the given preset balance to evaluate the degree of positive and negative sample imbalance of different target objects.

[0096] In practical applications, the positive and negative sample balance of the exemplary embodiment of the present disclosure can be determined by the linear method, the cosine method and the square method. j Indicates the balance of positive and negative samples of the jth attribute label of the target object, r j Greater than 0 and less than 1. For the value of j, see the previous text.

[0097] When the linear method is used to determine the positive and negative sample balance, the exemplary embodiment of the present disclosure can determine the correlation between the positive and negative sample balance of the j-th attribute label and the positive and negative sample attribute labels through the expression shown in Formula 3.

[0098]

[0099] Among them, y ij The values ​​of , M, N, i and j are as mentioned above and will not be repeated here. When the jth attribute label of the i-th target object is a positive sample attribute label, y ij =1; when the jth attribute label of the i-th target object is a negative sample attribute label, y ij =0.

[0100] Figure 3 FIG. 1 shows a characteristic curve diagram of the first positive and negative sample balance and the positive and negative sample attribute labels of an exemplary embodiment of the present disclosure. Figure 3 As shown in the figure, for the jth attribute label of the target object, the more positive sample attribute labels there are, the larger the numerator is, while the denominator remains unchanged. j The larger the value, the smaller the j When the attribute labels of positive and negative samples are evenly distributed, r j =0.5; when the number of positive sample attribute labels is N, the jth attribute label of the target object is all positive sample attribute labels. At this time, r j =1; when the number of positive sample attribute labels is 0, the jth attribute label of the target object is all negative sample attribute labels. At this time, r j =1. Figure 3 It can also be seen that the balance of positive and negative samples is linearly correlated with the probability of occurrence of positive sample attribute labels in different target objects, r j The characteristic curve along r j =0.5, symmetrically distributed on both sides.

[0101] When the cosine method is used to determine the positive and negative sample balance, the cosine function can be constructed using the positive and negative sample balance obtained by the linear method, and the positive and negative sample balance can be re-determined based on the cosine function. At this time, the exemplary embodiment of the present disclosure can determine the correlation between the positive and negative sample balance of the jth attribute label and the positive and negative sample attribute labels through the expression shown in Formula 4.

[0102]

[0103] Among them, y ij The values ​​of , M, N, i and j are as mentioned above and will not be repeated here. When the jth attribute label of the i-th target object is a positive sample attribute label, y ij =1; when the jth attribute label of the i-th target object is a negative sample attribute label, y ij =0.

[0104] Figure 4 FIG. 2 shows a characteristic curve diagram of the second positive and negative sample balance and the positive and negative sample attribute labels of an exemplary embodiment of the present disclosure. Figure 4 As shown, for the jth attribute label of the target object, when the positive and negative sample attribute labels are evenly distributed, r j =0.5; when the number of positive sample attribute labels is N, the jth attribute label of the target object is all positive sample attribute labels. At this time, r j =1; when the number of positive sample attribute labels is 0, the jth attribute label of the target object is all negative sample attribute labels. At this time, r j =1.

[0105] from Figure 4 It can also be seen that r j The characteristic curve along r j =0.5 Symmetrical distribution on both sides, r j The closer to 0.5, the smoother the curve, indicating that the positive and negative samples are equally balanced in different target objects. The neural network model is targeted at this r j The positive and negative sample attention of attributes near 0.5 is similar, so there is no need to increase the weight of the corresponding attribute loss. j The closer to 0 or 1, the steeper the curve, indicating that the imbalance between positive and negative samples is high. The neural network model is designed for this r j The positive and negative sample attention of attributes near 0 or 1 is quite different. At this time, the weight of the corresponding attribute loss can be increased to make the neural network model pay more attention to the impact of the distribution at both ends on the loss of the neural network model, so as to improve the neural network model's attention to the fewer positive sample attribute labels or positive sample attribute labels.

[0106] When the balance method is used to determine the positive and negative sample balance, the balance method may include two balance methods. For the first balance method, the exemplary embodiment of the present disclosure can determine the correlation between the positive and negative sample balance of the jth attribute label and the positive and negative sample attribute labels through the expression shown in Formula 5.

[0107]

[0108] Among them, y ij The values ​​of , M, N, i and j are as mentioned above and will not be repeated here. When the jth attribute label of the i-th target object is a positive sample attribute label, y ij =1; when the jth attribute label of the i-th target object is a negative sample attribute label, y ij =0.

[0109] It can be seen from the above formula 5 that formula 5 can use the positive and negative sample balance obtained by linear method formula 1 and then square it after -0.5. Figure 5 The third characteristic curve diagram of the positive and negative sample balance and the positive and negative sample attribute labels of the exemplary embodiment of the present disclosure is shown. The characteristic curve corresponding to the corresponding piecewise function in the dotted box in the figure is the characteristic curve of the third positive and negative sample balance and the positive and negative sample attribute labels of the exemplary embodiment of the present disclosure. Figure 5 As shown, for the jth attribute label of the target object, when the positive and negative sample attribute labels are evenly distributed, r j =0.5; when the number of positive sample attribute labels is N, the jth attribute label of the target object is all positive sample attribute labels. At this time, r j =1; when the number of positive sample attribute labels is 0, the jth attribute label of the target object is all negative sample attribute labels. At this time, r j =1.

[0110] from Figure 5 It can also be seen that r j along with As the value increases, r j The characteristic curve along r j =0.5 Symmetrical distribution on both sides, r j The closer to 0.5, the smoother the curve, indicating that the positive and negative samples are equally balanced in different target objects. The neural network model is targeted at this r j The positive and negative sample attention of attributes near 0.5 is similar, so there is no need to increase the weight of the corresponding attribute loss. j The closer to 0 or 1, the steeper the curve is. Figure 4 The steepness in is higher, indicating that the imbalance between positive and negative samples is higher. The neural network model is targeted at this r jThe positive and negative sample attention of attributes near 0 or 1 is quite different. At this time, the weight of the corresponding attribute loss can be increased to make the neural network model pay more attention to the impact of the distribution at both ends on the loss of the neural network model, so as to improve the neural network model's attention to the fewer positive sample attribute labels or positive sample attribute labels.

[0111] For the second balance method, the exemplary embodiment of the present disclosure can determine the correlation between the positive and negative sample balance degree of the j-th attribute label and the positive and negative sample attribute labels through the expression shown in Formula 6.

[0112]

[0113] Among them, y ij The values ​​of , M, N, i and j are as mentioned above and will not be repeated here. When the jth attribute label of the i-th target object is a positive sample attribute label, y ij =1; when the jth attribute label of the i-th target object is a negative sample attribute label, y ij =0.

[0114] It can be seen from the above formula 6 that formula 6 can use the square of the positive and negative sample balance obtained by the linear method formula 1. Figure 6 The fourth characteristic curve diagram of the positive and negative sample balance and the positive and negative sample attribute labels of the exemplary embodiment of the present disclosure is shown. The characteristic curve corresponding to the corresponding piecewise function in the dashed box in the figure is the fourth characteristic curve of the positive and negative sample balance and the positive and negative sample attribute labels of the exemplary embodiment of the present disclosure. Figure 6 As shown, for the jth attribute label of the target object, when the positive and negative sample attribute labels are evenly distributed, r j =0.5; when the number of positive sample attribute labels is N, the jth attribute label of the target object is all positive sample attribute labels. At this time, r j =1; when the number of positive sample attribute labels is 0, the jth attribute label of the target object is all negative sample attribute labels. At this time, r j =1.

[0115] from Figure 6 It can also be seen that r j along with As the value increases, r j The characteristic curve along r j =0.5 Symmetrical distribution on both sides, r j The closer to 0.5, the steeper the curve, r j The closer it is to 0 or 1, the smoother the curve is, which weakens the impact of outliers compared to the linear method.

[0116] In an optional manner, the multiple attribute prediction results of the exemplary embodiment of the present disclosure may include valid attribute prediction results, which match the corresponding existing attribute labels of the marked object, and the attribute loss is the existing attribute loss. However, considering that some target sample images have complete attribute labels, and some target sample images have missing attribute labels, in order to ensure that the attribute prediction model obtained after training the neural network model has universality, the multiple attribute prediction results of the exemplary embodiment of the present disclosure may include not only valid attribute prediction results but also invalid attribute prediction results, which match the missing attribute labels of the target object.

[0117] The above-mentioned valid attribute prediction results may have a clear output result, which may indicate that the target object has or does not have the attribute, while the invalid attribute prediction result has no clear output result or no output result. Based on this, the method of the exemplary embodiment of the present disclosure also includes: if there is an invalid attribute prediction result among the multiple attribute prediction results, blocking the influence of the missing attribute of the target object on the loss of the neural network model.

[0118] When the exemplary embodiment of the present disclosure blocks the influence of the missing attribute of the target object on the loss of the neural network model, the influence of the missing attribute on the loss of the neural network model can be controlled by gradient return. For example, the attribute loss corresponding to the missing attribute can be detected and directly set to 0, or the weight of the attribute loss corresponding to the missing attribute can be set to 0, both of which can achieve gradient return control.

[0119] In practical applications, there may be missing attribute labels in the training samples, that is, the attribute labels corresponding to multiple attribute prediction results of the target object included in the target sample image may all exist, or may all or partly be missing.

[0120] The multiple attribute prediction results of the exemplary embodiment of the present disclosure may include a valid attribute prediction result, the valid attribute prediction result matches the corresponding existing attribute label of the target object, and the attribute loss is the existing attribute loss. Here, after the exemplary embodiment of the present disclosure predicts the multiple attribute prediction results of the target object included in the target sample image based on the neural network model, each attribute prediction result may be compared with the corresponding existing attribute label of the target object respectively. If the attribute prediction result matches the corresponding existing attribute label of the target object, the attribute prediction result is a valid attribute prediction result. After determining that the attribute prediction result is a valid attribute prediction result, the method of the exemplary embodiment of the present disclosure may directly determine the corresponding existing attribute loss based on each valid attribute prediction result and the corresponding existing attribute label included in the target object.

[0121] In terms of the validity of the attribute prediction results, the multiple attribute prediction results of the exemplary embodiment of the present disclosure may also include invalid attribute prediction results, and the invalid attribute prediction results match the missing attribute label of the target object. It can be understood that when the exemplary embodiment of the present disclosure compares each attribute prediction result with the corresponding existing attribute label of the target object, if the attribute prediction result does not match the corresponding existing attribute label of the target object, that is, the target object has a missing attribute label, and the attribute prediction result matches the missing attribute label of the target object, then the attribute prediction result is an invalid attribute prediction result.

[0122] Based on this, the method of the exemplary embodiment of the present disclosure may further include: if there are invalid attribute prediction results among the multiple attribute prediction results, blocking the influence of the missing attribute loss of the target object on the loss of the neural network model.

[0123] If there are invalid attribute prediction results among multiple attribute prediction results, since the target object has a missing attribute label, the attribute label corresponding to the invalid attribute prediction result is missing, and the attribute loss corresponding to the invalid attribute prediction result cannot be determined. The learning value of the neural network model for the attribute corresponding to the invalid attribute prediction result is not great. At this time, the attribute loss corresponding to the invalid attribute prediction result can be regarded as the missing attribute loss. For example, the missing attribute loss can be reset to zero to block the gradient backpropagation, thereby blocking the influence of the missing attribute loss of the target object on the loss of the neural network model, so that the neural network model will not be affected by the missing attribute label during the training process.

[0124] At the same time, the exemplary embodiments of the present disclosure can also mark missing attribute tags. For example, [x 11 , x 12 , x 13 , x 14 , x 15 ] = [0, 1, 0, -100, -100] means that the first target object does not have the first attribute, has the second attribute, does not have the third attribute, the fourth attribute label is missing, and the fifth attribute label is missing. In each iteration, the influence of the missing attribute loss of the fourth and fifth attributes on the loss of the neural network model is blocked, and the effective attribute prediction results are guided to move in the direction of the corresponding existing attribute labels, thereby making the loss of the neural network model converge.

[0125] It can be seen that the exemplary embodiments of the present disclosure can block the influence of missing attribute loss on the loss of the neural network model when there are missing attribute labels in the target object, thereby improving the learning efficiency of the neural network model, shortening the training time, and saving training costs.

[0126] The exemplary embodiment of the present disclosure can predict multiple attributes at the same time. In terms of the category of the attribute prediction results, the multiple attribute prediction results of the exemplary embodiment of the present disclosure can include at least one of the clothing attribute prediction result, the posture attribute prediction result and the occlusion attribute prediction result. Here, the clothing attribute prediction result is the prediction result for the clothing attribute, the posture attribute prediction result is the prediction result for the posture attribute, and the occlusion attribute prediction result is the prediction result for the occlusion attribute.

[0127] The above-mentioned clothing attributes may include wearing a hat, a mask, glasses, top type (short sleeves, long sleeves, etc.), bottom type (shorts, trousers, skirts, etc.), shoe type (slippers, sandals, sneakers, others, etc.), backpack, etc., but are not limited to these.

[0128] The above-mentioned posture attributes may include standing, sitting, standing and leaning forward, squatting, walking, raising legs, lying on the ground, sitting on the ground, kneeling on the ground, stretching hands, raising hands, etc., but are not limited to these. Among them, the posture attributes can be determined by the attributes of the limbs. The attributes of the limbs include: the state of the left upper arm, the state of the left forearm, the state of the right upper arm, the state of the right forearm, the state of the upper body, the state of the left leg, the state of the right leg, the state of the head, etc., but are not limited to these.

[0129] The above-mentioned occlusion attribute may be that a certain clothing attribute or posture attribute is occluded at the position of the target object, including upper body occlusion and lower body occlusion.

[0130] If a certain clothing attribute or posture attribute is blocked at the position of the target object, the prediction result for the clothing attribute or posture attribute is not accurate. Therefore, when determining the loss of the neural network model, the corresponding attribute loss of the clothing attribute or posture attribute can be ignored, thereby improving the learning efficiency of the neural network model, shortening the training time, and saving training costs.

[0131] The exemplary embodiments of the present disclosure also provide an early warning method, which can be executed by an electronic device or by a chip applied to the electronic device. Here, the type of the electronic device can be found in the above text and will not be described in detail here.

[0132] Figure 7 FIG. 1 is a flow chart showing an early warning method according to an exemplary embodiment of the present disclosure. Figure 7 As shown, the early warning method of the exemplary embodiment of the present disclosure may include:

[0133] Step 701: inputting an image to be detected containing an object to be detected into an attribute prediction model to obtain multiple attribute detection results of the object to be detected.

[0134] Here, the above-mentioned attribute prediction model is the attribute prediction model of the exemplary embodiment of the present disclosure. The relevant content of the above-mentioned image to be detected can refer to the relevant content of the target sample image in the above text, and the relevant content of the above-mentioned object to be detected can refer to the relevant content of the target object in the above text, which will not be repeated here.

[0135] Step 702: Determine the detection result of the object to be detected based on multiple attribute detection results. The above detection result can be the detection result of the object to be detected that is finally determined based on multiple attribute detection results, and then the detection result of the object to be detected is exported, and step 703 is performed based on other neural network models or other methods.

[0136] Step 703: If the detection result of the object to be detected meets the warning condition, trigger a warning for the object to be detected. Here, the specific way of performing this step is not specifically limited.

[0137] The above-mentioned warning conditions can be pre-set conditions for triggering warnings, which are determined according to actual application scenarios and are not specifically limited here. After determining the detection result of the object to be detected based on multiple attribute detection results, it can be determined whether the detection result of the object to be detected meets the warning conditions. If the detection result of the object to be detected meets the warning conditions, it is necessary to trigger a warning for the object to be detected in a timely manner.

[0138] It can be seen that the method of the exemplary embodiment of the present disclosure can use the attribute prediction model trained by the exemplary embodiment of the present disclosure to obtain multiple attribute detection results of the object to be detected included in the image to be detected, determine the detection result of the object to be detected based on the multiple attribute detection results, and trigger an early warning for the object to be detected if the detection result of the object to be detected meets the early warning condition. The attribute prediction model can not only use the positive and negative sample balance of the attribute label in the training stage to balance the impact of the corresponding attribute loss determined based on the attribute label on the loss of the neural network model, thereby improving the learning efficiency of the neural network model, but also improve the attribute prediction accuracy of the attribute prediction model in the reasoning stage, thereby improving the accuracy of early warnings and reducing false alarms.

[0139] The above-mentioned multiple attribute detection results may include at least one of a dress attribute detection result, a posture attribute detection result, and an occlusion attribute detection result. The above-mentioned dress attribute detection result, posture attribute detection result, and occlusion attribute detection result refer to the dress attribute prediction result, posture attribute prediction result, and occlusion attribute prediction result in the above text, and will not be repeated here.

[0140] When multiple attribute detection results include at least one of a clothing attribute detection result and a posture attribute detection result, as well as an occlusion attribute detection result, the method of the exemplary embodiment of the present disclosure may further include: when the detection result of the object to be detected meets the warning condition, determining whether the multiple attribute detection results include the occlusion attribute detection result; if the multiple attribute detection results do not include the occlusion attribute detection result, triggering a warning for the object to be detected.

[0141] If the multiple attribute detection results of the target object do not include the occlusion attribute detection result, it means that the detection result of the object to be detected is not affected by the occlusion attribute detection result. At this time, an early warning can be triggered for the object to be detected directly when the detection result of the object to be detected meets the early warning condition.

[0142] Based on this, the method of the exemplary embodiment of the present disclosure may further include: if the multiple attribute detection results include an occlusion attribute detection result, determining that the object to be detected is in an occluded state based on the occlusion attribute detection result, and updating the image to be detected.

[0143] If multiple attribute detection results of the target object include occlusion attribute detection results, there is an occluded part of the target object. If there is an attribute whose position on the target object is the same as the occluded part, the attribute detection result corresponding to the attribute is not accurate. At this time, in order to reduce false alarms, there is no need to trigger an early warning for the object to be detected, and the image to be detected is updated to the next image to be detected, and attribute prediction and early warning detection continue.

[0144] One or more technical solutions provided in the exemplary embodiments of the present disclosure can predict multiple attribute prediction results of the target object included in the target sample image based on the neural network model, determine the corresponding attribute loss based on each attribute prediction result and the corresponding attribute label included in the target object, and then weight each attribute loss to obtain the loss of the neural network model, and the weight of each attribute loss is associated with the positive and negative sample balance of the corresponding attribute label, so that when determining the loss of the neural network model, the influence of the positive and negative sample balance of the attribute label corresponding to each attribute prediction result on the corresponding attribute loss can be considered, reducing the loss difference caused by the imbalance of positive and negative samples of the attribute label, improving the learning efficiency of the neural network model, and thus increasing the attribute prediction accuracy of the attribute prediction model finally obtained in the reasoning stage.

[0145] The above mainly introduces the scheme provided by the embodiment of the present disclosure. It is understandable that in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed in this article, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0146] The disclosed embodiment can divide the electronic device into functional units according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the disclosed embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0147] In the case of dividing each functional module according to each function, an exemplary embodiment of the present disclosure provides a training device for an attribute prediction model, and the training device for the attribute prediction model may be an electronic device or a chip applied to the electronic device. Figure 8 FIG. 1 is a schematic block diagram of a module of a training device for an attribute prediction model according to an exemplary embodiment of the present disclosure. Figure 8 As shown, the device 800 includes:

[0148] Prediction module 801, used for predicting multiple attribute prediction results of a target object included in a target sample image based on a neural network model;

[0149] A determination module 802 is used to determine a corresponding attribute loss based on each of the attribute prediction results and corresponding attribute labels included in the same target object;

[0150] The determination module 802 is further used to weight each of the attribute losses to obtain the loss of the neural network model, and the weight of each of the attribute losses is associated with the positive and negative sample balance of the corresponding attribute label;

[0151] The training module 803 is used to update the model parameters of the neural network model based on each of the attribute prediction results and the corresponding attribute labels included in the target object if the loss of the neural network model does not meet the convergence condition; otherwise, determine that the neural network model is an attribute prediction model.

[0152] As a possible implementation manner, the determination module 802 is further configured to determine the weight of the corresponding attribute loss based on the positive and negative sample balance of each attribute prediction result and the corresponding attribute label.

[0153] As a possible implementation manner, the balance degree of positive and negative samples of the attribute label is positively correlated with the occurrence probability of the positive sample attribute label in the corresponding attribute label in different target objects;

[0154] If the attribute prediction result matches the positive sample of the corresponding attribute, the weight of the attribute loss is negatively correlated with the balance of positive and negative samples of the attribute label;

[0155] If the attribute prediction result matches the negative sample of the corresponding attribute, the weight of the attribute loss is positively correlated with the balance of positive and negative samples of the attribute label.

[0156] As a possible implementation manner, in the same attribute label, the probability of occurrence of the positive sample attribute label in different target objects is greater than the probability of occurrence of the negative sample attribute label in different target objects, and the positive and negative sample balance is greater than a preset balance;

[0157] In the same attribute label, the occurrence probability of the positive sample attribute label in different target objects is less than the occurrence probability of the negative sample attribute label in different target objects, and the positive and negative sample balance is less than the preset balance.

[0158] As a possible implementation manner, the multiple attribute prediction results also include valid attribute prediction results, and the valid attribute prediction results match the existing attributes of the target object.

[0159] As a possible implementation method, the multiple attribute prediction results also include invalid attribute prediction results, and the invalid attribute prediction results match the missing attributes of the target object. The determination module 802 is also used to block the influence of the missing attributes of the target object on the loss of the neural network model if the invalid attribute prediction results exist in the multiple attribute prediction results.

[0160] As a possible implementation manner, the multiple attribute prediction results include at least one of a clothing attribute prediction result, a posture attribute prediction result, and an occlusion attribute prediction result.

[0161] As a possible implementation method, the neural network model includes: a target detection unit and an attribute recognition unit, and the prediction module 801 is also used to detect the target sample image based on the target detection unit to obtain the target object; and predict multiple attribute prediction results of the target object based on the attribute recognition unit.

[0162] The exemplary embodiments of the present disclosure further provide an early warning device, which may be an electronic device or a chip applied to an electronic device. Fig. 9 FIG. 1 shows a schematic block diagram of a module of an early warning device according to an exemplary embodiment of the present disclosure. Fig. 9 As shown, the device 900 includes:

[0163] A detection module 901, used for inputting an image to be detected containing an object to be detected into an attribute prediction model to obtain multiple attribute detection results of the object to be detected, wherein the attribute prediction model is the attribute prediction model trained according to claim 13;

[0164] A determination module 902 is used to determine the detection result of the object to be detected based on the multiple attribute detection results;

[0165] The determination module 902 is further configured to trigger an early warning for the object to be detected if the detection result of the object to be detected meets an early warning condition.

[0166] As a possible implementation manner, the multiple attribute detection results include at least one of a clothing attribute detection result, a posture attribute detection result, and an occlusion attribute detection result.

[0167] As a possible implementation method, the determination module 902 is also used to determine whether the multiple attribute detection results include occlusion attribute detection results when the detection result of the object to be detected meets the warning condition; if the multiple attribute detection results do not include occlusion attribute detection results, trigger a warning for the object to be detected.

[0168] As a possible implementation manner, the determination module 902 is further configured to, if the multiple attribute detection results include an occlusion attribute detection result, determine based on the occlusion attribute detection result that the object to be detected is in an occluded state, and update the image to be detected.

[0169] Fig.10 Schematic block diagram of a chip of an exemplary embodiment of the present disclosure is shown. Fig.10 As shown, the chip 1000 includes one or more (including two) processors 1001 and a communication interface 1002. The communication interface 1002 can support the server to perform the data sending and receiving steps in the above method, and the processor 1001 can support the server to perform the data processing steps in the above method.

[0170] Optional, such as Fig.10As shown, the chip 1000 also includes a memory 1003, which may include a read-only memory and a random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory (NVRAM).

[0171] In some embodiments, Fig.10 As shown, the processor 1001 performs corresponding operations by calling the operation instructions stored in the memory (the operation instructions may be stored in the operating system). The processor 1001 controls the processing operations of any one of the terminal devices, and the processor may also be called a central processing unit (CPU). The memory 1003 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1001. A portion of the memory 1003 may also include NVRAM. For example, in an application, the memory, the communication interface, and the memory are coupled together through a bus system, wherein the bus system may include a power bus, a control bus, and a status signal bus in addition to a data bus. However, for the sake of clarity, in Fig.10 Various buses are labeled as bus system 1004 .

[0172] The method disclosed in the above-mentioned embodiment of the present disclosure can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above-mentioned processor may be a general-purpose processor, a digital signal processor (digital signal processing, DSP), an ASIC, a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiment of the present disclosure can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware.

[0173] The exemplary embodiment of the present disclosure also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to cause the electronic device to perform the method according to the embodiment of the present disclosure when executed by the at least one processor.

[0174] The exemplary embodiments of the present disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform the method according to the embodiments of the present disclosure.

[0175] The exemplary embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, the computer is used to enable the computer to perform the method according to the embodiments of the present disclosure.

[0176] refer to Fig.11 , a block diagram of an electronic device 1100 that can be used as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0177] like Fig.11 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0178] Multiple components in the electronic device 1100 are connected to the I / O interface 1105, including: an input unit 1106, an output unit 1107, a storage unit 1108, and a communication unit 1109. The input unit 1106 can be any type of device that can input information to the electronic device 1100, and the input unit 1106 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 1107 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1108 can include but is not limited to a disk, an optical disk. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0179] like Fig.11 As shown, the computing unit 1101 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above. For example, in some embodiments, the method of the exemplary embodiments of the present disclosure may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. In some embodiments, the computing unit 1101 can be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0180] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0181] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0182] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0183] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0184] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0185] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0186] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present disclosure is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a terminal, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0187] Although the present disclosure has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present disclosure. Accordingly, this specification and the drawings are merely exemplary illustrations of the present disclosure as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present disclosure. Obviously, those skilled in the art may make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for training an attribute prediction model, characterized in that: include: Predicting multiple attribute prediction results of a target object included in a target sample image based on a neural network model; Determine the corresponding attribute loss based on the attribute prediction results and corresponding attribute labels included in the same target object; Weighting each of the attribute losses to obtain the loss of the neural network model, wherein the weight of each of the attribute losses is associated with the balance degree of positive and negative samples of the corresponding attribute label; If the loss of the neural network model does not meet the convergence condition, the model parameters of the neural network model are updated based on each of the attribute prediction results and the corresponding attribute labels included in the target object; otherwise, the neural network model is determined to be an attribute prediction model.

2. The method according to claim 1, characterized in that: The method further comprises: The weight of the corresponding attribute loss is determined based on the positive and negative sample balance of each attribute prediction result and the corresponding attribute label.

3. The method according to claim 1, characterized in that The balance degree of positive and negative samples of the attribute label is positively correlated with the occurrence probability of the positive sample attribute label in the corresponding attribute label in different target objects; If the attribute prediction result matches the positive sample of the corresponding attribute, the weight of the attribute loss is negatively correlated with the balance of positive and negative samples of the attribute label; If the attribute prediction result matches the negative sample of the corresponding attribute, the weight of the attribute loss is positively correlated with the balance of positive and negative samples of the attribute label.

4. The method according to claim 1, characterized in that In the same attribute label, the probability of occurrence of the positive sample attribute label in different target objects is greater than the probability of occurrence of the negative sample attribute label in different target objects, and the positive and negative sample balance is greater than the preset balance; In the same attribute label, the occurrence probability of the positive sample attribute label in different target objects is less than the occurrence probability of the negative sample attribute label in different target objects, and the positive and negative sample balance is less than the preset balance.

5. The method according to any one of claims 1 to 4, characterized in that: The plurality of attribute prediction results also include valid attribute prediction results, and the valid attribute prediction results match existing attributes of the target object.

6. The method according to claim 5, characterized in that The plurality of attribute prediction results further include an invalid attribute prediction result, the invalid attribute prediction result matches a missing attribute of the target object, and the method further includes: If the invalid attribute prediction result exists among the multiple attribute prediction results, the influence of the missing attribute of the target object on the loss of the neural network model is blocked.

7. The method according to any one of claims 1 to 4, characterized in that: The plurality of attribute prediction results include at least one of a clothing attribute prediction result, a posture attribute prediction result, and an occlusion attribute prediction result.

8. The method according to claim 7, characterized in that The neural network model includes: a target detection unit and an attribute recognition unit. The prediction results of multiple attributes of the target object included in the target sample image based on the neural network model include: Detecting the target sample image based on the target detection unit to obtain a target object; A plurality of attribute prediction results of the target object are predicted based on the attribute recognition unit.

9. An early warning method, characterized in that: include: Inputting an image to be detected containing an object to be detected into an attribute prediction model to obtain multiple attribute detection results of the object to be detected, wherein the attribute prediction model is the attribute prediction model according to any one of claims 1 to 8; Determine the detection result of the object to be detected based on the multiple attribute detection results; If the detection result of the object to be detected meets the warning condition, a warning is triggered for the object to be detected.

10. The method according to claim 9, characterized in that The plurality of attribute detection results include at least one of a clothing attribute detection result, a posture attribute detection result, and an occlusion attribute detection result.

11. The method according to claim 9, characterized in that The method further comprises: When the detection result of the object to be detected meets the warning condition, determining whether the multiple attribute detection results include an occlusion attribute detection result; If the multiple attribute detection results do not include an occlusion attribute detection result, an early warning is triggered for the object to be detected.

12. The method according to claim 11, characterized in that The method further comprises: If the multiple attribute detection results include an occlusion attribute detection result, it is determined based on the occlusion attribute detection result that the object to be detected is in an occluded state, and the image to be detected is updated.

13. A training device for an attribute prediction model, characterized in that: include: A prediction module, used for predicting multiple attribute prediction results of a target object included in the target sample image based on a neural network model; A determination module, configured to determine a corresponding attribute loss based on each of the attribute prediction results and corresponding attribute labels included in the same target object; The determination module is further used to weight each of the attribute losses to obtain the loss of the neural network model, and the weight of each of the attribute losses is associated with the positive and negative sample balance of the corresponding attribute label; A training module is used to update the model parameters of the neural network model based on each of the attribute prediction results and the corresponding attribute labels included in the target object if the loss of the neural network model does not meet the convergence conditions; otherwise, determine that the neural network model is an attribute prediction model.

14. An early warning device, characterized in that: include: A detection module, used for inputting an image to be detected containing an object to be detected into an attribute prediction model to obtain multiple attribute detection results of the object to be detected, wherein the attribute prediction model is the attribute prediction model trained according to claim 13; A determination module, used to determine the detection result of the object to be detected based on the multiple attribute detection results; The determination module is further configured to trigger an early warning for the object to be detected if the detection result of the object to be detected meets an early warning condition.

15. An electronic device, characterized in that: include: processor; as well as, A memory for storing programs; The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 12.

16. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 12.

Images