A training method for a network model and an image processing method

Abstract

This application provides a network model training method and an image processing method to improve the model's stability against environmental interference. The network model training method includes: inputting N images from a first training sample set into the network model to be trained; performing facial attribute recognition on the N images using the network model to obtain N predicted age values ​​and N predicted gender values; determining a first loss value and a first standard deviation loss value based on the N predicted age values ​​and age label values, and determining a first total loss value based on the first loss value and the first standard deviation loss value; determining a second loss value and a second standard deviation loss value based on the N predicted gender values ​​and gender label values, and determining a second total loss value based on the second loss value and the second standard deviation loss value; and adjusting the network parameters of the network model to be trained based on the first total loss value and the second total loss value to obtain a trained network model.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method for training a network model and an image processing method. Background Technology

[0002] Facial attributes include age and gender, and deep learning inference based on facial images can yield good results. In training attribute recognition models, manual annotations, such as age information, are often used as supervisory signals. However, the reliability of manual annotations is limited. Differences in perception among different annotators and interference from the image itself can lead to significant deviations between the labeled information and the actual values. Furthermore, in non-cooperative scenarios, images are affected by natural environmental factors (lighting, occlusion) and the individual's own features (angle, occlusion) (hereinafter collectively referred to as environmental interference), resulting in unclear faces and thus significantly inaccurate predictions. During training, environmental interference in the image can lead to substantial loss values, thereby deviating the entire model's training direction. Summary of the Invention

[0003] This application provides a network model training method, namely an image processing method, to improve the model's stability against environmental disturbances.

[0004] In a first aspect, embodiments of this application provide a network model training method, including:

[0005] The first training sample set includes N images, which are then input into the network model to be trained. The network model performs facial attribute recognition on the N images to obtain N predicted age values ​​and N predicted gender values. The N images in the first training sample set have the same age label value and gender label value. A first loss value and a first standard deviation loss value are determined based on the N predicted age values ​​and the age label values, and a first total loss value is determined based on the first loss value and the first standard deviation loss value. A second loss value and a second standard deviation loss value are determined based on the N predicted gender values ​​and the gender label values, and a second total loss value is determined based on the second loss value and the second standard deviation loss value. The network parameters of the network model to be trained are adjusted based on the first total loss value and the second total loss value to obtain a trained network model.

[0006] Based on the above scheme, this application introduces a standard deviation loss value when calculating the loss value. The network parameters are adjusted by using the weighted sum of the loss value and the standard deviation loss value as the total loss value. This can narrow the fluctuation difference of the predicted value under different environmental disturbances, thereby improving the model's ability to resist environmental disturbances.

[0007] In one possible implementation, determining the first standard deviation loss value based on the N predicted age values ​​and the age label value includes:

[0008] A first standard deviation is determined based on the N predicted age values ​​and the age label values;

[0009] When the first standard deviation is greater than a first set standard deviation threshold, the difference between the first standard deviation and the first set standard deviation threshold is taken as the first standard deviation loss value; or,

[0010] When the first standard deviation is less than or equal to the first standard deviation threshold, the first set value is used as the first standard deviation loss value;

[0011] The second standard deviation loss value is determined based on the N predicted gender values ​​and the gender label value, including:

[0012] The second standard deviation is determined based on the N predicted gender values ​​and the gender label values;

[0013] When the second standard deviation is greater than the second set standard deviation threshold, the difference between the second standard deviation and the second set standard deviation threshold is taken as the second standard deviation loss value; or,

[0014] When the second standard deviation is less than or equal to the second standard deviation threshold, the second set value is used as the second standard deviation loss value.

[0015] Based on the above scheme, by setting the standard deviation loss value, environmental interference can be controlled in a targeted manner, and by setting the standard deviation threshold, overfitting can be prevented when the model fits infinitely to zero.

[0016] In one possible implementation, the first training sample set is obtained in the following way:

[0017] Obtain M images, along with the age value, age confidence score, gender value, gender confidence score, and quality score for each of the M images. The M images are images of the same person in different scenes within a set time period.

[0018] N images that meet the set conditions are selected from the M images, and a first training sample set is generated based on the N images; the set conditions include: the age confidence of the image is greater than or equal to a first threshold, the gender confidence of the image is greater than or equal to a second threshold, and the quality score of the image is greater than or equal to a third threshold.

[0019] Based on the age and gender values ​​of the N images, determine the age label and gender label values ​​of the N images in the first training sample set.

[0020] In one possible implementation, determining the age and gender labels of the N images in the first training sample set based on the age and gender values ​​of the N images includes:

[0021] From the N age values ​​corresponding to the N images, select multiple age values ​​that fall within a set age range;

[0022] The age value that appears most frequently among the multiple age values ​​is used as the age label value of N images in the first training sample set;

[0023] The gender value that appears most frequently in the N images is used as the gender label value for the N images in the first training sample set.

[0024] Based on the above scheme, multiple quality controls are used to ensure the reliability of the label values ​​in the first training sample set, thereby preventing training bias.

[0025] In one possible implementation, the set age range is determined in the following way:

[0026] Calculate the mean and standard deviation of the N age values;

[0027] The set age range is determined based on the mean age and the standard deviation.

[0028] In one possible implementation, the N images include L first images with quality scores less than a first quality threshold. Determining the first loss value based on the N predicted age values ​​and the age label value includes: for each first image, when the difference between the predicted age value of the first image output by the network model to be trained and the age label value of the first image is greater than a first set deviation value, determining a first predicted age value of the first image based on the age label value of the first image, the predicted age value, and the first set deviation value; or, when the difference between the predicted age value of the first image and the age label value of the first image is less than or equal to the first set deviation value, using the predicted age value of the first image as the first predicted age value; and determining the first loss value based on the first predicted age values ​​of the L first images, the predicted age values ​​of the N images, and the age label value.

[0029] In one possible implementation, the first standard deviation is determined as follows:

[0030] The N predicted age values ​​determined by the network model to be trained are sorted and the median is taken as the first value. For each first image, when the difference between the predicted age value of the first image and the first value is greater than the second set deviation value, a second value is determined according to the first value, the age label value, and the second set deviation value; or, when the difference between the predicted age value of the first image and the first value is less than or equal to the second set deviation value, the predicted age value of the first image is taken as the second value. The first standard deviation is determined according to the N predicted age values, L second values, and the age label value.

[0031] With the above approach, when a low-quality image is input into the network model, if it is determined that the predicted value is too far from the label value, the predicted value can be modified according to the set deviation value and the label value. Then, the loss calculation can be determined by the modified predicted value, which can reduce the noise bias caused by the bright sample of the first bead.

[0032] Secondly, this application provides an image processing method, comprising: acquiring an image to be processed; inputting the image to be processed into a network model, and performing facial attribute recognition on the image to be processed through the network model to obtain the age value and gender value corresponding to the face in the image to be processed, wherein the network model is obtained after training by the training method of the network model described in the first aspect and different implementations of the first aspect.

[0033] Thirdly, embodiments of this application provide a network model training apparatus, comprising:

[0034] The input module is used to input the N images included in the first training sample set into the network model to be trained, and to perform facial attribute recognition on the N images through the network model to be trained to obtain N predicted age values ​​and N predicted gender values; the N images in the first training sample set have the same age label value and gender label value.

[0035] The determination module is used to determine a first loss value and a first standard deviation loss value based on the N predicted age values ​​and the age label values, and to determine a first total loss value based on the first loss value and the first standard deviation loss value.

[0036] A second loss value and a second standard deviation loss value are determined based on the N predicted gender values ​​and the gender label value, and a second total loss value is determined based on the second loss value and the second standard deviation loss value.

[0037] The adjustment module is used to adjust the network parameters of the network model to be trained according to the first total loss value and the second total loss value, so as to obtain the trained network model.

[0038] Fourthly, embodiments of this application provide an execution device, including:

[0039] Memory, used to store program instructions;

[0040] A processor is configured to acquire program instructions stored in the memory and execute the methods described in the first aspect, the second aspect, and different implementations of the first aspect according to the acquired program instructions.

[0041] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect, the second aspect, and different implementations of the first aspect of the claim.

[0042] The technical effects of any of the implementation methods in the second to fourth aspects can be found in the first aspect and the technical effects of different implementation methods of the first aspect, which will not be repeated here. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1A This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0045] Figure 1B A schematic diagram of a server structure provided in an embodiment of this application;

[0046] Figure 2 A flowchart illustrating a network model training method provided in an embodiment of this application;

[0047] Figure 3 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0048] Figure 4 A schematic diagram of a network model training device provided in an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of an execution device provided in an embodiment of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0051] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0052] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0053] Facial attributes include age and gender, and deep learning inference based on facial images can yield good results. In training attribute recognition models, manual annotations, such as age information, are often used as supervisory signals. However, the reliability of manual annotations is limited. Differences in perception among different annotators and interference from the image itself can lead to significant deviations between the labeled information and the actual values. Furthermore, in non-cooperative scenarios, images are affected by natural environmental factors (lighting, occlusion) and the individual's own features (angle, occlusion) (hereinafter collectively referred to as environmental interference), resulting in unclear faces and thus significantly inaccurate predictions. During training, environmental interference in the image can lead to substantial loss values, thereby deviating the entire model's training direction.

[0054] To address the aforementioned issues, this application provides a training method for a network model and an image processing method. During the training process, a standard deviation loss value is introduced. By fusing the basic loss value and the standard deviation loss value, the fluctuation difference of the predicted value under different environmental disturbances is directly reduced, thereby improving the stability of the model to environmental disturbances.

[0055] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0056] The image processing method provided in this application can be implemented by an execution device. In some embodiments, the execution device may be an electronic device, which may be implemented by one or more servers. Figure 1A Let's take a server with 100 servers as an example. (Reference) Figure 1A The diagram illustrates a possible application scenario provided by an embodiment of this application, including a server 100 and a data acquisition device 200. The server 100 can be implemented as a physical server or a virtual server. The server can be implemented as a single server or as a server cluster consisting of multiple servers; the image processing method provided in this application can be implemented using either a single server or a server cluster. The data acquisition device 200 is a device with image acquisition capabilities, including electronic police equipment, electronic monitoring equipment, surveillance cameras, video recorders, and terminal devices with video acquisition capabilities (such as laptops, computers, mobile phones, and televisions). The data acquisition device 200 can send the acquired images to be processed to the server 100 via a network. Optionally, the server 100 can be connected to a terminal device 300, receiving image processing tasks sent by the terminal device 300 and performing image processing based on the received images to be processed sent by the data acquisition device 200. In some scenarios, the server 100 can send the image processing results to the terminal device 300. The terminal device 300 can be a television, mobile phone, tablet computer, personal computer, etc. In some embodiments, after the acquisition device 200 acquires an image, it can send the acquired image to a server. The server then performs image analysis and portrait aggregation on the image to obtain multiple images and save them as image archives. Each image archive includes facial images of the same person in different scenes within a set time period. In some scenarios, the image archive also includes facial attributes, confidence scores, and quality scores of the images. The server 100 can train the network model using multiple image archives to implement the network model training method.

[0057] As an example, see Figure 1B As shown, server 100 may include processor 110, communication interface 120, and memory 130. Of course, server 100 may also include other components. Figure 1B Not shown in the image.

[0058] The communication interface 120 is used to communicate with the acquisition device 200 and the terminal device 300, to receive images to be processed sent by the acquisition device 200, or to receive image processing tasks sent by the terminal device 300, or to send image processing results to the terminal device 300.

[0059] In the embodiments of this application, the processor 110 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0060] Processor 110 is the control center of server 100, connecting various parts of server 100 through various interfaces and routes. It executes various functions and processes data by running or executing software programs and / or modules stored in memory 130, and by calling data stored in memory 130. Optionally, processor 110 may include one or more processing units. Processor 110 may be, for example, a processor, microprocessor, controller, or other control component. It may be a general-purpose central processing unit (CPU), a general-purpose processor, a digital signal processing unit (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0061] The memory 130 can be used to store software programs and modules. The processor 110 executes various functional applications and data processing by running the software programs and modules stored in the memory 130. The memory 130 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function, etc.; the data storage area may store data created according to business processing, etc. As a non-volatile computer-readable storage medium, the memory 130 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory 130 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. The memory 130 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. The memory 130 in this embodiment can also be a circuit or any other means capable of implementing storage functions for storing program instructions and / or data.

[0062] In other embodiments, the execution device may be a terminal device. In some scenarios, the terminal device can receive the image to be processed sent by the acquisition device, and perform image processing on the image to obtain the age and gender values ​​of the image. The terminal device may include a display device, which may be a liquid crystal display, an organic light-emitting diode (OLED) display, a projection display device, etc., and this application does not specifically limit it.

[0063] It should be noted that the above Figure 1A and Figure 1B The structure shown is merely an example, and the embodiments of this application are not limited thereto.

[0064] This application provides a method for training a network model. Figure 2 An exemplary flowchart of a method for training a network model is shown, which can be executed by a network model training device, such as... Figure 1BThe server 100 shown could be either processor 110 or server 100 itself. The specific process is as follows:

[0065] 201. Input the N images included in the first training sample set into the network model to be trained. The network model to be trained performs facial attribute recognition on the N images to obtain N predicted age values ​​and N predicted gender values.

[0066] In some embodiments, the first training sample set can be obtained as follows: M images are acquired, along with the age value, age confidence score, gender value, gender confidence score, and quality score corresponding to each of the M images. The M images represent the same person in different scenarios within a set time period, and these M images can form an image archive. In some scenarios, the image archive can be obtained as follows: City-level snapshot data within a set time period is acquired. This snapshot data has a short time span to ensure that facial attributes remain unchanged within the set time period. For example, the set time period can be 1 month, 3 months, etc. Preferably, to ensure the uniformity of facial attributes in the snapshot data, the set time period should not exceed six months. Facial attributes, facial quality, and a facial recognition model are analyzed to obtain attribute values ​​(age, gender), attribute confidence scores, quality scores, and facial recognition features. Further, based on the association information between the face and head / shoulder, body, and gait at the time of snapshot, a corresponding type of analysis algorithm is used to extract head / shoulder features, body features, and gait features from the detected person. Simultaneously, the time information and snapshot location information of the snapshot data are recorded.

[0067] After obtaining the above information, facial clustering can be performed based on features such as face, head and shoulders, body shape, and gait, as well as spatiotemporal information. This allows a large number of low-quality facial images to be retrieved into the same image archive. As a result, multiple image archives can be obtained, each containing facial images of the same person in different scenes within a set time period. Clustering processing based on massive city-level snapshot data ensures the richness of image data within the image archives, far exceeding the scale achievable through manual annotation.

[0068] In some embodiments, the image archive may include M images, from which N images that meet certain conditions can be selected, and a first training sample set can be generated based on the N images. The conditions include: the age confidence level of the images is greater than or equal to a first threshold, the gender confidence level of the images is greater than or equal to a second threshold, and the quality score of the images is greater than or equal to a third threshold.

[0069] As an example, confidence and quality filtering can be applied to multiple images within an image archive. For instance, with an age confidence threshold as the first threshold and a gender confidence threshold as the second threshold, images with very low confidence can be excluded based on these thresholds. For example, assuming the image archive originally contains M images, for the age attribute, the archive can be filtered based on the first threshold. If 3 out of the M images have an age confidence score less than the first threshold, then M-3 images with an age confidence score greater than or equal to the first threshold are retained. For the gender attribute, the archive can be filtered based on the second threshold. If 1 out of the M-3 images has an age confidence score less than the second threshold, then M-4 images with an age confidence score greater than or equal to the second threshold are retained. Furthermore, images can be filtered based on their quality scores, excluding images with quality scores less than a third threshold, resulting in N images with quality scores greater than or equal to the third threshold. For example, if 2 out of the M-4 images have quality scores less than the third threshold, then M-6 images are used as the first training sample set.

[0070] In some scenarios, an image quantity threshold can be set, and image files whose number of images after confidence and quality filtering exceeds the image quantity threshold can be used as training sample sets.

[0071] In some embodiments, the first training sample set is any one of multiple training sample sets. After obtaining the first training sample set, the age label value and gender label value of the N images in the first training sample set can be determined based on the age and gender values ​​of the N images. The N images in the first training sample set have the same age label value and gender label value. In some scenarios, the age label value and gender label value of the N images in the first training sample set can be determined based on the age and gender values ​​of the N images. Specifically, this can be done as follows: Select multiple age values ​​within a set age range from the N age values ​​corresponding to the N images. Then, use the age value with the highest frequency among the multiple age values ​​as the age label value of the N images in the first training sample set. Similarly, the gender value with the highest frequency among the N images can be used as the gender label value of the N images in the first training sample set.

[0072] In some scenarios, an age range can be determined as follows: calculate the mean and standard deviation of N age values; then determine the age range based on the mean and standard deviation. For example, the mean age can be represented by `mean`, and the standard deviation by `std`, thus the age range can be expressed as `mean ± std × 3`.

[0073] 202. Determine the first loss value and the first standard deviation loss value based on N predicted age values ​​and age label values, and determine the first total loss value based on the first loss value and the first standard deviation loss value.

[0074] In some embodiments, multiple loss values ​​can be determined using existing loss value calculation methods, and a first loss value can be determined based on these multiple loss values. In some scenarios, the first loss value can be the sum of N loss values. In other scenarios, the first loss value can be the average of the N loss values.

[0075] In some embodiments, a first standard deviation loss value is determined based on N predicted age values ​​and age label values. Specifically, this can be determined as follows: A first standard deviation is determined based on the N predicted age values ​​and age label values; when the first standard deviation is greater than a first set standard deviation threshold, the difference between the first standard deviation and the first set standard deviation threshold is used as the first standard deviation loss value. When the first standard deviation is less than or equal to the first standard deviation threshold, a first set value is used as the first standard deviation loss value. The first set value can be 0. The first standard deviation satisfies the condition described in the following formula:

[0076] Loss Std =max(0,Std-L);

[0077] Among them, Loss Std Std represents the first standard deviation loss value, L represents the first standard deviation, and L is the first set standard deviation threshold.

[0078] In some embodiments, the weighted sum of the first loss value and the first standard deviation loss value can be used as the first total loss value, or the average of the first loss value and the first standard deviation loss value can be used as the first total loss value. In some scenarios, the first total loss value satisfies the condition shown in the following formula:

[0079] Loss = Loss Basic +β·Loss Std ;

[0080] Where Loss represents the first total loss value, Loss Basic Let β represent the first loss value, and let β represent the hyperparameter.

[0081] 203. Determine the second loss value and the second standard deviation loss value based on N predicted gender values ​​and gender label values, and determine the second total loss value based on the second loss value and the second standard deviation loss value.

[0082] In some embodiments, the second standard deviation loss value is determined based on N predicted gender values ​​and gender label values. This can be achieved by: determining a second standard deviation based on the N predicted gender values ​​and gender label values; when the second standard deviation is greater than a second set standard deviation threshold, the difference between the second standard deviation and the second set standard deviation threshold is used as the second standard deviation loss value. When the second standard deviation is less than or equal to the second standard deviation threshold, a second set value is used as the second standard deviation loss value. The second set value can be 0.

[0083] In some embodiments, the method for determining the second total loss value is the same as the method for determining the first loss value, and will not be illustrated here.

[0084] 204. Adjust the network parameters of the network model to be trained based on the first total loss value and the second total loss value to obtain the trained network model.

[0085] Based on the above scheme, by fusing the basic loss value and the standard deviation loss value, the basic loss value ensures that the basic performance of the model does not degrade, while the standard deviation loss value can bring the distribution of predicted data under different environmental disturbances closer together, thereby improving the model's ability to resist disturbances. It does not need to care about the specific predicted value, but only the distribution deviation, so as to improve the stability of the trained network model to environmental disturbances.

[0086] In some embodiments, the N images include L first images whose quality scores are less than a first quality threshold. The first loss value is determined based on the N predicted age values ​​and age label values, which can be determined as follows: For each first image, when the difference between the predicted age value of the first image output by the network model to be trained and the age label value of the first image is greater than a first set deviation value, the first predicted age value of the first image is determined based on the age label value, the predicted age value, and the first set deviation value of the first image.

[0087] Specifically, the first predicted age value of the first image satisfies the following formula: S = Label + H or S = Label - H, where Label represents the label value, H represents the first set deviation value, and S represents the first predicted age value of the first image. Here, S is a value closer to the predicted age value. For example, when the predicted age value is determined to be 24, the age label value of the first image is 30, and the first set deviation is 4 years, S is either 26 or 34. Since the predicted age is 24, the first predicted age S is 26.

[0088] In some embodiments, when the difference between the predicted age value of the first image and the age label value of the first image is less than or equal to a first set deviation value, the predicted age value of the first image is used as the first predicted age value.

[0089] Furthermore, the first loss value can be determined based on the first predicted age values ​​of the L first images, the predicted age values ​​of the NL images, and the age label values.

[0090] In some embodiments, the first standard deviation can also be determined as follows: The N predicted age values ​​determined by the network model to be trained are sorted, and the median is taken as the first value. For each first image, when the difference between the predicted age value of the first image and the first value is greater than a second set deviation value, a second value is determined based on the first value, the age label value, and the second set deviation value. As an example, the first value can be represented as Median, and the second set deviation value can be represented as A. When the difference between the predicted age value of the first image and the first value is greater than the second set deviation value, the second value satisfies the condition shown in the following formula: X = Median + A or X = Median - A, where A represents the first set deviation value, and X represents the second value. Here, X is a value closer to the predicted age value. As an example, when the predicted age value is determined to be 30, the median of the N predicted age values ​​(i.e., the first value) is 25, and the second set deviation is 4 years, X is 29 or 21. Since the predicted age is 30, the second value X is 29.

[0091] In some embodiments, when the difference between the predicted age value of the first image and the first value is less than or equal to a second preset deviation value, the predicted age value of the first image is used as the second value; a first standard deviation is determined based on NL predicted age values, L second values, and the age label value. In some scenarios, the first standard deviation satisfies the condition shown in the following formula:

[0092]

[0093] Where Std represents the first standard deviation, and x represents the age label value. When the image is the first image with a quality score less than the first quality threshold, x... i The value is set to the second value, when the image has a quality score greater than or equal to the first quality threshold. i The value is the predicted age.

[0094] In some embodiments, a set deviation value can be set for the predicted gender value, which can be set to 1. The predicted gender value can be 0 or 1. When the distance between the predicted gender value and the label value is greater than the set deviation value, a second loss value and a second standard deviation can be determined according to the method for determining a first loss value and a first standard deviation.

[0095] In some embodiments, a test set can be used to test the network model after a certain number of training epochs. In some scenarios, multiple image files can be used as both the test set and the training sample set. For the test set, images can be extracted in equal segments according to the valid data range of attribute values. For example, the valid values ​​for age can be [0, 100]. In some scenarios, images can be extracted in 5-year intervals, with P images extracted from each interval based on representative values. Furthermore, the number of images in the extracted P images should be varied as much as possible. Low-quality images are then excluded, and valid data (including outlier images, the same below) and low-confidence data are retained as the test set. Outlier images are images that are not within the age range.

[0096] In some scenarios, it's possible to obtain the label values ​​of the test set data and calculate the difference between the test values ​​and the attribute label values ​​within the file. The calculation method is similar to the standard deviation, defined as:

[0097]

[0098] Where k represents the number of images in the image archive, Value represents the test value, and Label represents the label value.

[0099] In some embodiments, for each test, the average deviation of the test values ​​from the label values ​​on the test set and the Error can be calculated. std A preset threshold T is set. If the absolute value of the average deviation exceeds T during testing, training is immediately stopped, reverting to the base model and reporting training failure. Error values ​​for each test are also considered. std With a preset threshold E and number of rounds I, an error was found during testing for rounds exceeding I. std If the error value is greater than E, training should be stopped and rolled back immediately. In some scenarios, errors can be monitored. std The value is used to calculate its moving average. Training stops when the moving average converges to a preset threshold. The model that finished training last time becomes the base model for the next training iteration.

[0100] Based on the same technical concept, embodiments of this application provide an image processing method. Figure 3 An illustrative diagram illustrates the flow of an image processing method, which can be executed by an image processing apparatus, such as... Figure 1B The server 100 shown can be, for example, a processor 110 or the server 100 itself. The image processing device can also be located in the terminal device. The specific process is as follows:

[0101] 301, Get the image to be processed.

[0102] In some embodiments, the image to be processed is an image frame captured by an acquisition device. The acquisition device may be an electronic police device, electronic monitoring device, surveillance camera, video recorder, or terminal device with video acquisition function (such as a laptop, computer, mobile phone, television), etc.

[0103] For example, after the acquisition device acquires video frames, the server obtains the image to be processed from the acquisition device.

[0104] In some embodiments, the server receives a video file sent by the acquisition device, the video file including the image to be processed. This video file can be an encoded video file. The server can then decode the received video file to obtain the image to be processed. Encoding the video effectively reduces the file size, facilitating transmission. This improves the video transmission speed, thereby increasing the efficiency of subsequent confirmation of video events. The encoded bitstream data can be acquired using any applicable method, including but not limited to: Real-Time Streaming Protocol (RTSP), Open Network Video Interface Forum (ONVIF) standards, or proprietary protocols.

[0105] 302. Input the image to be processed into the network model. The network model performs facial attribute recognition on the image to obtain the age and gender values ​​corresponding to the faces in the image.

[0106] Based on the same technical concept, embodiments of this application provide a network model training device 400, such as... Figure 4 As shown. Device 400 can perform any step in the above-described network model training method; to avoid repetition, it will not be described again here. Device 400 includes an input module 401, a determination module 402, and an adjustment module 403.

[0107] The input module 401 is used to input the N images included in the first training sample set into the network model to be trained, and to perform facial attribute recognition on the N images through the network model to be trained to obtain N predicted age values ​​and N predicted gender values; the N images in the first training sample set have the same age label value and gender label value.

[0108] The determination module 402 is used to determine a first loss value and a first standard deviation loss value based on the N predicted age values ​​and the age label values, and to determine a first total loss value based on the first loss value and the first standard deviation loss value.

[0109] A second loss value and a second standard deviation loss value are determined based on the N predicted gender values ​​and the gender label value, and a second total loss value is determined based on the second loss value and the second standard deviation loss value.

[0110] The adjustment module 403 is used to adjust the network parameters of the network model to be trained according to the first total loss value and the second total loss value, so as to obtain the trained network model.

[0111] In some embodiments, the determining module 402, when determining the first standard deviation loss value based on the N predicted age values ​​and the age label value, is specifically configured to: determine the first standard deviation based on the N predicted age values ​​and the age label value; when the first standard deviation is greater than a first set standard deviation threshold, use the difference between the first standard deviation and the first set standard deviation threshold as the first standard deviation loss value; or, when the first standard deviation is less than or equal to the first standard deviation threshold, use a first set value as the first standard deviation loss value.

[0112] The determining module 402, in determining a second standard deviation loss value based on the N predicted gender values ​​and the gender label value, is specifically used for: determining a second standard deviation based on the N predicted gender values ​​and the gender label value; when the second standard deviation is greater than a second set standard deviation threshold, using the difference between the second standard deviation and the second set standard deviation threshold as the second standard deviation loss value; or, when the second standard deviation is less than or equal to the second standard deviation threshold, using a second set value as the second standard deviation loss value.

[0113] In some embodiments, the determining module 402 is further configured to obtain the first training sample set by: acquiring M images, and for each of the M images, an age value, age confidence score, gender value, gender confidence score, and quality score, wherein the M images are images of the same person in different scenarios within a set time period; selecting N images from the M images that meet set conditions, and generating a first training sample set based on the N images; the set conditions include: the age confidence score of the image is greater than or equal to a first threshold, the gender confidence score of the image is greater than or equal to a second threshold, and the quality score of the image is greater than or equal to a third threshold; and determining the age label value and gender label value of the N images in the first training sample set based on the age value and gender value of the N images.

[0114] In some embodiments, the determining module 402, when determining the age label values ​​and gender label values ​​of the N images in the first training sample set based on the age values ​​and gender values ​​of the N images, is specifically used for:

[0115] From the N age values ​​corresponding to the N images, select multiple age values ​​that fall within a set age range; take the age value that appears most frequently among the multiple age values ​​as the age label value of the N images in the first training sample set; take the gender value that appears most frequently among the N images as the gender label value of the N images in the first training sample set.

[0116] In some embodiments, the set age range is determined by: calculating the mean and standard deviation of the N age values; and determining the set age range based on the mean and standard deviation.

[0117] In some embodiments, the N images include L first images with quality scores less than a first quality threshold. The determining module 402, when determining the first loss value based on the N predicted age values ​​and the age label value, specifically performs the following: for each first image, when the difference between the predicted age value of the first image output by the network model to be trained and the age label value of the first image is greater than a first set deviation value, determines the first predicted age value of the first image based on the age label value of the first image, the predicted age value, and the first set deviation value; or, when the difference between the predicted age value of the first image and the age label value of the first image is less than or equal to the first set deviation value, uses the predicted age value of the first image as the first predicted age value.

[0118] A first loss value is determined based on the first predicted age values ​​of the L first images, the predicted age values ​​of the NL images, and the age label values.

[0119] In some embodiments, the determining module 402 is further configured to determine the first standard deviation by: sorting the N predicted age values ​​determined by the network model to be trained and taking the median as the first value; for each first image, when the difference between the predicted age value of the first image and the first value is greater than the second set deviation value, determining the second value based on the first value, the age label value, and the second set deviation value; or, when the difference between the predicted age value of the first image and the first value is less than or equal to the second set deviation value, taking the predicted age value of the first image as the second value; and determining the first standard deviation based on the N predicted age values, the L second values, and the age label value.

[0120] Based on the same technical concept, embodiments of this application provide an execution device 500, which can implement any step of the network model training method and image processing method described above. Please refer to... Figure 5 The device includes a memory 501 and a processor 502.

[0121] The memory 501 is used to store program instructions;

[0122] The processor 502 is used to call the program instructions stored in the memory and execute the training method or image processing method of the above network model according to the obtained program.

[0123] In the embodiments of this application, the processor 502 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, capable of implementing or executing the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0124] Memory 501, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 501 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 501 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. Memory 501 in the embodiments of this application may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0125] Based on the same technical concept, embodiments of this application provide a computer-readable storage medium, including: computer program code, which, when executed on a computer, causes the computer to perform the network model training method or image processing method as described above. Since the principle by which the above-described computer-readable storage medium solves the problem is similar to the network model training method or image processing method, the implementation of the above-described computer-readable storage medium can be referred to the implementation of the method, and repeated details will not be elaborated further.

[0126] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0128] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0129] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0130] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A network model training method, characterized in that, The method comprises the following steps: inputting N images included in a first training sample set into a network model to be trained respectively, and performing face attribute recognition on the N images by using the network model to be trained to obtain N predicted age values and N predicted gender values; the N images in the first training sample set have the same age label value and gender label value; determining a first loss value according to the N predicted age values and the age label value; determining a first standard deviation according to the N predicted age values and the age label value; when the first standard deviation is greater than a first set standard deviation threshold, taking the difference between the first standard deviation and the first set standard deviation threshold as a first standard deviation loss value; or, when the first standard deviation is less than or equal to the first set standard deviation threshold, taking a first set value as the first standard deviation loss value; determining a first total loss value according to the first loss value and the first standard deviation loss value; determining a second loss value according to the N predicted gender values and the gender label value; determining a second standard deviation according to the N predicted gender values and the gender label value; when the second standard deviation is greater than a second set standard deviation threshold, taking the difference between the second standard deviation and the second set standard deviation threshold as a second standard deviation loss value; or, when the second standard deviation is less than or equal to the second set standard deviation threshold, taking a second set value as the second standard deviation loss value; determining a second total loss value according to the second loss value and the second standard deviation loss value; adjusting network parameters of the network model to be trained according to the first total loss value and the second total loss value to obtain a trained network model.

2. The method of claim 1, wherein, The first training sample set is obtained in the following manner: obtaining M images, and age values, age confidences, gender values, gender confidences and quality scores corresponding to each of the M images, wherein the M images are images of a same person in different scenes within a set time length; selecting N images satisfying a set condition from the M images, and generating a first training sample set based on the N images; the set condition comprises: the age confidence of an image is greater than or equal to a first threshold value, the gender confidence of the image is greater than or equal to a second threshold value, and the quality score of the image is greater than or equal to a third threshold value; determining age label values and gender label values of the N images in the first training sample set according to the age values and the gender values of the N images.

3. The method of claim 2, wherein, The determination of the age label values and the gender label values of the N images in the first training sample set according to the age values and the gender values of the N images comprises: selecting multiple age values within a set age range from the N age values corresponding to the N images; taking an age value with the highest frequency in the multiple age values as the age label value of the N images in the first training sample set; taking a gender value with the highest frequency in the N images as the gender label value of the N images in the first training sample set.

4. The method of claim 3, wherein, The set age range is determined in the following manner: calculating the age mean and the standard deviation of the N age values; determining the set age range according to the age mean and the standard deviation.

5. The method of claim 2, wherein, The N images include L first images with a quality score less than a first quality threshold, and the first loss value is determined according to the N predicted age values and the age label values. For each first image, when a difference between the predicted age value of the first image output by the network model to be trained and the age label value of the first image is greater than a first set deviation value, a first predicted age value of the first image is determined according to the age label value, the predicted age value and the first set deviation value of the first image; or when the difference between the predicted age value of the first image and the age label value of the first image is less than or equal to the first set deviation value, the predicted age value of the first image is taken as the first predicted age value. The first loss value is determined according to the first predicted age values of the L first images, the predicted age values of N-L images and the age label values.

6. The method of claim 5, wherein, The first standard deviation is determined in the following manner: The N predicted age values determined by the network model to be trained are sorted and the median is taken as a first value; For each first image, when a difference between the predicted age value of the first image and the first value is greater than a second set deviation value, a second value is determined according to the first value, the age label value and the second set deviation value; or when the difference between the predicted age value of the first image and the first value is less than or equal to the second set deviation value, the predicted age value of the first image is taken as the second value; The first standard deviation is determined according to the N-L predicted age values, the L second values and the age label values.

7. An image processing method characterized by, The method comprises: Obtaining an image to be processed; Inputting the image to be processed into a network model, and performing face attribute recognition on the image to be processed by the network model to obtain an age value and a gender value corresponding to a face in the image to be processed, wherein the network model is obtained by training the network model according to any one of claims 1-6.

8. A network model training apparatus characterized by comprising: The method comprises: An input module is configured to input N images included in a first training sample set into a network model to be trained, and perform face attribute recognition on the N images by the network model to be trained to obtain N predicted age values and N predicted gender values. The N images in the first training sample set have the same age label value and gender label value. A determination module is configured to determine a first loss value according to the N predicted age values and the age label values. A first standard deviation is determined according to the N predicted age values and the age label values. When the first standard deviation is greater than a first set standard deviation threshold, a difference between the first standard deviation and the first set standard deviation threshold is taken as a first standard deviation loss value; or when the first standard deviation is less than or equal to the first set standard deviation threshold, a first set value is taken as the first standard deviation loss value; and a first total loss value is determined according to the first loss value and the first standard deviation loss value. determining a second loss value according to the N predicted gender values and the gender label values; determining a second standard deviation according to the N predicted gender values and the gender label values; when the second standard deviation is greater than a second set standard deviation threshold, taking a difference between the second standard deviation and the second set standard deviation threshold as a second standard deviation loss value; or, when the second standard deviation is less than or equal to the second set standard deviation threshold, taking a second set value as the second standard deviation loss value; determining a second total loss value according to the second loss value and the second standard deviation loss value; an adjusting module, configured to adjust network parameters of the network model to be trained according to the first total loss value and the second total loss value, to obtain a trained network model.

9. An execution device, characterized by comprising: a memory, configured to store program instructions; a processor, configured to acquire the program instructions stored in the memory, and perform the method in any one of claims 1-7 according to the acquired program instructions.

10. A computer readable storage medium characterized by, The computer readable storage medium stores computer instructions, when the computer instructions run on a computer, make the computer execute the method in any one of claims 1-7.

Images