Network training method and device, target recognition method and device, and electronic equipment

Abstract

The present disclosure relates to a network training method and device, a target recognition method and device, and an electronic device, the method comprising: according to the labeling result and the first recognition result of the first sample image, respectively training the target recognition network of the n-1th round of training through multiple first loss functions of the n-1th round, and obtaining multiple target recognition networks of the n-th intermediate state; determining the target recognition network of the n-th round of training according to the multiple target recognition networks of the n-th intermediate state; updating the hyperparameters of the multiple first loss functions of the n-1th round according to the hyperparameter optimization function and sampling distribution of the first loss function, and obtaining multiple first loss functions of the n-th round; obtaining the trained target recognition network when the training conditions are met. According to the embodiment of the present disclosure, the first loss function can be automatically designed without human intervention, greatly improving the deployment efficiency of the target recognition deployment system.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a network training method and device, a target recognition method and device, and an electronic device. Background Art

[0002] Target recognition refers to the process of distinguishing a specific target (or a type of target) from other targets (or other types of targets). It includes both the recognition of two very similar targets and the recognition of one type of target from other types of targets. Among them, target recognition technology may include target re-identification (ReID) technology, that is, for a specific target (which may include pedestrians, vehicles, faces or other specific objects, etc.), the target can be retrieved from the candidate image set. Target re-identification originated from multi-camera tracking and is used to determine whether targets in different images captured in non-overlapping fields of view belong to the same target. Target re-identification involves multiple disciplines such as computer vision, machine learning, and pattern recognition, and can be widely used in intelligent video surveillance, security, criminal investigation and other fields. In recent years, target re-identification technology has attracted widespread attention from academia and industry, and has become a research hotspot in the field of computer vision. However, taking person re-identification as an example, since pedestrians have the characteristics of both rigid and flexible objects, their appearance is easily affected by various complex factors such as clothing, posture and perspective changes, as well as lighting, occlusion, environment, etc., which makes pedestrian re-identification face huge technical challenges.

[0003] With the development of deep learning, more and more models based on deep convolutional neural networks have achieved good results in the field of computer vision. For the task of object re-identification, the learned features need to be not only separable like classification tasks, but also distinguishable. The loss function plays an important role in learning distinguishable features. Summary of the invention

[0004] In view of this, the present disclosure proposes a network training method and device, a target recognition method and device, and an electronic device.

[0005] According to one aspect of the present disclosure, a network training method is provided, the method comprising: for the nth round of training of a target recognition network, inputting a first sample image in a training set into a target recognition network for training in the n-1th round for processing, obtaining a first recognition result of the first sample image, the target recognition network being used to perform target recognition on the processed image, where n is a positive integer; according to the labeling result of the first sample image and the first recognition result, respectively training the target recognition network for training in the n-1th round through a plurality of first loss functions for the n-1th round, obtaining a plurality of target recognition networks for the nth intermediate states; determining a target recognition network for training in the nth round according to the plurality of target recognition networks for the nth intermediate states; updating the hyperparameters of the plurality of first loss functions for the n-1th round according to the hyperparameter optimization function and sampling distribution of the first loss function, obtaining a plurality of first loss functions for the nth round; obtaining a trained target recognition network when the training conditions are met;

[0006] Among them, the first loss function includes a linear piecewise function, the linear piecewise function includes M piecewise subintervals and M+1 hyperparameters, the domain of the linear piecewise function ranges from -1 to 1, and M is an integer greater than 1.

[0007] In one possible implementation, the method further includes: initializing the first loss function according to the sampling distribution to obtain an initial plurality of first loss functions; performing random initialization according to the target recognition network to obtain an initial plurality of target recognition networks; wherein, when n=1, the target recognition network trained in the n-1th round includes the initial plurality of target recognition networks, and the plurality of first loss functions in the n-1th round are the initial plurality of first loss functions.

[0008] In a possible implementation, determining the target recognition network for the nth round of training based on the target recognition networks of the multiple nth intermediate states includes: for any one of the target recognition networks of the multiple nth intermediate states, inputting a second sample image in a test set into the target recognition network of the nth intermediate state for processing to obtain a second recognition result of the second sample image; determining feedback information of the target recognition network of the nth intermediate state based on the labeling result of the second sample image and the second recognition result; and determining the target recognition network for the nth round of training from the target recognition networks of the multiple nth intermediate states based on the feedback information of the target recognition networks of the multiple nth intermediate states.

[0009] In a possible implementation, the hyperparameters of the multiple first loss functions of the n-1th round are updated according to the hyperparameter optimization function and the sampling distribution of the first loss function to obtain multiple first loss functions of the nth round, including: updating the parameters of the sampling distribution of the first loss function according to the reward information of the target recognition network of the multiple nth intermediate states and the hyperparameter optimization function; and updating the hyperparameters of the multiple first loss functions of the n-1th round respectively according to the updated sampling distribution to obtain multiple first loss functions of the nth round.

[0010] In a possible implementation, the linear piecewise function satisfies a first constraint, which includes: the first-order derivative of the linear piecewise function is greater than or equal to 0; the first-order derivative of the linear piecewise function is not a fixed value; when the independent variable is x, the function value of the linear piecewise function is less than or equal to the independent variable x; and when the independent variable is 1, the function value of the linear piecewise function is greater than or equal to 0. The M+1 hyperparameters of the linear piecewise function satisfy a second constraint, which includes: the jth hyperparameter is less than or equal to the j+1th hyperparameter, the M+1th hyperparameter is greater than or equal to 0 and less than or equal to 1; and the jth hyperparameter is less than or equal to the right endpoint of the jth segmented subregion, 1≤j≤M.

[0011] According to another aspect of the present disclosure, a target recognition method is provided, the method comprising: inputting an image to be processed into a target recognition network for processing to obtain a recognition result of the image to be processed, the recognition result including a target area in the image to be processed, wherein the target recognition network is trained according to the network training method according to any one of claims 1-6.

[0012] According to another aspect of the present disclosure, a network training device is provided, characterized in that the device includes: a result acquisition module, for inputting the first sample image in the training set into the target recognition network trained in the n-1th round for processing for the nth round of training of the target recognition network, and obtaining a first recognition result of the first sample image, the target recognition network is used to perform target recognition on the processed image, and n is a positive integer; a network training module, for training the target recognition network trained in the n-1th round respectively according to the labeling result of the first sample image and the first recognition result through multiple first loss functions of the n-1th round, and obtaining multiple target recognition networks in the nth intermediate state; a first network determination module, for determining the target recognition network trained in the nth round according to the multiple target recognition networks in the nth intermediate state; a loss function acquisition module, for updating the hyperparameters of the multiple first loss functions of the n-1th round according to the hyperparameter optimization function and sampling distribution of the first loss function, and obtaining multiple first loss functions of the nth round; a second network determination module, for obtaining a trained target recognition network when the training conditions are met;

[0013] Among them, the first loss function includes a linear piecewise function, the linear piecewise function includes M piecewise subintervals and M+1 hyperparameters, the domain of the linear piecewise function ranges from -1 to 1, and M is an integer greater than 1.

[0014] In one possible implementation, the device also includes an initialization module: used to initialize the first loss function according to the sampling distribution to obtain an initial multiple first loss function; randomly initialize according to the target recognition network to obtain an initial multiple target recognition network; wherein, when n=1, the target recognition network trained in the n-1th round includes the initial multiple target recognition networks, and the multiple first loss functions of the n-1th round are the initial multiple first loss functions.

[0015] In one possible implementation, the first network determination module includes: for any one of the target recognition networks of the multiple n-th intermediate states, inputting the second sample image in the test set into the target recognition network of the n-th intermediate state for processing to obtain a second recognition result of the second sample image; determining the return information of the target recognition network of the n-th intermediate state according to the labeling result of the second sample image and the second recognition result; and determining the target recognition network for the n-th round of training from the target recognition networks of the multiple n-th intermediate states according to the return information of the target recognition networks of the multiple n-th intermediate states.

[0016] In one possible implementation, the loss function acquisition module includes: updating the parameters of the sampling distribution of the first loss function according to the reward information of the target recognition network of the multiple n-th intermediate states and the hyperparameter optimization function; and updating the hyperparameters of the multiple first loss functions of the n-1th round respectively according to the updated sampling distribution to obtain the multiple first loss functions of the nth round.

[0017] In a possible implementation, the linear piecewise function satisfies a first constraint, which includes: the first-order derivative of the linear piecewise function is greater than or equal to 0; the first-order derivative of the linear piecewise function is not a fixed value; when the independent variable is x, the function value of the linear piecewise function is less than or equal to the independent variable x; and when the independent variable is 1, the function value of the linear piecewise function is greater than or equal to 0. The M+1 hyperparameters of the linear piecewise function satisfy a second constraint, which includes: the jth hyperparameter is less than or equal to the j+1th hyperparameter, the M+1th hyperparameter is greater than or equal to 0 and less than or equal to 1; and the jth hyperparameter is less than or equal to the right endpoint of the jth segmented subregion, 1≤j≤M.

[0018] According to another aspect of the present disclosure, a target recognition device is provided, comprising: a processing module, configured to process the image to be processed according to the target recognition network trained by the network training method described above, and obtain a target area in the image to be processed.

[0019] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0020] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.

[0021] In this way, the first loss function can be automatically obtained without human intervention, and the automatically obtained first loss function can be used to train the target recognition network. The hyperparameters of the first loss function can be dynamically adjusted during the training, which can improve the training efficiency of the target recognition network and the deployment efficiency of the target recognition network.

[0022] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0024] Figure 1 A flowchart showing a network training method according to an embodiment of the present disclosure is shown;

[0025] Figure 2 A schematic diagram showing a network training method according to an embodiment of the present disclosure;

[0026] Figure 3 A schematic diagram showing a method of determining hyperparameters of a first loss function according to a sampling distribution according to an embodiment of the present disclosure;

[0027] Figure 4 A schematic diagram showing the target recognition effect of the target recognition method according to an embodiment of the present disclosure;

[0028] Figure 5 A block diagram of a network training device according to an embodiment of the present disclosure is shown;

[0029] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0030] Figure 7 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0031] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0032] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0033] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0034] In target recognition tasks, such as pedestrian re-identification tasks, in related technologies, the target re-identification network can be trained based on a manually designed loss function, and the neural network model can be divided into three categories:

[0035] 1) Classification-based neural networks. This type of neural network treats the target re-identification task as a classification task, directly uses the cross entropy loss function to train the neural network, and finally uses the features input into the cross entropy loss function as the final learned features. This type of neural network does not take into account the difference between the classification task and the target re-identification task. In the target re-identification task, the training data set and the test set have no intersection, that is, the test target in the test set will not appear in the training data set. If it is directly regarded as a classification task, the learned features are inappropriate.

[0036] 2) Neural networks based on metric learning. This type of neural network uses the loss function in metric learning (such as contrast loss function, triple loss function, etc.). From the nature of the task, the target re-identification task is indeed very similar to metric learning, so it is more appropriate to use such a loss function. However, this type of loss function is sensitive to the hyperparameters in the function and has problems such as slow convergence speed.

[0037] 3) Neural networks based on classification and metric learning. In related technologies, the more popular loss function for training target re-identification networks is to use classification and metric learning loss functions for collaborative training. The most common combination is to combine the cross entropy loss function and the triple loss function. Even so, there are certain problems with this combination. When these two types of loss functions act on the same feature, the learned features will not be robust. Therefore, for this type of neural network, it is often necessary to carefully design the network architecture or dynamically adjust the weights of these two types of loss functions.

[0038] With the development of automated machine learning, more and more automated methods have surpassed manually designed methods in various fields. Compared with the loss function based on manual design, the loss function based on automated machine learning can indirectly optimize some non-differentiable indicators (such as mAP). In addition, the loss function based on automated machine learning does not require much professional knowledge for loss function design and is more automated.

[0039] However, in terms of automatic loss function design, there are two works in the related art, one is the AM-LFS method, and the other is the Search-Softmax method. However, for the AM-LFS method, the loss function automatically designed by this method may include a piecewise linear function, in which each segment independently maintains two parameters. In this case, the search space is very large, so the final search results are not very stable. For the Search-Softmax method, the loss function automatically designed by this method may include a linear function. Although the search space is simplified and only one parameter needs to be optimized, such a search space is too simple, which makes the form of the loss function contained in this search space too single.

[0040] In view of the irrationality of the above two search spaces, the present disclosure proposes a network training method, which can propose a more stable search space and automatically design the loss function without human intervention, thereby greatly improving the training efficiency of the target recognition network and thus improving the deployment efficiency of the target recognition deployment system.

[0041] Figure 1 FIG. 5 is a flow chart showing a network training method according to an embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0042] In step S1, for the nth round of training of the target recognition network, the first sample image in the training set is input into the target recognition network of the n-1th round of training for processing, and a first recognition result of the first sample image is obtained. The target recognition network is used to perform target recognition on the image to be processed, and n is a positive integer;

[0043] In step S2, according to the labeling result of the first sample image and the first recognition result, the target recognition network trained in the n-1th round is trained respectively by using multiple first loss functions in the n-1th round to obtain multiple target recognition networks in the nth intermediate state;

[0044] In step S3, determining the target recognition network for the nth round of training according to the target recognition networks in the nth intermediate states;

[0045] In step S4, the hyperparameters of the multiple first loss functions of the n-1th round are updated according to the hyperparameter optimization function and the sampling distribution of the first loss function to obtain the multiple first loss functions of the nth round;

[0046] In step S5, when the training conditions are met, a trained target recognition network is obtained.

[0047] In a possible implementation, the network training method may be executed by a terminal device or other processing device, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, etc. Other processing devices may be a server or a cloud server, etc. In some possible implementations, the network training method may be implemented by a processor calling computer-readable instructions stored in a memory.

[0048] In a possible implementation, the training set may include multiple first sample images, wherein the first sample image may be an image captured by an image acquisition device (e.g., a camera) and may include a target to be identified. The target may include, for example, a human body, a face, a vehicle, an object, etc. The present disclosure does not limit the method for acquiring the first sample image and the category of the target in the first sample image.

[0049] In a possible implementation, the first sample image in the training set may include the annotation result of the target to be identified, that is, the target is annotated in the first sample image in the data set by labeling, marking, coloring or highlighting. The higher the quality of the annotation result of the first sample image, the more accurate the result of the target recognition network learning.

[0050] In a possible implementation, the target recognition network is used to perform target recognition on the image to be processed. The target recognition network can be pre-set, and the image to be processed is input into the target recognition network, and the area where the target is located in the image to be processed can be identified. The target recognition network may include a convolutional neural network, and the present disclosure does not limit the specific network structure of the target recognition network.

[0051] In one possible implementation, the target recognition network can be used to perform target re-identification on the image to be processed, such as pedestrian re-identification, that is, for a known pedestrian target, the target can be retrieved in the image to be processed to determine whether the image to be processed includes an area where the target is the pedestrian.

[0052] In a possible implementation, before step S1, the hyperparameters of the first loss function may be initialized according to the initialized sampling distribution, for example, including multiple non-independent truncated Gaussian distributions, to obtain multiple initial first loss functions. The target recognition network is randomly initialized to obtain multiple initial target recognition networks.

[0053] For example, the first loss function may include multiple hyperparameters, and the hyperparameters may be sampled according to the order of the hyperparameters and the truncated Gaussian distribution corresponding to the current hyperparameters until the values ​​of the multiple hyperparameters are completed to obtain an initial first loss function. Among them, the truncation range of the truncated Gaussian distribution corresponding to the current hyperparameter (i.e., limiting the value range of the hyperparameter to be sampled) can be determined according to the value of the previous hyperparameter, so the truncated Gaussian distributions corresponding to the various hyperparameters are non-independent.

[0054] According to the above process of sampling the first loss function, the first loss function can be sampled multiple times according to the initialized sampling distribution to obtain multiple different initial first loss functions.

[0055] In a possible implementation, the training of the target recognition network can be completed through multiple rounds of iterative training, wherein, in each round of training, the target recognition network determined in the previous round can be trained in parallel through multiple automatically generated first loss functions to obtain multiple target recognition networks in intermediate states, and one target recognition network is selected from them as the target recognition network of the current round, that is, the target recognition network to be trained in the next round, until the condition for stopping iteration is reached. For example, until the number of iterations reaches a preset number, or until the performance of the obtained target recognition network meets the expected performance, etc. The present disclosure does not limit the specific conditions for stopping iteration.

[0056] In a possible implementation, in step S1, for the nth round of training of the target recognition network (n is a positive integer), the first sample image may be input into the target recognition network trained in the n-1th round for processing, and a first recognition result of the first sample image may be obtained;

[0057] Among them, each first sample image data can be input individually into the target recognition network of the n-1th round of training, or multiple first sample image data can be input into the target recognition network of the n-1th round of training in the form of batch data. Each first sample image can correspond to a first recognition result. The present disclosure does not limit the input method of the first sample image into the target recognition network.

[0058] In one possible implementation, in step S2, the method of obtaining multiple initial first loss functions can be referred to above, and according to the sampling distribution of the n-1th round, the hyperparameters of the multiple first loss functions of the n-1th round can be sampled to obtain multiple different first loss functions of the n-1th round.

[0059] The target recognition network trained in the n-1th round can be copied into multiple identical target recognition networks, and the number of the copied target recognition networks is the same as the number of the first loss functions.

[0060] For any of the copied target recognition networks, the first sample image can be input into the target recognition network to obtain a first recognition result corresponding to the target recognition network. According to the first recognition result of the target recognition network and the labeling result of the first sample image, the target recognition network is trained through a first loss function corresponding to the target recognition network to obtain a target recognition network in an nth intermediate state corresponding to the target recognition network.

[0061] By analogy, according to the labeling result and the first recognition result of the first sample image, the multiple copied target recognition networks are trained respectively through multiple first loss functions of the n-1th round to obtain multiple target recognition networks in the nth intermediate state.

[0062] In a possible implementation, in step S3, based on multiple n-th intermediate state target recognition networks, the n-th intermediate state target recognition network with the best recognition effect among the multiple n-th intermediate state target recognition networks can be determined as the target recognition network for the n-th round of training.

[0063] In a possible implementation, in step S4, the sampling distribution of the n-1th round can be updated according to the hyperparameter optimization function of the first loss function, and the sampling distribution of the nth round can be determined. By performing sampling processing on the sampling distribution of the nth round, the hyperparameters of multiple first loss functions of the n-1th round are updated to obtain multiple first loss functions of the nth round.

[0064] Among them, by sampling the sampling distribution of the nth round, multiple first loss functions of the nth round are obtained. The method of obtaining multiple initial first loss functions can be referred to the above, and will not be repeated here.

[0065] In step S5, when the training conditions are met, the trained target recognition network is obtained. The situation where the training conditions are met may include until the number of iterations reaches a preset number, or until the performance of the obtained target recognition network meets the expected performance, etc. The present disclosure does not limit the specific conditions for stopping iteration.

[0066] In this way, the first loss function can be automatically designed without human intervention, and the target recognition network can be obtained using the automatically designed first loss function, which can improve the training efficiency of the target recognition network and the deployment efficiency of the target recognition deployment system.

[0067] The network training method of the embodiment of the present disclosure is described below.

[0068] In a possible implementation, the first loss function includes a linear piecewise function, which includes M piecewise subintervals and M+1 hyperparameters. The domain of the linear piecewise function ranges from -1 to 1, and M is an integer greater than 1.

[0069] For example, the first loss function It can be expressed as:

[0070]

[0071] In formula (1), s represents the scale factor, and s>0, The feature w representing the category y y and the cosine similarity of feature x, The feature w representing class k k The cosine similarity with feature x, the value range of category k is from the 1st category to the Kth category and does not include the yth category, and t(x) represents a linear piecewise function with a domain of [-1, 1].

[0072] It should be understood that in the target recognition task, the similarity between samples can be judged by the distance between features, which can be cosine similarity, Euclidean distance, Mahalanobis distance, Manhattan distance or other distance measurement methods. The present disclosure does not limit the specific distance measurement method.

[0073] According to formula (1), the first loss function The framework of the interval-based softmax loss function is composed of the interval-based softmax loss function (ie, the initial loss function) and the linear piecewise function t(x).

[0074] In order to obtain a better first loss function The first loss function can be analyzed in terms of gradient, features, and optimization. Conduct analysis.

[0075] First, the first loss function can be analyzed from the gradient Consider, according to formula (1), the first loss function about and The two types of gradients can be expressed as:

[0076]

[0077]

[0078] In formula (2) and formula (3), and It can be expressed as:

[0079]

[0080]

[0081] As the training progresses, in order to obtain better classification results, the distance between classes can be enlarged while making the samples within the class more compact (i.e. reducing the distance within the class). The overall trend is to increase, and The overall trend is decreasing. Therefore, from the overall trend, we can make:

[0082]

[0083] According to formula (3), combined with formula (2) and formula (4), we can know that t′(x)≥0, x∈[-1,1].

[0084] Therefore, in order to obtain better classification effect, the linear piecewise function t(x) can be made to satisfy t′(x)≥0, x∈[-1,1].

[0085] Furthermore, according to formula (2) and formula (3), the relative gradient can also be defined as It can be expressed as:

[0086]

[0087] For simplicity, we can consider the binary classification case, and formula (7) can be further simplified as follows:

[0088]

[0089] In formula (8), t′(x) controls the relative gradient The magnitude of the relative gradient The magnitude of represents the positive gradient and negative gradient Maintain different relative gradients for samples of different difficulty levels The size is beneficial to the optimization of the target recognition network, so the relative gradient The value of needs to keep changing so that the linear piecewise function t(x) satisfies t′(x)≠constant, x∈[-1,1].

[0090] Next, the first loss function can be analyzed from the feature aspect To improve the distinguishability of features, the softmax probability can be reduced, where the softmax probability p y It can be expressed as:

[0091]

[0092] According to formula (9) and formula (4), we can get:

[0093]

[0094] In order to achieve the purpose of reducing the softmax probability, that is According to formula (10), a series of derivations can be obtained as shown below:

[0095]

[0096] According to the above analysis, in order to improve the distinguishability of features and reduce the softmax probability, the linear piecewise function t(x) can satisfy t(x)≤x, x∈[-1,1].

[0097] Finally, the first loss function can be optimized In formula (4), the maximum positive similarity is s·t(1), and the minimum negative similarity is s·t(-1), so we can get The value of in the whole optimization process is:

[0098]

[0099] Compared with the case of t(1)<0, in the case of t(1)≥0, The maximum value is closer to 1. If t(1) < 0, the entire interval is relatively small, which is not conducive to optimization. Therefore, in order to maintain a larger optimization interval, t(1) ≥ 0 can be used.

[0100] In order to make the first loss function The above characteristics can be satisfied in terms of gradient, features and optimization. The linear piecewise function t(x) can be expressed as:

[0101] t(x)=pwl(x;a0,a1,...,a M ), x∈[-1,1] (13)

[0102] In formula (13), pωl(x; a0, a1, ..., a M ) includes M+1 (M is an integer greater than 1) hyperparameters a0~a M , which can represent a linear piecewise function consisting of M+1 control points. The coordinates of these M+1 points are Specifically, the interval [-1, 1] can be evenly divided into M subintervals, and in the i-th subinterval In the above equation, the linear piecewise function t(x) can be defined as:

[0103] t(x)=k i x+b i , i∈[1,M] (14)

[0104] In formula (14), the slope k in the i-th subinterval is i and intercept b i It can be expressed as:

[0105]

[0106] In formula (13) to formula (15), a0 represents the first hyperparameter of the linear piecewise function t(x), a1 represents the second hyperparameter of the linear piecewise function t(x), and so on. M Represents the M+1th hyperparameter of the linear piecewise function t(x).

[0107] In this way, the composite first loss function includes a linear piecewise function t(x) with M+1 hyperparameters, which not only ensures that the first loss function in the search space is diverse in form and the stability of the search results of the first loss function, but also simplifies the search space and improves the generation efficiency of the first loss function.

[0108] In order to make the first loss function It has better performance in gradient, features and optimization, and can make linear piecewise functions satisfy the following constraints.

[0109] In one possible implementation, the linear piecewise function satisfies a first constraint, which includes: the first-order derivative of the linear piecewise function is greater than or equal to 0; the first-order derivative of the linear piecewise function is not a fixed value; when the independent variable is x, the function value of the linear piecewise function is less than or equal to the independent variable x; and when the independent variable is 1, the function value of the linear piecewise function is greater than or equal to 0.

[0110] Among them, the M+1 hyperparameters of the linear piecewise function satisfy the second constraint, which includes: the j-th hyperparameter is less than or equal to the j+1-th hyperparameter, the M+1-th hyperparameter is greater than or equal to 0 and less than or equal to 1; and the j-th hyperparameter is less than or equal to the right endpoint of the j-th segmented sub-region, 1≤j≤M.

[0111] For example, the M+1 hyperparameters a0~a of the linear piecewise function t(x) shown in formula (13) are M The second constraint is satisfied, namely:

[0112]

[0113] In formula (16), the jth (j=i+1) hyperparameter a i Less than or equal to the j+1th hyperparameter a i+1 (1≤j≤M), the M+1th hyperparameter a M ≥0;

[0114] And, the jth hyperparameter a i Less than or equal to the right endpoint of the jth segment sub-region The M+1th hyperparameter

[0115] In the M+1 hyperparameters a0~a of the linear piecewise function t(x) M , when the second constraint condition shown in formula (16) is satisfied, it is easy to verify that the linear piecewise function t(x) can satisfy the first constraint condition.

[0116] First, according to formula (16), a0≤a1≤…≤a M , a M ≥0, that is, a i-1 ≤a i According to formula (14) and formula (15), we can get Therefore, the first-order derivative of the linear piecewise function t(x) is greater than or equal to 0, that is, t′(x)≥0, x∈[-1,1].

[0117] Then, according to formula (15), the linear piecewise function t(x) can be divided into M segments (intervals), and the k of each segment is i The value of can be different. Therefore, the first-order derivative of the linear piecewise function t(x) is not a fixed value, that is, t′(x)=k i ≠constant, x∈[-1,1].

[0118] Next, the i-th subinterval Substituting the endpoints of the interval into the linear piecewise function t(x), we can get as well as According to the second constraint of formula (16), we can know that the function value of the linear piecewise function t(x) is less than or equal to the independent variable x, that is, t(x)≤x, x∈[-1,1].

[0119] Finally, because the coordinates of the M+1th point of the linear piecewise function t(x) are (1, a M ), then according to the second constraint condition a in formula (16) M ≥0, we know that t(1)=a M≥ 0. Therefore, when the independent variable x of the linear piecewise function t(x) is 1, the function value of the linear piecewise function t(x) is greater than or equal to 0, that is, t(1)≥0.

[0120] In this way, the hyperparameters of the linear piecewise function t(x) satisfy the second constraint, so that the linear piecewise function t(x) satisfies the first constraint, thereby ensuring that the first loss function in the search space has diverse forms, ensuring the stability of the search results of the first loss function, and improving the generation efficiency and performance of the first loss function.

[0121] After determining the constraints of the linear piecewise function t(x), when searching for the loss function, a sampling distribution can be designed to sample the hyperparameters of the linear piecewise function t(x) to ensure that the hyperparameters of the linear piecewise function t(x) sampled from this distribution meet the second constraints, so that the linear piecewise function t(x) meets the first constraints, and thus the first loss function Good performance on gradients, features, and optimization.

[0122] Specifically, we define the search space Θ = [a0, a1, ..., a M ] T , then the whole sampling process can be expressed as:

[0123]

[0124] In formula (17), a i Represents the hyperparameter of the linear piecewise function (that is, the hyperparameter of the first loss function), i ranges from 0 to M, and the mean μ of the sampling distribution is i , which can be expressed as:

[0125]

[0126] In formula (17), The probability density function of the truncated Gaussian distribution, that is, the value range of the variable x is limited to a~b, can be expressed as:

[0127]

[0128] In formula (19), α = (a-μ) / σ, β = (b-μ) / σ, -∞≤a<b≤∞, a≤x≤b, μ and σ represent the mean and standard deviation of the Gaussian distribution, respectively. φ (x) and Φ(x) represent the probability density function and cumulative distribution function of the standard normal distribution, respectively, which can be expressed as:

[0129]

[0130] Where erf(·) represents the error function. The inverse transformation method can be used to sample the truncated Gaussian distribution, that is:

[0131]

[0132] In formula (21), U is a uniform distribution on [0, 1]. In this way, we can get the derivative of x with respect to μ, σ, α, β. In this way, the whole is differentiable, which is convenient for subsequent optimization using the stochastic gradient descent method. In actual use, the above method cannot effectively sample the tail of the standard normal distribution, so it can be modified as follows:

[0133]

[0134] In formula (22), In the case of , the effective interval of the truncated distribution can be located at the right tail of the standard normal distribution. As α increases, the truncated Gaussian distribution gradually approaches the delta function (Dirac Delta Function); In the case of , the effective interval of the truncated distribution can be located at the left tail of the standard normal distribution. As β decreases, the truncated Gaussian distribution gradually approaches the δ function.

[0135] In theory, As large as possible, but if If it is too large, it may cause instability in the sampling space. In order to ensure the stability of the sampled hyperparameters, you can preset Among them, in satisfying In the case of and There is no restriction on the specific value of .

[0136] For the convenience of subsequent description, formula (17) can be defined as:

[0137]

[0138] In formula (23), μ includes μ0~μ M .

[0139] In determining the sampling distribution Afterwards, the first loss function can be automatically calculated without human intervention according to formula (1) and formula (13). Design, and based on the automatically generated first loss function Train the object recognition network.

[0140] Figure 2 FIG. 1 is a schematic diagram showing a network training method according to an embodiment of the present disclosure. Figure 2As shown, assuming that the target recognition network is M Ω , the network M Ω The parameter is Ω, and the training set and test set are and Among them, the training set For target recognition network M Ω Training and testing sets Used to train the target recognition network M Ω The recognition effect is verified.

[0141] Before steps S1 to S5, an initialization operation may be performed first.

[0142] In a possible implementation, the method further includes:

[0143] Initializing the first loss function according to the sampling distribution to obtain multiple initial first loss functions;

[0144] Performing random initialization according to the target recognition network to obtain multiple initial target recognition networks;

[0145] Among them, when n=1, the target recognition network trained in the n-1th round includes the initial multiple target recognition networks, and the multiple first loss functions in the n-1th round are the initial multiple first loss functions.

[0146] For example, when n=1, B target recognition networks can be randomly initialized Object recognition network trained in round 0 May include initial B target recognition networks And initialize the sampling distribution The sampling distribution of round 0 can be Perform sampling and sample the hyperparameters Θ1~Θ of the first loss function of group B B , used to initialize the hyperparameters of the B first loss functions of the 0th round, and obtain the B first loss functions of the 0th round

[0147] It should be understood that the present disclosure does not limit the number B of the first loss function and the value of the corresponding initial target recognition network data B.

[0148] In this way, the initialization of the 0th round can be completed, which is convenient for the subsequent iterative training of the target recognition network to obtain a trained target recognition network.

[0149] After the initialization operation, in step S1, for the target recognition network M Ω The first sample image in the training set is input into the target recognition network in the n-1th round of training. The target recognition network is used to perform target recognition on the image to be processed, and n is a positive integer.

[0150] In step S2, according to the annotation result and the first recognition result of the first sample image, the B first loss functions of the n-1th round are used, that is, For the target recognition network trained in the n-1th round Train and obtain B trained target recognition networks of the nth intermediate state, namely:

[0151] For example, the target recognition network trained in round n-1 can be (like Figure 2 20) into B target recognition networks (like Figure 2 21).

[0152] According to the annotation results of the first sample image and the target recognition network The first recognition result of the first sample image is obtained by passing the first loss function through n-1 rounds Target recognition network (like Figure 2 211) to obtain the target recognition network of the 1st nth intermediate state (like Figure 2 221 in the figure);

[0153] According to the annotation results of the first sample image and the target recognition network The first recognition result of the first sample image is obtained by passing the first loss function through n-1 rounds Target recognition network (like Figure 2 212) to obtain the target recognition network of the second nth intermediate state (like Figure 2 222);

[0154] Similarly, according to the annotation results of the first sample image and the target recognition network The first recognition result of the first sample image is obtained by passing the first loss function through n-1 rounds Target recognition network (like Figure 2 213) to obtain the target recognition network of the Bth nth intermediate state (like Figure 2 223).

[0155] It should be understood that the B first loss functions through the n-1th round For the target recognition network trained in round n-1 The training process can be performed asynchronously and in parallel to improve training efficiency.

[0156] In step S3, the target recognition network is constructed based on the B n-th intermediate states. Determine the target recognition network for the nth round of training

[0157] In a possible implementation, step S3 may include:

[0158] For any one of the plurality of n-th intermediate state object recognition networks, inputting a second sample image in a test set into the n-th intermediate state object recognition network for processing to obtain a second recognition result of the second sample image;

[0159] Determining target recognition network feedback information of the nth intermediate state according to the labeling result of the second sample image and the second recognition result;

[0160] According to the feedback information of the target recognition networks in the n-th intermediate states, the target recognition network for the n-th round of training is determined from the target recognition networks in the n-th intermediate states.

[0161] For example, based on the target recognition network of B n-th intermediate states You can focus your tests on The second sample image in is input into the target recognition network of B n-th intermediate states respectively. Processing, get B n-th intermediate state target recognition network A second recognition result of the second sample image.

[0162] Among them, the test set The second sample image in may include the annotation result.

[0163] Identify the network based on the target The second recognition result of the second sample image and the labeling result of the second sample image can determine the target recognition network of the nth intermediate state Report Information like Figure 2 As shown in 231;

[0164] Identify the network based on the target The second recognition result of the second sample image and the labeling result of the second sample image can determine the target recognition network of the nth intermediate state Report Information like Figure 2 As stated in 232;

[0165] By analogy, according to the target recognition network The second recognition result of the second sample image and the labeling result of the second sample image can determine the target recognition network of the nth intermediate state Report Information like Figure 2 As shown in 233.

[0166] The target recognition network can be based on B n-th intermediate states The Return Object recognition network from B n-th intermediate states In the above example, the B return The target recognition network in the nth intermediate state corresponding to the maximum return is determined as the target recognition network for the nth round of training. The formula is as follows:

[0167]

[0168] Through formula (24), the target recognition network of the i-th and n-th intermediate states can be Determine the target recognition network for the nth round of training like Figure 2 As shown in Figure 24.

[0169] In this way, in each round of training, an optimal n-th intermediate state target recognition network can be determined from multiple n-th intermediate state target recognition networks as the target recognition network to be trained in the next round, which helps to improve the accuracy of training.

[0170] In a possible implementation, step S4 may include:

[0171] According to the reward information of the target recognition network of the plurality of n-th intermediate states and the hyperparameter optimization function, updating the parameters of the sampling distribution of the first loss function;

[0172] According to the updated sampling distribution, the hyperparameters of the multiple first loss functions of the (n-1)th round are updated respectively to obtain the multiple first loss functions of the nth round.

[0173] For example, the hyperparameter optimization function of the first loss function can be expressed as:

[0174]

[0175] In formula (25), p(Θ b ) is the sampling distribution of the n-1th round The probability density function of is a standardized return, such as a standardized return with zero mean, Θ = [a0, a1, ..., a M ] T .

[0176] The parameter μ of the sampling distribution of the n-1th round can be updated by formula (1): n-1 , get the sampling distribution of the nth round The standard deviation parameter σ of the sampling distribution is a preset fixed value, and the present disclosure does not limit the specific value range of σ.

[0177] The sampling distribution of the nth round can be Perform sampling processing to sample the hyperparameters of the first loss function of group B, which are used to update the B first loss functions of the n-1th round The hyperparameters of the nth round are obtained by B first loss functions

[0178] Figure 3 FIG. 5 is a schematic diagram showing a method for determining the hyperparameters of the first loss function according to the sampling distribution according to an embodiment of the present disclosure. Figure 3 As shown, the broken line 31 represents the linear piecewise function t(x) of the first loss function, which may include 4 (M=4) segmented subintervals and 5 hyperparameters a0-a4. In the above example, a4 is sampled, μ4 can be determined according to formula (25), σ is a preset fixed value, and the hyperparameter a4 can be randomly sampled within the truncation interval 0 to 1 of the sampling distribution, and the sampled value is used as the value of the hyperparameter a4.

[0179] According to formula (18), we can substitute i=3 and M=4 into We can get the sampling distribution of the hyperparameter a3 as μ3=min(a4,0.5), and then determine the sampling distribution of a3 like Figure 3 As shown in 32. By sampling distribution Sampling is performed to determine the value of the hyperparameter a3.

[0180] Next, we can substitute i=2, M=4 into We can get the sampling distribution of the hyperparameter a2 as μ2=min(a3,0), and then determine the sampling distribution of a2 By sampling the distribution Sampling is performed to determine the value of the hyperparameter a2.

[0181] Substitute i=1, M=4 into We can get the sampling distribution of the hyperparameter a1 as μ1=min(a2,-0.5), and then determine the sampling distribution of a1 like Figure 3As shown in 33. By sampling distribution Sampling is performed to determine the value of the hyperparameter a1.

[0182] Finally, substitute i=0, M=4 into We can get the sampling distribution of the hyperparameter a0 as μ0=min(a1,-1), and then determine the sampling distribution of a0 By sampling the distribution Sampling is performed to determine the value of the hyperparameter a0.

[0183] Among them, the above method completes the update of the hyperparameters of a first loss function, and the hyperparameters of multiple first loss functions can be sampled asynchronously and in parallel according to the above method.

[0184] analyze Figure 3 The linear piecewise function t(x), x∈[-1,1] represented by the broken line 31 shows that t′(x)≥0, t′(x)≠constant, t(x)≤x and t(1)≥0, which satisfies the first constraint. Therefore, the first loss function after updating the hyperparameters can have better performance.

[0185] In this way, based on the sampling distribution of the non-independent truncated Gaussian distribution, the first loss function can be automatically designed without human intervention, and the hyperparameters of the first loss function can be dynamically adjusted during training to improve the training effect of the first loss function on the target recognition network.

[0186] In step S5, when the training conditions are met, the trained target recognition network is obtained. The situation where the training conditions are met may include until the number of iterations reaches a preset number, or until the performance of the obtained target recognition network meets the expected performance, etc. The present disclosure does not limit the specific conditions for stopping iteration.

[0187] In summary, the goal of searching the first loss function (i.e., sampling the hyperparameters of the first loss function multiple times) is to obtain the optimal hyperparameters of the first loss function. Make In the test set Return on The maximum, while the target recognition network M Ω The parameter Ω is obtained by minimizing the current Θ = [a0, a1, ..., a M ] T The first loss function under Right now:

[0188]

[0189] In formula (26), Ω *Represents the current first loss function for the target recognition network M Ω Train and obtain the trained target recognition network M Ω The optimal parameters of The first sample image, y represents the training set The first recognition result of the first sample image. This method does not keep a fixed loss function hyperparameter Θ = [a0, a1, ..., a M ] T Instead, we can adopt a method to dynamically adjust the hyperparameters of the first loss function during training.

[0190] In this way, in each round of training, the sampling distribution based on the non-independent truncated Gaussian distribution can be sampled and processed to determine multiple first loss functions, so that the first loss function can be variable during the training process, and the first loss function does not need to be manually designed, and the corresponding search space is not only large but also more stable. Furthermore, the target recognition network finally determined has better performance, which can greatly improve the deployment efficiency of the target recognition network.

[0191] In a possible implementation, experimental comparisons can be performed on four datasets (including Market1501, DukeMTMC, CUHK03, and MSMT17). The results on each dataset are shown in Table 1.

[0192] Table 1 Comparison of the disclosed method with other methods on four data sets.

[0193]

[0194] In Table 1, “R1” represents the accuracy of the first recognition result obtained by recognizing the images in the dataset, and “mAP” represents the average accuracy of the recognition results obtained by recognizing the images in the dataset.

[0195] In one possible implementation, the present disclosure also provides a target recognition method, which includes: inputting an image to be processed into a target recognition network for processing, and obtaining a recognition result of the image to be processed, wherein the recognition result includes a target area in the image to be processed, wherein the target recognition network is trained according to the network training method described above, and the target may include, for example, a human body, a face, a vehicle, an object, etc.

[0196] Figure 4 A schematic diagram showing the target recognition effect of the target recognition method according to an embodiment of the present disclosure. Figure 4 As shown in Figure 2, taking pedestrian re-identification as an example, Figure 4The white solid line box in the middle is the object to be identified, the first and second rows are the recognition results of the AM-LFS method and the Search-Softmax method in the related art, respectively, and the third row is the recognition result of the target recognition method disclosed in the present invention. The white dotted line box in the figure indicates that the label of the recognition result is different from the label of the object to be identified. Compared with the AM-LFS method and the Search-Softmax method, the target recognition accuracy of the disclosed method is higher.

[0197] For the AM-LFS method, it approximates t(x) as a piecewise linear function, and each segment independently maintains two parameters. For example, assuming that the piecewise linear function is divided into M segments, 2M parameters are required. This search space is very large, so the final search results are not very stable. Search-Softmax greatly simplifies this search space, and only one parameter needs to be optimized, but such a search space is too simple, so the form of the loss function contained in this search space is too single. The linear piecewise function t(x) of the first loss function of the method disclosed in the present invention can be divided into M segments, and only M+1 parameters are required. On the basis of ensuring that the search space is large enough, the stability of the search space can also be guaranteed. The target recognition network trained by the first loss function can obtain better performance.

[0198] In this way, the method can realize the re-identification of pedestrians in the processed image, improving the accuracy of pedestrian re-identification. Moreover, it can automatically design the loss function without human intervention, greatly improving the deployment efficiency of the pedestrian re-identification deployment system.

[0199] Therefore, according to the network training method of the embodiment of the present disclosure, the first loss function can be automatically obtained without human intervention, and the target recognition network can be trained using the automatically obtained loss function, and the hyperparameters of the first loss function can be dynamically adjusted during training, which can improve the training efficiency of the target recognition network and the deployment efficiency of the target recognition deployment system. Moreover, the trained target recognition network obtained by the network training method has excellent performance and can accurately identify the target.

[0200] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0201] In addition, the present disclosure also provides an image processing device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any image processing method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0202] Figure 5 A block diagram of a network training device according to an embodiment of the present disclosure is shown as follows: Figure 5 As shown, the device comprises:

[0203] A result acquisition module 51 is used for inputting the first sample image in the training set into the target recognition network trained in the n-1th round for processing, and obtaining a first recognition result of the first sample image, wherein the target recognition network is used to perform target recognition on the image to be processed, and n is a positive integer;

[0204] A network training module 52 is used to train the target recognition network trained in the n-1th round respectively according to the labeling result of the first sample image and the first recognition result through multiple first loss functions of the n-1th round to obtain multiple target recognition networks in the nth intermediate state;

[0205] A first network determination module 53, configured to determine the target recognition network for the nth round of training according to the plurality of target recognition networks in the nth intermediate state;

[0206] A loss function acquisition module 54, configured to update the hyperparameters of the multiple first loss functions of the n-1th round according to the hyperparameter optimization function and the sampling distribution of the first loss function, and obtain the multiple first loss functions of the nth round;

[0207] A second network determination module 55 is used to obtain a trained target recognition network when the training conditions are met;

[0208] Among them, the first loss function includes a linear piecewise function, the linear piecewise function includes M piecewise subintervals and M+1 hyperparameters, the domain of the linear piecewise function ranges from -1 to 1, and M is an integer greater than 1.

[0209] In one possible implementation, the device also includes an initialization module: used to initialize the first loss function according to the sampling distribution to obtain an initial multiple first loss function; randomly initialize according to the target recognition network to obtain an initial multiple target recognition network; wherein, when n=1, the target recognition network trained in the n-1th round includes the initial multiple target recognition networks, and the multiple first loss functions of the n-1th round are the initial multiple first loss functions.

[0210] In one possible implementation, the first network determination module 53 includes: for any one of the target recognition networks of the multiple n-th intermediate states, inputting the second sample image in the test set into the target recognition network of the n-th intermediate state for processing to obtain a second recognition result of the second sample image; determining the return information of the target recognition network of the n-th intermediate state according to the labeling result of the second sample image and the second recognition result; and determining the target recognition network for the n-th round of training from the target recognition networks of the multiple n-th intermediate states according to the return information of the target recognition networks of the multiple n-th intermediate states.

[0211] In one possible implementation, the loss function acquisition module 54 includes: updating the parameters of the sampling distribution of the first loss function according to the reward information of the target recognition network of the multiple n-th intermediate states and the hyperparameter optimization function; and updating the hyperparameters of the multiple first loss functions of the n-1th round respectively according to the updated sampling distribution to obtain the multiple first loss functions of the nth round.

[0212] In a possible implementation, the linear piecewise function satisfies a first constraint, which includes: the first-order derivative of the linear piecewise function is greater than or equal to 0; the first-order derivative of the linear piecewise function is not a fixed value; when the independent variable is x, the function value of the linear piecewise function is less than or equal to the independent variable x; and when the independent variable is 1, the function value of the linear piecewise function is greater than or equal to 0. The M+1 hyperparameters of the linear piecewise function satisfy a second constraint, which includes: the jth hyperparameter is less than or equal to the j+1th hyperparameter, the M+1th hyperparameter is greater than or equal to 0 and less than or equal to 1; and the jth hyperparameter is less than or equal to the right endpoint of the jth segmented subregion, 1≤j≤M.

[0213] In one possible implementation, an embodiment of the present disclosure provides a target recognition device, which includes: a processing module, used to process the image to be processed according to the recognition network trained by the network training device described above, to obtain the target area in the image to be processed.

[0214] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0215] The embodiment of the present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0216] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0217] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0218] Figure 6 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0219] Reference Figure 6 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0220] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0221] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0222] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0223] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0224] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0225] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0226] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0227] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G), a third generation mobile communication technology (3G), a fourth generation mobile communication technology (4G) or a fifth generation mobile communication technology (5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0228] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0229] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above method.

[0230] Figure 71 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Figure 7 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0231] The electronic device 1900 may further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Microsoft's server operating system (Windows ServerTM), Apple's graphical user interface-based operating system (Mac OSXTM), a multi-user multi-process computer operating system (UnixTM), a free and open source Unix-like operating system (LinuxTM), an open source Unix-like operating system (FreeBSDTM), or the like.

[0232] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0233] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0234] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0235] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0236] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0237] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0238] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0239] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0240] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0241] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0242] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A network training method, characterized in that: The method comprises: For the nth round of training of the target recognition network, the first sample image in the training set is input into the target recognition network of the n-1th round of training for processing, and a first recognition result of the first sample image is obtained, and the target recognition network is used to perform target recognition on the image to be processed, where n is a positive integer; According to the labeling result of the first sample image and the first recognition result, the target recognition network trained in the n-1th round is trained respectively by using multiple first loss functions in the n-1th round to obtain multiple target recognition networks in the nth intermediate state; Determining a target recognition network for an n-th round of training according to the plurality of target recognition networks in the n-th intermediate states; According to the hyperparameter optimization function and sampling distribution of the first loss function, the hyperparameters of the multiple first loss functions of the n-1th round are updated to obtain the multiple first loss functions of the nth round; When the training conditions are met, a trained target recognition network is obtained; Among them, the first loss function includes a linear piecewise function, the linear piecewise function includes M piecewise sub-intervals and M+1 hyperparameters, the domain of the linear piecewise function ranges from -1 to 1, M is an integer greater than 1, and the linear piecewise function satisfies a first constraint condition, and the first constraint condition includes: the first-order derivative of the linear piecewise function is greater than or equal to 0; the first-order derivative of the linear piecewise function is not a fixed value; when the independent variable is x, the function value of the linear piecewise function is less than or equal to the independent variable x; and when the independent variable is 1, the function value of the linear piecewise function is greater than or equal to 0.

2. The method according to claim 1, characterized in that The method further comprises: Initializing the first loss function according to the sampling distribution to obtain multiple initial first loss functions; Performing random initialization according to the target recognition network to obtain multiple initial target recognition networks; Among them, when n=1, the target recognition network trained in the n-1th round includes the initial multiple target recognition networks, and the multiple first loss functions in the n-1th round are the initial multiple first loss functions.

3. The method according to claim 1, characterized in that The step of determining the target recognition network for the nth round of training according to the target recognition networks in the nth intermediate states includes: For any one of the plurality of n-th intermediate state object recognition networks, inputting a second sample image in a test set into the n-th intermediate state object recognition network for processing to obtain a second recognition result of the second sample image; Determining target recognition network feedback information of the nth intermediate state according to the labeling result of the second sample image and the second recognition result; According to the feedback information of the target recognition networks in the n-th intermediate states, the target recognition network for the n-th round of training is determined from the target recognition networks in the n-th intermediate states.

4. The method according to claim 3, characterized in that According to the hyperparameter optimization function and the sampling distribution of the first loss function, the hyperparameters of the multiple first loss functions of the n-1th round are updated to obtain the multiple first loss functions of the nth round, including: According to the reward information of the target recognition network of the plurality of n-th intermediate states and the hyperparameter optimization function, updating the parameters of the sampling distribution of the first loss function; According to the updated sampling distribution, the hyperparameters of the multiple first loss functions of the (n-1)th round are updated respectively to obtain the multiple first loss functions of the nth round.

5. The method according to claim 1, characterized in that: in, The M+1 hyperparameters of the linear piecewise function satisfy the second constraint, which includes: the j-th hyperparameter is less than or equal to the j+1-th hyperparameter, the M+1-th hyperparameter is greater than or equal to 0 and less than or equal to 1; and the j-th hyperparameter is less than or equal to the right endpoint of the j-th segmented sub-region, 1≤j≤M.

6. A target recognition method, characterized in that: The method comprises: The image to be processed is input into the target recognition network for processing to obtain a recognition result of the image to be processed, wherein the recognition result includes a target area in the image to be processed. Wherein, the target recognition network is trained according to the network training method according to any one of claims 1-5.

7. A network training device, characterized in that: The device comprises: A result acquisition module is used for inputting the first sample image in the training set into the target recognition network trained in the n-1th round for processing, and obtaining a first recognition result of the first sample image, wherein the target recognition network is used to perform target recognition on the image to be processed, and n is a positive integer; A network training module, configured to train the target recognition network trained in the n-1th round respectively according to the labeling result of the first sample image and the first recognition result through multiple first loss functions of the n-1th round, so as to obtain multiple target recognition networks in the nth intermediate state; A first network determination module, used to determine the target recognition network for the nth round of training according to the plurality of target recognition networks in the nth intermediate state; A loss function acquisition module, used to update the hyperparameters of the multiple first loss functions of the n-1th round according to the hyperparameter optimization function and sampling distribution of the first loss function, and obtain the multiple first loss functions of the nth round; A second network determination module is used to obtain a trained target recognition network when the training conditions are met; Among them, the first loss function includes a linear piecewise function, the linear piecewise function includes M piecewise sub-intervals and M+1 hyperparameters, the domain of the linear piecewise function ranges from -1 to 1, M is an integer greater than 1, and the linear piecewise function satisfies a first constraint condition, and the first constraint condition includes: the first-order derivative of the linear piecewise function is greater than or equal to 0; the first-order derivative of the linear piecewise function is not a fixed value; when the independent variable is x, the function value of the linear piecewise function is less than or equal to the independent variable x; and when the independent variable is 1, the function value of the linear piecewise function is greater than or equal to 0.

8. A target recognition device, characterized in that: The device comprises: The processing module is used to process the image to be processed according to the target recognition network trained by the network training device of claim 7 to obtain the target area in the image to be processed.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images