Target detection model processing method and device, equipment and medium

Abstract

The embodiment of the invention provides a target detection model processing method and device, equipment and a medium. A network structure searching method comprises the steps that an image data set is acquired; data groups in the image data set comprise original image data, noise data and mixed image data; according to a first network corresponding to a target detection model, performing forward operation and reverse operation on a data group in the image data set to obtain a corresponding forward operation result and a corresponding reverse operation result; wherein one data group corresponds to one forward operation and one reverse operation; determining a score corresponding to the first network according to the forward operation result and the reverse operation result; determining a target network from a plurality of networks corresponding to the target detection model according to the score corresponding to the first network; the network comprises a first network. The embodiment of the invention can improve the search efficiency of the network structure.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer vision technology, and in particular, to a method, device, equipment and medium for processing a target detection model. Background Art

[0002] Object detection is a computer vision technology based on machine learning. It is used to complete tasks such as counting people and tracking pedestrians by analyzing pedestrians, vehicles and other moving objects in the detection scene. Neural Architecture Search (NAS) is used to search for a network structure with better performance from the search space, thereby obtaining the network structure of the object detection model.

[0003] Current network structure search methods usually perform iterative training on each subnet included in the search space to obtain a score for each subnet; and based on the score of each subnet, the optimal subnet is selected from the subnets included in the search space. The optimal subnet can be used as the network structure of the target detection model.

[0004] In practical applications, iterative training is performed on each subnet included in the search space, and the weight parameters of each subnet need to be continuously adjusted until the iteration end condition is met; therefore, the above iterative training takes a lot of time, resulting in low search efficiency of the network structure. Summary of the invention

[0005] An embodiment of the present application provides a network structure search method for a target detection model, which can improve the search efficiency of the network structure.

[0006] Correspondingly, an embodiment of the present application also provides a network structure search device for a target detection model, an electronic device and a machine-readable medium to ensure the implementation and application of the above method.

[0007] In order to solve the above problems, the present application discloses a network structure search method for a target detection model, the method comprising:

[0008] Acquire an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data;

[0009] According to the first network corresponding to the target detection model, a forward operation and a reverse operation are performed on the data groups in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation;

[0010] Determining a score corresponding to the first network according to the forward operation result and the reverse operation result;

[0011] According to the score corresponding to the first network, a target network is determined from multiple networks corresponding to the target detection model; the network includes: a first network.

[0012] In order to solve the above problems, the present application discloses a method for training a target detection model, the method comprising:

[0013] Determine a target network from multiple networks corresponding to the target detection model;

[0014] Training the target network according to the second training data;

[0015] Wherein, determining the target network from the multiple networks corresponding to the target detection model includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, performing forward operation and reverse operation on the data group in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, determining the score corresponding to the first network; according to the score corresponding to the first network, determining the target network from the multiple networks corresponding to the target detection model; the network includes: a first network.

[0016] In order to solve the above problems, an embodiment of the present application discloses a target detection method, which includes:

[0017] receiving an input image;

[0018] Determine the detection result corresponding to the input image by using the trained target network;

[0019] Output the test results;

[0020] Among them, the determination process of the target network includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, the data group in the image data set is subjected to forward operation and reverse operation to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, the score corresponding to the first network is determined; according to the score corresponding to the first network, the target network is determined from the multiple networks corresponding to the target detection model; the network includes: a first network.

[0021] In order to solve the above problems, the embodiment of the present application discloses a network structure search device for a target detection model, the device comprising:

[0022] An acquisition module, used for acquiring an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data;

[0023] A forward and reverse operation module, used to perform forward operation and reverse operation on the data groups in the image data set according to the first network corresponding to the target detection model, so as to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation;

[0024] A network score determination module, used to determine a score corresponding to the first network according to the forward operation result and the reverse operation result;

[0025] A target network determination module is used to determine a target network from a plurality of networks corresponding to the target detection model according to a score corresponding to the first network; the network includes: a first network.

[0026] Optionally, the first network includes: a plurality of network units, the network units including: at least one network node;

[0027] The network score determination module comprises:

[0028] A unit score determination module, used to determine a score corresponding to a network unit according to a forward operation result and a reverse operation result corresponding to a network node in a network unit;

[0029] The weighted processing module is used to perform weighted processing on the scores corresponding to the multiple network units to obtain the score corresponding to the first network.

[0030] Optionally, the unit score determination module includes:

[0031] A first processing module, configured to perform a first processing on a forward operation result corresponding to a network node in a network unit to obtain a first processing result;

[0032] A second processing module, used for performing a second processing on a reverse operation result corresponding to a network node in a network unit to obtain a second processing result;

[0033] A fusion processing module is used to fuse the first processing result and the second processing result to obtain a score corresponding to the network unit.

[0034] Optionally, the forward operation result and the reverse operation result corresponding to the one network node include: a forward operation result corresponding to the one network node under the condition of mixed image data, and a reverse operation result corresponding to the one network node under the condition of original image data.

[0035] Optionally, the network further includes: a second network; and the target network determination module includes:

[0036] A training data determination module, used to determine first training data; the first training data includes: a code and a score corresponding to the first network;

[0037] A training module, used to train a score prediction model according to the first training data; the score prediction model is used to characterize the mapping relationship between the code corresponding to the network and the score;

[0038] A prediction module, used for inputting the code corresponding to the second network into the score prediction model to obtain the score corresponding to the second network;

[0039] The first target network determination module is used to determine a target network from a plurality of first networks and a plurality of second networks corresponding to the target detection model according to the scores corresponding to the first network and the second network respectively.

[0040] Optionally, the target network determination module includes:

[0041] A selection module, configured to select a plurality of candidate networks from the plurality of networks according to the time consumption information respectively corresponding to the plurality of networks;

[0042] A target candidate network determination module, configured to determine a target candidate network from the multiple candidate networks according to scores corresponding to the candidate networks;

[0043] The evolutionary processing module is used to process the target candidate network using an evolutionary method to obtain a target network.

[0044] Optionally, the target candidate network determination module includes:

[0045] A first target candidate network determination module is configured to select, in ascending order of time consumption information, a plurality of candidate networks whose time consumption information does not exceed a preset time length from the plurality of networks; or

[0046] The second target candidate network determination module is used to determine a plurality of time-consuming intervals that do not exceed a preset time length, and select a plurality of candidate networks whose time-consuming information is respectively within the plurality of time-consuming intervals from the plurality of networks.

[0047] In order to solve the above problems, the present application discloses a training device for a target detection model, the device comprising:

[0048] A determination module, used to determine a target network from multiple networks corresponding to the target detection model;

[0049] A training module, used for training the target network according to the second training data;

[0050] Wherein, determining the target network from the multiple networks corresponding to the target detection model includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, performing forward operation and reverse operation on the data group in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, determining the score corresponding to the first network; according to the score corresponding to the first network, determining the target network from the multiple networks corresponding to the target detection model; the network includes: a first network.

[0051] In order to solve the above problems, an embodiment of the present application discloses a target detection device, the device comprising:

[0052] A receiving module, used for receiving an input image;

[0053] A detection module, used to determine a detection result corresponding to the input image using the trained target network;

[0054] An output module, used for outputting the detection results;

[0055] Among them, the determination process of the target network includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, the data group in the image data set is subjected to forward operation and reverse operation to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, the score corresponding to the first network is determined; according to the score corresponding to the first network, the target network is determined from the multiple networks corresponding to the target detection model; the network includes: a first network.

[0056] The embodiment of the present application further discloses an electronic device, including: a processor; and a memory, on which executable code is stored. When the executable code is executed, the processor executes the method described in the embodiment of the present application.

[0057] The embodiment of the present application further discloses a machine-readable medium on which executable code is stored. When the executable code is executed, a processor executes the method described in the embodiment of the present application.

[0058] The embodiments of the present application include the following advantages:

[0059] In an embodiment of the present application, a first network corresponding to a target detection model is first used to perform a forward operation and a reverse operation on a data group in an image data set to obtain a forward operation result and a reverse operation result corresponding to the data group; then, a score corresponding to the first network is determined based on the forward operation result and the reverse operation result corresponding to the data group; then, a target network is determined from multiple networks corresponding to the target detection model based on the score corresponding to the first network.

[0060] The embodiment of the present application determines the score corresponding to the first network based on one forward operation and one reverse operation corresponding to one data group. Since iterative training usually requires multiple operations for one type of data, and one forward operation and one reverse operation in the embodiment of the present application can reduce the number of operations, the embodiment of the present application can improve the search efficiency of the network structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 is a schematic diagram of the structure of a target detection model according to an embodiment of the present application;

[0062] Figure 2 This is a schematic diagram of the steps of a network structure search method for a target detection model according to an embodiment of the present application;

[0063] Figure 3 is a schematic diagram of the structure of an initial network according to an embodiment of the present application;

[0064] Figure 4 is a schematic diagram of a score determination process corresponding to a network unit according to an embodiment of the present application;

[0065] FIG5( a ) is an example corresponding to selection method 1 of an embodiment of the present application;

[0066] FIG5( b ) is an example corresponding to selection method 2 of an embodiment of the present application;

[0067] Figure 6 This is a schematic diagram of the steps of a network structure search method for a target detection model according to an embodiment of the present application;

[0068] Figure 7 It is a schematic diagram of the steps of a method for training a target detection model according to an embodiment of the present application;

[0069] Figure 8 This is a schematic diagram of the steps of a target detection method according to an embodiment of the present application;

[0070] Fig. 9 It is a structural diagram of a network structure search device for a target detection model according to an embodiment of the present application;

[0071] Fig.10 It is a structural schematic diagram of a training device for a target detection model according to an embodiment of the present application;

[0072] Fig.11 is a schematic structural diagram of a target detection device according to an embodiment of the present application;

[0073] Fig.12 It is a schematic diagram of the structure of a device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0075] The embodiments of the present application can be applied to target detection scenarios. In target detection scenarios, the target detection model can extract feature representations of the input image, and detect whether the input image contains targets such as pedestrians based on the feature representations. If so, the location information of the targets such as pedestrians can be given. Targets can include moving objects such as pedestrians, vehicles, and animals. It can be understood that the embodiments of the present application do not limit the specific targets to be detected.

[0076] The embodiment of the present application can train the mathematical model to obtain the target detection model. A mathematical model is a scientific or engineering model constructed using mathematical logic methods and mathematical language. A mathematical model is a mathematical structure that is generally or approximately expressed using mathematical language with reference to the characteristics or quantitative dependencies of a certain system of things. This mathematical structure is a relational structure depicted with the help of mathematical symbols. Among them, machine learning, deep learning methods and other methods can be used to train the mathematical model. The machine learning method may include: linear regression, decision tree, random forest, etc. The deep learning method may include: CNN (Convolutional Neural Networks), LSTM (Long Short-Term Memory Network), GRU (Gated Recurrent Unit), etc.

[0077] Reference Figure 1 , shows a schematic diagram of the structure of a target detection model of an embodiment of the present application, and the target detection model may specifically include: a feature extraction unit 101, a feature fusion unit 102 and a detection unit 103.

[0078] Among them, the feature extraction unit 101 can be used to extract features from the input image. The feature extraction unit 101 can be used to receive an input image and extract image features of the image from the input image. The image features can refer to deep image features. The feature extraction unit 101 can be a backbone network, and the network types corresponding to the feature extraction unit 101 can include: VGG (Visual Geometry Group Network), ResNet (Residual Network), lightweight network (mobilenet), etc. It can be understood that the embodiment of the present application does not limit the specific network type corresponding to the feature extraction unit 101.

[0079] A convolutional network can be a deep feedforward artificial neural network that has good performance in image recognition. A convolutional network can specifically include a convolutional layer and a pooling layer. The convolutional layer is used to automatically extract features from the input image to obtain a feature map. The pooling layer is used to perform pooling processing on the feature map to reduce the number of features in the feature map. The pooling processing of the pooling layer can include: maximum pooling, average pooling, or random pooling, etc., and the appropriate method can be selected according to actual needs.

[0080] The feature fusion unit 102 is a connecting unit in the target detection model, which can fuse the image features extracted by the feature extraction unit 101 to obtain fused image features, which can improve the diversity of features and the performance of the target detection model.

[0081] The detection unit 103 is used to perform target detection according to the fused image features to obtain a detection result. Specifically, the detection unit 103 can detect whether the input image contains a target and output a corresponding detection result.

[0082] In response to the technical problem that related technologies iteratively train each subnet included in the search space, resulting in low search efficiency of the network structure, an embodiment of the present application provides a network structure search method for a target detection model, which method may specifically include: obtaining an image data set; the data group in the image data set specifically includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to a first network corresponding to the target detection model, performing forward operation and reverse operation on the data group in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, determining the score corresponding to the first network; according to the score corresponding to the first network, determining a target network from multiple networks corresponding to the target detection model; the network specifically includes: a first network.

[0083] The embodiment of the present application first uses the first network corresponding to the target detection model to perform a forward operation and a reverse operation on a data group in the image data set to obtain the forward operation result and the reverse operation result corresponding to the data group; then, according to the forward operation result and the reverse operation result corresponding to the data group, the score corresponding to the first network is determined; then, according to the score corresponding to the first network, the target network is determined from the multiple networks corresponding to the target detection model.

[0084] The embodiment of the present application determines the score corresponding to the first network based on one forward operation and one reverse operation corresponding to a data set. Since the embodiment of the present application can save millions of forward and reverse propagation operations included in the iterative training of traditional technologies; therefore, the embodiment of the present application can improve the search efficiency of the network structure.

[0085] Method Example 1

[0086] Reference Figure 2 , shows a schematic flow chart of the steps of a network structure search method for a target detection model according to an embodiment of the present application, and the method may specifically include the following steps:

[0087] Step 201, obtaining an image data set; the data group in the image data set may specifically include: original image data, noise data, and mixed image data; the mixed image data may be a mixture of the original image data and the noise data;

[0088] Step 202: According to the first network corresponding to the target detection model, a forward operation and a reverse operation are performed on the data groups in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation;

[0089] Step 203: Determine a score corresponding to the first network according to the forward operation result and the reverse operation result;

[0090] Step 204: determine a target network from multiple networks corresponding to the target detection model according to the score corresponding to the first network; the network may specifically include: the first network.

[0091] In step 201, raw image data for target detection may be obtained. For example, the raw image data may include the raw image and a label corresponding to the raw image. The raw image may include targets such as pedestrians, vehicles, and animals, or the raw image may not include a target. The label corresponding to the raw image may represent the correct output of the target detection model for the raw image.

[0092] The label may include: region-level annotation information, such as the boundary information of the target region where the target is located in the original image. For example, when the target region is a rectangular region, the boundary information may be: the coordinates of the upper left corner and the lower right corner of the rectangular region. Of course, the label may include: pixel-level annotation information, such as the target to which the pixel in the original image belongs.

[0093] In the embodiment of the present application, noise data can be added to the original image data to obtain mixed image data, wherein the noise data can include: normally distributed noise with a mean of 0 and a variance of 1.

[0094] In an application example of the present application, a first preset number of original images and corresponding labels can be randomly acquired from a data set of a target detection scene, that is, the original image data can be obtained. For example, the acquisition factors of the original image data may include: illumination factors, scene factors, and target size factors, etc. The embodiment of the present application can randomly acquire 1024 original images and their corresponding labels from the data set according to the above acquisition factors, so that the original image data can cover the conditions corresponding to the above acquisition factors, and the original image data can be referred to as D. The embodiment of the present application can also construct noise data that matches the length and dimension of the original image data, and the noise data can obey the (0, 1) normal distribution, and the noise data can be referred to as N. The original image data D is fused with the corresponding noise data N to obtain mixed image data P.

[0095] In step 202, an initial network corresponding to the target detection model may be first constructed, and then a first network may be sampled from the initial network.

[0096] The initial network may include: networks corresponding to the feature extraction unit, the feature fusion unit and the detection unit respectively, a connection relationship between the feature extraction unit and the feature fusion unit, and a connection relationship between the feature fusion unit and the detection unit.

[0097] In a specific implementation, the feature extraction unit may include multiple stages, and the network structure parameters of each stage may include: stage parameters, operator parameters, channel amplification parameters, depth parameters, channel parameters, and convolution kernel parameters. Among them, the target network structure parameters such as depth parameters, channel parameters, and convolution kernel parameters are variable. Specifically, the target network structure parameters have a certain range, or the target network structure parameters may include multiple options.

[0098] In practical applications, the initial network may be a network corresponding to a case where the parameter value of the target network structure parameter is an upper limit value. Further, the first network may be sampled from the initial network by utilizing the variability of the target network structure parameter.

[0099] Referring to Table 1, a schematic diagram of the network structure parameters of the feature extraction unit of an embodiment of the present application is shown. Among them, the feature extraction unit may include: 7 stages, and the operator parameters corresponding to each stage may be preset operators. The preset operator may be determined by a person skilled in the art according to actual application requirements. For example, the preset operator may be an operator with a convolution function, and for another example, the preset operator may be an operator with a depthwise separable convolution function.

[0100] The channel amplification parameter may be a fixed value. The depth parameter, channel parameter and convolution kernel parameter may be variable. Among them, the channel parameter may have a corresponding range, for example, the range of the channel parameter of stage 2 may be [16,24]. The depth parameter and the convolution kernel parameter may have corresponding multiple options. For example, the multiple options of the convolution kernel parameter of stage 2 may include: 3, 5, 7, 9, which means that the operator of stage 2 may include 3 convolution kernels, or the operator of stage 2 may include 5 convolution kernels, or the operator of stage 2 may include 7 convolution kernels, or the operator of stage 2 may include 9 convolution kernels.

[0101] Table 1

[0102] Phase parameters Operator Parameters Channel Amplification Parameters Depth Parameters Channel parameters Convolution kernel parameters Phase 1 Preset Operators 1 1 16-24 3,5,7,9 Phase 2 Preset Operators 6 2,3 24-32 3,5,7,9 Phase 3 Preset Operators 6 2,3 32-48 3,5,7,9 Phase 4 Preset Operators 6 2,3,4 64-88 3,5,7,9 Phase 5 Preset Operators 6 2,3,4,5,6 96,128 3,5,7,9 Phase 6 Preset Operators 6 2,3,4,5,6 160-216 3,5,7,9 Phase 7 Preset Operators 1 1,2 320-352 3,5,7,9

[0103] The network structure parameters of the feature fusion unit may include: level parameters, depth parameters, convolution kernel parameters, etc. Among them, the depth parameters and convolution kernel parameters may have corresponding multiple options.

[0104] Referring to Table 2, a schematic diagram of the network structure parameters of the feature fusion unit of an embodiment of the present application is shown. Among them, the network structure parameters of the feature extraction unit may include: level parameters, depth parameters, and convolution kernel parameters. For example, multiple options for the convolution kernel parameters of level 2 may include: 3, 5, 7, 9, which means that the operator of level 2 may include 3 convolution kernels, or the operator of level 2 may include 5 convolution kernels, or the operator of level 2 may include 7 convolution kernels, or the operator of level 2 may include 9 convolution kernels.

[0105] Table 2

[0106]

[0107]

[0108] The network structure parameters of the detection unit may include: level parameters, depth parameters, convolution kernel parameters, etc. Among them, the depth parameters and convolution kernel parameters may have corresponding multiple options.

[0109] Referring to Table 3, a schematic diagram of the network structure parameters of the detection unit of an embodiment of the present application is shown. Among them, the network structure parameters of the feature extraction unit may include: identification parameters, depth parameters, and convolution kernel parameters. The depth parameter may indicate the number of network layers. For example, multiple options for the depth parameter of identification 2 may include: 1, 2, 3.

[0110] Table 3

[0111] Identification parameters Depth Parameters Convolution kernel parameters Detection unit 1 1 3,5,7,9 Detection unit 2 1,2,3 3,5,7,9 Detection unit 3 1,2,3 3,5,7,9 Detection unit 4 1,2,3 3,5,7,9 Detection unit 5 1,2,3 3,5,7,9

[0112] In practical applications, the feature extraction unit and the feature fusion unit can be connected using a fully connected topology; in this way, the diversity of the first network can be improved. The feature fusion unit and the detection unit can be connected one-to-one to reduce the amount of calculation of the detection unit. Of course, it is also feasible to connect the feature fusion unit and the detection unit one-to-many, many-to-one, or many-to-many.

[0113] Reference Figure 3 , shows a schematic diagram of the structure of the initial network of an embodiment of the present application, wherein the feature extraction units of stages 1 to 7 can be fully connected with the feature fusion units of levels 1 to 5, and the feature fusion units of levels 1 to 5 can be connected one by one with detection units 1 to detection units 5 to obtain the initial network.

[0114] In addition, the initial network may be a network corresponding to the case where the parameter value of the target network structure parameter is the upper limit value. Figure 3 The depth parameter of stage 2 included in the initial network can be 2, the channel parameter can be 32, and the convolution kernel parameter can be 9.

[0115] The process of sampling the first network from the initial network in the embodiment of the present application may specifically include: reducing the number of connections between the feature extraction unit and the feature fusion unit, and / or reducing the parameter value of the target network structure parameter of at least one of the feature extraction unit, the feature fusion unit and the detection unit to obtain the first network.

[0116] The sampling in the embodiment of the present application may be random sampling. Specifically, parameter values ​​of different target network structure parameters may be determined at the network layer to obtain the first network.

[0117] Specifically, for the feature extraction unit, a corresponding first search space can be set according to the restrictions of the target hardware on time-consuming information. The first search space can include: different convolution kernel parameters, different depth parameters, different channel parameters, etc.

[0118] For the feature fusion unit, a corresponding second search space can be set according to the restrictions of the target hardware on time-consuming information. The second search space may include: different convolution kernel parameters and different depth parameters, etc.

[0119] For the detection unit, a corresponding third search space can be set according to the restrictions of the target hardware on the time-consuming information. The third search space may include: different convolution kernel parameters and different depth parameters, etc.

[0120] The target hardware may be a hardware device for running the target detection model. The types of the target hardware may include: a PC (personal computer) or a mobile phone, etc. Of course, the target hardware may also be a server.

[0121] Optionally, the embodiment of the present application may also randomly select the number of connections of a feature extraction unit to a feature fusion unit to obtain a first network. Assuming that the number of feature fusion units connected to a feature extraction unit in the initial network is Q, the randomly selected number of connections may be within the range corresponding to [1, Q]. Figure 3 Taking the initial network shown as an example, a feature extraction unit is connected to 5 feature fusion units, and the number of randomly selected connections can be in the range corresponding to [1,5].

[0122] In the embodiment of the present application, M (M can be a positive integer) networks can be sampled from the initial network, and the M networks can include: a first network, or a first network and a second network. In the embodiment of the present application, the first network can be randomly obtained from the M networks, and the number of the first networks can be a second preset number, and the second preset number can be a value such as 5000.

[0123] The embodiment of the present application can use the first network to perform forward operation and reverse operation on the data groups in the image data set to obtain corresponding forward operation results and reverse operation results. Specifically, the three types of data included in each data group can be input into the first network respectively, and the first network outputs the corresponding forward operation results and reverse operation results.

[0124] Among them, the forward operation result corresponds to the forward propagation of data in the first network, which can be output feature information. The reverse operation result corresponds to the reverse propagation of data in the first network, which can be gradient information. In the forward propagation process, the data passes through the input layer and the hidden layer, and is processed layer by layer and transmitted to the output layer. If the expected output value is not obtained in the output layer, the loss information is determined according to the output value and the label, and the reverse propagation is turned on. The partial derivative of the loss function to each neuron weight is calculated layer by layer to form the gradient of the loss function to the weight vector.

[0125] The noise data of the embodiment of the present application may conform to a normal distribution, and the forward operation result and the reverse operation result obtained by the embodiment of the present application may also conform to a normal distribution. In this way, the embodiment of the present application can determine the score corresponding to the first network based on one forward operation and one reverse operation of a data group, and the score corresponding to the first network can indicate whether the network structure of the first network accurately restores the information contained in the original image.

[0126] In practical applications, the first network may include multiple network units, and the network unit may be a block. The above network unit may include: at least one network node. Then the forward operation result and the reverse operation result may include: the forward operation result and the reverse operation result corresponding to a network node in a network unit. The reverse operation result corresponding to a network node may be the gradient information of the loss function to the weight vector of the network node.

[0127] In an example of the present application, the network unit may include: an activation unit, a convolution unit, and a normalization unit.

[0128] The data group in the embodiment of the present application specifically includes: original image data D, noise data N, and mixed image data P. The forward operation result and reverse operation result corresponding to a network node may specifically include: the forward operation result and reverse operation result corresponding to a network node under the condition of original image data D, the forward operation result and reverse operation result corresponding to a network node under the condition of noise data N, and the forward operation result and reverse operation result corresponding to a network node under the condition of mixed image data P. In other words, a network node may correspond to 3 forward operation results and 3 reverse operation results.

[0129] In step 203, the process of determining the score corresponding to the first network based on the forward operation result and the reverse operation result may specifically include: determining the score corresponding to the network unit based on the forward operation result and the reverse operation result corresponding to a network node in a network unit; and performing weighted processing on the scores corresponding to multiple network units to obtain the score corresponding to the first network.

[0130] In practical applications, any one or combination of the three forward operation results and any one or combination of the three reverse operation results can be used to determine the score corresponding to a network unit. Optionally, the forward operation result and the reverse operation result used for the score corresponding to a network unit can include: the forward operation result corresponding to a network node under the condition of the mixed image data P, and the reverse operation result corresponding to a network node under the condition of the original image data.

[0131] Reference Figure 4 , showing a schematic diagram of the score determination process corresponding to the network unit of an embodiment of the present application. Among them, the network unit may include: an activation unit, a convolution unit and a normalization unit. T1, T2, T3, T4 respectively represent the input of the activation unit, the output of the activation unit (forward operation result), the output of the convolution unit (forward operation result) and the output of the normalization unit (forward operation result); T1G, T2G, T3G, T4G respectively represent the reverse operation results corresponding to the corresponding forward input in the network node.

[0132] The embodiment of the present application can determine the score corresponding to the network unit according to the forward operation result T3 corresponding to the normalization unit and the reverse operation result T4G corresponding to the normalization unit. The data corresponding to the forward operation result T3 can be the mixed image data P; the data corresponding to the reverse operation result T4G can be the original image data D, so that the influence of noise data on the reverse operation result T4G can be avoided.

[0133] The process of determining the score corresponding to the network unit in the embodiment of the present application may specifically include: performing a first processing on a forward operation result corresponding to a network node in a network unit to obtain a first processing result; performing a second processing on a reverse operation result corresponding to a network node in a network unit to obtain a second processing result; and fusing the first processing result and the second processing result to obtain a score corresponding to the network unit.

[0134] In a specific implementation, the first processing may specifically include: absolute value logarithm operation. The forward operation result may include: matrix elements. The absolute value logarithm operation may calculate the absolute value of the matrix elements and then take the logarithm. The second processing may include: positive and negative operations. When the element value of the matrix element is greater than 0, the second processing result obtained by the positive and negative operations may be 1; when the element value of the matrix element is equal to 0, the second processing result obtained by the positive and negative operations may be 0; when the element value of the matrix element is less than 0, the second processing result obtained by the positive and negative operations may be -1.

[0135] The above-mentioned fusion processing may specifically include: single-channel fusion or multi-channel fusion. In single-channel fusion, the element values ​​corresponding to the first processing result and the second processing result of a channel may be summed to obtain a single-channel result. Multi-channel fusion may perform weighted averaging on the single-channel results according to the weights corresponding to the multiple channels respectively to obtain a multi-channel result. In particular, the weights corresponding to the multiple channels may be determined by those skilled in the art according to actual application requirements. For example, the weights corresponding to the multiple channels may be the same; assuming the number of channels is T, the weight corresponding to one channel may be 1 / T.

[0136] Figure 4 The score of the network unit is determined according to the forward operation results and reverse operation results corresponding to the normalization unit. Since the normalization unit is located after the convolution unit, the score of the network unit can reflect the stability of the convolution unit.

[0137] In practical applications, the first network may include multiple network units. In the embodiment of the present application, weighted processing is performed on the scores corresponding to the multiple network units to obtain the score corresponding to the first network.

[0138] Since different network units in the first network may have different importance for the target detection task, different weights may be used for different network units. Optionally, the weight corresponding to the network unit may be a value determined based on a random number method. A number may be randomly selected from (0, 1) as the weight corresponding to the network unit.

[0139] In step 204, a target network may be determined from a plurality of networks corresponding to the target detection model according to the score corresponding to the first network.

[0140] In practical applications, the multiple networks may include: a first network. The score corresponding to the first network may be determined based on step 202 and step 203. In the embodiment of the present application, all the M networks sampled from the initial network may be used as the first network, and the score of the first network may be determined by using step 202 and step 203.

[0141] Alternatively, the multiple networks may include: a first network and a second network. The score of the second network may be determined based on the score of the first network. In the embodiment of the present application, a portion of the M networks sampled from the initial network may be first selected as the first network, and the score of the first network may be determined by using steps 202 and 203; then, the score of the second network may be determined based on the score of the first network.

[0142] The process of determining the score of the second network is provided here:

[0143] The network also includes: a second network; determining the target network from multiple networks corresponding to the target detection model includes: determining first training data; the first training data includes: the code and score corresponding to the first network; training a score prediction model based on the first training data; the score prediction model is used to characterize the mapping relationship between the code corresponding to the network and the score; the code corresponding to the second network is input into the score prediction model to obtain the score corresponding to the second network.

[0144] The embodiment of the present application can determine the encoding corresponding to the first network according to the network structure corresponding to the first network. For example, a binary number can be used to represent the encoding corresponding to the first network. It can be understood that the embodiment of the present application does not limit the specific encoding corresponding to the first network. Similarly, the embodiment of the present application can determine the encoding corresponding to the second network according to the network structure corresponding to the second network.

[0145] The score corresponding to the first network can be used as a label of the score prediction model, which can characterize the correct output of the score prediction model for the encoding corresponding to the first network; therefore, it can be used to determine the loss information of the score prediction model during the training process.

[0146] In a specific implementation, the score prediction model can be a fully connected network with S layers. The fully connected network may include: an input layer, a hidden layer, and an output layer. For example, S may be 3, and the number of neurons contained in the hidden layer may be: 200, 800, and 200, respectively. The second preset number of groups of first training data may be sent to the score prediction model so that the score prediction model has the ability to output a score according to the encoding corresponding to the second network.

[0147] Therefore, in the embodiment of the present application, when the multiple networks include only the first network, the target network can be determined from the multiple first networks corresponding to the target detection model according to the scores corresponding to the first network.

[0148] Alternatively, in the embodiment of the present application, when the multiple networks include a first network and a second network, the target network can be determined from the multiple first networks and multiple second networks corresponding to the target detection model according to the scores corresponding to the first network and the second network respectively.

[0149] The process of determining the target network from the multiple networks corresponding to the target detection model in the embodiment of the present application may specifically include:

[0150] Step S1, selecting multiple candidate networks from the multiple networks according to the time consumption information respectively corresponding to the multiple networks;

[0151] Step S2: determining a target candidate network from the multiple candidate networks according to the scores corresponding to the candidate networks;

[0152] Step S3: Using an evolutionary method, the target candidate network is processed to obtain a target network.

[0153] The time consumption information may refer to the time information consumed by a network for forward propagation (or forward propagation and backward propagation) of a type of data.

[0154] In a specific implementation, the selection method adopted by the above-mentioned selecting multiple candidate networks from the multiple networks may specifically include:

[0155] Selection method 1: selecting, in ascending order of time consumption information, multiple candidate networks whose time consumption information does not exceed a preset time length from the multiple networks; or

[0156] Selection method 2: determining multiple time-consuming intervals that do not exceed a preset time length, and selecting multiple candidate networks whose time-consuming information is respectively within the multiple time-consuming intervals from the multiple networks.

[0157] The preset time length can be determined by those skilled in the art according to actual application requirements. For example, the preset time length can be 100 ms.

[0158] 5(a) shows an example corresponding to selection method 1 of an embodiment of the present application. In this example, multiple candidate networks whose time consumption information does not exceed a preset time length can be selected from the multiple networks in ascending order of time consumption information, and the number of the multiple candidate networks can be a third preset number W.

[0159] Referring to Figure 5(b), an example corresponding to selection method 2 of an embodiment of the present application is shown. Among them, multiple time-consuming intervals that do not exceed a preset time length can be determined. In Figure 5(b), the number of time-consuming intervals can be 5. Multiple candidate networks whose time-consuming information is respectively located in the multiple time-consuming intervals can be selected from multiple networks. The number of candidate networks in a time-consuming interval can be W / 5, or a value close to W / 5. It can be understood that 5 is only an example of the number of time-consuming areas, and is not to be understood as a limitation on the number of time-consuming intervals in the embodiment of the present application. In fact, those skilled in the art can determine the number of time-consuming intervals according to actual application requirements.

[0160] Selection method 2 can, to a certain extent, avoid the concentration of candidate networks in a specific time-consuming interval, and can make the candidate networks and their network structures in each time-consuming interval have equal or identical probabilities to participate in the determination of the target network. Therefore, the embodiment of the present application can improve the corresponding time-consuming range of the target network, and thus can improve the performance of the target network.

[0161] According to the scores corresponding to the candidate networks, the process of determining the target candidate network from the multiple candidate networks may include: selecting a fourth preset number of target candidate networks from the multiple candidate networks in descending order of scores. The value of the fourth preset number can be determined by those skilled in the art according to actual application requirements.

[0162] Furthermore, the embodiment of the present application may utilize an evolutionary method to process the target candidate network to obtain a target network.

[0163] Evolutionary methods may include: crossover methods and / or mutation methods, etc. Among them, the crossover method can exchange the first code and the second code corresponding to any two target candidate networks to generate a new network. Assuming that the number of bits of the code is X, a crossover point can be selected from the X code bits, and the A1 part of the first code before the crossover point is exchanged with the B2 part of the second code after the crossover point, and the A2 part of the first code before the crossover point is exchanged with the B1 part of the second code before the crossover point to obtain a new network. The crossover method can increase the diversity of the networks involved in the operation.

[0164] The mutation method can randomly select a mutation point in the code corresponding to any target candidate network, and randomly change the value corresponding to the mutation point to obtain a new network. The mutation method can increase the diversity of the networks involved in the operation.

[0165] The process of processing the target candidate network using the evolutionary method may include:

[0166] Step S31, using an evolutionary method to determine a new network corresponding to the target candidate network;

[0167] Step S32: selecting a plurality of candidate networks from the plurality of new networks according to the time consumption information respectively corresponding to the plurality of new networks;

[0168] Step S33: determining a target candidate network from the multiple candidate networks according to the scores corresponding to the candidate networks;

[0169] Step S34, determine whether the number of iterations reaches a preset value, if so, end the iteration and use the target candidate network with the highest score as the target network; otherwise, return to step S31.

[0170] In summary, the network structure search method of the target detection model of the embodiment of the present application first uses the first network corresponding to the target detection model to perform a forward operation and a reverse operation on a data group in the image data set to obtain the forward operation result and the reverse operation result corresponding to this data group; then, according to the forward operation result and the reverse operation result corresponding to the data group, the score corresponding to the first network is determined; then, according to the score corresponding to the first network, the target network is determined from the multiple networks corresponding to the target detection model.

[0171] The embodiment of the present application determines the score corresponding to the first network based on one forward operation and one reverse operation corresponding to one data group. Since iterative training usually requires multiple operations for one type of data, and one forward operation and one reverse operation in the embodiment of the present application can reduce the number of operations, the embodiment of the present application can improve the search efficiency of the network structure.

[0172] Method Example 2

[0173] Reference Figure 6 , shows a schematic flow chart of the steps of a network structure search method for a target detection model according to an embodiment of the present application, and the method may specifically include the following steps:

[0174] Step 601: Acquire an image data set; the data group in the image data set may specifically include: original image data, noise data, and mixed image data; the mixed image data may be a mixture of the original image data and the noise data;

[0175] Step 602: construct an initial network corresponding to the target detection model, and sample a first network and a second network from the initial network;

[0176] Step 603: Perform a forward operation and a reverse operation on the data groups in the image data set according to the first network corresponding to the target detection model to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation;

[0177] Step 604: Determine a score corresponding to the first network according to the forward operation result and the reverse operation result;

[0178] Step 605: determine first training data according to the score corresponding to the first network; train a score prediction model according to the first training data, and input the code corresponding to the second network into the score prediction model to obtain the score corresponding to the second network;

[0179] Step 606: Determine a target network from the plurality of first networks and the plurality of second networks corresponding to the target detection model according to the scores corresponding to the first network and the second network respectively.

[0180] The sampling results obtained based on the initial network in the embodiment of the present application may include a first network and a second network. The embodiment of the present application performs forward and reverse operations on the data group in the image data set according to the first network corresponding to the target detection model to obtain the score of the first network; and trains the score prediction model according to the first training data corresponding to the score corresponding to the first network to obtain the score corresponding to the second network. Since the embodiment of the present application can save the forward and reverse operations corresponding to the second network, the embodiment of the present application can save the amount of calculation corresponding to the second network, thereby further improving the search efficiency of the network structure.

[0181] Method Example 3

[0182] Reference Figure 7 , shows a schematic flow chart of the steps of a method for training a target detection model according to an embodiment of the present application, and the method may specifically include the following steps:

[0183] Step 701: determine a target network from multiple networks corresponding to the target detection model;

[0184] Step 702: training the target network according to the second training data;

[0185] Wherein, determining the target network from the multiple networks corresponding to the target detection model includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, performing forward operation and reverse operation on the data group in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, determining the score corresponding to the first network; according to the score corresponding to the first network, determining the target network from the multiple networks corresponding to the target detection model; the network includes: a first network.

[0186] The second training data may be data corresponding to the training of the target network. Those skilled in the art may determine the second training data according to actual application requirements, and are not limited to the specific second training data. For example, the second training data may include: image samples, and labels corresponding to the image samples, etc.

[0187] The training process of the target network can include: forward propagation and back propagation.

[0188] Among them, the forward propagation can calculate the output information in sequence from the input layer to the output layer according to the parameters of the target network. Among them, the output information can be used to determine the loss information.

[0189] Back propagation can calculate and update the parameters of the target network in sequence from the output layer to the input layer according to the loss information. The target network usually adopts the structure of a neural network, and the parameters of the target network may include: parameters such as the weights of the neural network. Among them, during the back propagation process, the gradient information of the parameters of the target network can be determined, and the parameters of the target network can be updated using the gradient information. For example, back propagation can calculate and store the gradient information of the parameters of the processing layers (including the input layer, the intermediate layer, and the output layer) of the target network in sequence from the output layer to the input layer according to the chain rule in calculus.

[0190] Method Example 4

[0191] Reference Figure 8 , shows a schematic flow chart of the steps of a target detection method according to an embodiment of the present application, which method may specifically include the following steps:

[0192] Step 801: receiving an input image;

[0193] Step 802: using the trained target network, determine the detection result corresponding to the input image;

[0194] Step 803: output the detection result;

[0195] Among them, the determination process of the target network includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, the data group in the image data set is subjected to forward operation and reverse operation to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, the score corresponding to the first network is determined; according to the score corresponding to the first network, the target network is determined from the multiple networks corresponding to the target detection model; the network includes: a first network.

[0196] The input image may be an image to be detected. The trained target network may be a target network trained using the second training data. The embodiment of the present application may use the trained target network to determine the detection result corresponding to the input image and output the detection result.

[0197] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0198] Based on the above embodiment, this embodiment also provides a network structure search device for a target detection model, referring to Fig. 9 The device may specifically include: an acquisition module 901, a forward and reverse operation module 902, a network score determination module 903 and a target network determination module 904.

[0199] Wherein, the acquisition module 901 is used to acquire an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data;

[0200] A forward and reverse operation module 902 is used to perform a forward operation and a reverse operation on the data groups in the image data set according to the first network corresponding to the target detection model to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation;

[0201] A network score determination module 903, configured to determine a score corresponding to the first network according to the forward operation result and the reverse operation result;

[0202] The target network determination module 904 is used to determine the target network from the multiple networks corresponding to the target detection model according to the score corresponding to the first network; the network includes: a first network.

[0203] Optionally, the first network includes: a plurality of network units, the network units including: at least one network node;

[0204] The network score determination module 903 may specifically include:

[0205] A unit score determination module, used to determine a score corresponding to a network unit according to a forward operation result and a reverse operation result corresponding to a network node in a network unit;

[0206] The weighted processing module is used to perform weighted processing on the scores corresponding to the multiple network units to obtain the score corresponding to the first network.

[0207] Optionally, the unit score determination module includes:

[0208] A first processing module, configured to perform a first processing on a forward operation result corresponding to a network node in a network unit to obtain a first processing result;

[0209] A second processing module, used for performing a second processing on a reverse operation result corresponding to a network node in a network unit to obtain a second processing result;

[0210] A fusion processing module is used to fuse the first processing result and the second processing result to obtain a score corresponding to the network unit.

[0211] Optionally, the forward operation result and the reverse operation result corresponding to the one network node include: a forward operation result corresponding to the one network node under the condition of mixed image data, and a reverse operation result corresponding to the one network node under the condition of original image data.

[0212] Optionally, the network may further include: a second network; the target network determination module 904 may specifically include:

[0213] A training data determination module, used to determine first training data; the first training data includes: a code and a score corresponding to the first network;

[0214] A training module, used to train a score prediction model according to the first training data; the score prediction model is used to characterize the mapping relationship between the code corresponding to the network and the score;

[0215] A prediction module, used for inputting the code corresponding to the second network into the score prediction model to obtain the score corresponding to the second network;

[0216] The first target network determination module is used to determine a target network from a plurality of first networks and a plurality of second networks corresponding to the target detection model according to the scores corresponding to the first network and the second network respectively.

[0217] Optionally, the target network determination module 904 may specifically include:

[0218] A selection module, configured to select a plurality of candidate networks from the plurality of networks according to the time consumption information respectively corresponding to the plurality of networks;

[0219] A target candidate network determination module, configured to determine a target candidate network from the multiple candidate networks according to scores corresponding to the candidate networks;

[0220] The evolutionary processing module is used to process the target candidate network using an evolutionary method to obtain a target network.

[0221] Optionally, the target candidate network determination module may specifically include:

[0222] A first target candidate network determination module is configured to select, in ascending order of time consumption information, a plurality of candidate networks whose time consumption information does not exceed a preset time length from the plurality of networks; or

[0223] The second target candidate network determination module is used to determine a plurality of time-consuming intervals that do not exceed a preset time length, and select a plurality of candidate networks whose time-consuming information is respectively within the plurality of time-consuming intervals from the plurality of networks.

[0224] Based on the above embodiment, this embodiment also provides a training device for a target detection model. Fig.10 , the device may specifically include:

[0225] A determination module 1001 is used to determine a target network from multiple networks corresponding to the target detection model;

[0226] A training module 1002 is used to train the target network according to the second training data;

[0227] Wherein, determining the target network from the multiple networks corresponding to the target detection model includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, performing forward operation and reverse operation on the data group in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, determining the score corresponding to the first network; according to the score corresponding to the first network, determining the target network from the multiple networks corresponding to the target detection model; the network includes: a first network.

[0228] Based on the above embodiment, this embodiment further provides a target detection device, referring to Fig.11 , the device may specifically include:

[0229] A receiving module 1101 is used to receive an input image;

[0230] A detection module 1102 is used to determine a detection result corresponding to the input image using the trained target network;

[0231] Output module 1103, used to output the detection result;

[0232] Among them, the determination process of the target network includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, the data group in the image data set is subjected to forward operation and reverse operation to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, the score corresponding to the first network is determined; according to the score corresponding to the first network, the target network is determined from the multiple networks corresponding to the target detection model; the network includes: a first network.

[0233] The embodiment of the present application also provides a non-volatile readable storage medium, which stores one or more modules (programs). When the one or more modules are applied to a device, the device can execute instructions (instructions) of each method step in the embodiment of the present application.

[0234] The present application embodiment provides one or more machine-readable media on which instructions are stored, and when executed by one or more processors, an electronic device executes one or more of the methods described in the above embodiments. In the present application embodiment, the electronic device includes various types of devices such as terminal devices and servers (clusters).

[0235] The embodiments of the present disclosure may be implemented as a device that uses any appropriate hardware, firmware, software, or any combination thereof to perform a desired configuration, and the device may include: electronic devices such as terminal devices and servers (clusters). Fig.12 An exemplary apparatus 1300 that can be used to implement various embodiments described in this application is schematically shown.

[0236] For one embodiment, Fig.12 An exemplary apparatus 1300 is shown having one or more processors 1302, a control module (chip set) 1304 coupled to at least one of the (one or more) processors 1302, a memory 1306 coupled to the control module 1304, a non-volatile memory (NVM) / storage device 1308 coupled to the control module 1304, one or more input / output devices 1310 coupled to the control module 1304, and a network interface 1312 coupled to the control module 1304.

[0237] The processor 1302 may include one or more single-core or multi-core processors, and the processor 1302 may include any combination of general-purpose processors or special-purpose processors (such as graphics processors, application processors, baseband processors, etc.). In some embodiments, the device 1300 can be used as a terminal device, server (cluster), and other devices described in the embodiments of the present application.

[0238] In some embodiments, the device 1300 may include one or more computer-readable media (e.g., memory 1306 or NVM / storage device 1308) having instructions 1314 and one or more processors 1302 configured to execute the instructions 1314 in combination with the one or more computer-readable media to implement a module to perform the actions described in the present disclosure.

[0239] For one embodiment, the control module 1304 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 1302 and / or any suitable device or component in communication with the control module 1304 .

[0240] The control module 1304 may include a memory controller module to provide an interface to the memory 1306. The memory controller module may be a hardware module, a software module, and / or a firmware module.

[0241] The memory 1306 may be used, for example, to load and store data and / or instructions 1314 for the device 1300. For one embodiment, the memory 1306 may include any suitable volatile memory, such as a suitable DRAM. In some embodiments, the memory 1306 may include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0242] For one embodiment, control module 1304 may include one or more input / output controllers to provide an interface to NVM / storage device 1308 and input / output device(s) 1310 .

[0243] For example, NVM / storage 1308 may be used to store data and / or instructions 1314. NVM / storage 1308 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0244] NVM / storage device 1308 may include storage resources that are physically part of the device on which apparatus 1300 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 1308 may be accessed via input / output device(s) 1310 over a network.

[0245] (One or more) input / output devices 1310 may provide an interface for the apparatus 1300 to communicate with any other appropriate device, and the input / output device 1310 may include a communication component, an audio component, a sensor component, etc. The network interface 1312 may provide an interface for the apparatus 1300 to communicate through one or more networks, and the apparatus 1300 may wirelessly communicate with one or more components of a wireless network according to any of one or more wireless network standards and / or protocols, for example, accessing a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, etc., or a combination thereof for wireless communication.

[0246] For one embodiment, at least one of the processor(s) 1302 may be packaged together with the logic of one or more controllers (e.g., a memory controller module) of the control module 1304. For one embodiment, at least one of the processor(s) 1302 may be packaged together with the logic of one or more controllers of the control module 1304 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with the logic of one or more controllers of the control module 1304. For one embodiment, at least one of the processor(s) 1302 may be integrated on the same die with the logic of one or more controllers of the control module 1304 to form a system-on-chip (SoC).

[0247] In various embodiments, the device 1300 may be, but is not limited to, a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, the device 1300 may have more or fewer components and / or a different architecture. For example, in some embodiments, the device 1300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touch screen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0248] Among them, the main control chip can be used as a processor or control module in the detection device, sensor data, location information corresponding to the camera device, etc. are stored in the memory or NVM / storage device, the sensor group can be used as an input / output device, and the communication interface may include a network interface.

[0249] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0250] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0251] The present application embodiment is described with reference to the flowchart and / or block diagram of the method, terminal device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for implementing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0252] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0253] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0254] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0255] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0256] The above is a detailed introduction to a network structure search method and device for a target detection model, a training method and device for a target detection model, a target detection method and device, an electronic device and a machine-readable medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A network structure search method for a target detection model, characterized in that: The method comprises: Acquire an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; According to the first network corresponding to the target detection model, a forward operation and a reverse operation are performed on the data groups in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; Determining a score corresponding to the first network according to the forward operation result and the reverse operation result; According to the score corresponding to the first network, a target network is determined from multiple networks corresponding to the target detection model; the network includes: a first network.

2. The method according to claim 1, characterized in that The first network includes: a plurality of network units, wherein the network units include: at least one network node; Determining a score corresponding to the first network according to the forward operation result and the reverse operation result includes: Determine a score corresponding to a network unit according to a forward operation result and a reverse operation result corresponding to a network node in the network unit; The scores corresponding to the multiple network units are weighted to obtain a score corresponding to the first network.

3. The method according to claim 2, characterized in that The determining the score corresponding to the network unit includes: Performing a first process on a forward operation result corresponding to a network node in a network unit to obtain a first processing result; Performing a second process on a reverse operation result corresponding to a network node in a network unit to obtain a second processing result; The first processing result and the second processing result are fused to obtain a score corresponding to the network unit.

4. The method according to claim 2, characterized in that: The forward operation result and the reverse operation result corresponding to a network node include: a forward operation result corresponding to a network node under the condition of mixed image data, and a reverse operation result corresponding to a network node under the condition of original image data.

5. The method according to any one of claims 1 to 4, characterized in that: The network further includes: a second network; and determining the target network from the multiple networks corresponding to the target detection model includes: Determine first training data; the first training data includes: the encoding and score corresponding to the first network; According to the first training data, a score prediction model is trained; the score prediction model is used to characterize the mapping relationship between the encoding corresponding to the network and the score; Inputting the code corresponding to the second network into the score prediction model to obtain the score corresponding to the second network; According to the scores corresponding to the first network and the second network respectively, a target network is determined from a plurality of first networks and a plurality of second networks corresponding to the target detection model.

6. The method according to any one of claims 1 to 4, characterized in that: The determining of the target network from the multiple networks corresponding to the target detection model includes: Selecting a plurality of candidate networks from the plurality of networks according to the time consumption information respectively corresponding to the plurality of networks; Determining a target candidate network from the multiple candidate networks according to the scores corresponding to the candidate networks; The target candidate network is processed using an evolutionary method to obtain a target network.

7. The method according to claim 6, characterized in that The selecting a plurality of candidate networks from the plurality of networks comprises: Selecting, in ascending order of the time consumption information, a plurality of candidate networks whose time consumption information does not exceed a preset time length from the plurality of networks; or A plurality of time-consuming intervals that do not exceed a preset time length are determined, and a plurality of candidate networks whose time-consuming information is respectively within the plurality of time-consuming intervals are selected from the plurality of networks.

8. A method for training a target detection model, characterized in that: The method comprises: Determine a target network from multiple networks corresponding to the target detection model; Training the target network according to the second training data; Wherein, determining the target network from the multiple networks corresponding to the target detection model includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, performing forward operation and reverse operation on the data group in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, determining the score corresponding to the first network; according to the score corresponding to the first network, determining the target network from the multiple networks corresponding to the target detection model; the network includes: a first network.

9. A target detection method, characterized in that: The method comprises: receiving an input image; Determine the detection result corresponding to the input image by using the trained target network; Output the test results; Among them, the determination process of the target network includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, the data group in the image data set is subjected to forward operation and reverse operation to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, the score corresponding to the first network is determined; according to the score corresponding to the first network, the target network is determined from the multiple networks corresponding to the target detection model; the network includes: a first network.

10. A network structure search device for a target detection model, characterized in that: The device comprises: An acquisition module, used for acquiring an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; A forward and reverse operation module, used to perform forward operation and reverse operation on the data groups in the image data set according to the first network corresponding to the target detection model, so as to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; A network score determination module, used to determine a score corresponding to the first network according to the forward operation result and the reverse operation result; A target network determination module is used to determine a target network from a plurality of networks corresponding to the target detection model according to a score corresponding to the first network; the network includes: a first network.

11. A training device for a target detection model, characterized in that: The device comprises: A determination module, used to determine a target network from multiple networks corresponding to the target detection model; A training module, used for training the target network according to the second training data; Wherein, determining the target network from the multiple networks corresponding to the target detection model includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, performing forward operation and reverse operation on the data group in the image data set to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, determining the score corresponding to the first network; according to the score corresponding to the first network, determining the target network from the multiple networks corresponding to the target detection model; the network includes: a first network.

12. A target detection device, characterized in that: The device comprises: A receiving module, used for receiving an input image; A detection module, used to determine a detection result corresponding to the input image using the trained target network; An output module, used for outputting the detection results; Among them, the determination process of the target network includes: obtaining an image data set; the data group in the image data set includes: original image data, noise data, and mixed image data; the mixed image data is a mixture of the original image data and the noise data; according to the first network corresponding to the target detection model, the data group in the image data set is subjected to forward operation and reverse operation to obtain corresponding forward operation results and reverse operation results; wherein one data group corresponds to one forward operation and one reverse operation; according to the forward operation result and the reverse operation result, the score corresponding to the first network is determined; according to the score corresponding to the first network, the target network is determined from the multiple networks corresponding to the target detection model; the network includes: a first network.

Images