A model generation method and device, electronic equipment and storage medium

Abstract

This application provides a model generation method, apparatus, electronic device, and storage medium. The solution is as follows: A first network model is obtained, and a second training set is obtained; each sample image in the second training set is input into the first network model, and a first output result corresponding to each convolutional layer is obtained; for each convolutional layer in the first network model, based on the first output result and the first weight matrix corresponding to that convolutional layer, a second weight matrix corresponding to that convolutional layer in the target scene is calculated to obtain a second network model; for each convolutional layer in the second network model, the convolutional layer is decomposed to obtain a third network model. Through the technical solution provided by this application, a convolutional neural network model corresponding to the target scene is generated, redundant information in large-scale convolutional neural network models is removed, and the convolution operation speed of the generated convolutional neural network model is improved.

Description

Technical Field

[0001] This application relates to the field of radar image processing technology, and in particular to a model generation method, apparatus, electronic device and storage medium. Background Technology

[0002] Currently, the network architecture corresponding to Convolutional Neural Networks (CNNs) has demonstrated outstanding performance in 3D computer vision tasks, especially large-scale CNN models. For example, large-scale CNNs can be used to process 3D images in processes such as object detection, point-based semantic segmentation, and object recognition. Summary of the Invention

[0003] The purpose of this application is to provide a model generation method, apparatus, electronic device, and storage medium to generate a convolutional neural network model corresponding to a target scene, remove redundant information in large-scale convolutional neural network models, and improve the convolution operation speed of the generated convolutional neural network model. The specific technical solution is as follows:

[0004] This application provides a model generation method, the method comprising:

[0005] Obtain a convolutional neural network model trained on a first training set as a first network model, and obtain a second training set corresponding to the target scene in the first training set; wherein, the first training set includes sample images corresponding to multiple scenes;

[0006] Each sample image in the second training set is input into the first network model, and the first output result corresponding to each convolutional layer in the first network model is obtained.

[0007] For each convolutional layer in the first network model, based on the first output result of each sample image in the second training set corresponding to the convolutional layer and the first weight matrix corresponding to the convolutional layer, the second weight matrix corresponding to the convolutional layer in the target scene is calculated to obtain the second network model.

[0008] For each convolutional layer in the second network model, the third weight matrix obtained by decomposing the second weight matrix corresponding to the convolutional layer is used to decompose the convolutional layer to obtain the third network model corresponding to the target scene.

[0009] Optionally, the step of calculating the second weight matrix corresponding to the convolutional layer in the target scene for each convolutional layer in the first network model, based on the first output result corresponding to the convolutional layer for each sample image in the second training set and the first weight matrix corresponding to the convolutional layer, to obtain the second network model, includes:

[0010] For each convolutional layer in the first network model, based on the first output result corresponding to each sample image in the second training set, the target transformation matrix corresponding to the convolutional layer is calculated using a preset objective function;

[0011] Calculate the product of the target transformation matrix corresponding to the convolutional layer and the first weight matrix to obtain the second weight matrix corresponding to the convolutional layer in the target scene;

[0012] Based on the second weight matrix corresponding to each convolutional layer, the weights of each convolutional layer in the first network model are updated to obtain the second network model.

[0013] Optionally, the preset objective function is a calculation function that calculates the optimal solution when the output error corresponding to the same convolutional layer in the first network model and the second network model is minimized under the preset rank constraint of the matrix.

[0014] Optionally, the preset objective function is:

[0015]

[0016] st(M)≤k

[0017] or,

[0018]

[0019] st(M)≤k

[0020] Where min represents the minimum value operation, and n is the number of sample images in the second training set. For the Frobenius norm calculation, M is the target transformation matrix corresponding to the convolutional layer, and Y is... l is the first output result corresponding to the l-th sample image in the convolutional layer, st is the constraint condition, rank() is the rank of M, k is the rank of the preset matrix, B is the bias to be optimized, and r() is the nonlinear operator.

[0021] Optionally, the step of decomposing each convolutional layer in the second network model based on the third weight matrix obtained by decomposing the second weight matrix corresponding to that convolutional layer to obtain the third network model corresponding to the target scene includes:

[0022] For each convolutional layer in the second network model, singular value decomposition is performed on the second weight matrix corresponding to the convolutional layer to obtain two third weight matrices corresponding to the convolutional layer.

[0023] Based on the two third weight matrices corresponding to the convolutional layer, the convolutional layer is decomposed into two sub-convolutional layers to obtain the third network model corresponding to the target scene.

[0024] Optionally, the second training set also includes label information corresponding to each sample image;

[0025] The method further includes:

[0026] Each sample image in the second training set is input into the third network model to obtain the third output result corresponding to each sample image;

[0027] The loss value of the third network model is calculated based on the third output result and label information corresponding to each sample image.

[0028] When it is determined that the third network model has not converged based on the loss value, the parameters of the third network model are adjusted according to a preset step size, and based on the adjusted parameters, the step of inputting each sample image in the second training set into the third network model to obtain the third output result corresponding to each sample image is returned to be executed until the third network model converges.

[0029] Optionally, if the third network model is an object detection model, the method further includes:

[0030] Acquire the image to be detected collected in the target scene;

[0031] The image to be detected is input into the third network model to obtain the target detection result.

[0032] This application embodiment also provides a model generation apparatus, the apparatus comprising:

[0033] The first acquisition module is used to acquire a convolutional neural network model trained based on a first training set as a first network model, and to acquire a second training set corresponding to the target scene in the first training set; wherein, the first training set includes sample images corresponding to multiple scenes.

[0034] The second acquisition module is used to input each sample image in the second training set into the first network model and acquire the first output result corresponding to each convolutional layer in the first network model.

[0035] The first calculation module is used to calculate the second weight matrix corresponding to the convolutional layer in the target scene for each convolutional layer in the first network model, based on the first output result corresponding to the convolutional layer for each sample image in the second training set and the first weight matrix corresponding to the convolutional layer, so as to obtain the second network model.

[0036] The decomposition module is used to decompose each convolutional layer in the second network model based on the third weight matrix obtained by decomposing the second weight matrix corresponding to the convolutional layer, thereby obtaining the third network model corresponding to the target scene.

[0037] Optionally, the first calculation module is specifically used to calculate the target transformation matrix corresponding to each convolutional layer in the first network model based on the first output result corresponding to each sample image in the second training set at that convolutional layer using a preset objective function.

[0038] Calculate the product of the target transformation matrix corresponding to the convolutional layer and the first weight matrix to obtain the second weight matrix corresponding to the convolutional layer in the target scene;

[0039] Based on the second weight matrix corresponding to each convolutional layer, the weights of each convolutional layer in the first network model are updated to obtain the second network model.

[0040] Optionally, the preset objective function is a calculation function that calculates the optimal solution when the output error corresponding to the same convolutional layer in the first network model and the second network model is minimized under the preset rank constraint of the matrix.

[0041] Optionally, the preset objective function is:

[0042]

[0043] st(M)≤k

[0044] or,

[0045]

[0046] st(M)≤k

[0047] Where min represents the minimum value operation, and n is the number of sample images in the second training set. For the Frobenius norm calculation, M is the target transformation matrix corresponding to the convolutional layer, and Y is... l is the first output result corresponding to the l-th sample image in the convolutional layer, st is the constraint condition, rank() is the rank of M, k is the rank of the preset matrix, B is the bias to be optimized, and r() is the nonlinear operator.

[0048] Optionally, the decomposition module is specifically used to perform singular value decomposition on the second weight matrix corresponding to each convolutional layer in the second network model to obtain two third weight matrices corresponding to the convolutional layer.

[0049] Based on the two third weight matrices corresponding to the convolutional layer, the convolutional layer is decomposed into two sub-convolutional layers to obtain the third network model corresponding to the target scene.

[0050] Optionally, the second training set also includes label information corresponding to each sample image;

[0051] The device further includes:

[0052] The determination module is used to input each sample image in the second training set into the third network model to obtain a third output result corresponding to each sample image;

[0053] The second calculation module is used to calculate the loss value of the third network model based on the third output result and label information corresponding to each sample image.

[0054] The adjustment module is used to adjust the parameters of the third network model according to a preset step size when it is determined that the third network model has not converged based on the loss value, and based on the adjusted parameters, call the determination module to return to the step of inputting each sample image in the second training set into the third network model to obtain the third output result corresponding to each sample image, until the third network model converges.

[0055] Optionally, if the third network model is an object detection model, the device further includes:

[0056] The third acquisition module is used to acquire the image to be detected collected in the target scene;

[0057] The detection module is used to input the image to be detected into the third network model to obtain the target detection result.

[0058] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0059] Memory, used to store computer programs;

[0060] When a processor executes a program stored in memory, it implements any of the steps of the model generation method described above.

[0061] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the model generation method steps described above.

[0062] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the model generation methods described above.

[0063] Beneficial effects of the embodiments in this application:

[0064] The technical solution provided in this application, after obtaining a convolutional neural network model trained on a first training set (i.e., a first network model) and a second training set corresponding to the target scene in the first training set, inputs each sample data from the second training set into the first network model to obtain a first output result corresponding to each convolutional layer in the first network model. For each convolutional layer in the first network model, based on the first output result and the first weight matrix corresponding to that convolutional layer, the first network model is converted into a second network model under the target scene. Then, each convolutional layer in the second network model is decomposed to obtain a third network model, i.e., the convolutional neural network model corresponding to the target scene, thus realizing the generation of the convolutional neural network model corresponding to the target scene.

[0065] Furthermore, since the decomposition of each convolutional layer in the second network model is based on the decomposition of the second weight matrix corresponding to each convolutional layer to obtain the third weight matrix, the rank of the matrix corresponding to the third weight matrix obtained by decomposing the second weight matrix is ​​reduced. This can remove redundant information in the second weight matrix corresponding to each convolutional layer in the second network model, that is, remove parameters in the second weight matrix that have low correlation with the target scene, and retain parameters in the second weight matrix that have high correlation with the target scene. As a result, the redundant information in the original first network model is removed in the decomposed third network model, that is, the redundant information in the large-scale convolutional neural network model is removed, which speeds up the convolution operation speed of the decomposed third network model, that is, improves the convolution operation speed of the generated convolutional neural network model.

[0066] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0068] Figure 1 This is a schematic diagram of a first flowchart of the model generation method provided in an embodiment of this application;

[0069] Figure 2 This is a second flowchart illustrating the model generation method provided in the embodiments of this application;

[0070] Figure 3 This is a schematic diagram of a third process for the model generation method provided in the embodiments of this application;

[0071] Figure 4 This is a schematic diagram of the fourth process of the model generation method provided in the embodiments of this application;

[0072] Figure 5 This is a schematic flowchart of a target detection method provided in an embodiment of this application;

[0073] Figure 6 A schematic diagram of the structure of the model generation apparatus provided in the embodiments of this application;

[0074] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0075] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0076] In related technologies, for training large-scale convolutional neural network models applicable to various scenarios, two methods can be used to obtain large-scale convolutional neural network models.

[0077] Method 1: Obtain training data (also known as training sets) corresponding to various scenarios, train a pre-defined large-scale convolutional neural network model, and obtain a convolutional neural network model suitable for various scenarios.

[0078] Method 2: Obtain training data corresponding to multiple scenarios. For each scenario, use the training data corresponding to that scenario to train a preset large-scale convolutional neural network model to obtain a convolutional neural network model suitable for each scenario.

[0079] To facilitate understanding, let's take the autonomous driving process as an example. During vehicle operation, the scenarios in which the vehicle operates can be diverse, such as sunny / rainy weather, curves, tunnels, highways, and urban roads. When generating object detection models for these scenarios, a pre-defined large-scale convolutional neural network model can be trained. For example, using method one above, the convolutional neural network model is trained using training data corresponding to all scenarios, resulting in an object detection model applicable to all scenarios. As another example, using method two above, the convolutional neural network model is trained using training data corresponding to each scenario, resulting in multiple object detection models, each suitable for object detection in images acquired within a specific scenario.

[0080] Regarding Method 1, because the training data includes a large amount of training data from different scenarios, the trained object detection model, while applicable to various scenarios, will not actually be used by a vehicle in all scenarios during its actual operation. For example, a vehicle in country A will not travel on highways in country B; that is, the highway scene in country B will not appear during the vehicle's journey. In this case, the information related to the highway scene in country B in the trained object detection model is redundant. Therefore, the convolutional neural network model trained using Method 1 contains a large amount of redundant information.

[0081] Regarding the second method, since the convolutional neural network model corresponding to each scene is trained separately, the training process does not take into account the coupling effect between training data of different scenes. This will result in low coupling between the various convolutional neural network models trained, and relatively low target detection accuracy.

[0082] To address the problems in related technologies, embodiments of this application provide a model generation method. For example... Figure 1 As shown, Figure 1 This is a schematic diagram of a first flowchart of the model generation method provided in an embodiment of this application. This method can be applied to any electronic device. Figure 1 The method shown includes the following steps.

[0083] Step S101: Obtain the convolutional neural network model trained based on the first training set as the first network model, and obtain the second training set corresponding to the target scene in the first training set; wherein, the first training set includes sample images corresponding to multiple scenes.

[0084] Step S102: Input each sample image in the second training set into the first network model and obtain the first output result corresponding to each convolutional layer in the first network model.

[0085] Step S103: For each convolutional layer in the first network model, based on the first output result of each sample image in the second training set corresponding to the convolutional layer and the first weight matrix corresponding to the convolutional layer, calculate the second weight matrix corresponding to the convolutional layer in the target scene to obtain the second network model.

[0086] Step S104: For each convolutional layer in the second network model, decompose the convolutional layer based on the third weight matrix obtained by decomposing the second weight matrix corresponding to the convolutional layer to obtain the third network model corresponding to the target scene.

[0087] In the embodiments of this application, the aforementioned electronic device can be an image acquisition device, such as a camera or radar, or other devices that are communicatively connected to the image acquisition device, such as a server. No specific limitations are imposed on the aforementioned electronic device.

[0088] pass Figure 1 The method described above, after obtaining a convolutional neural network model trained on a first training set (i.e., a first network model) and a second training set corresponding to the target scene in the first training set, inputs each sample data from the second training set into the first network model to obtain the first output result corresponding to each convolutional layer in the first network model. For each convolutional layer in the first network model, based on the first output result and the first weight matrix corresponding to that convolutional layer, the first network model is converted into a second network model under the target scene. Then, each convolutional layer in the second network model is decomposed to obtain a third network model, i.e., the convolutional neural network model corresponding to the target scene, thus realizing the generation of the convolutional neural network model corresponding to the target scene.

[0089] Furthermore, since the decomposition of each convolutional layer in the second network model is based on the decomposition of the second weight matrix corresponding to each convolutional layer to obtain the third weight matrix, the rank of the matrix corresponding to the third weight matrix obtained by decomposing the second weight matrix is ​​reduced. This can remove redundant information in the second weight matrix corresponding to each convolutional layer in the second network model, that is, remove parameters in the second weight matrix that have low correlation with the target scene, and retain parameters in the second weight matrix that have high correlation with the target scene. As a result, the redundant information in the original first network model is removed in the decomposed third network model, that is, the redundant information in the large-scale convolutional neural network model is removed, which speeds up the convolution operation speed of the decomposed third network model, that is, improves the convolution operation speed of the generated convolutional neural network model.

[0090] The embodiments of this application will be described below through specific examples.

[0091] For step S101 above, namely, obtaining the convolutional neural network model trained based on the first training set as the first network model, and obtaining the second training set corresponding to the target scene in the first training set; wherein, the first training set includes sample images corresponding to multiple scenes.

[0092] In this embodiment, a pre-set large-scale convolutional neural network model is trained using a first training set to obtain a first network model. The first training set may include sample images corresponding to multiple scenes, and label information corresponding to each sample image. After the first network model is trained, the electronic device can acquire it.

[0093] The training of the first network model described above can be performed by the aforementioned electronic device or by other devices. The training steps for the first network model can be referred to the retraining process of the third network model below, and will not be repeated here.

[0094] The convolutional neural network model obtained through the above training (i.e., the first network model) can be an object detection model or an object recognition model. For example, in the above autonomous driving process, the first network model can be used to identify vehicles, traffic lights, obstacles, etc. in the driving scene.

[0095] Each sample image in the first training set mentioned above can be a two-dimensional image, such as an image frame captured by an in-vehicle camera; or a three-dimensional image, such as a point cloud frame captured by an in-vehicle radar. The aforementioned scenarios can represent various application scenarios corresponding to the convolutional neural network model trained above under different conditions. For ease of understanding, the following explanation only uses the above-mentioned autonomous driving process scenarios (sunny / rainy day scenario, curve scenario, tunnel scenario, highway scenario, and urban road scenario) as examples, and does not constitute any limitation. Here, no specific limitations are made on the sample images and the scenarios corresponding to the sample images.

[0096] Depending on the different application scenarios and functions of the convolutional neural network models trained above, the label information corresponding to each sample image in the first training set is also completely different.

[0097] For example, when the first network model is an object detection model, the label information corresponding to each sample image in the first training set can be: the location information of the target object in the sample image, such as a detection box indicating the location of the target object.

[0098] For example, when the first network model is a target recognition model, the label information corresponding to each sample image in the first training set can be: the location information of the target object in the sample image, and the category of the target object, such as vehicle, tree, building, etc.

[0099] The label information corresponding to each of the above sample images can be generated through automatic perception calculation and manual transfer.

[0100] In this embodiment, depending on the application scenario and purpose of the trained convolutional neural network model, the sample images, the corresponding scenes, the label information, and the target objects in the sample images will vary. Therefore, no specific limitations are imposed on the sample images, the corresponding scenes, the label information, and the target objects in the sample images.

[0101] While acquiring the first network model, the electronic device can also acquire training data corresponding to the target scene from the first training set, namely, sample images and label information corresponding to the target scene, to obtain a second training set. This second training set is a subset of the first training set.

[0102] In this embodiment, the target scenario will vary depending on the application scenario and user needs of the third network model. The target scenario can be one scenario or a combination of multiple scenarios. The number of scenarios included in the combination scenario is less than the number of scenarios corresponding to the first training set.

[0103] For ease of understanding, we will continue to use the aforementioned autonomous driving process as an example. If the third network model is applied to an indoor parking lot, then the sample images in the second training set are all sample images collected in the indoor parking lot; if the third network model is applied to urban roads, then the sample images in the second training set can be sample images of urban road scenes collected under various weather conditions.

[0104] In an optional embodiment, the first training set S described above sum It can be represented as: S sum =(S1,S2,…,S) m That is, S. sum It consists of a training set corresponding to m scenes, namely S1 to S2. m The training set for each scene includes multiple sample images and their corresponding label information. The second training set can be S. sum One or more training sets corresponding to the target scene. For example, if the target scene is the scene corresponding to S2, then S2 is the second training set mentioned above.

[0105] In the embodiments of this application, the above S1, S2, ..., S mThe number of sample images included in each training set can be the same or different. This can be set according to different scenarios and user needs, and is not specifically limited here. For ease of understanding, the following will only use S1, S2, ..., S... m This example uses n sample images and does not serve as a limitation. Specifically, it can be represented as: S j = [X1,X2,X3,…,X] n ]. X1,X2,X3,…,X n For training set S j n sample images in the dataset.

[0106] In this embodiment of the application, the target scenario and the second training set are not specifically limited.

[0107] In one optional embodiment, the model scale of the convolutional neural network model described above is larger than a preset scale value. The convolutional neural network model may include multiple convolutional layers. Here, there are no specific limitations on the preset scale value or the number of convolutional layers.

[0108] In an optional embodiment, the convolutional neural network model described above can be a large-scale model with good fit, such as Visual Geometry Group Network-16 (VGG-16) or ResNet50. VGG-16 includes 16 layers, namely 13 convolutional layers and 3 fully connected layers, while ResNet50 contains 49 convolutional layers and one fully connected layer. The network architecture of the above convolutional neural network model is not specifically limited here.

[0109] In step S101 above, the acquisition of the first network model and the acquisition of the second training set are performed simultaneously. Alternatively, the acquisition of the first network model can be performed before or after the acquisition of the second training set. Here, no specific limitation is made on the timing of the acquisition of the first network model and the acquisition of the second training set.

[0110] In step S101 above, the second training set is obtained from the first training set. Alternatively, the electronic device can also reacquire sample images corresponding to the target scene, using the same method as the sample images in the first training set. Here, no specific limitations are placed on the sample images in the second training set.

[0111] For step S102 above, each sample image in the second training set is input into the first network model, and the first output result corresponding to each convolutional layer in the first network model is obtained.

[0112] In this embodiment, the first network model includes multiple convolutional layers. For each sample image in the second training set, the electronic device can input the sample image into the first network model, whereby the sample image undergoes convolution calculation in each convolutional layer of the first network model to obtain the output result corresponding to each convolutional layer (denoted as the first output result). The electronic device can obtain the first output result corresponding to each sample image output by each convolutional layer.

[0113] In this embodiment, the convolutional layers in the first network model described above can be used to extract features from the input sample image to obtain corresponding feature data. Here, the output results corresponding to each convolutional layer in the first network model are not specifically limited.

[0114] For step S103 above, that is, for each convolutional layer in the first network model, based on the first output result corresponding to the convolutional layer for each sample image in the second training set and the first weight matrix corresponding to the convolutional layer, the second weight matrix corresponding to the convolutional layer in the target scene is calculated to obtain the second network model.

[0115] In this embodiment of the application, the electronic device can obtain the network model (denoted as the second network model) corresponding to the target scenario based on the first network model described above. The model structure of the second network model is the same as that of the first network model, but the weight matrix corresponding to the same convolutional layer is different.

[0116] In an optional embodiment, the weight matrix (denoted as the second weight matrix) corresponding to each convolutional layer in the second network model and the weight matrix (denoted as the first weight matrix) corresponding to the same convolutional layer in the first network model satisfy a preset transformation relationship. Specifically, it can be expressed as: w j = j W j Among them, w j W is the second weight matrix corresponding to the j-th convolutional layer in the second network model. j M is the first weight matrix corresponding to the j-th convolutional layer in the first network model. j For W j with w j The target transformation matrix between.

[0117] Given that the weight matrices between each convolutional layer in the first network model and each convolutional layer in the second network model satisfy the aforementioned preset transformation relationship, that is, when w j = j W jIn this case, the electronic device can calculate the second weight matrix corresponding to the convolutional layer in the target scene based on the first output result of each sample image in the second training set corresponding to the convolutional layer and the first weight matrix corresponding to the convolutional layer, thereby obtaining the second network model. The calculation of the second weight matrix corresponding to each convolutional layer in the second network model is described below and will not be repeated here.

[0118] For step S104 above, that is, for each convolutional layer in the second network model, the third weight matrix obtained by decomposing the second weight matrix corresponding to the convolutional layer is used to decompose the convolutional layer to obtain the third network model corresponding to the target scene.

[0119] In this step, after determining the second network model, the electronic device can decompose each convolutional layer in the second network model to obtain the third network model corresponding to the target scene. That is, using the third weight matrix obtained by decomposing the second weight matrix corresponding to each convolutional layer in the second network model, the third convolutional layer is decomposed to obtain the third network model.

[0120] In one optional embodiment, for each convolutional layer in the second network model, the convolutional layer is decomposed based on the two third weight matrices obtained by performing singular value decomposition on the second weight matrix corresponding to the convolutional layer, to obtain the third network model corresponding to the target scene.

[0121] The aforementioned third weight matrix is ​​obtained by decomposing the second weight matrix, and the rank of the resulting third weight matrix is ​​less than the rank of the second weight matrix. The specific method of decomposing the second weight matrix is ​​not limited here.

[0122] In an optional embodiment, according to the above... Figure 1 The method shown in this application embodiment also provides a model generation method. For example... Figure 2 As shown, Figure 2 This is a schematic diagram of a second flowchart of the model generation method provided in an embodiment of this application. Figure 2 In the method shown, the above step S103 is refined into the following steps, namely step S1031-step S1033.

[0123] Step S1031: For each convolutional layer in the first network model, based on the first output result corresponding to each sample image in the second training set, calculate the target transformation matrix corresponding to the convolutional layer using a preset objective function.

[0124] In this embodiment, the aforementioned preset objective function is a calculation function that calculates the optimal solution when the output error corresponding to the same convolutional layer in the first network model and the second network model is minimized under the preset matrix rank constraint.

[0125] In an optional embodiment, the above-mentioned preset objective function can be expressed as:

[0126]

[0127] strank(M)≤k

[0128] Where min represents the minimum value operation, and n is the number of sample images in the second training set. For the Frobenius norm calculation, M is the target transformation matrix corresponding to the convolutional layer, and Y is... l is the first output result corresponding to the l-th sample image in the convolutional layer, st is the constraint condition, rank(M) is the rank of M, and k is the rank of the preset matrix.

[0129] In this embodiment, regarding the first network model and the second network model, since the first network model is a convolutional neural network model trained on the first training set, the first output result corresponding to each convolutional layer is accurate when processing each sample image in the second training set based on the first network model. At this time, when constructing the preset objective function, in order to ensure the accuracy of the output result corresponding to each convolutional layer in the transformed second network model (denoted as the second output result), it is necessary to ensure that the error between the second output result corresponding to each convolutional layer in the second network model and the first output result corresponding to the same convolutional layer in the first network model is as small as possible.

[0130] For each convolutional layer in the first network model described above, the first output result Y corresponding to that convolutional layer is... 1l It can be represented as: Y 1l =W j X l , where X l For the second training set S j The l-th sample image in the above second network model. For each convolutional layer in the above second network model, the second output result Y corresponding to that convolutional layer. 2l It can be represented as: Y 2l =w j X l =M l W j X l =M l Y 1l In order to minimize the error between the second output of each convolutional layer in the second network model and the first output of the same convolutional layer in the first network model, the above-mentioned preset objective function can be constructed.

[0131] The rank k of the aforementioned preset matrix can be set based on the rank of the first weight matrix corresponding to each convolutional layer in the first network model or a user-experienced value. The preset rank k of the matrix is ​​less than the rank of the first weight matrix corresponding to each convolutional layer in the first network model. Here, no specific limitation is made on the preset rank of the matrix.

[0132] In another optional embodiment, considering that the output data corresponding to each convolutional layer in a convolutional neural network model is generally represented in the form of feature vectors, and the output result corresponding to the last convolutional layer is input to a rectified linear unit (ReLU) for processing to obtain the output result, in order to reduce the computational load of the above-mentioned target transformation matrix calculation process and improve the computational efficiency of the target transformation matrix, the electronic device can input the output results of each convolutional layer into a ReLU for processing, thereby calculating the target transformation matrix based on the output result of the ReLU. In this case, the above-mentioned preset objective function can be expressed as:

[0133]

[0134] strank(M)≤k

[0135] Where min represents the minimum value operation, and n is the number of sample images in the second training set. For the Frobenius norm calculation, M is the target transformation matrix corresponding to the convolutional layer, and Y is... l Let be the first output result corresponding to the l-th sample image in the convolutional layer, st be the constraint condition, rank(M) be the rank of M, k be the rank of the preset matrix, B be the bias to be optimized, and r() be the nonlinear operator.

[0136] The above nonlinear operators are denoted as the computation process in ReLU.

[0137] In an optional embodiment, given a preset objective function and the first output result of each sample image in the second training set corresponding to each convolutional layer, the electronic device can use the Alternating Direction Method of Multipliers (ADMM), a precise penalty function based on augmented Lagrange multipliers, to calculate the target transformation matrix corresponding to each convolutional layer. The specific calculation process of the target transformation matrix is ​​not described here.

[0138] Through the aforementioned ADMM, the electronic device can accurately and quickly calculate the optimal solution of the preset objective function, ensuring stable convergence of the calculated optimal solution. This improves the computational efficiency of the target transformation matrix corresponding to each convolutional layer, as well as the accuracy of the calculated target transformation matrix.

[0139] Step S1032: Calculate the product of the target transformation matrix and the first weight matrix corresponding to the convolutional layer to obtain the second weight matrix corresponding to the convolutional layer in the target scene.

[0140] In this step, for each convolutional layer in the first network model described above, the electronic device can perform the transformation according to the preset transformation relationship, i.e., w j =M j W j The product of the target transformation matrix corresponding to the convolutional layer and the first weight matrix is ​​calculated to obtain the second weight matrix corresponding to the convolutional layer, which is to determine the second weight matrix corresponding to the same convolutional layer in the second network model mentioned above.

[0141] Step S1033: Based on the second weight matrix corresponding to each convolutional layer, update the weights of each convolutional layer in the first network model to obtain the second network model.

[0142] In this step, after calculating the second weight matrix corresponding to each convolutional layer in the first network model, the electronic device can update the weights corresponding to each convolutional layer in the first network model based on the second weight matrix to obtain the second network model. That is, according to the second weight matrix corresponding to each convolutional layer, the weights corresponding to the same convolutional layer in the first network model are updated to obtain the second network model.

[0143] Through steps S1031-S1033, the electronic device can use the preset objective function to calculate the second weight matrix corresponding to each convolutional layer, thereby updating the weights of each convolutional layer in the first network model and obtaining the second network model. This minimizes the error between the output results of each convolutional layer in the second network model and the same convolutional layer in the first network model when processing the same image, effectively ensuring the accuracy of the processing results obtained by the second network model in processing the image.

[0144] Furthermore, since the second weight matrix corresponding to each convolutional layer in this second network model is determined solely based on the second training set in the target scene, this second network model is better suited for processing images acquired in the target scene, improving its specificity and thus making it applicable to the target scene. In other words, while ensuring the accuracy of the image processing results, it also makes the second network model more suitable for the target scene.

[0145] In an optional embodiment, according to the above... Figure 1 The method shown in this application embodiment also provides a model generation method. For example... Figure 3 As shown, Figure 3This is a schematic diagram of a third process for the model generation method provided in an embodiment of this application. Figure 3 In the method shown, step S104 is further refined into the following steps, namely step S1041-step S1042.

[0146] Step S1041: For each convolutional layer in the second network model, perform singular value decomposition on the second weight matrix corresponding to the convolutional layer to obtain two third weight matrices corresponding to the convolutional layer.

[0147] In one optional embodiment, for each convolutional layer in the second network model, the electronic device may perform singular value decomposition on the second weight matrix corresponding to that convolutional layer using the following formula.

[0148] w=P*Σ* T

[0149] Where w is the second weight matrix mentioned above, P is an orthogonal matrix obtained from singular value decomposition, Σ is a diagonal matrix, P*Σ is one of the third weight matrices mentioned above, and Q T This is another orthogonal matrix obtained from singular value decomposition, namely the other one in the third weight matrix mentioned above.

[0150] For ease of understanding, let's take the above example where the size of w is m*n. Then, the size of P is m*m, the size of Σ is m*n, and each element on the main diagonal of Σ is a symmetric matrix ww. T The arithmetic square root of the non-zero eigenvalues, Q T The specification is n*n. T is the transpose operation.

[0151] In an optional embodiment, the number of dimensions corresponding to the two third weight matrices mentioned above can be expressed as:

[0152]

[0153] Where o is the number of output channels of the convolutional layer, i is the number of input channels of the convolutional layer, and k w k represents the width of the convolutional kernel corresponding to the convolutional layer. h This represents the height of the convolutional kernel corresponding to the convolutional layer.

[0154] Step S1042: Based on the two third weight matrices corresponding to the convolutional layer, decompose the convolutional layer into two sub-convolutional layers to obtain the third network model corresponding to the target scene.

[0155] In this step, for each convolutional layer in the second network model described above, the electronic device can decompose the convolutional layer into two sub-convolutional layers. The weight matrices corresponding to each sub-convolutional layer are the two third weight matrices corresponding to the convolutional layer. At this point, the electronic device can obtain the third network model corresponding to the target scene.

[0156] Through the above step S1042, for each convolutional layer, the convolutional kernel corresponding to that convolutional layer can be changed from the original i*o*k... w * h Convert to: i*k*k w1 * h1 and k*o*k w2 * h2 For example, if a convolutional layer has the specification of 256*256*3*3, when k is 64, the convolutional kernel will be converted to 256*64*3*3 and 64*256*1*1.

[0157] Regarding the above i*o*k w * h If the above k w1 * h1 For k w * h k w2 * h2 If the value is 1*1, then k*o*k w2 * h2 The convolution kernel calculation only requires i*o*k. w * h of

[0158] Through steps S1041-S1042, the electronic device can decompose the second network model to obtain a third network model for the target scene. In this third network model, since the third weight matrix corresponding to every two convolutional layers is obtained by performing singular value decomposition on the second weight matrix corresponding to one convolutional layer in the second network model, this effectively reduces the rank of the weight matrix corresponding to each convolutional layer in the third network model. This ensures the accuracy of the output results of each convolutional layer while eliminating redundant information in the second weight matrix—that is, deleting parameters in the second weight matrix that are highly relevant to the target scene, and retaining parameters in the second weight matrix that are highly relevant to the target scene—effectively accelerating the convolution operation efficiency of the third network model.

[0159] Regarding the second network model mentioned above, although the second weight matrix corresponding to each convolutional layer in the second network model is determined based on the first output result corresponding to each sample image in the second training set, meaning that the second network model is applicable to the target scene, the second weight matrix corresponding to each convolutional layer in the second network model satisfies the aforementioned preset transformation relationship with the first weight matrix corresponding to the same convolutional layer in the first network model, and the first network model is applicable to all scenes. Therefore, the second weight matrix corresponding to each convolutional layer in the second network model contains parameters that are unrelated to or have low relevance to the target scene. These parameters will not be used in the process of image convolution processing using the third network model. For example, when the target scene is a rainy scene, since the features of a sunny scene cannot be extracted from the image of a rainy scene, the parameters corresponding to the sunny scene do not need to be reflected in the convolution operation of the image of the rainy scene. In this embodiment of the application, by using the above-mentioned singular value decomposition method, parameters (i.e. redundant information) that are irrelevant to the target scene or have low relevance to the second weight matrix can be effectively removed, while parameters that are highly relevant to the target scene in the second weight matrix are retained. This reduces the impact of redundant information on the image convolution operation process of the third network model and improves the convolution operation efficiency of the third network model.

[0160] In an optional embodiment, when the second training set further includes label information corresponding to each sample image, according to the above... Figure 1 The method shown in this application embodiment also provides a model generation method. For example... Figure 4 As shown, Figure 4 This is a schematic diagram of the fourth process of the model generation method provided in the embodiments of this application. Figure 4 The method shown has been augmented with the following steps, namely steps S105-S107.

[0161] Step S105: Input each sample image in the second training set into the third network model to obtain the third output result corresponding to each sample image.

[0162] In this step, for each sample image in the second training set, the electronic device can input the sample image into the third network model to obtain the output result of the third network model for the sample image (denoted as the third output result).

[0163] In the embodiments of this application, the third output result will vary depending on the purpose of the third network model. For example, when the third network model is an object detection model, the third output result can be the location of the detected target object in the sample image. As another example, when the third network model is an object recognition model, the third output result can be the location of the recognized target object in the sample image, as well as the category of the target object. Here, no specific limitation is made on the third output result.

[0164] Step S106: Calculate the loss value of the third network model based on the third output result and label information corresponding to each sample image.

[0165] In this step, the electronic device can calculate the loss value of the third network model based on the third output result and label information corresponding to each sample image, using a preset loss function.

[0166] The aforementioned preset loss function can be, for example, the squared loss function or the cross-entropy loss function. No specific limitations are imposed on the preset loss function here.

[0167] Step S107: When it is determined that the third network model has not converged based on the loss value, the parameters of the third network model are adjusted according to the preset step size. Based on the adjusted parameters, the process returns to the step of inputting each sample image in the second training set into the third network model to obtain the third output result corresponding to each sample image, until the third network model converges.

[0168] In this embodiment of the application, after calculating the loss value of the third network model, the electronic device can compare the loss value with a preset loss value. When the loss value is greater than the preset loss value, the electronic device can determine that the third network model has not converged; when the loss value is not greater than the preset loss value, the electronic device can determine that the third network model has converged.

[0169] When the third network model fails to converge, the electronic device can adjust the parameters of the third network model according to a preset step size. Based on the adjusted parameters, the electronic device can return to step S105, which is the step of inputting each sample image in the second training set into the third network model to obtain the third output result corresponding to each sample image, until the third network model converges.

[0170] The parameters in the aforementioned third network model can be numerical values ​​such as weights and biases. The parameters of this third network model can be adjusted using methods such as gradient descent or backpropagation. Here, no specific limitations are made regarding the parameters of the aforementioned third network model or the methods for adjusting them.

[0171] In an optional embodiment, the parameter adjustments to the third network model described above are merely fine-tuning. Therefore, the preset step size is a small value, such as less than a preset threshold.

[0172] Through steps S105-S107 above, after generating the third network model corresponding to the target scene, the electronic device retrains the third network model using the second training set corresponding to the target scene. Since the target scene is clear and the amount of data in the second training set is much smaller than that in the first training set, the retraining process of the third network model is relatively less complex and computationally intensive than the training process of the first network model, making it easier to train large-scale convolutional neural network models suitable for the target scene.

[0173] In addition, since the training data used in the retraining process of the third network model is only the sample images corresponding to the target scene, the retrained convolutional neural network model can be more applicable to the processing of images collected in the target scene, thus improving the accuracy and relevance of the retrained convolutional neural network model.

[0174] In the above Figure 4 In the illustrated embodiment, the retraining process of the third network model is based on the second training set. In addition, the electronic device can also retrain the third network model using a validation set or test set corresponding to the target scene. The validation set or test set also includes multiple sample images corresponding to the target scene and label information corresponding to each sample image. The method for obtaining the validation set or test set can refer to the method for obtaining the training set described above, and will not be specifically described here.

[0175] In the above Figures 1-4 In the illustrated embodiment, the generation of the convolutional neural network model corresponding to the target scene is used as an example for explanation. In addition, the electronic device can also generate the corresponding convolutional neural network model for any one or any number of the above-mentioned scenarios using the same method, following the model generation method described above.

[0176] In the embodiments of this application, the convolutional neural network models for different scenarios generated by the above method are highly coupled because each convolutional neural network model is obtained by decomposing the model structure and model parameters of the first network model.

[0177] Based on the same inventive concept, when the aforementioned third network model is a target detection model, according to the model generation method provided in the embodiments of this application, the embodiments of this application also provide a target detection method. For example... Figure 5 As shown, Figure 5This is a schematic flowchart of a target detection method provided in an embodiment of this application. The method includes the following steps.

[0178] Step S501: Obtain the image to be detected collected in the target scene.

[0179] In this step, the image acquisition device can acquire images in the target scene to obtain acquired images. The electronic device can then use these acquired images as the images to be detected.

[0180] The image acquisition equipment mentioned above can be cameras, radar, or similar devices. The images to be detected are described in the sample images above and will not be further detailed here.

[0181] Step S502: Input the image to be detected into the third network model to obtain the target detection result.

[0182] In this step, after acquiring the image to be detected, the electronic device can input the image to be detected into the third network model for target detection and obtain the output result of the third network model, that is, the target detection result of the third network model for the image to be detected. The target detection process for the image to be detected will not be described in detail here.

[0183] The third network model mentioned above can be a convolutional neural network model obtained by decomposing the convolutional layers in the second network model, or it can be a convolutional neural network model obtained by retraining the third network model.

[0184] Through the above Figure 5 The method shown allows the electronic device to perform target detection on the images collected in the target scene using the third network model described above, and obtain the detection results. Compared with using the first network model to perform target detection on the images collected in the target scene, the efficiency of convolution calculation is significantly improved in the target detection process shown in steps S501-S502.

[0185] Furthermore, when the aforementioned third network model is the above... Figure 4 In the embodiment shown, the third network model obtained through retraining is more suitable for the target scene than the first network model described above, which makes the target detection results obtained by the target detection process shown in steps S501-S502 more accurate.

[0186] In the above Figure 5 In the illustrated embodiments, the application of the third network model described above is only illustrated through the process of detecting target objects in an image. Besides this, the third network model can also be used for target object recognition in images, point-based semantic segmentation, and other processes. The application of the third network model is not specifically limited here.

[0187] Based on the same inventive concept, and according to the model generation method provided in the above embodiments of this application, this application also provides a model generation apparatus. For example... Figure 6 As shown, Figure 6 This is a schematic diagram of a model generation apparatus provided in an embodiment of this application. The apparatus includes the following modules.

[0188] The first acquisition module 601 is used to acquire a convolutional neural network model trained based on a first training set as a first network model, and to acquire a second training set corresponding to the target scene in the first training set; wherein, the first training set includes sample images corresponding to multiple scenes.

[0189] The second acquisition module 602 is used to input each sample image in the second training set into the first network model and acquire the first output result corresponding to each convolutional layer in the first network model.

[0190] The first calculation module 603 is used to calculate the second weight matrix corresponding to the convolutional layer in the target scene for each convolutional layer in the first network model, based on the first output result corresponding to the convolutional layer for each sample image in the second training set and the first weight matrix corresponding to the convolutional layer, so as to obtain the second network model.

[0191] The decomposition module 604 is used to decompose each convolutional layer in the second network model based on the third weight matrix obtained by decomposing the second weight matrix corresponding to the convolutional layer, and obtain the third network model corresponding to the target scene.

[0192] Optionally, the first calculation module 603 described above can be used to calculate the target transformation matrix corresponding to each convolutional layer in the first network model based on the first output result corresponding to each sample image in the second training set and using a preset objective function for each convolutional layer.

[0193] Calculate the product of the target transformation matrix corresponding to the convolutional layer and the first weight matrix to obtain the second weight matrix corresponding to the convolutional layer in the target scene;

[0194] Based on the second weight matrix corresponding to each convolutional layer, the weights of each convolutional layer in the first network model are updated to obtain the second network model.

[0195] Optionally, the aforementioned preset objective function is a calculation function for the optimal solution when the output error corresponding to the same convolutional layer in the first network model and the second network model is minimized under the preset matrix rank constraint.

[0196] Optionally, the above-mentioned preset objective function is:

[0197]

[0198] strank(M)≤k

[0199] or,

[0200]

[0201] strank(M)≤k

[0202] Where min represents the minimum value operation, and n is the number of sample images in the second training set. For the Frobenius norm calculation, M is the target transformation matrix corresponding to the convolutional layer, and Y is... l Let be the first output result corresponding to the l-th sample image in the convolutional layer, st be the constraint condition, rank(M) be the rank of M, k be the rank of the preset matrix, B be the bias to be optimized, and r() be the nonlinear operator.

[0203] Optionally, the decomposition module 604 can be used to perform singular value decomposition on the second weight matrix corresponding to each convolutional layer in the second network model to obtain two third weight matrices corresponding to the convolutional layer.

[0204] Based on the two third weight matrices corresponding to the convolutional layer, the convolutional layer is decomposed into two sub-convolutional layers to obtain the third network model corresponding to the target scene.

[0205] Optionally, the second training set mentioned above also includes label information corresponding to each sample image;

[0206] The above-mentioned model generation apparatus may further include:

[0207] The determination module is used to input each sample image in the second training set into the third network model to obtain the third output result corresponding to each sample image;

[0208] The second calculation module is used to calculate the loss value of the third network model based on the third output result and label information corresponding to each sample image.

[0209] The adjustment module is used to adjust the parameters of the third network model according to a preset step size when it is determined that the third network model has not converged based on the loss value. Based on the adjusted parameters, the determination module is called to return and execute the step of inputting each sample image in the second training set into the third network model to obtain the third output result corresponding to each sample image, until the third network model converges.

[0210] Optionally, if the third network model is an object detection model, the model generation device may further include:

[0211] The third acquisition module is used to acquire the image to be detected collected in the target scene;

[0212] The detection module is used to input the image to be detected into the third network model to obtain the target detection result.

[0213] The apparatus provided in this application, after obtaining a convolutional neural network model trained on a first training set (i.e., a first network model) and a second training set corresponding to the target scene in the first training set, inputs each sample data from the second training set into the first network model to obtain a first output result corresponding to each convolutional layer in the first network model. For each convolutional layer in the first network model, based on the first output result and the first weight matrix corresponding to that convolutional layer, the first network model is converted into a second network model under the target scene. Then, each convolutional layer in the second network model is decomposed to obtain a third network model, i.e., the convolutional neural network model corresponding to the target scene, thus realizing the generation of the convolutional neural network model corresponding to the target scene.

[0214] Furthermore, since the decomposition of each convolutional layer in the second network model is based on the decomposition of the second weight matrix corresponding to each convolutional layer to obtain the third weight matrix, the rank of the matrix corresponding to the third weight matrix obtained by decomposing the second weight matrix is ​​reduced. This can remove redundant information in the second weight matrix corresponding to each convolutional layer in the second network model, that is, remove parameters in the second weight matrix that have low correlation with the target scene, and retain parameters in the second weight matrix that have high correlation with the target scene. As a result, the redundant information in the original first network model is removed in the decomposed third network model, that is, the redundant information in the large-scale convolutional neural network model is removed, which speeds up the convolution operation speed of the decomposed third network model, that is, improves the convolution operation speed of the generated convolutional neural network model.

[0215] Based on the same inventive concept, and according to the model generation method provided in the above embodiments of this application, this application also provides an electronic device, such as... Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703, and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0216] Memory 703 is used to store computer programs;

[0217] When processor 701 executes a program stored in memory 703, it performs the following steps:

[0218] Obtain a convolutional neural network model trained on a first training set as the first network model, and obtain a second training set corresponding to the target scene in the first training set; wherein, the first training set includes sample images corresponding to multiple scenes;

[0219] Each sample image in the second training set is input into the first network model, and the first output result corresponding to each convolutional layer in the first network model is obtained.

[0220] For each convolutional layer in the first network model, based on the first output result of each sample image in the second training set corresponding to the convolutional layer and the first weight matrix corresponding to the convolutional layer, the second weight matrix corresponding to the convolutional layer in the target scene is calculated to obtain the second network model.

[0221] For each convolutional layer in the second network model, the third weight matrix obtained by decomposing the second weight matrix corresponding to the convolutional layer is used to decompose the convolutional layer to obtain the third network model corresponding to the target scene.

[0222] The electronic device provided in this application, after obtaining a convolutional neural network model trained on a first training set (i.e., a first network model) and a second training set corresponding to the target scene in the first training set, inputs each sample data from the second training set into the first network model to obtain a first output result corresponding to each convolutional layer in the first network model. For each convolutional layer in the first network model, based on the first output result and the first weight matrix corresponding to that convolutional layer, the first network model is converted into a second network model under the target scene. Then, each convolutional layer in the second network model is decomposed to obtain a third network model, i.e., the convolutional neural network model corresponding to the target scene, thus realizing the generation of the convolutional neural network model corresponding to the target scene.

[0223] Furthermore, since the decomposition of each convolutional layer in the second network model is based on the decomposition of the second weight matrix corresponding to each convolutional layer to obtain the third weight matrix, the rank of the matrix corresponding to the third weight matrix obtained by decomposing the second weight matrix is ​​reduced. This can remove redundant information in the second weight matrix corresponding to each convolutional layer in the second network model, that is, remove parameters in the second weight matrix that have low correlation with the target scene, and retain parameters in the second weight matrix that have high correlation with the target scene. As a result, the redundant information in the original first network model is removed in the decomposed third network model, that is, the redundant information in the large-scale convolutional neural network model is removed, which speeds up the convolution operation speed of the decomposed third network model, that is, improves the convolution operation speed of the generated convolutional neural network model.

[0224] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0225] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0226] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0227] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0228] Based on the same inventive concept, and according to the model generation method provided in the above embodiments of this application, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above model generation methods.

[0229] Based on the same inventive concept, and according to the model generation method provided in the above embodiments of this application, this application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute any of the model generation methods in the above embodiments.

[0230] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0231] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0232] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, embodiments such as apparatuses, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0233] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A model generation method, characterized in that, The method includes: Obtain a convolutional neural network model trained on a first training set as a first network model, and obtain a second training set corresponding to the target scene in the first training set; wherein, the first training set includes sample images corresponding to multiple scenes; Each sample image in the second training set is input into the first network model, and the first output result corresponding to each convolutional layer in the first network model is obtained. For each convolutional layer in the first network model, based on the first output result corresponding to each sample image in the second training set, a target transformation matrix corresponding to the convolutional layer is calculated using a preset objective function; the product of the target transformation matrix corresponding to the convolutional layer and the first weight matrix is ​​calculated to obtain the second weight matrix corresponding to the convolutional layer in the target scene; based on the second weight matrix corresponding to each convolutional layer, the weights of each convolutional layer in the first network model are updated to obtain the second network model; the preset objective function is a calculation function that calculates the optimal solution when the output error corresponding to the same convolutional layer in the first network model and the second network model is minimized under a preset matrix rank constraint; For each convolutional layer in the second network model, the third weight matrix obtained by decomposing the second weight matrix corresponding to the convolutional layer is used to decompose the convolutional layer to obtain the third network model corresponding to the target scene.

2. The method according to claim 1, characterized in that, The preset objective function is: ； ； or, ； ； in, For minimum value operation, The number of sample images in the second training set. Calculate the Frobenius norm. This is the target transformation matrix corresponding to the convolutional layer. This is the first output result corresponding to the l-th sample image in the convolutional layer. As constraints, for rank, The rank of the matrix is ​​given by a predefined formula. The bias to be optimized. It is a nonlinear operator.

3. The method according to claim 1, characterized in that, The step of decomposing each convolutional layer in the second network model based on the third weight matrix obtained by decomposing the second weight matrix corresponding to that convolutional layer to obtain the third network model corresponding to the target scene includes: For each convolutional layer in the second network model, singular value decomposition is performed on the second weight matrix corresponding to the convolutional layer to obtain two third weight matrices corresponding to the convolutional layer. Based on the two third weight matrices corresponding to the convolutional layer, the convolutional layer is decomposed into two sub-convolutional layers to obtain the third network model corresponding to the target scene.

4. The method according to claim 1, characterized in that, The second training set also includes label information for each sample image; The method further includes: Each sample image in the second training set is input into the third network model to obtain the third output result corresponding to each sample image; The loss value of the third network model is calculated based on the third output result and label information corresponding to each sample image. When it is determined that the third network model has not converged based on the loss value, the parameters of the third network model are adjusted according to a preset step size. Based on the adjusted parameters, the process of inputting each sample image in the second training set into the third network model to obtain the third output result corresponding to each sample image is repeated until the third network model converges.

5. The method according to any one of claims 1-4, characterized in that, If the third network model is an object detection model, then the method further includes: Acquire the image to be detected collected in the target scene; The image to be detected is input into the third network model to obtain the target detection result.

6. A model generation apparatus, characterized in that, The device includes: The first acquisition module is used to acquire a convolutional neural network model trained based on a first training set as a first network model, and to acquire a second training set corresponding to the target scene in the first training set; wherein, the first training set includes sample images corresponding to multiple scenes. The second acquisition module is used to input each sample image in the second training set into the first network model and acquire the first output result corresponding to each convolutional layer in the first network model. The first calculation module is used to calculate the target transformation matrix corresponding to each convolutional layer in the first network model based on the first output result corresponding to each sample image in the second training set, using a preset objective function; calculate the product of the target transformation matrix corresponding to the convolutional layer and the first weight matrix to obtain the second weight matrix corresponding to the convolutional layer in the target scene; and update the weights of each convolutional layer in the first network model based on the second weight matrix corresponding to each convolutional layer to obtain the second network model; the preset objective function is the calculation function of the optimal solution when the output error corresponding to the same convolutional layer in the first network model and the second network model is minimized under the preset matrix rank constraint. The decomposition module is used to decompose each convolutional layer in the second network model based on the third weight matrix obtained by decomposing the second weight matrix corresponding to the convolutional layer, thereby obtaining the third network model corresponding to the target scene.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-5.

Images