Neural network generation method and device, electronic device and storage medium

Abstract

The present disclosure relates to a neural network generation method and device, an electronic device and a storage medium, the method comprising: in response to a network setting operation for a target neural network, performing network setting according to a selected network block to obtain the target neural network; in response to a training operation for the target neural network, training the target neural network according to a preset target data set to obtain a trained target neural network; encapsulating the trained target neural network to obtain an encapsulation file of the target neural network; and sending the encapsulation file to a second electronic device so that the second electronic device deploys the target neural network according to the encapsulation file. The embodiments of the present disclosure can improve the training efficiency and accuracy of the target neural network.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a neural network generation method and device, an electronic device, and a storage medium. Background Art

[0002] With the rapid development of computer vision, object detection technology has important applications in many areas of life. For example, the first step of the camera's smile capture function is to detect the face, and then smile recognition and capture. Another example: in the field of autonomous driving, it is necessary to detect and identify pedestrians, vehicles, and obstacles at all times to achieve safe driving. Summary of the invention

[0003] The present disclosure proposes a technical solution for generating a neural network.

[0004] According to one aspect of the present disclosure, a neural network generation method is provided, which is applied to a first electronic device, and the method includes:

[0005] In response to a network setting operation for a target neural network, performing network setting according to the selected network block to obtain the target neural network;

[0006] In response to a training operation on the target neural network, training the target neural network according to a preset target data set to obtain a trained target neural network;

[0007] Encapsulating the trained target neural network to obtain a package file of the target neural network;

[0008] The packaging file is sent to a second electronic device so that the second electronic device deploys the target neural network according to the packaging file.

[0009] In a possible implementation, before training the target neural network according to a preset target data set, the method further includes:

[0010] In response to a data setting operation for a target neural network, a selected data set is confirmed, and the target data set is generated based on the selected data set.

[0011] In a possible implementation, the target neural network includes a feature extraction subnetwork and a task branch subnetwork.

[0012] The step of performing network settings according to the selected network block to obtain the target neural network includes:

[0013] According to the first network block corresponding to the feature extraction subnetwork and the second network block corresponding to the task branch subnetwork in the network block, respectively generating the feature extraction subnetwork and the task branch subnetwork;

[0014] The target neural network is obtained according to the feature extraction subnetwork and the task branch subnetwork.

[0015] In a possible implementation, the method further includes:

[0016] For a third network block implemented by using a third-party language, encapsulating the third network block by using a preset format to obtain a fourth network block;

[0017] The performing network settings according to the selected network block includes:

[0018] When the selected network blocks include the third network block, a fourth network block corresponding to the third network block is called.

[0019] In a possible implementation, the training of the target neural network according to a preset target data set to obtain a trained target neural network includes:

[0020] splitting the target data set into a plurality of sub-data sets according to a preset number of processing processes of a third electronic device, the third electronic device comprising a distributed processing device;

[0021] Sending each sub-data set and the target neural network to the processing process of the third electronic device respectively, so that the processing process of the third electronic device trains the target neural network in parallel to obtain network parameters of the target neural network;

[0022] Upon receiving the network parameters sent by the third electronic device, a trained target neural network is determined.

[0023] In a possible implementation, encapsulating the trained target neural network to obtain an encapsulation file of the target neural network includes:

[0024] According to a target format supported by the second electronic device, converting the trained target neural network into a target format to obtain a target neural network in the target format;

[0025] The target neural network in the target format is packaged to obtain the packaged file.

[0026] In a possible implementation, converting the format of the trained target neural network according to the target format supported by the second electronic device to obtain the target neural network in the target format includes:

[0027] Based on the open neural network exchange ONNX protocol, the trained target neural network is converted from an original format to a target format supported by the second electronic device to obtain a target neural network in the target format.

[0028] According to one aspect of the present disclosure, a neural network generation device is provided, which is applied to a first electronic device, and the device includes:

[0029] A network setting module, configured to respond to a network setting operation for a target neural network, perform network setting according to a selected network block, and obtain the target neural network;

[0030] A training module, configured to, in response to a training operation on the target neural network, train the target neural network according to a preset target data set to obtain a trained target neural network;

[0031] A packaging module is used to package the trained target neural network, obtain a packaging file of the target neural network, and send the packaging file to a second electronic device so that the second electronic device deploys the target neural network according to the packaging file.

[0032] In a possible implementation manner, the device further includes:

[0033] The data definition module is used to confirm the selected data set in response to the data setting operation for the target neural network, and generate the target data set based on the selected data set.

[0034] In a possible implementation, the target neural network includes a feature extraction subnetwork and a task branch subnetwork, and the network setting module is further used to:

[0035] According to the first network block corresponding to the feature extraction subnetwork and the second network block corresponding to the task branch subnetwork in the network block, respectively generating the feature extraction subnetwork and the task branch subnetwork;

[0036] The target neural network is obtained according to the feature extraction subnetwork and the task branch subnetwork.

[0037] In a possible implementation manner, the device further includes:

[0038] An extension module, configured to encapsulate a third network block implemented in a third-party language in a preset format to obtain a fourth network block;

[0039] The network setting module is also used for:

[0040] When the selected network blocks include the third network block, a fourth network block corresponding to the third network block is called.

[0041] In a possible implementation, the training module is further used to:

[0042] splitting the target data set into a plurality of sub-data sets according to a preset number of processing processes of a third electronic device, the third electronic device comprising a distributed processing device;

[0043] Sending each sub-data set and the target neural network to the processing process of the third electronic device respectively, so that the processing process of the third electronic device trains the target neural network in parallel to obtain network parameters of the target neural network;

[0044] Upon receiving the network parameters sent by the third electronic device, a trained target neural network is determined.

[0045] In a possible implementation, the packaging module is further used to:

[0046] According to a target format supported by the second electronic device, converting the trained target neural network into a target format to obtain a target neural network in the target format;

[0047] The target neural network in the target format is packaged to obtain the packaged file.

[0048] In a possible implementation, the packaging module is further used to:

[0049] Based on the open neural network exchange ONNX protocol, the trained target neural network is converted from an original format to a target format supported by the second electronic device to obtain a target neural network in the target format.

[0050] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0051] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0052] In an embodiment of the present disclosure, a first electronic device may respond to a network setup operation for a target neural network, perform network setup according to a selected network block, obtain the target neural network, and respond to a training operation for the target neural network, train the target neural network according to a preset target data set, obtain a trained target neural network, and package the trained target neural network to obtain a package file of the target neural network. The first electronic device may send the package file to a second electronic device, so that the second electronic device deploys the target neural network according to the package file. According to the neural network generation method and apparatus, electronic device, and storage medium provided in an embodiment of the present disclosure, a target neural network can be quickly defined through pre-integrated network blocks, thereby improving the training efficiency and accuracy of the target neural network.

[0053] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0055] Figure 1 A flowchart showing a method for generating a neural network according to an embodiment of the present disclosure is shown;

[0056] Figure 2 A schematic diagram showing a neural network generation method according to an embodiment of the present disclosure;

[0057] Figure 3 A schematic diagram showing a neural network generation method according to an embodiment of the present disclosure;

[0058] Figure 4 A block diagram showing a neural network generating device according to an embodiment of the present disclosure;

[0059] Figure 5 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown;

[0060] Figure 6 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0061] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0062] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0063] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0064] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0065] Providing an efficient and accurate object detection network requires consideration of many aspects. First, there is the problem of detection accuracy. Traditional object detection methods are difficult to meet the needs of practical applications due to limitations in methods and data. Secondly, the training process of the object detection network based on deep learning requires training on massive sample data, so an efficient training method is needed. Finally, in the deployment of the network, it is often necessary to convert the network format so that it can be deployed on different platforms to accelerate reasoning.

[0066] The disclosed embodiments provide a neural network generation method that can efficiently generate a high-precision, high-performance, and easily deployable target neural network.

[0067] Figure 1 FIG. 1 is a flowchart of a neural network generation method according to an embodiment of the present disclosure, and the neural network generation method can be executed by a server. Figure 1 As shown, the neural network generation method may include:

[0068] In step S11, in response to a network setting operation for a target neural network, network setting is performed according to the selected network block to obtain the target neural network.

[0069] For example, the first electronic device may be a server running a network framework, and the network framework is a system for developing neural networks. The user may call the setting interface (config interface) of the target neural network and set the target neural network in the setting interface. Among them, the network setting operation for the target neural network may include a selection operation for the network block called in the target neural network, and the user may select the network block to be called by means of a command line, for example: writing identification information such as the name of the selected network block in the command line. Furthermore, according to the setting operation for the target neural network, the network blocks used by each module in the target neural network may be selected, and the target neural network may be formed according to the selected network blocks.

[0070] Exemplarily, a plurality of network blocks may be pre-integrated in the network framework, wherein the network block may include a network structure and / or a network algorithm, the network structure may include at least one of the ResNet series, the ResNeXt series, the SENet series, the MobileNet, the ShuffleNet, and the EfficientNet series, and the network algorithm may include at least one of the Faster R-CNN, the Mask R-CNN, the Keypoint R-CNN, the Cascade R-CNN, the Grid R-CNN, the RFCN, the RetinaNet, the GHM, the ATSS, the EfficientDet, the OHEM, the DCN, and the EQL series. The present disclosure does not limit the specific types of network structures and network algorithms.

[0071] The neural network generation method provided by the embodiment of the present disclosure pre-integrates network structures and network algorithms that have their own advantages and disadvantages in terms of detection accuracy and detection efficiency. The selected network blocks can be called from the pre-integrated network blocks to obtain the target neural network. This can reduce the limitations of the network algorithm and network structure, flexibly design the target neural network, and quickly carry out algorithm innovation.

[0072] In step S12, in response to a training operation for the target neural network, the target neural network is trained according to a preset target data set to obtain a trained target neural network.

[0073] For example, after obtaining the target neural network, the target neural network can be trained according to a preset target data set, wherein the target data set includes a plurality of sample data, each of which has labeling information.

[0074] The above-mentioned training operation for the target neural network may be a trigger operation for a control, and the control may be a control for triggering a training instruction. Alternatively, the above-mentioned training operation for the target neural network may be a training instruction input through a command line. In response to the training instruction, the target data set may be called to train the target neural network. For example, the sample data in the target data set may be used as input information of the target neural network, and the network loss of the target neural network may be determined based on the output information of the target neural network and the annotation information of the sample data, and then the network parameters of the target neural network may be adjusted based on the network loss. The specific training process of the target neural network will not be described in detail in the embodiments of the present disclosure.

[0075] In step S13, the trained target neural network is packaged to obtain a package file of the target neural network;

[0076] In step S14, the encapsulation file is sent to a second electronic device so that the second electronic device deploys the target neural network according to the encapsulation file.

[0077] For example, the second electronic device may be a terminal device or a server, etc., and the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. After the training is completed, the first electronic device may encapsulate the trained target neural network to obtain an encapsulation file of the target neural network, and may send the encapsulation file to the second electronic device, so that the second electronic device may directly deploy the target neural network according to the encapsulation file, for example: the second electronic device may deploy the target neural network into a corresponding application.

[0078] In this way, the first electronic device can respond to the network setting operation for the target neural network, perform network setting according to the selected network block to obtain the target neural network, and respond to the training operation for the target neural network, train the target neural network according to a preset target data set to obtain the trained target neural network, and encapsulate the trained target neural network to obtain the encapsulation file of the target neural network. The first electronic device can send the encapsulation file to the second electronic device so that the second electronic device deploys the target neural network according to the encapsulation file. According to the neural network generation method provided by the embodiment of the present disclosure, the target neural network can be quickly defined through pre-integrated network blocks, thereby improving the training efficiency and accuracy of the target neural network.

[0079] In a possible implementation, before training the target neural network in step S12, the method may further include:

[0080] In response to a data setting operation for a target neural network, a selected data set is confirmed, and the target data set is generated based on the selected data set.

[0081] For example, the network framework or the first electronic device may pre-store at least one data set locally, each data set including a plurality of sample data, each sample data having annotation information. In response to a user's selection operation on the target data set, a target data set for training the target neural network may be determined. Exemplarily, the user may specify the target data set by means of a command line, for example, by inputting identification information such as the name of the target data set through the command line, or by inputting path information of the target data set through the command line.

[0082] For example, when a mobile phone manufacturer is developing a face clustering function for an album, they can collect a large number of album pictures and store the collected album images in a local album folder. They can then enter the path information of the album folder in the command line to identify the album folder as a target data set, train a face detection network with the target data set, and then encapsulate the trained face detection network into the face clustering function to implement the face clustering function for the album.

[0083] In this way, the embodiments of the present disclosure can provide a rich training set for training the target neural network, and can also customize the target data set for training the target neural network, alleviating the limitations of data on the development of the target neural network and facilitating the flexible development of the target neural network.

[0084] In a possible implementation, the target neural network includes a feature extraction subnetwork and a task branch subnetwork. In step S11, network settings are performed according to the selected network blocks to obtain the target neural network, which may include:

[0085] According to the first network block corresponding to the feature extraction subnetwork and the second network block corresponding to the task branch subnetwork in the network block, respectively generating the feature extraction subnetwork and the task branch subnetwork;

[0086] The target neural network is obtained according to the feature extraction subnetwork and the task branch subnetwork.

[0087] For example, the target neural network can be divided into a feature extraction subnetwork and at least one task branch subnetwork, that is, the target neural network can be divided into a "feature extraction subnetwork + task branch subnetwork" structure. The feature extraction subnetwork is a network used for feature extraction, which may include a backbone and a neck, wherein the backbone is a main network used for feature extraction, and the neck is a network for fusing the extracted features. The task branch subnetwork is a network that uses the features extracted by the feature extraction subnetwork for reasoning, and may include different heads.

[0088] Exemplarily, the user can configure the first network block corresponding to the feature extraction subnetwork and the second network block corresponding to the task branch subnetwork in the config interface corresponding to the target neural network. For example, the identification information of the first network block can be input in the command line corresponding to the feature subnetwork in the config interface to configure the feature subnetwork; the identification information of the second network block can be input in the command line corresponding to the task branch subnetwork to configure the task branch subnetwork; and then the target neural network can be obtained according to the feature subnetwork and the task branch subnetwork.

[0089] In this way, the disclosed embodiment designs the target neural network through a modular, low-coupling, and high-cohesion design structure, flexibly supports a variety of network constructions and algorithm extensions, and facilitates user operation to generate the corresponding target neural network.

[0090] In a possible implementation, the method may further include:

[0091] For a third network block implemented by using a third-party language, encapsulating the third network block by using a preset format to obtain a fourth network block;

[0092] In step S12, performing network settings according to the selected network block may include:

[0093] When the selected network blocks include the third network block, a fourth network block corresponding to the third network block is called.

[0094] For example, assuming that the development language used by the network framework is Python, for some computationally intensive operations, using Python will reduce the efficiency of the program, so a third language can be used for appropriate optimization, that is, a third network block in a third-party language is introduced and encapsulated in Python format, and a fourth network block can be obtained for the framework to call. For example: For a third network block implemented using C++ and CUDA, Python can be used for encapsulation for external interface calls.

[0095] When the user selects the third network block to build the target neural network based on factors such as operating efficiency, the second electronic device can call the fourth network block corresponding to the third network block. In this way, a compilation and linking process can be provided for the implementation of the third-party language, so that the third network block implemented by the third-party language becomes a dynamic library that can be directly called by Python.

[0096] In a possible implementation, in step S13, training the target neural network according to a preset target data set to obtain a trained target neural network may include:

[0097] splitting the target data set into a plurality of sub-data sets according to a preset number of processing processes of a third electronic device, the third electronic device comprising a distributed processing device;

[0098] Sending each sub-data set and the target neural network to multiple processing processes of the third electronic device respectively, so that the processing processes of the third electronic device train the target neural network in parallel to obtain network parameters of the target neural network;

[0099] Upon receiving the network parameters sent by the third electronic device, a trained target neural network is determined.

[0100] For example, in order to improve the training speed of the target neural network, a distributed training method can be used to train the target neural network. The third electronic device can include a distributed processing device, which can determine the preset number of processing processes of the third electronic device according to demand, split the target data set, and obtain a preset number of sub-data sets. The first electronic device can send each sub-data set and the target neural network to the processing process of the third electronic device respectively to train the target neural network in parallel.

[0101] It should be noted that when the first electronic device has a distributed training function, the third electronic device can be the same device as the first electronic device, for example, when the first electronic device is a server, the third electronic device can be the first electronic device. Alternatively, the third electronic device and the first electronic device can also be different devices, for example, when the first electronic device is a terminal device that does not have a distributed training function, the third electronic device can be a server, or when the first electronic device is a server, the third electronic device can be a server different from the first electronic device. Alternatively, when the first electronic device is a distributed cluster, the third electronic device can be a training node in the first electronic device.

[0102] For example, Figure 2As shown, the third electronic device can start n processing processes at the same time, and the first electronic device can divide the target data set into n equal parts to obtain n sub-data sets, and send each sub-data set and the target neural network to each processing process of the third electronic device respectively, and each processing process can independently perform an iterative training in parallel (for example: in processing process 1, the first sub-data set is used to iteratively train the target neural network, in processing process 2, the second sub-data set is used to iteratively train the target neural network, ..., in processing process n, the nth sub-data set is used to iteratively train the target neural network), and the first electronic device can synchronize (for example: calculate the average value) the network parameters of the target neural network obtained by the training of each processing process to obtain the network parameters of the target neural network and complete a complete distributed iterative training.

[0103] In this way, the disclosed embodiments provide a target neural network that can be trained in a distributed manner, which can improve the training speed of the target neural network, shorten the training cycle, obtain higher accuracy in a short time, and meet the production needs of large-scale training in the context of deep learning.

[0104] In a possible implementation, in step S13, encapsulating the trained target neural network to obtain an encapsulation file of the target neural network may include:

[0105] According to a target format supported by the second electronic device, converting the trained target neural network into a target format to obtain a target neural network in the target format;

[0106] The target neural network in the target format is packaged to obtain the packaged file.

[0107] For example, the formats supported when deploying the target neural network on different platforms may be different (the format of the target neural network may include at least one of PyTorch, Caffe (Convolutional Architecture for Fast Feature Embedding), C++ (the C++ Programming Language / cplus plus, object-oriented programming), and CUDA (Compute Unified Device Architecture)). The first electronic device can convert the format of the trained target neural network according to the target format supported by the second electronic device to obtain the target neural network in the target format.

[0108] In a possible implementation, converting the format of the trained target neural network according to the target format supported by the second electronic device to obtain the target neural network in the target format may include:

[0109] Based on the open neural network exchange ONNX protocol, the trained target neural network is converted from an original format to a target format supported by the second electronic device to obtain a target neural network in the target format.

[0110] Exemplarily, if the trained target neural network is in Python version, that is, the original format of the target neural network is PyTorch format, the target neural network of Python version can be converted into a target format based on the target format supported by the second electronic device, so that the trained target neural network can be converted from PyTorch format to target format based on the Open Neural Network Exchange (ONNX) protocol. Assuming that the target format is Caffe format, the model conversion tool can be used first to convert the target neural network in PyTorch format into a target neural network in Caffe format through the ONNX protocol, and then the converted target neural network in Caffe format can be accelerated and network integrated through the model serialization tool provided by the model conversion tool, and then the integrated target neural network in Caffe format can be packaged to obtain a lightweight network that can be directly run on the second electronic device. Among them, after the trained target neural network is converted into C++ format or CUDA format, the target neural network can be easily deployed on different platforms through dynamic linking technology.

[0111] In this way, the neural network generation method provided by the embodiment of the present disclosure can convert the format of the trained target neural network according to the target format supported by the second electronic device, and can realize end-to-end training and deployment of the target neural network, making the deployment of the target neural network easier.

[0112] In fact, the first electronic device can also directly convert the trained target neural network into multiple formats, and after respectively packaging the target neural networks in multiple formats, send the packaging file corresponding to the target format supported by the second electronic device to the second electronic device for corresponding deployment. The embodiments of the present disclosure are not limited to this.

[0113] In order to enable those skilled in the art to better understand the embodiments of the present disclosure, the embodiments of the present disclosure are described below through specific examples.

[0114] Reference Figure 3In the initialization environment, the user can construct a data set, and construct a target neural network through pre-integrated network blocks in the network framework, and use the constructed data set as the target data set for training the target neural network. The target neural network is trained using distributed training to obtain a trained target neural network. After obtaining the trained target neural network, the trained target neural network can be tested. After the test passes, the trained target neural network is converted to a corresponding format and packaged to obtain a packaged file of the target neural network. The packaged file of the target neural network is sent to the second electronic device to complete the delivery. The second electronic device can deploy the target neural network according to the packaged file.

[0115] According to the network generation method provided by the embodiment of the present disclosure, not only the target neural network is designed through a modular, low-coupling, and high-cohesion design structure, but also network structures and network algorithms with their own advantages and disadvantages in terms of detection accuracy and detection efficiency are pre-integrated, and an extension interface of a third-party language is provided, as well as a target data set for training the target neural network is constructed. This can alleviate the limitations of the method and data, flexibly construct the target neural network and train the target neural network, and improve the training efficiency and accuracy of the target neural network.

[0116] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0117] In addition, the present disclosure also provides a neural network generation device, an electronic device, a computer-readable storage medium, and a program, all of which can be used to implement any neural network generation method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0118] Figure 4 A block diagram of a neural network generating device according to an embodiment of the present disclosure is shown, which is applied to a first electronic device, such as Figure 4 As shown, the device may include:

[0119] A network setting module 41 may be used to respond to a network setting operation for a target neural network, perform network setting according to a selected network block, and obtain the target neural network;

[0120] A training module 42 may be used to respond to a training operation on the target neural network and train the target neural network according to a preset target data set to obtain a trained target neural network;

[0121] The packaging module 43 can be used to package the trained target neural network, obtain a packaging file of the target neural network, and send the packaging file to a second electronic device so that the second electronic device deploys the target neural network according to the packaging file.

[0122] In this way, the first electronic device can respond to the network setting operation for the target neural network, perform network setting according to the selected network block to obtain the target neural network, and respond to the training operation for the target neural network, train the target neural network according to a preset target data set to obtain the trained target neural network, and encapsulate the trained target neural network to obtain a package file of the target neural network. The first electronic device can send the package file to the second electronic device so that the second electronic device deploys the target neural network according to the package file. According to the network generation device provided in the embodiment of the present disclosure, the target neural network can be quickly defined through pre-integrated network blocks, thereby improving the training efficiency and accuracy of the target neural network.

[0123] In a possible implementation, the device may further include:

[0124] The data definition module 44 can be used to confirm the selected data set in response to the data setting operation for the target neural network, and generate the target data set based on the selected data set.

[0125] In a possible implementation, the target neural network includes a feature extraction subnetwork and a task branching subnetwork, and the network setting module 41 can also be used to:

[0126] According to the first network block corresponding to the feature extraction subnetwork and the second network block corresponding to the task branch subnetwork in the network block, respectively generating the feature extraction subnetwork and the task branch subnetwork;

[0127] The target neural network is obtained according to the feature extraction subnetwork and the task branch subnetwork.

[0128] In a possible implementation, the device may further include:

[0129] The expansion module 45 may be used to encapsulate the third network block implemented in a third-party language in a preset format to obtain a fourth network block;

[0130] The network setting module 41 can also be used for:

[0131] When the selected network blocks include the third network block, a fourth network block corresponding to the third network block is called.

[0132] In a possible implementation, the training module 42 may also be used to: split the target data set into a plurality of sub-data sets according to a preset number of processing processes of a third electronic device, the third electronic device including a distributed processing device;

[0133] Sending each sub-data set and the target neural network to the processing process of the third electronic device respectively, so that the processing process of the third electronic device trains the target neural network in parallel to obtain network parameters of the target neural network;

[0134] Upon receiving the network parameters sent by the third electronic device, a trained target neural network is determined.

[0135] In a possible implementation, the packaging module 43 may also be used to:

[0136] According to a target format supported by the second electronic device, converting the trained target neural network into a target format to obtain a target neural network in the target format;

[0137] The target neural network in the target format is packaged to obtain the packaged file.

[0138] In a possible implementation, the packaging module 43 may also be used to:

[0139] Based on the open neural network exchange ONNX protocol, the trained target neural network is converted from an original format to a target format supported by the second electronic device to obtain a target neural network in the target format.

[0140] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0141] The embodiment of the present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented. The computer-readable storage medium may be a non-volatile computer-readable storage medium.

[0142] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0143] The embodiments of the present disclosure also provide a computer program product, including computer-readable codes. When the computer-readable codes are executed on a device, a processor in the device executes instructions for implementing the neural network generation method provided in any of the above embodiments.

[0144] The embodiments of the present disclosure also provide another computer program product for storing computer-readable instructions, which, when executed, enable a computer to perform the operations of the neural network generation method provided in any of the above embodiments.

[0145] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0146] Figure 5 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0147] Reference Figure 5 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0148] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0149] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0150] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0151] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0152] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0153] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0154] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0155] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0156] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0157] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above method.

[0158] Figure 6 1 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Figure 6 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0159] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (Mac OSX TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0160] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0161] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0162] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0163] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0164] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0165] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0166] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0167] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0168] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0169] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0170] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for generating a neural network, characterized in that: Applied to a first electronic device, the method includes: In response to a network setting operation for a target neural network, network setting is performed according to a selected network block to obtain the target neural network, wherein the network setting operation includes: a selection operation of the network block called by the target neural network, the network block including a network structure and / or a network algorithm; In response to a training operation on the target neural network, training the target neural network according to a preset target data set to obtain a trained target neural network; Encapsulating the trained target neural network to obtain a package file of the target neural network; Sending the package file to a second electronic device so that the second electronic device deploys the target neural network according to the package file; Wherein, in response to the network setting operation for the target neural network, performing network setting according to the selected network block to obtain the target neural network includes: According to the selection operation, determining the selected network block used by each module in the target neural network; According to the selected network blocks, the target neural network is formed; Before training the target neural network according to the preset target data set, the method further includes: In response to a data setting operation for a target neural network, confirming a selected data set, and generating the target data set according to the selected data set; The step of training the target neural network according to a preset target data set to obtain a trained target neural network includes: splitting the target data set into a plurality of sub-data sets according to a preset number of processing processes of a third electronic device, the third electronic device comprising a distributed processing device; Sending each sub-data set and the target neural network to the processing process of the third electronic device respectively, so that the processing process of the third electronic device trains the target neural network in parallel to obtain network parameters of the target neural network; Upon receiving the network parameters sent by the third electronic device, a trained target neural network is determined.

2. The method according to claim 1, characterized in that The target neural network includes a feature extraction subnetwork and a task branch subnetwork. The step of performing network settings according to the selected network block to obtain the target neural network includes: According to the first network block corresponding to the feature extraction subnetwork and the second network block corresponding to the task branch subnetwork in the network block, respectively generating the feature extraction subnetwork and the task branch subnetwork; The target neural network is obtained according to the feature extraction subnetwork and the task branch subnetwork.

3. The method according to claim 1, characterized in that The method further comprises: For a third network block implemented by using a third-party language, encapsulating the third network block by using a preset format to obtain a fourth network block; The performing network settings according to the selected network block includes: When the selected network blocks include the third network block, a fourth network block corresponding to the third network block is called.

4. The method according to claim 1, characterized in that The step of encapsulating the trained target neural network to obtain an encapsulation file of the target neural network includes: According to a target format supported by the second electronic device, converting the trained target neural network into a target format to obtain a target neural network in the target format; The target neural network in the target format is packaged to obtain the packaged file.

5. The method according to claim 4, characterized in that The step of converting the format of the trained target neural network according to the target format supported by the second electronic device to obtain the target neural network in the target format includes: Based on the open neural network exchange ONNX protocol, the trained target neural network is converted from an original format to a target format supported by the second electronic device to obtain a target neural network in the target format.

6. A neural network generation device, characterized in that: Applied to a first electronic device, the device comprises: A network setting module, configured to respond to a network setting operation for a target neural network, perform network setting according to a selected network block, and obtain the target neural network, wherein the network setting operation includes: a selection operation of the network block called by the target neural network, the network block including a network structure and / or a network algorithm; A training module, configured to, in response to a training operation on the target neural network, train the target neural network according to a preset target data set to obtain a trained target neural network; A packaging module, used to package the trained target neural network to obtain a packaging file of the target neural network, and send the packaging file to a second electronic device so that the second electronic device deploys the target neural network according to the packaging file; Wherein, the network setting module is specifically used for: According to the selection operation, determining the selected network block used by each module in the target neural network; According to the selected network blocks, the target neural network is formed; The device also includes a data definition module, which is used to: In response to a data setting operation for a target neural network, confirming a selected data set, and generating the target data set according to the selected data set; The training module is also used to: splitting the target data set into a plurality of sub-data sets according to a preset number of processing processes of a third electronic device, the third electronic device comprising a distributed processing device; Sending each sub-data set and the target neural network to the processing process of the third electronic device respectively, so that the processing process of the third electronic device trains the target neural network in parallel to obtain network parameters of the target neural network; Upon receiving the network parameters sent by the third electronic device, a trained target neural network is determined.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images