A method and device for constructing a neural network

By deforming and training the starting point network and updating the weights with lightweight data sets, the problem of insufficient feature extraction capabilities of backbone networks is solved, and efficient and accurate neural network construction is achieved to adapt to the complex task needs of different scenarios.

CN111797983BActive Publication Date: 2024-12-03HUAWEI TECH CO LTD

Patent Information

Application Number
CN202010448652.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-05-25
Publication Date
2024-12-03
Estimated Expiration
2040-05-25

AI Technical Summary

Technical Problem

In the prior art, the feature extraction capability of the backbone network is insufficient, the output accuracy is low, the efficiency is low, and the generalization capability is weak, making it difficult to adapt to complex tasks in different scenarios.

Method used

By obtaining the starting point network, deform it based on the preset search space, train it if the conditions are met, use the lightweight data set to update the weight, improve the training efficiency, and search for suitable serial networks in different scenarios to build the target nerve network.

Benefits of technology

It improves the output accuracy and efficiency of neural networks, enhances generalization capabilities, is suitable for a variety of visual tasks, reduces the cost of building different neural networks, and improves user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111797983B_ABST
    Figure CN111797983B_ABST
Patent Text Reader

Abstract

The present application discloses a neural network construction method and device in the field of artificial intelligence, which are used to accurately and efficiently construct a target neural network. The constructed target neural network has high output accuracy and can also be applied to different application scenarios with strong generalization ability. The method includes: obtaining a starting network, which includes multiple serial subnetworks; deforming the starting network at least once based on a preset first search space to obtain a serial network, and the first search space includes a range of parameters used to deform the starting network; if the serial network meets a preset condition, the serial network is trained through a preset data set to obtain a trained serial network; if the trained serial network meets a termination condition, the target neural network is obtained according to the trained serial network.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and device for constructing a neural network. Background Art

[0002] In the field of artificial intelligence, computer vision technology has been widely used in scenarios such as autonomous driving, mobile terminals, and security monitoring. The backbone network is an important network architecture in the neural network. It is usually not related to the specific tasks of the neural network. It can be a feature extraction network or residual network required for most tasks. Common computer vision tasks are mainly used for target detection and semantic segmentation. Usually, the backbone network can be used to extract features from the input information first, and then the extracted features can be input into the prediction module to obtain the prediction results. As the complexity of the scene to be processed increases, the demand for the backbone network's feature extraction capabilities also increases.

[0003] However, in existing solutions, a feature extraction network of a classification network designed manually or obtained by search is usually used as the backbone network, or the backbone network is obtained by stacking the building units obtained by search. The backbone network obtained in the prior art often has low output accuracy, low efficiency in obtaining the backbone network, and the backbone network obtained by stacking can usually only be used in fixed scenarios. In different scenarios, it is necessary to search for building units again and stack the backbone network, and the generalization ability is weak. Summary of the invention

[0004] The present application provides a neural network construction method and device for accurately and efficiently constructing a target neural network. The constructed target neural network output has high accuracy and can be applied to different application scenarios with strong generalization ability.

[0005] In a first aspect, the present application provides a method for constructing a neural network, comprising: obtaining a starting network, the starting network comprising a plurality of serial networks; deforming the starting network at least once based on a preset first search space to obtain a serial network, the first search space comprising a range of parameters used to deform the starting network; if the serial network meets a preset condition, training the serial network using a preset data set to obtain a trained serial network; if the trained serial network meets a termination condition, obtaining a target neural network based on the trained serial network.

[0006] In the implementation manner of the present application, the starting point network can be deformed at least once to obtain a serial network. When the serial network meets the preset conditions, the serial network is trained through a preset data set, and there is no need to train the deformed starting point network every time it is deformed, thereby reducing the workload and improving the efficiency of obtaining the target neural network. In addition, the deformed starting point network can inherit the weight value of the starting point network before deformation, and the inherited weight value can be updated through a lightweight data set, so as to achieve rapid training of the deformed starting point network, quickly obtain a serial network that meets the requirements, and further improve the efficiency of obtaining the target neural network. In addition, the present application can make the output of the target neural network more accurate by deforming the starting point network. In addition, the method for constructing a neural network provided by the present application can adapt to different scenarios and can search for serial networks according to different requirements. It can be oriented to more complex visual tasks and has strong generalization ability. Thereby, the cost of constructing different neural networks can be saved, the accuracy of the output of the constructed neural network can be improved, and the user experience can be improved.

[0007] In a possible implementation, one of the aforementioned at least one deformations of the starting network based on the first search space includes: searching for a deformation strategy in the first search space, the deformation strategy including a set of parameters used to deform the starting network; deforming the starting network according to the deformation strategy to obtain a deformed starting network, the weight of the deformed starting network inherits the weight of the starting network before deformation; updating the weight of the deformed starting network through a lightweight data set to obtain a starting network after weight update, the preset data set includes more data than the lightweight data set; evaluating the starting network after weight update to obtain a first evaluation result; if the first evaluation result meets the preset update condition, re-searching the deformation strategy in the first search space, and re-deforming the starting network before deformation according to the deformation strategy obtained by the re-search; if the first evaluation result does not meet the preset update condition, treating the starting network after weight update as a serial network.

[0008] Therefore, in the implementation manner of the present application, when the starting network is deformed, the deformed starting network can inherit the weight value of the starting network before the deformation, and update the weight according to the lightweight data set, which can reduce the training process of the weight value, improve the efficiency of the subsequent serial network, and thus improve the efficiency of obtaining the target neural network. In addition, the starting network can be deformed by searching for deformation strategies in the first search space, and the first search space includes information related to each feature level in the starting network, such as depth, width, etc., and usually, the deformation strategy re-searched in the first search space is different from the deformation strategy used for the last deformation of the starting network, thereby avoiding meaningless deformation, and thus improving the efficiency of obtaining a starting network that does not meet the update conditions.

[0009] In a possible implementation, the method further includes: if the serial network does not meet the preset conditions, using the serial network as a new starting network, or if the trained serial network does not meet the termination conditions, using the trained serial network as a new starting network; updating the first search space, searching for a deformation strategy in the updated first search space, and then deforming the new starting network according to the searched deformation strategy.

[0010] Therefore, in the implementation of the present application, if the serial network does not meet the preset conditions, the serial network can be used as a new starting network, and the new starting network can be re-deformed until a serial network that meets the preset conditions or the termination conditions is selected. Therefore, the embodiment of the present application can be applied to a variety of different scenarios, and a suitable serial network can be selected for the target task, with strong generalization ability.

[0011] In one possible implementation, the update conditions may include but are not limited to: the number of deformations of the starting network does not exceed a preset number, or the first evaluation result does not meet a preset convergence condition, and the convergence condition includes but is not limited to one or more of the following: the first evaluation result is better than a threshold, or the difference between the first evaluation result and the evaluation result obtained by the last evaluation of the starting network is less than a preset value.

[0012] Therefore, in the implementation manner of the present application, if the number of deformations of the starting network does not exceed a preset number, or the first evaluation result does not meet the preset convergence condition, the starting network can continue to be deformed until a starting network that meets the update condition is obtained.

[0013] In one possible implementation, the preset conditions may include but are not limited to: the second evaluation result of the serial network satisfies the first preset evaluation condition, or the number of deformations of the serial network exceeds the preset number of deformations, or the time since the last deformation of the starting network exceeds a preset value, and the second evaluation result is obtained by evaluating the serial network.

[0014] Therefore, in an embodiment of the present application, when the evaluation result of the serial network meets the first preset evaluation condition, the number of deformation times of the serial network exceeds the preset number of deformation times, or the time since the last deformation of the starting network exceeds the preset value, the serial network can be trained to obtain a trained serial network. Usually, the output result of the trained serial network is more accurate.

[0015] Optionally, the first preset evaluation condition may include but is not limited to the loss value of the serial network being no greater than a first loss threshold, the accuracy value of the output result being greater than a first accuracy threshold, or the inference time being less than a first inference threshold, etc.

[0016] In a possible implementation, each serial subnetwork in the starting network is composed of one or more basic units. The deformation of the starting network may specifically include: exchanging the basic units included in two adjacent serial subnetworks in the multiple serial subnetworks; or inserting processed basic units into one or more of the multiple serial subnetworks. The processed basic units may include initializing the unit matrix of the weights of the basic units and / or grouping and normalizing the basic units.

[0017] In the implementation manner of the present application, the starting point network can be deformed by exchanging or transferring basic units included in adjacent serial subnetworks in the serial subnetwork, or by adding basic units in the serial subnetwork, thereby obtaining a deformed starting point network.

[0018] In a possible implementation, the termination condition may include but is not limited to: the second evaluation result of the serial network meets the second preset evaluation condition, or the number of training times for the serial network exceeds the preset number of training times.

[0019] In an implementation manner of the present application, when the evaluation result of the serial network meets the second preset evaluation condition, or the number of training times of the serial network exceeds the preset number of training times, the deformation of the serial network can be terminated, and a target neural network for performing the target task can be constructed based on the serial network.

[0020] Optionally, the second preset evaluation condition may include that the loss value of the serial network is not greater than a second loss threshold, the accuracy value of the output result is greater than a second accuracy threshold, or the inference time is less than a second inference threshold, etc.

[0021] In a possible implementation, the aforementioned obtaining of the target neural network based on the trained serial network may include: if the trained serial network meets the output condition, then using the trained serial network as the target neural network; if the trained serial network does not meet the output condition, then obtaining a parallel subnetwork corresponding to the trained serial network, and constructing the target neural network based on the trained serial network and the parallel subnetwork corresponding to the trained serial network.

[0022] In the implementation manner of the present application, if the trained serial network meets the output condition, the trained serial network can be used as the target neural network, thereby obtaining the target neural network efficiently and accurately. If the trained serial network meets the termination condition but does not meet the output condition, a parallel subnetwork corresponding to the trained serial network can be obtained, and the target neural network can be constructed based on the trained serial network and the parallel subnetwork corresponding to the trained serial network. It can be understood that a parallel subnetwork corresponding to the trained serial network can be obtained, and a target neural network can be constructed based on the serial network and the parallel subnetwork, thereby improving the accuracy of the output result of the obtained target neural network, so that the neural network constructed based on the serial network and the parallel subnetwork can meet the output condition, thereby obtaining a target neural network with a more accurate output result.

[0023] In a possible implementation, a serial network may include multiple serial serial sub-networks, each serial sub-network includes one or more basic units, and the above-mentioned obtaining a parallel sub-network corresponding to the trained serial network may include: determining the number of parallel sub-networks corresponding to each serial sub-network in the multiple serial sub-networks within a preset number range; and constructing a parallel sub-network corresponding to each serial sub-network according to the number of parallel sub-networks corresponding to each serial sub-network.

[0024] Therefore, in an embodiment of the present application, the number of parallel subnetworks corresponding to each serial subnetwork can be searched within a preset number range, and then the number is used to construct the parallel subnetwork corresponding to each serial subnetwork, so that a target neural network with more accurate output results can be constructed.

[0025] In a possible implementation, the aforementioned construction of a target neural network based on a serial network and a parallel subnetwork corresponding to each serial subnetwork may include: constructing a temporary neural network based on the serial network and a parallel subnetwork corresponding to each serial subnetwork; evaluating the temporary neural network to obtain a third evaluation result; if the third evaluation result meets the output condition, using the temporary neural network as the target neural network; if the third evaluation result does not meet the output condition, reacquiring the parallel subnetwork corresponding to the serial network.

[0026] Therefore, in the implementation mode of the present application, a target neural network that meets the output conditions can be constructed based on the serial network and the corresponding parallel sub-network, so that the target neural network can be obtained accurately and efficiently, can be applied to different scenarios, and has strong generalization ability.

[0027] In a possible implementation, the output condition includes one or more of the following: the accuracy is greater than the first threshold, or the average accuracy is greater than the second threshold, or the loss value is not greater than the third threshold, or the inference time is not greater than the fourth threshold, the number of floating-point operations per second is not greater than the fifth threshold, the average accuracy is the average of multiple accuracies obtained by multiple evaluations of the neural network, and the inference time is the time it takes to obtain the output result from the neural network based on the input. Usually, the output condition can be determined based on the target task, for example, the output condition can be obtained from the user input data, so as to obtain the target neural network that can be applied to different scenarios and improve the user experience.

[0028] In a possible implementation, the types of parameters included in the first search space include: width, depth, number of downsampling and location of downsampling, the width is the number of input channels and output channels of the basic unit included in each serial subnetwork in the starting network, the depth is the number of basic units included in each serial subnetwork in the starting network, and downsampling is used to split two adjacent serial subnetworks in the starting network. Therefore, in the implementation of the present application, parameters such as width, depth, number of downsampling or location of downsampling can be sampled in the first search space, so that the serial network can be deformed according to the searched parameters, providing a variety of different deformation methods, thereby enriching the deformation network for deforming the serial network, so that a serial network that meets the conditions can be accurately obtained.

[0029] In a possible implementation, the target neural network is used to perform at least one of image recognition, semantic segmentation, or target detection, and the preset data set includes the ImageNet data set. Therefore, in the implementation of the present application, the serial network can be trained using the ImageNet data set to obtain a serial network with more accurate output results.

[0030] In a possible implementation, a specific method of obtaining the starting network may include: obtaining a target task through user input data, and then obtaining a starting network according to the target task, and the constructed target neural network may be used to execute the target task.

[0031] In a second aspect, the present application provides a neural network construction device, comprising:

[0032] An acquisition module is used to acquire a starting point network, where the starting point network includes a plurality of serial sub-networks;

[0033] A deformation module, used to deform the starting network at least once based on a preset first search space to obtain a serial network, wherein the first search space includes a range of parameters used to deform the starting network;

[0034] A training module, used for training the serial network using a preset data set to obtain a trained serial network if the serial network meets preset conditions;

[0035] A construction module is used to obtain a target neural network according to the trained serial network if the trained serial network meets the termination condition.

[0036] The beneficial effects produced by the second aspect and any possible implementation manner of the second aspect can be referred to the description of the first aspect and any possible implementation manner of the first aspect.

[0037] In a possible implementation, the deformation module can be specifically used to: search for a deformation strategy in a preset first search space, the deformation strategy includes a set of parameters used to deform the starting network; deform the starting network according to the deformation strategy to obtain a deformed starting network, the weight of the deformed starting network is inherited from the weight of the starting network before deformation; update the weight of the deformed starting network through a lightweight data set to obtain a starting network after weight update, the preset data set includes more data than the lightweight data set; evaluate the starting network after weight update to obtain a first evaluation result; if the first evaluation result meets the preset update condition, the deformation strategy can be re-searched in the first search space, and the starting network before deformation can be re-deformed according to the re-searched deformation strategy; if the first evaluation result does not meet the preset update condition, the starting network after weight update is used as a serial network.

[0038] In a possible implementation, the neural network construction device may further include:

[0039] An updating module is used to use the serial network as a new starting network if the serial network does not meet the preset conditions, or to use the trained serial network as a new starting network if the trained serial network does not meet the termination conditions, update the first search space, and re-search the deformation strategy according to the updated first search space, and re-deform the new starting network according to the re-searched deformation strategy.

[0040] In one possible implementation, the update conditions include: the number of deformations of the starting network does not exceed a preset number, or the first evaluation result does not meet a preset convergence condition, and the convergence condition includes one or more of the following: the first evaluation result is better than a threshold, or the difference between the first evaluation result and the last evaluation result obtained by evaluating the starting network is less than a preset value.

[0041] In one possible implementation, the preset conditions may include but are not limited to: the second evaluation result of the serial network satisfies the first preset evaluation condition, or the number of deformations of the serial network exceeds the preset number of deformations, or the time since the last deformation of the starting network exceeds the preset value, the second evaluation result is obtained by evaluating the serial network, and the first preset evaluation condition includes one or more of the following: the loss value of the serial network is not greater than the first loss threshold, the accuracy value of the output result of the serial network is greater than the first accuracy threshold, or the inference time of the serial network is less than the first inference threshold.

[0042] In a possible implementation, the starting network is composed of multiple serial sub-networks, wherein each serial sub-network is composed of one or more basic units, and the deformation module can be specifically used to: exchange the basic units included in two adjacent serial sub-networks in the multiple serial sub-networks; or, insert processed basic units into one or more of the multiple serial sub-networks, and the inserted processed basic units are weights obtained by initializing the weights of the basic units with a unit matrix and / or grouping and normalizing the basic units.

[0043] In a possible implementation, the termination condition may include but is not limited to: the second evaluation result of the serial network meets the second preset evaluation condition, or the number of training times for the serial network exceeds the preset number of training times.

[0044] In one possible implementation, the construction module can be specifically used to: if the trained serial network meets the output condition, then use the trained serial network as the target neural network; if the trained serial network does not meet the output condition, then obtain the parallel sub-network corresponding to the trained serial network, and construct the target neural network based on the trained serial network and the parallel sub-network corresponding to the trained serial network.

[0045] In one possible implementation, a serial network includes multiple serial sub-networks, each serial sub-network includes one or more basic units, and a building module can be specifically used to: determine the number of parallel sub-networks corresponding to each serial sub-network in the multiple serial sub-networks within a preset number range; construct a parallel sub-network corresponding to each serial sub-network according to the number of parallel sub-networks corresponding to each serial sub-network, and the structure of each serial sub-network is the same as the structure of the parallel sub-network corresponding to each serial sub-network.

[0046] In one possible implementation, the construction module can be specifically used to: construct a temporary neural network based on the serial network and the parallel subnetwork corresponding to each serial subnetwork; evaluate the temporary neural network to obtain a third evaluation result; if the third evaluation result meets the output condition, use the temporary neural network as the target neural network; if the third evaluation result does not meet the output condition, re-acquire the parallel subnetwork corresponding to the serial network.

[0047] In one possible implementation, the output conditions include but are not limited to one or more of the following: the accuracy is greater than a first threshold, or the average accuracy is greater than a second threshold, or the loss value is not greater than a third threshold, or the inference time is not greater than a fourth threshold, the number of floating-point operations per second is not greater than a fifth threshold, the average accuracy is the average of multiple accuracies obtained by performing multiple evaluations of the neural network, and the inference time is the time it takes to obtain the output result from the neural network based on the input.

[0048] In one possible implementation, the types of parameters included in the first search space may include, but are not limited to: width, depth, number of downsampling, and location of downsampling, the width being the number of input channels and output channels of the basic units included in each serial subnetwork in the starting network, the depth being the number of basic units included in each serial subnetwork in the starting network, and downsampling being used to split two adjacent serial subnetworks in the starting network.

[0049] In a possible implementation, the target neural network is used to perform at least one of image recognition, semantic segmentation, or target detection, and the preset data set includes the ImageNet data set.

[0050] In a possible implementation, a specific method of obtaining the starting network may include: obtaining a target task through user input data, and then obtaining a starting network according to the target task, and the constructed target neural network may be used to execute the target task.

[0051] In a third aspect, an embodiment of the present application provides a neural network construction device, comprising: a processor and a memory, wherein the processor and the memory are interconnected via a line, and the processor calls the program code in the memory to execute the processing-related functions in the neural network construction method shown in any one of the first aspects above. Optionally, the neural network construction device can be a chip.

[0052] In a fourth aspect, an embodiment of the present application provides a neural network construction device, which can also be called a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface, and the program instructions are executed by the processing unit. The processing unit is used to perform functions related to processing in the above-mentioned first aspect or any optional embodiment of the first aspect.

[0053] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect or any optional implementation manner of the first aspect.

[0054] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect or any optional implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A schematic diagram of an artificial intelligence main framework for this application;

[0056] Figure 2 A schematic diagram of a system architecture provided for this application;

[0057] Figure 3 A schematic diagram of a convolutional neural network structure provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of another convolutional neural network structure provided in an embodiment of the present application;

[0059] Figure 5 Another system architecture diagram provided for this application;

[0060] Figure 6 A flowchart of a neural network construction method provided in an embodiment of the present application;

[0061] Figure 7 A schematic diagram of a network deformation method provided in an embodiment of the present application;

[0062] Figure 8 A schematic diagram of another network deformation method provided in an embodiment of the present application;

[0063] Fig. 9 A schematic diagram of a neural network construction method provided in an embodiment of the present application;

[0064] Fig.10 A schematic diagram of a process flow of another neural network construction method provided in an embodiment of the present application;

[0065] Fig.11 A schematic diagram of another network deformation method provided in an embodiment of the present application;

[0066] Fig.12 A schematic diagram of another neural network construction method provided in an embodiment of the present application;

[0067] Fig.13A schematic diagram of a neural network structure provided in an embodiment of the present application;

[0068] Fig.14 A schematic diagram of the structure of a neural network construction device provided in an embodiment of the present application;

[0069] Fig.15 A schematic diagram of the structure of another neural network construction device provided in an embodiment of the present application;

[0070] Fig.16 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The following will describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] The neural network construction method provided in this application can be applied to artificial intelligence (AI) scenarios. AI is a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. Research in the field of artificial intelligence includes robots, natural language processing, computer vision, decision-making and reasoning, human-computer interaction, recommendation and search, basic AI theory, etc.

[0073] Figure 1 A schematic diagram of an artificial intelligence main framework is shown, which describes the overall workflow of the artificial intelligence system and is applicable to general artificial intelligence field requirements.

[0074] The following is an explanation of the above artificial intelligence theme framework from two dimensions: "intelligent information chain" (horizontal axis) and "IT value chain" (vertical axis).

[0075] The "intelligent information chain" reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, data undergoes a condensed process of "data-information-knowledge-wisdom".

[0076] The "IT value chain" reflects the value that artificial intelligence brings to the information technology industry, from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the system's industrial ecological process.

[0077] (1) Infrastructure:

[0078] Infrastructure provides computing power support for artificial intelligence systems, enables communication with the outside world, and supports it through the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips, such as central processing units (CPU), neural-network processing units (NPU), graphics processing units (GPU), application specific integrated circuits (ASIC) or field programmable gate arrays (FPGA) and other hardware acceleration chips; the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.

[0079] (2) Data

[0080] The data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence. The data involves graphics, images, voice, video, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.

[0081] (3) Data processing

[0082] Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.

[0083] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0084] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0085] Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.

[0086] (4) General ability

[0087] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing (such as image recognition, target detection, etc.), speech recognition, etc.

[0088] (5) Smart products and industry applications

[0089] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart manufacturing, smart transportation, smart home, smart medical care, smart security, autonomous driving, safe cities, smart terminals, etc.

[0090] See attached Figure 2 , the embodiment of the present application provides a system architecture 200. The system architecture includes a database 230 and a client device 240. The data acquisition device 260 is used to collect data and store it in the database 230, and the construction module 202 generates a target model / rule 201 based on the data maintained in the database 230. The following will describe in more detail how the construction module 202 obtains the target model / rule 201 based on the data. The target model / rule 201 is the neural network constructed in the following implementation of the present application. For details, refer to the following Figure 6-13 Related description in .

[0091] The calculation module may include a construction module 202. The target model / rule obtained by the construction module 202 may be applied to different systems or devices. Figure 2 In the embodiment, the execution device 210 is configured with a transceiver 212, which can be a wireless transceiver, an optical transceiver or a wired interface (such as an I / O interface), etc., to interact with external devices. The "user" can input data to the transceiver 212 through the client device 240. For example, in the following implementation of the present application, the client device 240 can send a target task to the execution device 210, request the execution device to build a neural network, and send a database for training to the execution device 210.

[0092] The execution device 210 can call data, codes, etc. in the data storage system 250 , and can also store data, instructions, etc. in the data storage system 250 .

[0093] The calculation module 211 uses the target model / rule 201 to process the input data. Specifically, the calculation module 211 is used to: obtain the starting network based on the target task and deform the starting network at least once based on the preset first search space to obtain a serial network, and the serial search space includes parameters used to deform the starting network; if the serial network meets the preset conditions, the serial network is trained through the preset data set to obtain a trained serial network; if the trained serial network meets the termination conditions, the target neural network is obtained according to the trained serial network, and the target neural network is used to perform the target task. It can be understood that the calculation module 211 can be used to construct the backbone network of the neural network.

[0094] The association function module 213 and the association function module 214 can be used to search for other networks other than the backbone network that are associated with the target task, such as a region proposal network (RPN), a feature pyramid network (FPN), and the like.

[0095] Finally, the transceiver 212 returns the constructed neural network to the client device 240 so that the neural network can be deployed in the client device 240 or other devices.

[0096] More deeply, the construction module 202 can obtain corresponding target models / rules 201 based on different data for different target tasks to provide users with better results.

[0097] In the attached Figure 2 In the case shown in , the data input into the execution device 210 can be determined based on the user's input data. For example, the user can operate in the interface provided by the transceiver 212. In another case, the client device 240 can automatically input data into the transceiver 212 and obtain the result. If the client device 240 automatically inputs data and needs to obtain the user's authorization, the user can set the corresponding authority in the client device 240. The user can view the results output by the execution device 210 on the client device 240, and the specific presentation form can be a specific method such as display, sound, action, etc. The client device 240 can also serve as a data collection terminal to store the collected data associated with the target task into the database 230.

[0098] It should be noted that the attached Figure 2 This is only an exemplary schematic diagram of a system architecture provided by an embodiment of the present application, and the positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. Figure 2 In the embodiment, the data storage system 250 is an external memory relative to the execution device 210. In other scenarios, the data storage system 250 may also be placed in the execution device 210.

[0099] The training or updating process mentioned in the present application can be performed by the construction module 202. It is understandable that the training process of the neural network is to learn the way to control the spatial transformation, more specifically, to learn the weight matrix. The purpose of training the neural network is to make the output of the neural network as close to the expected value as possible. Therefore, the weight vector of each layer of the neural network in the neural network can be updated according to the difference between the predicted value and the expected value of the current network by comparing the predicted value and the expected value of the current network (of course, the weight vector can usually be initialized before the first update, that is, the parameters of each layer in the deep neural network are pre-configured). For example, if the predicted value of the network is too high, the value of the weight in the weight matrix is ​​adjusted to reduce the predicted value, and after continuous adjustment, the value output by the neural network is close to or equal to the expected value. Specifically, the difference between the predicted value and the expected value of the neural network can be measured by a loss function or an objective function. Taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference. The training of the neural network can be understood as the process of minimizing the loss as much as possible. The process of updating the weight of the starting network and training the serial network in the following embodiments of the present application can refer to this process, which will not be repeated below.

[0100] The neural networks mentioned in this application may include various types, such as deep neural networks (DNN), convolutional neural networks (CNN), recurrent neural networks (RNN), highway networks or residual networks, and other neural networks.

[0101] Exemplarily, a convolutional neural network (CNN) is taken as an example below.

[0102] CNN is a deep neural network with a convolutional structure. CNN is a deep learning architecture, which refers to multiple levels of learning at different levels of abstraction through machine learning algorithms. As a deep learning architecture, CNN is a feed-forward artificial neural network in which each neuron responds to overlapping areas in the image input to it. Convolutional neural network contains a feature extractor consisting of a convolution layer and a subsampling layer. The feature extractor can be regarded as a filter, and the convolution process can be regarded as using a trainable filter to convolve an input image or convolution feature plane (feature map). Convolutional layer refers to the neuron layer in the convolutional neural network that performs convolution processing on the input signal. In the convolutional layer of the convolutional neural network, a neuron can only be connected to some neurons in the adjacent layers. A convolutional layer usually contains several feature planes, each of which can be composed of some neural units arranged in a rectangular shape. The neural units in the same feature plane share weights, and the shared weights here are convolution kernels. Shared weights can be understood as the way to extract image information is independent of position. The underlying principle is that the statistical information of a certain part of the image is the same as that of other parts. This means that the image information learned in one part can also be used in another part. So for all positions on the image, we can use the same learned image information. In the same convolution layer, multiple convolution kernels can be used to extract different image information. Generally speaking, the more convolution kernels there are, the richer the image information reflected by the convolution operation.

[0103] The convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network. In addition, the direct benefit of shared weights is to reduce the connections between the layers of the convolutional neural network, while reducing the risk of overfitting.

[0104] Convolutional neural networks can use the error back propagation (BP) algorithm to correct the size of the parameters in the initial super-resolution model during the training process, so that the reconstruction error loss of the super-resolution model becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the error loss information is back-propagated to update the parameters in the initial super-resolution model, so that the error loss converges. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal parameters of the super-resolution model, such as the weight matrix.

[0105] like Figure 3 As shown, the convolutional neural network (CNN) 100 may include an input layer 110 , a convolutional layer / pooling layer 120 , wherein the pooling layer is optional, and a neural network layer 130 .

[0106] like Figure 3 The convolution layer / pooling layer 120 shown may include layers 121-126 as shown in the examples. In one implementation, layer 121 is a convolution layer, layer 122 is a pooling layer, layer 123 is a convolution layer, layer 124 is a pooling layer, layer 125 is a convolution layer, and layer 126 is a pooling layer. In another implementation, layers 121 and 122 are convolution layers, layer 123 is a pooling layer, layers 124 and 125 are convolution layers, and layer 126 is a pooling layer. That is, the output of a convolution layer can be used as the input of a subsequent pooling layer, or as the input of another convolution layer to continue the convolution operation.

[0107] Taking the convolution layer 121 as an example, the convolution layer 121 may include a plurality of convolution operators, which are also called kernels. The convolution operator plays a role in image processing that is equivalent to a filter that extracts specific information from an input image matrix. The convolution operator can essentially be a weight matrix, which is usually predefined. In the process of performing a convolution operation on an image, the weight matrix is ​​usually processed one pixel after another (or two pixels after two pixels... depending on the value of the stride) in the horizontal direction on the input image to complete the work of extracting specific features from the image. The size of the weight matrix should be related to the size of the image. It should be noted that the depth dimension of the weight matrix is ​​the same as the depth dimension of the input image. In the process of performing a convolution operation, the weight matrix extends to the entire depth of the input image. Therefore, convolution with a single weight matrix will produce a convolution output with a single depth dimension, but in most cases a single weight matrix is ​​not used, but multiple weight matrices of the same dimension are applied. The output of each weight matrix is ​​stacked to form the depth dimension of the convolution image. Different weight matrices can be used to extract different features in the image, for example, one weight matrix is ​​used to extract image edge information, another weight matrix is ​​used to extract specific colors of the image, and another weight matrix is ​​used to blur unnecessary noise points in the image, etc. The multiple weight matrices have the same dimensions, and the feature maps extracted by the multiple weight matrices with the same dimensions also have the same dimensions. The extracted feature maps with the same dimensions are then merged to form the output of the convolution operation.

[0108] Usually, the weight values ​​in the weight matrix need to be obtained through a lot of training in practical applications. The weight matrices formed by the weight values ​​obtained through training can extract information from the input image, thereby helping the convolutional neural network 100 to make correct predictions.

[0109] When the convolutional neural network 100 has multiple convolutional layers, the initial convolutional layer (for example, 121) often extracts more general features, which can also be called low-level features. As the depth of the convolutional neural network 100 increases, the features extracted by the later convolutional layers (for example, 126) become more and more complex, such as high-level semantic features. Features with higher semantics are more suitable for the problem to be solved.

[0110] Pooling layer:

[0111] Since it is often necessary to reduce the number of training parameters, it is often necessary to periodically introduce a pooling layer after the convolution layer, such as Figure 3 The layers 121-126 illustrated in 120 may be a convolution layer followed by a pooling layer, or may be multiple convolution layers followed by one or more pooling layers. In the image processing process, the only purpose of the pooling layer is to reduce the spatial size of the image. The pooling layer may include an average pooling operator and / or a maximum pooling operator for sampling the input image to obtain an image of smaller size. The average pooling operator may calculate the pixel values ​​in the image within a specific range to generate an average value. The maximum pooling operator may take the pixel with the largest value within a specific range as the result of the maximum pooling. In addition, just as the size of the weight matrix used in the convolution layer should be related to the image size, the operator in the pooling layer should also be related to the image size. The size of the image output after processing by the pooling layer may be smaller than the size of the image input to the pooling layer, and each pixel in the image output by the pooling layer represents the average value or maximum value of the corresponding sub-region of the image input to the pooling layer.

[0112] Neural Network Layer 130:

[0113] After being processed by the convolution layer / pooling layer 120, the convolution neural network 100 is not sufficient to output the required output information. As mentioned above, the convolution layer / pooling layer 120 only extracts features and reduces the parameters brought by the input image. However, in order to generate the final output information (the required class information or other related information), the convolution neural network 100 needs to use the neural network layer 130 to generate one or a group of outputs of the required number of classes. Therefore, the neural network layer 130 may include multiple hidden layers (such as Figure 3 131, 132 to 13n) and output layer 140. In the present application, the convolutional neural network is: the selected starting network is deformed at least once to obtain a serial network, and then obtained according to the trained serial network. The convolutional neural network can be used for image recognition, image classification, image super-resolution reconstruction, etc.

[0114] After the multiple hidden layers in the neural network layer 130, that is, the last layer of the entire convolutional neural network 100 is the output layer 140, which has a loss function similar to the classification cross entropy, specifically used to calculate the prediction error. Once the forward propagation of the entire convolutional neural network 100 (such as Figure 3 The propagation from 110 to 140 is forward propagation), and the reverse propagation (such as Figure 3 The propagation from 140 to 110 is called back propagation) and then the weight values ​​and biases of the aforementioned layers will begin to be updated to reduce the loss of the convolutional neural network 100 and the error between the result output by the convolutional neural network 100 through the output layer and the ideal result.

[0115] It should be noted that if Figure 3 The convolutional neural network 100 shown is only an example of a convolutional neural network. In specific applications, the convolutional neural network can also exist in the form of other network models, for example, Figure 4 The multiple convolutional layers / pooling layers shown are operated in parallel, and the features extracted respectively are input to the full neural network layer 130 for processing.

[0116] See attached Figure 5 , the embodiment of the present application also provides a system architecture 300. The execution device 210 is implemented by one or more servers, and optionally cooperates with other computing devices, such as data storage, routers, load balancers, etc.; the execution device 210 can be arranged on a physical site, or distributed on multiple physical sites. The execution device 210 can use the data in the data storage system 250, or call the program code in the data storage system 250 to implement the following of the present application Figure 6-13 The corresponding steps of the neural network construction method.

[0117] Users can operate their respective user devices (e.g., local device 301 and local device 302) to interact with execution device 210. Each local device can represent any computing device, such as a personal computer, a computer workstation, a smart phone, a tablet computer, a smart camera, a smart car or other type of cellular phone, a media consumption device, a wearable device, a set-top box, a game console, etc.

[0118] The local device of each user can interact with the execution device 210 through a communication network of any communication mechanism / communication standard, and the communication network can be a wide area network, a local area network, a point-to-point connection, or any combination thereof. Specifically, the communication network may include a wireless network, a wired network, or a combination of a wireless network and a wired network. The wireless network includes, but is not limited to: a fifth generation mobile communication technology (5th-Generation, 5G) system, a long term evolution (long termevolution, LTE) system, a global system for mobile communication (global system for mobile communication, GSM) or a code division multiple access (code division multiple access, CDMA) network, a wideband code division multiple access (wideband code division multiple access, WCDMA) network, wireless fidelity (wireless fidelity, WiFi), Bluetooth (bluetooth), Zigbee protocol (Zigbee), radio frequency identification technology (radio frequency identification, RFID), long range (Lora) wireless communication, near field communication (near field communication, NFC) in any one or more combinations. The wired network may include a fiber optic communication network or a network composed of a coaxial cable, etc.

[0119] In another implementation, one or more aspects of the execution device 210 may be implemented by each local device. For example, the local device 301 may provide local data or feedback calculation results to the execution device 210 .

[0120] It should be noted that all functions of the execution device 210 can also be implemented by the local device. For example, the local device 301 implements the functions of the execution device 210 and provides services to its own user, or provides services to the user of the local device 302.

[0121] Based on the above Figure 1-5 The system architecture or neural network provided, the neural network construction method provided in this application is introduced in detail below.

[0122] First, to facilitate understanding, some terms involved in this application are explained.

[0123] Automatic machine learning (AutoML): refers to designing a series of advanced control systems to operate machine learning models so that the models can automatically learn appropriate parameters and configurations without human intervention. In learning models based on deep neural networks, automatic computational learning mainly includes network architecture search and global parameter setting. Among them, network architecture search is used to allow computers to generate the most suitable neural network architecture for the problem based on data, which has the characteristics of high training complexity and great performance improvement.

[0124] Residual Networks: A type of convolutional neural network. In image recognition, it has a higher recognition accuracy than traditional convolutional neural networks. In residual network design, there are many submodules with the same structure. Usually, ResNet is connected with a number to indicate the number of times the submodule is repeated, such as ResNet50, which means there are 50 submodules.

[0125] Backbone: A network structure used to extract features from input information in detectors, segmenters, or classifiers. Usually, in addition to the backbone network, neural networks can also include other functional networks, such as region proposal networks (RPNs) and feature pyramid networks (FPNs), which are used to further process the features extracted by the backbone network, such as identifying the classification of features and performing semantic segmentation on features.

[0126] Basic unit (block): Usually a basic unit consists of a convolutional layer, or a basic unit can be understood as a convolutional module.

[0127] See also Figure 6 , a flowchart of a neural network construction method provided in this application is as follows.

[0128] 601. Obtain the starting point network.

[0129] The starting point network may include a plurality of serial sub-networks, the plurality of serial sub-networks are connected in a serial manner, and each serial sub-network is composed of one or more basic units.

[0130] Specifically, the starting network can be a backbone network commonly used in classification, segmentation or detection tasks, such as a classification network, segmentation network or detection network without a prediction head, and can be adjusted according to actual application scenarios.

[0131] Optionally, the starting network can be selected according to the target task, that is, the task to be performed by the neural network to be constructed. Of course, the starting network can also be a randomly selected network, which can be adjusted according to the actual application. This application selects the starting network through the target task, so that the selected starting network is close to the requirements of the target task, which can reduce the training process and improve the efficiency of obtaining the target neural network.

[0132] In a possible implementation, before step 601, a target task may be obtained, which may be a task to be performed by the target neural network to be constructed. If the present application can be performed by the aforementioned execution device 210, the target task may be a task sent by other devices (such as the aforementioned client device 240 or local device 301 or local device 302, etc.), or may be a target task generated according to the needs of the execution device 210 itself.

[0133] In addition, information associated with the target task can also be obtained, such as the data set associated with the target task, the accuracy, computational complexity, reasoning speed, reasoning duration, etc., which are satisfied by the target neural network that performs the target task, and the starting network can be selected specifically based on the information associated with the target task. For example, the starting network can be selected based on the accuracy, computational complexity, reasoning speed, reasoning duration, etc., which are satisfied by the target neural network, so that the starting network meets or approaches the accuracy, computational complexity, reasoning speed, reasoning duration, etc., which the target neural network needs to meet. For example, if the target task is to build a picture recognition neural network with an average accuracy of 80%, then when selecting the starting network, a classification network with an average accuracy greater than 80% or close to 80% can be selected as the starting network. Therefore, in the implementation manner of the present application, the starting network can be selected according to the requirements corresponding to the target task, and the starting network that can obtain the maximum benefit can be selected under the constraints of given computing resources. For example, by selecting the starting network based on accuracy, computational complexity, video memory usage, floating-point operations per second (FLOPS), CPU usage, inference speed, inference time, etc., the optimal starting network under the conditions can be obtained, and then the serial network can be quickly obtained, which can reduce the training process of obtaining the target neural network and improve the efficiency of obtaining the target neural network later.

[0134] In some possible scenarios, the target neural network may be used to perform one or more of image recognition, semantic segmentation, or object detection, etc. For example, the client device 240 may send a target task to the execution device 210, and the target task may be an image recognition task with an accuracy of 80%, or a semantic segmentation task with an accuracy of 95%, or an object detection task with an accuracy of 100ms and an inference time, etc.

[0135] 602. Perform at least one transformation on the starting network based on the first search space to obtain a serial network.

[0136] After the starting network is selected, or the serial network does not meet the preset condition, or the trained serial network does not meet the termination condition, the starting network can be deformed at least once based on the first search space, and the deformed starting network is used as the serial network. The first search space includes the range of parameters used to deform the starting network, such as the value range of depth or the value range of width and depth.

[0137] Among them, the deformation mentioned in this step can be that after selecting the starting network, the starting network is deformed at least once based on the first search space; or, after determining that the serial network does not meet the preset conditions (see the following step 603) or the trained serial network does not meet the termination conditions (see the following step 605), the serial network that does not meet the preset conditions or the trained serial network that does not meet the termination conditions can be used as a new starting network, and after updating the first search space, the new starting network is deformed. It can be understood that if it is determined that the serial network does not meet the preset conditions or the trained serial network does not meet the termination conditions, the serial network or the trained serial network can be used as a new starting network for deformation, so as to obtain a serial network again.

[0138] Specifically, the specific process of performing at least one deformation on the starting network may include: searching for a deformation strategy in a first search space, that is, a set of parameters used to deform the starting network, and then deforming the starting network according to the deformation strategy to obtain a deformed starting network, and the weight of the deformed starting network inherits the weight of the starting network before the deformation. Then, the weight value of the deformed starting network is updated by a lightweight data set to obtain a starting network with updated weights. Then, the starting network with updated weights is evaluated to obtain a first evaluation result. It is determined whether the first evaluation result meets a preset update condition. If the first evaluation result meets the update condition, the deformation strategy is re-searched in the first search space, and the starting network is re-deformed according to the re-searched deformation strategy, wherein when the starting network is re-deformed, the deformation strategy can be re-searched in the first search space for deformation, and the re-searched deformation strategy is usually different from the deformation strategy used in the previous deformation.

[0139] It is understandable that in this step, the starting network may be deformed multiple times and the weights may be updated until a starting network that does not meet the update conditions is obtained. In this process, multiple starting networks with updated weights are obtained. If the number of updates to the starting network has reached a limit value, an optimal starting network can be determined from the multiple starting networks with updated weights as the final serial network. For example, if the starting network is deformed multiple times to obtain multiple starting networks with updated weights, and the output results of the multiple starting networks with updated weights have an accuracy of 50%, 60%, and 80%, respectively, the starting network with an accuracy of 80% is used as the final serial network.

[0140] The following is a more detailed explanation of the deformation process of the starting network.

[0141] The data included in the lightweight data set is used to update the weight value of the deformed starting point network. It can be understood that the deformed starting point network can be quickly trained according to the data included in the lightweight data set, so as to efficiently update the weight value of the deformed starting point network. Specifically, the lightweight data set can be data collected by the data acquisition device 260, or it can be data sent from the client device to the execution device 210. For example, the data included in the lightweight data set can be part of the data in the local ImageNet data set, or the data included in the lightweight data set can be part of the data included in the database 230.

[0142] Specifically, the first evaluation result may include but is not limited to the loss value of the starting network, the accuracy value of the output result, or the inference time, etc. Accordingly, the specific process of evaluating the starting network may include: calculating the value of the loss function of the starting network. Calculating the accuracy of the output result of the starting network, or calculating the inference time of the starting network from input to output, etc., to obtain the first evaluation result.

[0143] Optionally, the update condition may include but is not limited to one or more of the following: the number of deformations of the starting network does not exceed a preset number, or the first evaluation result does not meet a preset convergence condition, and the preset convergence condition may be determined based on the data included in the first evaluation result, specifically, the data included in the first evaluation result is better than a threshold, or the difference between the data included in the first evaluation result and the evaluation result obtained by the last evaluation of the starting network is less than a preset value. For example, if the first evaluation result includes the loss value of the starting network, the convergence condition may be that the loss value is less than 20, or the difference between the first evaluation result and the loss value obtained by the last evaluation of the starting network is less than 10. For another example, if the first evaluation result includes the accuracy value of the classification of the starting network, the convergence condition may be that the accuracy value is greater than 80%, or the convergence condition may be that the difference between the accuracy value obtained by this evaluation and the accuracy value obtained by the last evaluation of the starting network is less than 5%, etc.

[0144] The first search space includes the range of parameters used to transform the starting network. Specifically, it may include information related to each feature level (stage) in the starting network, such as the value range of the width, depth or downsampling position of a stage. Among them, a feature level in the starting network can be understood as a serial subnetwork.

[0145] Generally, the starting network or serial network can be composed of multiple serial sub-networks, each of which can include one or more basic units (blocks). Specifically, the basic units included in the starting network can be divided into basic blocks (BB), bottleneck blocks (BNB), and bottleneck grouping (ResNeXt) units.

[0146] In a possible implementation, the types of parameters included in the first search space may include: width, depth, number of downsampling, and location of downsampling, etc. Among them, the width is the number of input channels and output channels of the basic unit included in each serial subnetwork in the starting network, the depth is the number of basic units included in each serial subnetwork in the starting network, the downsampling is used to split two adjacent serial subnetworks in the starting network, the number of downsampling is used to determine the number of serial subnetworks in the starting network, and the location of downsampling is used to split two adjacent serial subnetworks. Downsampling can be understood as each serial subnetwork of the starting network can downsample the image before processing it in the process of processing the image, so as to reduce the amount of data of the image to be processed, thereby reducing the workload. The content included in the first search space is, for example: network depth: 8-60 basic units; number of feature levels: 5-7 feature levels, that is, the number of serial subnetworks; network width: represents the number of input channels and output channels of the basic unit in the serial subnetwork, and further, it can also be represented as the position where the channel size is doubled in the serial network.

[0147] For example, Figure 7 As shown, the starting point network can include 4 serial sub-networks, namely Figure 7 Serial subnetwork 1, serial subnetwork 2, serial subnetwork 3 and serial subnetwork 4 are shown in . "1" means that the output channel of the corresponding basic unit is 1 times the input channel. Of course, "1" here can also be replaced by "2" or "3", etc., which can be adjusted according to the actual application scenario. "," means downsampling. For example, there is "," between block2 and block3, which means that block2 and block are in different serial subnetworks. Specifically, in the process of processing the picture, the picture can be downsampled, thereby reducing the amount of data of the picture to be processed, thereby reducing the workload. At present, each serial subnetwork in the serial network may not downsample the picture, that is, the amount of data of the picture processed by each serial subnetwork is the same, thereby improving the accuracy of the detection result. Whether to downsample can be determined according to the actual application scenario, and this application does not limit this.

[0148] Furthermore, the specific process of deforming the starting network may include: exchanging the basic units included in two adjacent serial subnetworks in the multiple serial subnetworks, or it can also be understood as exchanging the positions of downsampling and basic units; or, inserting basic units in one or more of the multiple serial subnetworks, and the inserted basic units are obtained by weights initialized according to the unit matrix and group normalization. For ease of understanding, the process of inserting basic units in one or more of the multiple serial subnetworks can be called expansion. That is, the deformation operation of the starting network can include exchange or expansion. For example, Figure 8As shown, block2 in serial subnetwork 1 can be adjusted to serial subnetwork 2, and block5 in serial subnetwork 3 can be adjusted to serial subnetwork 2, that is, the position of the downsampling "," is adjusted to adjust block2 in serial subnetwork 1 to serial subnetwork 2 and block5 in serial subnetwork 3 to serial subnetwork 2. In addition, block8 can be expanded in serial subnetwork 4.

[0149] Accordingly, the specific method of deforming the starting network may include: searching for a deformation strategy for deforming the starting network in the first search space, wherein the deformation strategy includes the parameters used for deforming the starting network, such as the width and depth of each serial subnetwork in the starting network, the position of downsampling in the starting network, or the number of downsampling, etc. Then, according to the deformation strategy, the width, depth, position of downsampling, or the number of downsampling, etc. of each serial subnetwork in the starting network are adjusted to obtain the deformed starting network. For example, when the depth of a serial subnetwork is less than the depth searched in the deformation strategy, a new basic unit can be inserted into the serial subnetwork to increase the depth of the serial subnetwork. For another example, when the width of a serial subnetwork is less than the width in the deformation strategy, the input channel and output channel of the basic unit in the serial subnetwork can be adjusted so that the deformed serial subnetwork meets the width included in the deformation strategy.

[0150] Moreover, when exchanging and expanding, in order to make the deformed starting network inherit the weight value of the starting network before deformation, the exchanged or expanded basic units need to be processed. Specifically, in the process of exchanging, the convolution step size of the adjacent basic units can be changed, so that the starting network after the exchange can reload the network parameters of the starting network before deformation, such as the weight value, and reduce the workload of adjusting the network parameters of the starting network. In the process of expansion, the newly inserted unit is obtained by processing the weight initialized by the unit matrix and group normalization, so that the deformed starting network can inherit the parameters of the starting network before deformation. It can be understood that after the basic units included in the exchange or expansion of the starting network, for the exchanged or expanded basic units, the structure of the exchanged or expanded basic units can be changed so that the structure of the basic units in the corresponding serial subnetwork in the starting network is similar, for example, the width of the newly inserted unit is the same as the width of the inserted serial subnetwork, or the width of the deformed basic unit is the same as the width of the basic unit before deformation, etc., so that the weight value of the starting network before deformation can be inherited, thereby reducing the weight training process. Therefore, in the implementation manner of the present application, the deformed starting point network can be quickly obtained in a short time, and the evaluation result of the deformed starting point network can be obtained, thereby improving the efficiency of obtaining the serial network.

[0151] 603. Determine whether the serial network meets the preset conditions, if yes, execute step 604, if no, execute step 602.

[0152] After the starting network is deformed at least once to obtain a serial network, it is determined whether the serial network meets the preset conditions. If the serial network meets the preset conditions, the serial network can be trained using a preset data set. If the serial network does not meet the preset conditions, the serial network can be used as a new starting network, and the deformation strategy is continued to be searched in the first search space, thereby continuing to deform the new starting network.

[0153] In a possible implementation, the preset conditions may include but are not limited to: the second evaluation result of the serial network satisfies the first preset evaluation condition, or the number of deformations of the serial network exceeds the preset deformation number, or the number of trainings of the serial network does not exceed the preset training number, and the second evaluation result is obtained by evaluating the serial network. The second evaluation result may be the first evaluation result of the starting network after at least one deformation in the aforementioned step 602, or the evaluation result obtained by re-evaluating the serial network after obtaining the serial network. The second evaluation result may include but is not limited to the loss value of the starting network, the accuracy value of the output result, or the inference time, etc. Correspondingly, the first preset evaluation condition may include that the loss value of the starting network or the serial network is not greater than the first loss threshold, the accuracy value of the output result is greater than the first accuracy threshold, or the inference time is less than the first inference threshold, etc. The number of deformations of the serial network is the number of deformations of the starting network in the aforementioned step 602. The number of trainings of the serial network does not exceed the preset training number, that is, the number of times the serial network is trained by the preset data set in the following step 604.

[0154] 604. Train the serial network using a preset data set to obtain a trained serial network.

[0155] After determining that the starting network meets the preset conditions, the serial network is trained using a preset data set to obtain a trained serial network.

[0156] Among them, the preset data set can be understood as a full data set, which includes a large amount of training data. For example, if the target neural network is used for at least one of image recognition, semantic segmentation or target detection, the full data set can be an ImageNet data set, which includes a large amount of image data. Therefore, in the implementation mode of the present application, instead of training with the ImageNet data set every time the starting network is deformed, the serial network is trained only after the starting network is deformed at least once and the preset conditions are met. This can reduce the workload of training the serial network and improve the efficiency of obtaining the target neural network.

[0157] It is understandable that, usually, in order to improve the accuracy of the output results of the final serial network, it is necessary to train with a large number of data sets, such as the ImageNet data set or other data sets including a large amount of data, and training with a large number of data sets requires more resources. In the present application, the serial network is trained with a large amount of data only after the serial network meets the preset conditions, and there is no need to train after each deformation, thereby reducing the workload of training the serial network and improving the efficiency of obtaining the final serial network.

[0158] 605. Determine whether the trained serial network meets the termination condition. If so, execute step 606; if not, execute step 602.

[0159] After the serial data is trained by the preset data set, it is determined whether the trained serial network meets the termination condition. If the trained serial network meets the termination condition, the target network can be obtained according to the trained serial network, that is, step 606 is executed. If the trained serial network does not meet the termination condition, after updating the first search space, the trained serial network can be used as a new starting point network, and the new starting point network can be deformed at least once, and then the deformed starting point network can be used as the serial network again.

[0160] In a possible implementation, the termination condition may include but is not limited to: the second evaluation result of the serial network meets the second preset evaluation condition, or the number of training times for the serial network exceeds the preset number of training times.

[0161] Among them, the second evaluation result can be the first evaluation result of the starting network after at least one deformation in the aforementioned step 602, or it can be the evaluation result obtained by re-evaluating the serial network after obtaining the serial network. The second evaluation result may include but is not limited to the loss value of the serial network, the accuracy value of the output result, or the inference time, etc. Correspondingly, the second preset evaluation condition may include that the loss value of the serial network is not greater than the second loss threshold, the accuracy value of the output result is greater than the second accuracy threshold, or the inference time is less than the second inference threshold, etc. The number of training times for the serial network does not exceed the preset training times, that is, the number of times the serial network is trained using the preset data set in step 604.

[0162] It should be understood that the second preset evaluation condition here may be the same as or different from the aforementioned first preset evaluation condition. When the second preset evaluation condition is different from the aforementioned first preset evaluation condition, the network corresponding to the second preset evaluation condition is usually better than the network corresponding to the first preset evaluation condition. The second loss threshold may be lower than the first loss threshold, the second accuracy threshold may be higher than the first accuracy threshold, or the second reasoning threshold may be lower than the first reasoning threshold, etc.

[0163] 606. Obtain a target neural network based on the trained serial network.

[0164] After determining that the trained serial network meets the termination condition, the target neural network can be obtained according to the trained serial network.

[0165] In a possible implementation, a target task may also be obtained, and the target neural network is used to perform the target task. For example, before step 601, the target task may be obtained, and in step 601, a starting network may be selected according to the target task, and the target neural network obtained subsequently may be used to perform the target task.

[0166] Specifically, obtaining a target neural network based on a trained serial network may include: if the trained serial network meets the output condition, using the trained serial network as the target neural network; if the trained serial network does not meet the output condition, obtaining a parallel subnetwork corresponding to the trained serial network, and constructing a target neural network based on the trained serial network and the parallel subnetwork corresponding to the trained serial network.

[0167] Typically, a serial network includes a plurality of serial subnetworks, each of which includes one or more basic units. The above-mentioned acquisition of a parallel subnetwork corresponding to a trained serial network may specifically include: determining the number of parallel subnetworks corresponding to each serial subnetwork in the serial network within a preset number range, and then constructing a parallel subnetwork corresponding to each serial subnetwork according to the number of parallel subnetworks corresponding to each serial subnetwork. Furthermore, the structure of the parallel subnetwork corresponding to each serial subnetwork is usually the same as that of the serial subnetwork. For example, if the number of parallel subnetworks corresponding to serial subnetwork 1 is 1, then a parallel subnetwork corresponding to serial subnetwork 1 may be constructed, and the structure of the parallel subnetwork is the same as that of serial subnetwork 1, or is obtained by deforming serial subnetwork 1.

[0168] In a specific embodiment, the above-mentioned construction of a target neural network based on a serial network and a parallel subnetwork corresponding to each serial subnetwork may include: constructing a temporary neural network based on the serial network and the parallel subnetwork corresponding to each serial subnetwork; evaluating the temporary neural network to obtain a third evaluation result; if the third evaluation result meets the output condition, the temporary neural network is used as the target neural network; if the third evaluation result does not meet the output condition, the parallel subnetwork corresponding to the serial network is re-acquired. Among them, the third evaluation result is similar to the aforementioned first evaluation result or the second evaluation result, which will not be repeated here. Furthermore, the specific process of obtaining the target neural network by constructing a parallel subnetwork corresponding to each serial subnetwork in the serial network can refer to the relevant description of the following steps 1012-1017, which will not be repeated here.

[0169] It can be understood that when the serial network meets the output condition, the serial network can be directly used as the target neural network, so as to obtain the target neural network efficiently and accurately.

[0170] For example, Fig. 9 As shown, the serial network includes serial subnetwork 1, serial subnetwork 2, serial subnetwork 3 and serial subnetwork 4. The number of parallel subnetworks corresponding to serial subnetwork 1 is 2, the number of parallel subnetworks corresponding to serial subnetwork 2 is 3, the number of parallel subnetworks corresponding to serial subnetwork 3 is 0, and the number of parallel subnetworks corresponding to serial subnetwork 4 is 2. Fig. 9 In the neural network shown in FIG. 1 , the transmission direction from serial subnetwork 1 to serial subnetwork 4 is called a row, and the serial subnetwork 1 and the parallel subnetworks parallel to it are called columns. Usually, after constructing the structure of the parallel subnetwork corresponding to each serial subnetwork, two adjacent rows in the same column are usually connected, for example, Fig. 9 As shown in , the input of block30 is the output of block20. When there is no adjacent parallel network in the same column of a parallel subnetwork, it can be connected to the serial subnetwork in the adjacent serial network. For example, the output of block6 in serial subnetwork 3 can be used as the input of block70 and block71, which are the basic units in the parallel subnetwork corresponding to serial subnetwork 4.

[0171] In an optional implementation, the output condition may include but is not limited to: the accuracy is greater than the first threshold, or the average accuracy is greater than the second threshold, or the loss value is not greater than the third threshold, or the inference time is not greater than the fourth threshold, the number of floating-point operations per second is not greater than the fifth threshold, etc. Among them, the average accuracy is the average value of multiple accuracies obtained by evaluating the neural network multiple times, and the inference time is the time taken to obtain the output result from the neural network according to the input.

[0172] Therefore, in the implementation manner of the present application, the starting point network can be deformed at least once to obtain a serial network. When the serial network meets the preset conditions, the serial network is trained through a preset data set, and there is no need to train the deformed starting point network every time it is deformed, thereby reducing the workload and improving the efficiency of obtaining the target neural network. In addition, the deformed starting point network can inherit the weight value of the starting point network before deformation, and update the inherited weight value through a lightweight data set, so as to achieve rapid training of the deformed starting point network, quickly obtain a serial network that meets the requirements, and further improve the efficiency of obtaining the target neural network. In addition, the method for constructing a neural network provided by the present application can adapt to different scenarios, and can search for serial networks according to different requirements. It can be oriented to more complex visual tasks and has strong generalization ability. Thereby, the cost of constructing different neural networks can be saved, the accuracy of the output of the constructed neural network can be improved, and the user experience can be improved.

[0173] The above is a detailed introduction to the process of the neural network construction method provided by the present application. For ease of understanding, the following is an exemplary introduction to the complete process of the neural network construction method provided by the present application in more detail. Fig.10 , a flow chart of another neural network construction method provided in this application is as follows.

[0174] It should be noted that, for ease of understanding, this application divides the construction of the target neural network into multiple stages, including the first stage and the second stage. Among them, the first stage is the serial search stage, which is the process of obtaining a serial network, and the second stage is the parallel search stage, which is the process of obtaining a parallel subnetwork and constructing a target neural network. The second stage is an optional stage. When the serial network obtained in the first stage meets the output conditions, the serial network can be used as the target neural network. It can be adjusted according to the actual application scenario, which will be introduced below.

[0175] Phase 1: Serial Search Phase

[0176] 1001. Obtain the starting point network.

[0177] 1002. Search for a deformation strategy in a first search space.

[0178] 1003. Deform the starting network to obtain a deformed starting network.

[0179] 1004. Update the weight of the deformed starting network using the lightweight data set to obtain the starting network with updated weights.

[0180] 1005. Evaluate the starting point network after the weight update to obtain a first evaluation result. If the update condition is met, execute step 1003; if the update condition is not met, execute step 1006.

[0181] 1006. The starting network after weight update is used as a serial network.

[0182] 1007 . Determine whether the serial network meets the preset conditions. If not, execute step 1002 . If yes, execute step 1008 .

[0183] 1008. Train the serial network using a preset data set to obtain a trained serial network

[0184] 1009. Determine whether the trained serial network meets the termination condition. If so, execute step 1010; if not, execute step 1002.

[0185] 1010. Determine whether the trained serial network meets the output condition. If so, execute step 1011; if not, execute step 1012.

[0186] 1011. Use the trained serial network as the backbone network of the target neural network.

[0187] Among them, steps 1001-1011 can refer to the relevant descriptions in the aforementioned steps 601-606, which will not be repeated here.

[0188] For example, take a scenario as an example. Fig.11As shown, the starting network includes serial subnetwork 1, serial subnetwork 2, serial subnetwork 3 and serial subnetwork 4. After searching for the deformation strategy in the first search space, the serial subnetwork included in the starting network is deformed, which may specifically include exchange or expansion. Among them, the convolution step size of the basic unit after the exchange can be changed, so that the basic unit after the convolution step size is changed can inherit the weight value of the serial subnetwork in the starting network before the deformation, and the expanded basic unit is obtained after grouping and normalization according to the unit matrix, so the expanded basic unit can also inherit the weight value of the corresponding serial subnetwork. When the serial network, that is, the deformed starting network meets the preset conditions, the serial network can be further trained, that is, the re-ignition process is started, and then the target neural network is obtained. The preset conditions can be understood as the triggering conditions for re-igniting the serial network. If the deformed starting network does not meet the preset conditions, the trained serial network is used as a new starting network, and the new starting network is re-deformed, so that a serial network that meets the post-point conditions can be obtained later.

[0189] Phase 2: Parallel search phase

[0190] 1012. Determine the number of parallel sub-networks corresponding to each serial sub-network in the plurality of serial sub-networks within a preset number range.

[0191] After obtaining a serial network that meets the termination condition but does not meet the output condition, the number of parallel sub-networks corresponding to each serial sub-network is determined within a preset number range, and the parallel sub-network is constructed according to the number.

[0192] For example, a serial network may include L feature layers, and the parallel subnetwork has the same structure as the serial subnetwork at the same feature layer. If the maximum number of parallel subnetworks allowed at each feature layer is K, the structure of the parallel subnetwork can be encoded as “k1, k2, …, k L ", a second search space can be constructed, which includes the range of the number of parallel subnetworks corresponding to each serial subnetwork, and is used to search and determine the number of parallel subnetworks corresponding to each serial subnetwork. The size of the second search space is K L For example, if the number of parallel subnetworks allowed to be added at each feature level is 3, that is, K = 3, and the number of feature levels is 10, then the size of the second search space can be 3 10 . In addition, "," can also be used to separate feature levels of different resolutions, that is, to indicate the location of downsampling. Usually, between two adjacent feature levels, the resolution corresponding to the latter feature level is halved compared to the resolution corresponding to the previous feature level. Of course, the resolution corresponding to the latter feature level can also be the same as the resolution corresponding to the previous feature level.

[0193] In addition, when searching for the number of parallel subnetworks corresponding to each feature level within a preset number range, a random search method can be used, or a preset resource constraint condition can be used as a constraint for sampling. For example, FLOPS, inference time, video memory occupancy rate, or CPU occupancy rate can be used as resource constraints for sampling. For example, the probability of adding a parallel subnetwork to each feature level is usually inversely proportional to the FLOPS of the serial subnetwork to be added, or the number of parallel subnetworks added to each feature level is inversely proportional to the inference time, video memory occupancy rate, or CPU occupancy rate. Therefore, in the implementation of the present application, random sampling, evolutionary algorithms, or resource constraints can be used for sampling, so that the number of parallel subnetworks corresponding to the requirements of the target task can be obtained, which speeds up the search process for the number of parallel subnetworks and improves the efficiency of obtaining the target neural network.

[0194] 1013. Construct a parallel subnetwork corresponding to each serial subnetwork according to the number of parallel subnetworks corresponding to each serial subnetwork.

[0195] After obtaining the number of parallel subnetworks corresponding to each serial subnetwork in the serial network, construct the parallel subnetwork corresponding to each serial subnetwork. Usually, the structure of the parallel subnetwork corresponding to each serial subnetwork is the same as the structure of the serial subnetwork in the corresponding serial network. For example, if a serial subnetwork in the serial network includes block2 and block3, and the number of corresponding parallel subnetworks is 3, block2 and block3 can be copied 3 times to obtain 3 parallel subnetworks.

[0196] 1014. Construct a temporary neural network according to the serial network and the parallel sub-network corresponding to each serial sub-network.

[0197] After constructing the parallel subnetwork corresponding to each serial subnetwork in the serial network, the parallel subnetwork is added on the basis of the serial network to obtain a temporary neural network.

[0198] For example, Fig.12 As shown, the serial network can include L feature levels, that is, L stages, and the serial network can be expressed as: SerialNet = s L (s l-1 (..s1(x))), l = 1, 2…L, where x is the input data of the serial network; L is the characteristic level of the serial network, that is, the number of serial subnetworks included in the serial network; s lRepresents the structure of the serial subnetwork at the lth feature level. The parallel subnetwork has the same structure as the serial subnetwork in the serial network at the same feature level. It can be understood that Satage1 in the serial network has the same structure as the parallel subnetwork Stage1 at the same level. For example, the structure of the parallel subnetwork corresponding to the serial subnetwork Stage1 can be the same as the structure of Stage1. After constructing the parallel subnetwork corresponding to each serial subnetwork, construct the connection mode of each parallel subnetwork. Usually, the input of the parallel subnetwork can be the output of the parallel subnetwork in the same column in the previous feature level. For example, represents the kth parallel subnetwork at the lth feature level. Usually, The input is The output of , and so on. If it does not exist, you can replace s l-1 The output is Therefore, in the implementation manner of the present application, after obtaining the serial network and the parallel sub-network corresponding to each serial sub-network, a temporary neural network can be constructed.

[0199] 1015. Evaluate the temporary neural network to obtain a third evaluation result.

[0200] After obtaining the temporary neural network, the temporary neural network may be evaluated to obtain a third evaluation result.

[0201] Specifically, the process of evaluating the temporary neural network is similar to the process of evaluating the starting network or the serial network, which will not be repeated here. The third evaluation result may also include the loss value, accuracy, inference time, FLOPS, average accuracy, etc. corresponding to the output result of the temporary neural network.

[0202] 1016. Determine whether the third evaluation result meets the output condition. If so, execute step 1017; if not, execute step 1012.

[0203] After obtaining the third evaluation result of the temporary neural network, determine whether the third evaluation result meets the output condition. If the third evaluation result meets the output condition, the temporary neural network can be used as the target neural network or the backbone network of the target neural network; if the third evaluation result does not meet the output condition, the number of parallel subnetworks corresponding to each serial subnetwork can be searched in a preset number range, that is, steps 1012-1016 are repeated until the evaluation result of the temporary neural network meets the output condition.

[0204] Specifically, the output condition can refer to the relevant introduction in the aforementioned step 606, which will not be repeated here.

[0205] 1017. Use the temporary neural network as the backbone network of the target neural network.

[0206] Among them, after determining that the output result of the temporary neural network meets the output condition, the temporary neural network can be used as the backbone network of the target neural network.

[0207] Usually, the backbone network can complete the target detection of the image, and the target neural network can be used for one or more of image recognition, semantic segmentation or target detection. Therefore, on the basis of the backbone network, other networks can be added to enable the target neural network to realize different functions. For example, heads with different functions can be added to enable the target neural network to have different functions, which can be adjusted according to the actual application scenario.

[0208] For example, Fig.13 As shown, after the backbone network is obtained through the serial search phase and the parallel search phase, FPN, RPN and Head are added on the basis of the backbone network. The added FPN, RPN and Head can be determined according to the specific needs of the target task. For example, if image recognition, semantic segmentation or target detection is required, one or more Heads corresponding to image recognition, semantic segmentation or target detection can be added. Therefore, compared with using the existing classification network as the backbone network, in the implementation mode of the present application, different functional networks can be added on the basis of the backbone network to adapt to different scenarios, so that the backbone network constructed by the present application has strong generalization ability, and can also have high accuracy in different scenarios, thereby improving the efficiency of constructing the target neural network.

[0209] Therefore, in the implementation manner of the present application, the process of constructing the target neural network can be divided into two stages. In the serial search stage, the starting point network can be deformed at least once to obtain a serial network. When the serial network meets the preset conditions, the serial network is trained through a preset data set, and there is no need to train the deformed starting point network every time it is deformed, thereby reducing the workload and improving the efficiency of obtaining the target neural network. In addition, the deformed starting point network can inherit the weight value of the starting point network before deformation, and the inherited weight value can be updated through a lightweight data set, thereby realizing rapid training of the deformed starting point network, quickly obtaining a serial network that meets the requirements, and further improving the efficiency of obtaining the target neural network. In the parallel search stage, the number of parallel subnetworks corresponding to each serial subnetwork in the serial network can be constrained by searching the resource constraint conditions corresponding to the target task, so that the target serial subnetwork constructed by the searched parallel subnetworks can meet the output conditions while meeting the resource constraints, and obtain a neural network with more accurate output results.

[0210] For ease of understanding, the effects achieved by the neural network construction method provided in this application are introduced below with some specific application scenarios.

[0211] In the scenario of the backbone network trained with different data sets, the proportion of the computational amount of different feature levels in the backbone network may be different. As shown in Table 1 below, for example, different scenarios, such as the scenario of training with ECP, BDD (Berkeley deep drive), COCO, VOC (PASCAL VOC) and other data sets, use different basic units, such as bottleneck units (bottle neck block, BNB), basic units (Basic Block, BB) or bottleneck grouping units (ResNeXt Block, XB), as an example for explanation.

[0212]

[0213] Table 1

[0214] Among them, the first column is the scene of different data sets, the second column is the selection of basic network units; the third column is the serial network structure searched in the serial stage, "," means that different resolutions separate each feature level stage, and the resolution of the latter stage is halved compared to the previous stage; "1" means a regular block that does not change the input and output channels, that is, the output channel is 1 times the input channel, "d" means that the number of basic channels in the block is doubled, that is, "2"; the last column represents the parallel network structure searched, "," means different feature level stages, and the number indicates the number of parallel sub-networks added to the feature level.

[0215] The following is a brief introduction to the different datasets.

[0216] COCO: The COCO 2017 dataset contains 80 target object categories, corresponding to approximately 118,000 training images and 5,000 test images.

[0217] BDD: is an autonomous driving dataset containing 10 types of target objects. It includes 70,000 training images and 10,000 test images.

[0218] VOC: Contains 20 types of target objects. It includes 10,000 training images and 4,900 test images.

[0219] ECP: It is the largest pedestrian detection dataset, divided into two categories: pedestrians and cyclists. It includes 24,000 training images and 4,300 test images.

[0220] Evaluation criteria may include: average precision (AP), which considers the accuracy at different thresholds, and the average precision and average recall of objects of different sizes. Running speed (Inf time), which compares all methods on the same graphics card to obtain a network architecture with higher accuracy at the same inference time. Log-average miss rate (LAMR), which represents the number of positive examples in the test set that are judged as negative examples / the number of desired positive examples retrieved in the test set plus the log index of undetected unwanted examples. The lower the standard, the better.

[0221] From the structures of the serial network and parallel sub-network of the target neural network obtained in Table 1, it can be seen that when facing the COCO dataset with complex targets, more complex semantic information and a larger field of view are required to detect the targets in the image. In the pedestrian detection (Euro City Persons, ECP) ​​dataset, there are a large number of small targets with a single category, so more computation needs to be invested in the feature level that retains more spatial information.

[0222] In addition, in different data sets, the backbone network (hereinafter referred to as SPNet) obtained by searching through the neural network construction method provided in this application is significantly better than the models obtained by existing manual designs (such as ResNext, HRNet, DetNet) and automatic search algorithms (such as DetNAS, NAS-FPN, etc.) in terms of FLOPS and detection accuracy.

[0223] Moreover, on the COCO dataset, the architectures SPNet (XB) and SPNet (BNB) searched by this application are compared with the classic manually designed classification backbone network ResNext101, and the FLOPS is reduced by nearly half (8.1G vs 15.2G) while improving mAP by 7%. Compared with another backbone network HRNet-w48 manually designed for detection tasks, the architecture E5 searched by this application improves mAP by nearly 6% at a similar running speed without using additional target segmentation annotations. Compared with target detection methods (such as AlignDet, NAS-FPN, etc.), it has significant advantages in both running speed and detection performance. Based on the FPN detector on the public datasets VOC and BDD, the backbone network searched by this application has significant performance improvements in accuracy and speed over the manually designed ResNet network or other automatic search algorithms. For example, in the VOC scenario, SPNet outperforms DetNAS, Auto-FPN AP50 and other solutions by 3.2% and 2.8% respectively; in the BDD scenario, SPNet runs faster than Auto-FPN, and its mAP is improved by 14%. Moreover, in the scenario of the public data set VOC, compared with DetNAS and NAS-FPN, this application can search for a better-performing architecture in less time. Moreover, the network architecture searched by this application also has good generalization performance on other data sets (all better than the manually designed backbone network ResNet101), and the more similar the migration scenario is (autonomous driving scenarios BDD, ECP), the better the migration effect.

[0224] The above has introduced the method provided by the present application in detail. The following is an introduction to the device provided by the present application.

[0225] See also Fig.14 , a schematic diagram of the structure of a neural network construction device provided in the present application. The neural network construction device can be used to perform the aforementioned Figure 6-13 The neural network construction method shown in . The neural network construction device includes:

[0226] An acquisition module 1401 is used to acquire a starting point network, where the starting point network includes a plurality of serial sub-networks;

[0227] A deformation module 1402 is used to deform the starting network at least once based on a preset first search space to obtain a serial network, wherein the first search space includes a range of parameters used to deform the starting network;

[0228] The training module 1403 is used to train the serial network using a preset data set to obtain a trained serial network if the serial network meets the preset conditions;

[0229] The construction module 1404 is used to obtain a target neural network according to the trained serial network if the trained serial network meets the termination condition.

[0230] In a possible implementation, the deformation module 1402 can be specifically used to: search for a deformation strategy in a preset first search space, the deformation strategy includes a set of parameters used to deform the starting network; deform the starting network according to the deformation strategy to obtain a deformed starting network, the weight of the deformed starting network is inherited from the weight of the starting network before deformation; update the weight of the deformed starting network through a lightweight data set to obtain a starting network after weight update, the preset data set includes more data than the lightweight data set; evaluate the starting network after weight update to obtain a first evaluation result; if the first evaluation result meets the preset update condition, re-search the deformation strategy in the first search space, and re-deform the starting network according to the re-searched deformation strategy; if the first evaluation result does not meet the preset update condition, the starting network after weight update is used as a serial network.

[0231] In a possible implementation, the neural network construction device may further include:

[0232] The updating module 1405 is used to use the serial network as a new starting network if the serial network does not meet the preset conditions, or use the trained serial network as a new starting network if the trained serial network does not meet the termination conditions, update the first search space, and re-search the deformation strategy according to the updated first search space, and re-deform the new starting network according to the re-searched deformation strategy.

[0233] In one possible implementation, the update conditions include: the number of deformations of the starting network does not exceed a preset number, or the first evaluation result does not meet a preset convergence condition, and the convergence condition includes one or more of the following: the first evaluation result is better than a threshold, or the difference between the first evaluation result and the last evaluation result obtained by evaluating the starting network is less than a preset value.

[0234] In a possible implementation, the preset conditions may include but are not limited to: the second evaluation result of the serial network satisfies the first preset evaluation condition, or the number of deformations of the serial network exceeds the preset number of deformations, or the time since the last deformation of the starting network exceeds the preset value, and the second evaluation result is obtained by evaluating the serial network;

[0235] The first preset evaluation condition includes one or more of the following: the loss value of the serial network is not greater than the first loss threshold, the accuracy value of the output result of the serial network is greater than the first accuracy threshold, or the inference time of the serial network is less than the first inference threshold.

[0236] In a possible implementation, each serial subnetwork of the starting point network is composed of one or more basic units, and the deformation module 1402 can be specifically used to: exchange the basic units included in two adjacent serial subnetworks in the multiple serial subnetworks; or, insert a basic unit into one or more of the multiple serial subnetworks, and the inserted basic unit is obtained by weighting and grouping normalization initialized according to the unit matrix.

[0237] In a possible implementation, the termination condition may include but is not limited to: the second evaluation result of the serial network meets the second preset evaluation condition, or the number of training times for the serial network exceeds the preset number of training times.

[0238] In one possible implementation, construction module 1404 can be specifically used to: if the trained serial network meets the output condition, then use the trained serial network as the target neural network; if the trained serial network does not meet the output condition, then obtain the parallel sub-network corresponding to the trained serial network, and construct the target neural network based on the trained serial network and the parallel sub-network corresponding to the trained serial network.

[0239] In one possible implementation, the serial network includes multiple serial sub-networks, each serial sub-network includes one or more basic units, and the construction module 1404 can be specifically used to: determine the number of parallel sub-networks corresponding to each serial sub-network in the multiple serial sub-networks within a preset number range; construct a parallel sub-network corresponding to each serial sub-network according to the number of parallel sub-networks corresponding to each serial sub-network, and the structure of each serial sub-network is the same as the structure of the parallel sub-network corresponding to each serial sub-network.

[0240] In one possible implementation, construction module 1404 can be specifically used to: construct a temporary neural network based on the serial network and the parallel subnetwork corresponding to each serial subnetwork; evaluate the temporary neural network to obtain a third evaluation result; if the third evaluation result meets the output condition, use the temporary neural network as the target neural network; if the third evaluation result does not meet the output condition, re-acquire the parallel subnetwork corresponding to the serial network.

[0241] In one possible implementation, the output conditions include but are not limited to one or more of the following: the accuracy is greater than a first threshold, or the average accuracy is greater than a second threshold, or the loss value is not greater than a third threshold, or the inference time is not greater than a fourth threshold, the number of floating-point operations per second is not greater than a fifth threshold, the average accuracy is the average of multiple accuracies obtained by performing multiple evaluations of the neural network, and the inference time is the time it takes to obtain the output result from the neural network based on the input.

[0242] In one possible implementation, the types of parameters included in the first search space may include, but are not limited to: width, depth, number of downsampling, and location of downsampling, the width being the number of input channels and output channels of the basic units included in each serial subnetwork in the starting network, the depth being the number of basic units included in each serial subnetwork in the starting network, and downsampling being used to split two adjacent serial subnetworks in the starting network.

[0243] In a possible implementation, the target neural network is used to perform at least one of image recognition, semantic segmentation, or target detection, and the preset data set may include an ImageNet data set.

[0244] In a possible implementation, a specific method of obtaining the starting network may include: obtaining the target task through user input data, and then obtaining the starting network according to the target task, and the constructed target neural network can be used to perform the target task. Fig.15 , a structural schematic diagram of another neural network construction device provided in the present application is described as follows.

[0245] The neural network construction device may include a processor 1501 and a memory 1502. The processor 1501 and the memory 1502 are interconnected via a line. The memory 1502 stores program instructions and data.

[0246] The memory 1502 stores the aforementioned Figure 4 or Figure 8 The program instructions and data corresponding to the steps in .

[0247] Processor 1501 is used to execute the above Figure 4 or Figure 8 The method steps are performed by the neural network construction device shown in any embodiment.

[0248] Optionally, the neural network construction device may further include a transceiver 1503 for receiving or sending data.

[0249] In an embodiment of the present application, a computer-readable storage medium is also provided, in which a program for generating a vehicle driving speed is stored. When the computer is running on a computer, the computer executes the above-mentioned Figures 4 to 8 The illustrated embodiments describe steps in a method.

[0250] Optionally, the aforementioned Fig.15 The neural network construction device shown in is a chip.

[0251] The embodiment of the present application also provides a neural network construction device, which can also be called a digital processing chip or chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface. The program instructions are executed by the processing unit. The processing unit is used to execute the aforementioned Figure 6 or Fig.10 The method steps are performed by the neural network construction device shown in any embodiment.

[0252] The embodiment of the present application also provides a digital processing chip. The digital processing chip integrates a circuit and one or more interfaces for implementing the above-mentioned processor 1501, or the functions of the processor 1501. When a memory is integrated in the digital processing chip, the digital processing chip can complete the method steps of any one or more of the aforementioned embodiments. When the digital processing chip does not integrate a memory, it can be connected to an external memory through a communication interface. The digital processing chip implements the actions performed by the neural network construction device in the above-mentioned embodiment according to the program code stored in the external memory.

[0253] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the above Figures 6 to 13 The illustrated embodiment describes the steps performed by the neural network construction device in the method.

[0254] The neural network construction device provided in the embodiment of the present application may be a chip, and the chip includes: a processing unit and a communication unit, the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit. The processing unit may execute the computer execution instructions stored in the storage unit, so that the chip in the server executes the above Figures 4 to 8 The neural network construction method described in the illustrated embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0255] Specifically, the aforementioned processing unit or processor may be a central processing unit (CPU), a neural-network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0256] For example, see Fig.16 , Fig.16 A schematic diagram of the structure of a chip provided in an embodiment of the present application, the chip can be expressed as a neural network processor NPU 160, NPU 160 is mounted on the host CPU (Host CPU) as a coprocessor, and the host CPU assigns tasks. The core part of the NPU is the operation circuit 1603, which is controlled by the controller 1604 to extract matrix data from the memory and perform multiplication operations.

[0257] In some implementations, the operation circuit 1603 includes multiple processing units (process engines, PEs) inside. In some implementations, the operation circuit 1603 is a two-dimensional systolic array. The operation circuit 1603 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition. In some implementations, the operation circuit 1603 is a general-purpose matrix processor.

[0258] For example, assume there is an input matrix A, a weight matrix B, and an output matrix C. The operation circuit takes the corresponding data of matrix B from the weight memory 1602 and caches it on each PE in the operation circuit. The operation circuit takes the matrix A data from the input memory 1601 and performs matrix operation with matrix B, and the partial result or final result of the matrix is ​​stored in the accumulator 1608.

[0259] The unified memory 1606 is used to store input data and output data. The weight data is directly transferred to the weight memory 1602 through the direct memory access controller (DMAC) 1605. The input data is also transferred to the unified memory 1606 through the DMAC.

[0260] The bus interface unit (BIU) 1610 is used for the interaction between the AXI bus and the DMAC and the instruction fetch buffer (IFB) 1609 .

[0261] The bus interface unit 1610 (BIU) is used for the instruction fetch memory 1609 to obtain instructions from the external memory, and is also used for the storage unit access controller 1605 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0262] DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 1606 or to transfer weight data to the weight memory 1602 or to transfer input data to the input memory 1601.

[0263] The vector calculation unit 1607 includes multiple operation processing units, and further processes the output of the operation circuit when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, upsampling of feature planes, etc.

[0264] In some implementations, the vector calculation unit 1607 can store the processed output vector to the unified memory 1606. For example, the vector calculation unit 1607 can apply a linear function and / or a nonlinear function to the output of the operation circuit 1603, such as linear interpolation of the feature plane extracted by the convolution layer, and then, for example, a vector of accumulated values ​​to generate an activation value. In some implementations, the vector calculation unit 1607 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1603, for example, for use in a subsequent layer in a neural network.

[0265] An instruction fetch buffer 1609 connected to the controller 1604 is used to store instructions used by the controller 1604;

[0266] Unified memory 1606, input memory 1601, weight memory 1602 and instruction fetch memory 1609 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0267] Among them, the operations of each layer in the recurrent neural network can be performed by the operation circuit 1603 or the vector calculation unit 1607.

[0268] The processor mentioned in any of the above can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more processors for controlling the above Figure 6-Figure 13 The method is performed by a program of an integrated circuit.

[0269] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0270] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0271] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0272] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

[0273] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0274] Finally, it should be noted that the above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for constructing a neural network, characterized in that: include: Acquire a starting point network, wherein the starting point network includes a plurality of serial sub-networks; The starting network is deformed at least once based on a first search space to obtain a serial network, wherein the first search space includes a range of parameters used to deform the starting network, and the types of parameters included in the first search space include: width, depth, number of downsampling, and position of downsampling, wherein the width is the number of input channels and output channels of a basic unit included in each serial sub-network in the starting network, the depth is the number of basic units included in each serial sub-network in the starting network, and the downsampling is used to split two adjacent serial sub-networks in the starting network; If the serial network meets the preset conditions, the serial network is trained using a preset data set to obtain a trained serial network; If the trained serial network meets the termination condition, a target neural network is obtained according to the trained serial network; Each serial sub-network in the starting point network is composed of one or more basic units. The deformation of the starting point network includes: exchanging the basic units included in two adjacent serial sub-networks among the plurality of serial sub-networks; or, Inserting processed basic units into at least one of the multiple serial sub-networks, wherein the processed basic units are obtained by initializing the unit matrix of the weights of the basic units and / or grouping and normalizing the basic units.

2. The method according to claim 1, characterized in that The performing at least one transformation of the starting point network based on the first search space comprises: Searching for a deformation strategy in the first search space, the deformation strategy comprising a set of parameters used to deform the starting network; Deforming the starting point network according to the deformation strategy to obtain a deformed starting point network, wherein the weight of the deformed starting point network inherits the weight of the starting point network before the deformation; updating the weight of the deformed starting point network by using a lightweight data set to obtain a starting point network after weight update, wherein the preset data set includes more data than the lightweight data set; Evaluate the starting point network after the weight update to obtain a first evaluation result; If the first evaluation result satisfies a preset update condition, re-searching a deformation strategy in the first search space, and re-deforming the starting point network by using the deformation strategy obtained by the re-search; If the first evaluation result does not meet the preset update condition, the starting network after the weight update is used as the serial network.

3. The method according to claim 2, characterized in that The method further comprises: If the serial network does not meet the preset condition, the serial network is used as a new starting point network, or if the trained serial network does not meet the termination condition, the trained serial network is used as a new starting point network; The first search space is updated, and a deformation strategy is searched according to the updated first search space, and the new starting point network is deformed according to the searched deformation strategy.

4. The method according to claim 2 or 3, characterized in that: The update condition includes: the number of deformations of the starting network does not exceed a preset number, or the first evaluation result does not meet a preset convergence condition, and the convergence condition includes one or more of the following: the first evaluation result is better than a threshold, or the difference between the first evaluation result and the evaluation result obtained by the last evaluation of the starting network is less than a preset value.

5. The method according to any one of claims 1 to 3, characterized in that: The preset conditions include: the second evaluation result of the serial network meets the first preset evaluation condition, or the number of deformations of the serial network exceeds the preset number of deformations, or the time since the last deformation of the starting network exceeds the preset value, and the second evaluation result is obtained by evaluating the serial network; The first preset evaluation condition includes one or more of the following: the loss value of the serial network is not greater than a first loss threshold, the accuracy value of the output result of the serial network is greater than a first accuracy threshold, or the inference time of the serial network is less than a first inference threshold.

6. The method according to any one of claims 1 to 3, characterized in that The termination conditions include: The second evaluation result of the serial network satisfies a second preset evaluation condition, or the number of training times for the serial network exceeds a preset number of training times.

7. The method according to any one of claims 1 to 3, characterized in that The step of obtaining a target neural network according to the trained serial network comprises: If the trained serial network meets the output condition, the trained serial network is used as the target neural network; If the trained serial network does not meet the output condition, a parallel sub-network corresponding to the trained serial network is obtained, and the target neural network is constructed according to the trained serial network and the parallel sub-network corresponding to the trained serial network.

8. The method according to claim 7, characterized in that The serial network includes a plurality of serial sub-networks, each of which includes one or more basic units. The obtaining of a parallel sub-network corresponding to the trained serial network comprises: Determine the number of parallel sub-networks corresponding to each serial sub-network in the plurality of serial sub-networks within a preset number range; The parallel subnetworks corresponding to each serial subnetwork are constructed according to the number of the parallel subnetworks corresponding to each serial subnetwork, and the structure of each serial subnetwork is the same as the structure of the parallel subnetwork corresponding to each serial subnetwork.

9. The method according to claim 8, characterized in that The step of constructing the target neural network according to the trained serial network and the parallel sub-network corresponding to the trained serial network includes: Constructing a temporary neural network according to the trained serial network and the parallel sub-network corresponding to each serial sub-network; Evaluating the temporary neural network to obtain a third evaluation result; If the third evaluation result satisfies the output condition, the temporary neural network is used as the target neural network; If the third evaluation result does not meet the output condition, the parallel sub-network corresponding to each serial sub-network is re-acquired.

10. The method according to claim 7, characterized in that The output conditions include one or more of the following: the accuracy is greater than a first threshold, or the average accuracy is greater than a second threshold, or the loss value is not greater than a third threshold, or the inference time is not greater than a fourth threshold, and the number of floating-point operations per second is not greater than a fifth threshold. The average accuracy is the average of multiple accuracies obtained by multiple evaluations of the neural network, and the inference time is the time it takes to obtain an output result from the neural network based on the input.

11. The method according to any one of claims 1 to 3, characterized in that: The target neural network is used to perform at least one of image recognition, semantic segmentation or target detection, and the preset data set includes the ImageNet data set.

12. A neural network construction device, characterized in that: include: An acquisition module, used for acquiring a starting point network, wherein the starting point network includes a plurality of serial sub-networks; A deformation module, used for deforming the starting network at least once based on a first search space to obtain a serial network, wherein the first search space includes a range of parameters used to deform the starting network, and the types of parameters included in the first search space include: width, depth, number of downsampling and position of downsampling, wherein the width is the number of input channels and output channels of the basic unit included in each serial sub-network in the starting network, the depth is the number of basic units included in each serial sub-network in the starting network, and the downsampling is used to split two adjacent serial sub-networks in the starting network; A training module, used for training the serial network using a preset data set to obtain a trained serial network if the serial network meets a preset condition; A construction module, configured to obtain a target neural network according to the trained serial network if the trained serial network meets a termination condition; Each serial sub-network in the starting point network is composed of one or more basic units, and the deformation module is specifically used to: exchanging the basic units included in two adjacent serial sub-networks among the plurality of serial sub-networks; or, Inserting processed basic units into one or more of the plurality of serial sub-networks, wherein the processed basic units are obtained by initializing the weights of the basic units with a unit matrix and / or performing group normalization on the basic units.

13. The neural network construction device according to claim 12, characterized in that: The deformation module is specifically used for: Searching for a deformation strategy in the first search space, the deformation strategy comprising a set of parameters used to deform the starting network; Deforming the starting point network according to the deformation strategy to obtain a deformed starting point network, wherein the weight of the deformed starting point network inherits the weight of the starting point network before the deformation; updating the weight of the deformed starting point network by using a lightweight data set to obtain a starting point network after weight update, wherein the preset data set includes more data than the lightweight data set; Evaluate the starting point network after the weight update to obtain a first evaluation result; If the first evaluation result satisfies a preset update condition, re-searching a deformation strategy in the first search space, and re-deforming the starting point network by using the deformation strategy obtained by the re-search; If the first evaluation result does not meet the preset update condition, the starting network after the weight update is used as the serial network.

14. The neural network construction device according to claim 13, characterized in that: The neural network construction device also includes: An updating module, configured to use the serial network as a new starting network if the serial network does not meet the preset condition, or use the trained serial network as a new starting network to update the first search space if the trained serial network does not meet the termination condition; The deformation module is further used to search for a deformation strategy according to the updated first search space, and to deform the new starting point network according to the searched deformation strategy.

15. The neural network construction device according to claim 13 or 14, characterized in that: The update condition includes: the number of deformations of the starting network does not exceed a preset number, or the first evaluation result does not meet a preset convergence condition, and the convergence condition includes one or more of the following: the first evaluation result is better than a threshold, or the difference between the first evaluation result and the evaluation result obtained by the last evaluation of the starting network is less than a preset value.

16. The neural network construction device according to any one of claims 12 to 14, characterized in that: The preset conditions include: the second evaluation result of the serial network meets the first preset evaluation condition, or the number of deformations of the serial network exceeds the preset number of deformations, or the time since the last deformation of the starting network exceeds the preset value, and the second evaluation result is obtained by evaluating the serial network; The first preset evaluation condition includes one or more of the following: the loss value of the serial network is not greater than a first loss threshold, the accuracy value of the output result of the serial network is greater than a first accuracy threshold, or the inference time of the serial network is less than a first inference threshold.

17. The neural network construction device according to any one of claims 12 to 14, characterized in that: The termination conditions include: The second evaluation result of the serial network satisfies a second preset evaluation condition, or the number of training times for the serial network exceeds a preset number of training times.

18. The neural network construction device according to any one of claims 12 to 14, characterized in that: The building blocks are specifically used for: If the trained serial network meets the output condition, the trained serial network is used as the target neural network; If the trained serial network does not meet the output condition, a parallel sub-network corresponding to the trained serial network is obtained, and the target neural network is constructed according to the trained serial network and the parallel sub-network corresponding to the trained serial network.

19. The neural network construction device according to claim 18, characterized in that: The serial network includes a plurality of serial sub-networks, each of which includes one or more basic units. The building blocks are specifically used for: Determine the number of parallel sub-networks corresponding to each serial sub-network in the plurality of serial sub-networks within a preset number range; The parallel subnetworks corresponding to each serial subnetwork are constructed according to the number of the parallel subnetworks corresponding to each serial subnetwork, and the structure of each serial subnetwork is the same as the structure of the parallel subnetwork corresponding to each serial subnetwork.

20. The neural network construction device according to claim 19, characterized in that: The building blocks are specifically used for: Constructing a temporary neural network according to the trained serial network and the parallel sub-network corresponding to each serial sub-network; Evaluating the temporary neural network to obtain a third evaluation result; If the third evaluation result satisfies the output condition, the temporary neural network is used as the target neural network; If the third evaluation result does not meet the output condition, the parallel sub-network corresponding to the serial network is re-acquired.

21. The neural network construction device according to claim 18, characterized in that: The output conditions include one or more of the following: the accuracy is greater than a first threshold, or the average accuracy is greater than a second threshold, or the loss value is not greater than a third threshold, or the inference time is not greater than a fourth threshold, and the number of floating-point operations per second is not greater than a fifth threshold. The average accuracy is the average of multiple accuracies obtained by multiple evaluations of the neural network, and the inference time is the time it takes to obtain an output result from the neural network based on the input.

22. The neural network construction device according to any one of claims 12 to 14, characterized in that: The target neural network is used to perform at least one of image recognition, semantic segmentation or target detection, and the preset data set includes the ImageNet data set.

23. A neural network construction device, characterized in that: The method comprises a processor, wherein the processor is coupled to a memory, wherein the memory stores a program, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 11 is implemented.

24. A computer-readable storage medium comprising a program, which, when executed by a processing unit, performs the method according to any one of claims 1 to 11.

25. A neural network construction device, characterized in that: The method comprises a processing unit and a communication interface, wherein the processing unit obtains program instructions through the communication interface, and when the program instructions are executed by the processing unit, the method according to any one of claims 1 to 11 is implemented.

Citation Information

Patent Citations

  • Neural network structure generation method and device, electronic equipment and storage medium

    CN108229647A

  • New deformation monitoring method for super high-rise building with troposphere delay influence taken into account

    CN108917583A

  • Neural network construction method and device and image processing method and device

    CN110175671A

  • Neural network model construction method, device and equipment, and storage medium

    CN110334802A

  • Method and device for establishing image detection model, equipment and computer storage medium

    CN110363810A

Cited By

  • Neural network construction method and apparatus

    WO2021238366A1