A network acquisition method and device

Abstract

The present application provides a network acquisition method and device. In this method, the second sub-network can be adjusted according to the loss value of the first sub-network, the loss value of the second sub-network and the loss value of the initial network to obtain a target network corresponding to the basic network, that is, the server combines the loss values ​​of multiple networks related to the basic network to adjust the second sub-network, which can not only reduce the power consumption and delay of the target network, but also improve the accuracy of the target network.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a network acquisition method and device. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, neural networks are widely used in various fields. Generally speaking, the network structure of neural networks is relatively complex and there are many parameters of neural networks. Therefore, when running neural networks, power consumption is high and latency is also high. This causes the terminal device to be unable to support the operation of the neural network, so such a neural network cannot be directly deployed on the terminal device. Therefore, how to reduce the power consumption and latency of neural networks has become a technical problem that needs to be solved urgently at the current stage. Summary of the invention

[0003] The present application provides a network acquisition method and device, which can reduce network power consumption and latency.

[0004] In a first aspect, a method for acquiring a network is provided, the method being applied to a server, the method comprising: constructing a super network based on a basic network, the super network comprising at least one subnetwork, each subnetwork in the at least one subnetwork comprising at least one block; determining a first subnetwork from the at least one subnetwork, the spatial degree of freedom of the first subnetwork satisfying preset spatial degree of freedom information and the number of blocks in the first subnetwork being the same as the number of blocks in an initial network corresponding to the basic network; constructing a second subnetwork based on the first subnetwork and the initial network, the number of blocks in the second subnetwork being the same as the number of blocks in the first subnetwork, the network structure of the second subnetwork being different from the network structure of the first subnetwork and the network structure of the initial network; adjusting the second subnetwork according to the loss value of the first subnetwork, the loss value of the second subnetwork and the loss value of the initial network to obtain a target network corresponding to the basic network.

[0005] By implementing the network acquisition method provided in the first aspect, the server can adjust the second subnetwork according to the loss value of the first subnetwork, the loss value of the second subnetwork and the loss value of the initial network to obtain the target network corresponding to the basic network, that is, the server combines the loss values ​​of multiple networks related to the basic network to adjust the second subnetwork, which can not only reduce the power consumption and latency of the target network, but also improve the accuracy of the target network.

[0006] In combination with the method provided in the first aspect, in some embodiments, the method further includes: sending parameters of the target network to the terminal device, the parameters of the target network are used by the terminal device to operate the target network.

[0007] In combination with the method provided in the first aspect, in some embodiments, before sending the parameters of the target network to the terminal device, the method further includes: receiving indication information from the terminal device, where the indication information is used to request the parameters of the target network.

[0008] In combination with the method provided in the first aspect, in some embodiments, constructing a super network based on a basic network includes: establishing a search space based on the basic network, the search space being used to indicate a range of changes in the network structure of the basic network; and constructing a super network based on the search space.

[0009] In combination with the method provided in the first aspect, in some embodiments, the spatial degrees of freedom of the first sub-network include at least one of the following: the depth of the convolution kernel in the first sub-network, the width of the convolution kernel in the first sub-network, the depth of the first sub-network, and the width of the first sub-network.

[0010] In combination with the method provided in the first aspect, in some embodiments, the input data of the first subnetwork is an image, the output data of the first subnetwork is a depth map obtained by depth estimation based on the input data of the first subnetwork, and the ratio between the pixels with incorrect estimation in the depth map and all the pixels in the depth map is less than or equal to a preset threshold.

[0011] In combination with the method provided in the first aspect, in some embodiments, the network structure of the second subnetwork is different from the network structure of the first subnetwork and the network structure of the initial network, including: the number of channels in the same block in the second subnetwork, the first subnetwork and the initial network are all different.

[0012] In a second aspect, a network acquisition method is provided, which is applied to a terminal device, and the method includes: receiving parameters of a target network from a server; running the target network according to the parameters of the target network; wherein the target network is obtained by adjusting the second subnetwork according to the loss value of the first subnetwork, the loss value of the second subnetwork and the loss value of the initial network corresponding to the basic network, the first subnetwork is determined from at least one subnetwork included in the super network, the spatial degree of freedom of the first subnetwork satisfies preset spatial degree of freedom information and the number of blocks in the first subnetwork is the same as the number of blocks in the initial network, the second subnetwork is constructed according to the first subnetwork and the initial network, the number of blocks in the second subnetwork is the same as the number of blocks in the first subnetwork, and the network structure of the second subnetwork is different from both the network structure of the first subnetwork and the network structure of the initial network.

[0013] In combination with the method provided in the second aspect, in some embodiments, before receiving the parameters of the target network from the server, the method further includes: sending indication information to the server, where the indication information is used to request the parameters of the target network.

[0014] In combination with the method provided in the second aspect, in some embodiments, the spatial degrees of freedom of the first sub-network include at least one of the following: the depth of the convolution kernel in the first sub-network, the width of the convolution kernel in the first sub-network, the depth of the first sub-network, and the width of the first sub-network.

[0015] In combination with the method provided in the second aspect, in some embodiments, the input data of the first subnetwork is an image, and the output data of the first subnetwork is a depth map obtained by depth estimation based on the input data of the first subnetwork, and the ratio between the pixels with incorrect estimation in the depth map and all the pixels in the depth map is less than or equal to a preset threshold.

[0016] In combination with the method provided in the second aspect, in some embodiments, the network structure of the second subnetwork is different from the network structure of the first subnetwork and the network structure of the initial network, including: the number of channels in the same block in the second subnetwork, the first subnetwork and the initial network are all different.

[0017] In a third aspect, a server is provided, comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the method described in the first aspect and any possible implementation method of the first aspect is executed.

[0018] In a fourth aspect, a terminal device is provided, comprising one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, and the computer program code includes computer instructions, and when the one or more processors execute the computer instructions, the method described in the second aspect and any possible implementation method of the second aspect is executed.

[0019] In a fifth aspect, a computer program product comprising instructions is provided. When the computer program product is run on a server, the server executes the first aspect and any possible implementation method of the first aspect.

[0020] In a sixth aspect, a computer program product comprising instructions is provided. When the computer program product is run on a terminal device, the terminal device executes the second aspect and any possible implementation method of the second aspect.

[0021] In a seventh aspect, a computer-readable storage medium is provided, comprising instructions, which, when executed on a server, enable execution of the first aspect and any possible implementation of the first aspect.

[0022] In an eighth aspect, a computer-readable storage medium is provided, comprising instructions, which, when executed on a terminal device, enable execution of the second aspect and any possible implementation of the second aspect.

[0023] It can be understood that the beneficial effects of the above-mentioned second aspect can refer to the beneficial effects of the first aspect, which will not be repeated here. In addition, the server provided in the above-mentioned third aspect, the computer program product provided in the fifth aspect, and the computer-readable storage medium provided in the seventh aspect are all used to execute the method provided in the first aspect of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. The terminal device provided in the above-mentioned fourth aspect, the computer program product provided in the sixth aspect, and the computer-readable storage medium provided in the eighth aspect are all used to execute the method provided in the second aspect of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0025] Figure 2 It is a software architecture diagram of a terminal device provided in an embodiment of the present application;

[0026] Figure 3 A flowchart of a method for acquiring a network provided in an embodiment of the present application;

[0027] Figure 4 It is a schematic diagram of the interior of a block provided in an embodiment of the present application;

[0028] Figure 5 It is a schematic diagram of the connection relationship between different blocks provided in an embodiment of the present application;

[0029] Figure 6 It is a hardware structure diagram of the terminal device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to facilitate the understanding of the content of this solution, some of the terms involved in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art. This part is only for ease of understanding and cannot be regarded as a specific limitation of the present application.

[0031] 1. Neural Networks

[0032] A neural network can be composed of neural units. A neural unit can be a s The output of the operation unit with the intercept 1 as input can be:

[0033] Where, s = 1, 2, ..., n, n is a natural number greater than 1, W s For X s , b is the bias of the neural unit. f() is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0034] 2. Deep neural network (DNN)

[0035] DNN, also known as multi-layer neural network, can be understood as a neural network with multiple hidden layers. According to the position of different layers, DNN can be divided into three categories: input layer, hidden layer and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer.

[0036] Although DNN looks complicated, the work of each layer is not complicated. In simple terms, it can be expressed as the following linear relationship: in, is the input vector, is the output vector, is the offset vector, w is the weight matrix (also called coefficient), and α() is the activation function. Each layer performs such a simple operation on the input vector to obtain the output vector. Due to the large number of DNN layers, the coefficient w and the offset vector The number of these parameters is also relatively large. The definitions of these parameters in DNN are as follows: Taking the coefficient w as an example, assuming that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of coefficient w, while the subscripts correspond to the third layer index 2 of the output and the second layer index 4 of the input.

[0037] In summary, the coefficients from the Kth neuron in the L-1th layer to the jth neuron in the Lth layer are defined as

[0038] It should be pointed out that the input layer does not have a w parameter. In a deep neural network, more hidden layers allow the network to better describe complex situations in the real world. Theoretically, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Training a deep neural network is the process of learning the weight matrix, and its ultimate goal is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by many layers of vectors w).

[0039] 3. Convolutional neural network (CNN)

[0040] CNN is a deep neural network with a convolutional structure. Convolutional neural networks contain a feature extractor consisting of a convolutional layer and a subsampling layer, which can be regarded as a filter. A convolutional layer refers to a neuron layer in a convolutional neural network that performs convolution processing on the input signal. In a convolutional layer of a convolutional neural network, a neuron can only be connected to some neurons in the adjacent layers. A convolutional layer usually contains several feature planes, each of which can be composed of some rectangularly arranged neural units. The neural units in the same feature plane share weights, and the shared weights here are convolution kernels. Shared weights can be understood as the way to extract image information is independent of position. The convolution kernel can be initialized in the form of a matrix of random size, and the convolution kernel can obtain reasonable weights through learning during the training process of the convolutional neural network. In addition, the direct benefit of shared weights is to reduce the connection between the layers of the convolutional neural network, while reducing the risk of overfitting.

[0041] 4. Loss Function

[0042] In the process of training DNN, because we hope that the output of DNN is as close as possible to the value we really want to predict, we can compare the predicted value of the current network with the target value we really want, and then update the weight vector of each layer of the neural network according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in DNN). For example, if the predicted value of the network is high, adjust the weight vector to make it predict lower, and keep adjusting until DNN can predict the target value we really want or a value very close to the target value we really want. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function, which are important equations used to measure the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training of DNN becomes a process of minimizing this loss as much as possible.

[0043] 5. Back Propagation Algorithm

[0044] Neural networks can use the error back propagation (BP) algorithm to correct the size of the initial neural network parameters during the training process, so that the reconstruction error loss of the neural network becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the error loss information is back-propagated to update the initial neural network parameters, so that the error loss converges. The back propagation algorithm is a back propagation movement dominated by error loss, aiming to obtain the optimal neural network parameters, such as the weight matrix.

[0045] Among them, the neural network in the embodiments of the present application may refer to DNN or CNN.

[0046] Generally speaking, neural networks can be trained on servers. The network structure of trained neural networks is relatively complex, and the parameters of neural networks are relatively large. Therefore, when running neural networks, power consumption is high and latency is also high. This results in the terminal device being unable to support the operation of the neural network, so such a neural network cannot be directly deployed on the terminal device. Therefore, how to reduce the power consumption and latency of the neural network has become a technical problem that needs to be solved urgently at this stage. Based on this, the present application provides a network acquisition method, which can be applied to Figure 1 The system architecture is shown in Figure 1. Figure 1 As shown, it may include a server 10 and a terminal device 20, specifically:

[0047] The server 10 has data processing capabilities and data transceiving capabilities. The data processing here may include at least one of the following: data training, machine learning, deep learning, search, reasoning, decision-making, etc. Among them, machine learning and deep learning can symbolize and formalize intelligent information modeling, extraction, preprocessing, training, etc. for data. Reasoning refers to the process of simulating human intelligent reasoning methods in a computer or intelligent system, using formalized information for machine thinking and problem solving based on reasoning control strategies, and the typical functions are search and matching. Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction. Optionally, the server mentioned in this application can be a server or a server cluster composed of multiple servers. The server cluster can include cloud computing servers, content delivery network (Content Delivery Network, CDN) servers, network time protocol (Network Time Protocol, NTP), domain name resolution system (Domain Name System, DNS) servers, etc., and the various servers can coordinate with each other to jointly complete functions such as computing, data storage, and communication. For the convenience of description, a single server and a server cluster are collectively referred to as a server in the embodiment of this application.

[0048] The terminal device 20 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the terminal device.

[0049] In some embodiments, the server 10 and the terminal device 20 are connected via a communication link, which may be various types of media, specifically a wired link, a wireless link, or a combination of a wired link and a wireless link, etc. The wired link is, for example, an Ethernet-based bus. The wireless link is, for example, a short-distance connection technology, including 802.11b / g, etc., and may also be a long-distance connection technology, including long-term evolution (LTE) communication technology, fifth-generation mobile communication technology (5th generation mobile networks or 5th generation wireless systems, 5th-Generation, referred to as 5G or 5G technology), etc. Of course, it is not excluded that other technologies can be used to support communication between the server 10 and the terminal device 20.

[0050] It can be understood that in the network acquisition method provided in the present application, the server 10 can construct a super network based on the basic network, and determine the first sub network from at least one sub network included in the super network. In this way, the server 10 can construct a second sub network based on the first sub network and the initial network corresponding to the basic network, so that the second sub network can be adjusted according to the loss value of the first sub network, the loss value of the second sub network and the loss value of the initial network to obtain the target network corresponding to the basic network. Finally, the server 10 can also send the parameters of the target network to the terminal device 20. After the terminal device 20 obtains the parameters of the target network, it can run the target network according to the parameters of the target network.

[0051] The following specifically introduces the specific process of the terminal device 20 implementing the network acquisition method provided in this application.

[0052] first, Figure 2 The software architecture of the terminal device 20 is exemplarily shown.

[0053] The software system of the terminal device 20 can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the terminal device 20.

[0054] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, the hardware abstraction layer (HAL), and the kernel layer.

[0055] The application layer can include a series of application packages. Figure 2 As shown, the application package may include applications such as camera, gallery, video, music, navigation, calendar, map, and wireless local area networks (WLAN).

[0056] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0057] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0058] The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build the display interface of an application. The display interface can be composed of one or more views. For example, the display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0059] The phone manager is used to provide communication functions for the terminal device 20, such as management of call status (including connection, disconnection, etc.).

[0060] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0061] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is issued, the terminal device vibrates, the indicator light flashes, etc. Android Runtime includes a core library and a virtual machine. Android runtime is responsible for the scheduling and management of the Android system. The core library consists of two parts: one is the function function that needs to be called by the Java language, and the other is the Android core library.

[0062] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0063] The system library can include multiple functional modules. For example: surface manager, media library, 3D graphics processing library (such as OpenGL or OpenGL ES, etc.), 2D graphics engine (such as SGL), etc. The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0064] The Hardware Abstraction Layer (HAL) provides a standard interface that can be used to control the actions of the hardware. The HAL includes multiple library modules, each of which implements a set of interfaces for a specific type of hardware component, such as a WLAN module, a Bluetooth module, etc.

[0065] In some embodiments, the WLAN module can obtain images from a server. The images here may include multiple images of at least one user. Each user corresponds to a specified number of images (for example, 3, 8, etc.), and each image may include, for example, a facial image of the corresponding user. Optionally, the number of images corresponding to each user may also be different. This application does not limit the type of image. In addition, the WLAN module can also receive parameters of a target network from a server so that the target network can be deployed in the HAL.

[0066] In some embodiments, the HAL may also include a camera hardware abstraction layer (Camera HAL), etc. The camera hardware abstraction layer may process the image acquired by the WLAN module (such as image quality enhancement processing, etc.), so that the processed image can be used as input data of the target network. Among them, image quality enhancement refers to processing one or more of the brightness, color, contrast, saturation, dynamic range, etc. of the image. Image quality enhancement can also be called image enhancement, which is equivalent to improving the quality of the image and making the image clearer.

[0067] The kernel layer is the foundation of the Android system. For example, ART relies on the kernel layer to perform low-level functions such as threads and low-level memory management. The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, sensor drivers, GPU drivers, etc.

[0068] Combine the following Figure 2 The software architecture of the terminal device 20 shown in FIG. 1 is used to describe the embodiment of the present application in detail. Figure 3 As shown, a network acquisition method provided in an embodiment of the present application includes but is not limited to the following steps:

[0069] 301. The server 10 constructs a super network based on a basic network. The super network includes at least one sub network. Each sub network in the at least one sub network includes at least one block.

[0070] Among them, the performance parameters (including latency and / or power consumption) of the basic network running on the terminal device do not meet the preset performance requirements, and the basic network is a trained network. The basic network can be used for depth estimation, target detection, target recognition, semantic segmentation or intent recognition, etc. Depth estimation can be divided into monocular depth estimation or binocular depth estimation, etc. The specific task of depth estimation refers to predicting the depth of each pixel in an image. Among them, the picture composed of the depth value of each pixel is also called a depth map. The depth value is the distance (depth) from the camera to each point in the scene. Generally speaking, the difference between monocular depth estimation and binocular depth estimation is that the image used for monocular depth estimation is an image of a single perspective, and the image used for binocular depth estimation is an image of the left and right viewpoints in the same scene taken by a binocular camera. In some embodiments, when the basic network is used for monocular depth estimation, the basic network can be called a monocular depth estimation network. When the basic network is used for binocular depth estimation, the basic network can be called a binocular depth estimation network. Target detection refers to detecting a target object such as a specific person or object in an image, such as marking the area of ​​the target object in the image. In some embodiments, when the base network is used for target detection, the base network can be called a target detection network, such as a face detection network. Target recognition refers to identifying target objects such as specific people or objects in an image. In some embodiments, when the base network is used for target recognition, the base network can be called a target recognition network, such as a face recognition network. Semantic segmentation refers to dividing an image into regions with different semantic information and labeling each region with corresponding semantic labels. In some embodiments, when the base network is used for semantic segmentation, the base network can be called a semantic segmentation network. Intent recognition refers to identifying intent contained in text (such as retrieval, etc.). In some embodiments, when the base network is used for intent recognition, the base network can be called an intent recognition network.

[0071] It should be pointed out that a certain network mentioned in this application (such as a basic network) is a trained network, which can be understood as: the accuracy of the network (also known as the accuracy) meets the preset accuracy, and / or the network meets the training convergence conditions, which may include, for example, the loss value of the network converges to a preset loss value, etc.

[0072] In some embodiments, step 302 may include the following steps (1) and (2):

[0073] Step (1) The server 10 establishes a search space according to the basic network, where the search space is used to indicate a range of changes in the network structure of the basic network.

[0074] Among them, the search space is used to indicate the range of changes in the network structure of the basic network, which can be understood as at least one of the following: the range of changes in the number of blocks in the basic network, the range of changes in the number of nodes in each block, the operations on each node in the block and the ratio of changes in the number of channels, and the change of the connection relationship between each block. Generally speaking, a block contains one or more nodes and operations (OP) on each node. The operation refers to some basic operation units of the neural network, such as convolution, pooling and other operations. The nodes here can be understood as the layers of the basic network, such as input layer, output layer, convolution layer, pooling layer, fully connected layer, etc.

[0075] To facilitate understanding, the following example illustrates a block and its internal operation relationships:

[0076] like Figure 4 As shown, Figure 4 The diagram shows a block and the internal operation relationship of the block. Each block can set the number of nodes, operations on each node, and changes in the number of channels. Figure 4 , which shows 5 nodes, including input nodes and output nodes (also called input layer and output layer), C represents the number of input channels, and the number of channels of the three middle nodes represented by (0.25~4)×C can vary according to the ratio of 0.25~4 of C. The variation ratio interval can be set by yourself, and the number of channels of each middle node can also be different. It should be pointed out that the number of channels of the input node and the output node is the same, and there is a jump connection (i.e. the direction of data flow) between the two by default. If the resolution of the input layer and the output layer is inconsistent, an operator can be inserted in the middle. When the output results of different nodes are merged, they can be directly added (add) or the channels can be merged (concat). The operator here can refer to NxN convolution, etc., and the size of N is not limited. For example, N can be an integer greater than 1.

[0077] In some embodiments, the number of nodes included in a block can range from 1 to 5, each node can include 7 different operations, and the number of channels generally varies in 5 ways. For example, it can be 7 operations and 5 changes in the number of channels as shown in Table 1 (characterized by the ratio of the change in the number of channels). Among them, conv3×3 in Table 1 means that the kernel size of the current operation (such as the convolution kernel size, etc.) is 3×3, and the rest are similar; the group in Table 1 means grouping the number of input channels and the number of output channels of the current operation; c in Table 1 represents the number of input channels of the current operation. It should be noted that Table 1 is only an illustration of the changes in the number of operations and channels. In specific applications, the changes in the number of operations and channels on the nodes can also be in other forms, which are not limited here.

[0078] Table 1: 7 operations and 5 channel number changes

[0079]

[0080] In some embodiments, the number of channels in different blocks may be the same or different, which is not limited here.

[0081] To facilitate understanding, the following example illustrates the connection relationship between blocks. Specifically:

[0082] like Figure 5 As shown, Figure 5 It illustrates the connection relationship (also called stacking relationship) between multiple identical or different blocks. The combined structure formed by connecting the blocks can be a sub-network in the super-network mentioned in the embodiment of the present application. Figure 5 The diagram shows a stacked structure of 4432, that is, in the stacked sub-network, the first stage (i.e., stage 1) includes 4 blocks with c channels each; the second stage (i.e., stage 2) includes 4 blocks, of which 2 blocks have c channels and the other 2 blocks have 2c channels; the third stage (i.e., stage 3) includes 3 blocks with 2c channels; the fourth stage (i.e., stage 4) includes 2 blocks with 4c channels. It should be noted that the stacked sub-network can include multiple stages, such as, Figure 5 There are four stages in , each of which can include blocks with the same or different internal structures, such as Figure 5 Phase 1 in the example consists of four blocks with the same internal structure. Figure 5 The four blocks of stage 2 in the example include two blocks with different internal structures. In the embodiment of the present application, the number of stages included in the stacked subnetwork and the type and number of channels of the blocks included in each stage are all set within the range indicated by the search space and are not limited here. Figure 5 In the figure, it is shown that each stage may include 1 to 10 identical or different blocks.

[0083] Step (2) The server 10 constructs a super network according to the search space.

[0084] In some implementations, step (2) may include: the server 10 may randomly sample from the search space to obtain multiple initial sub-networks; the server 10 may fuse the multiple initial sub-networks and train the fused network using the training sample set to obtain a super network. This application does not limit the specific training process.

[0085] In some embodiments, any sub-network in the super-network can have the same function as the base network, such as the base network is used for monocular depth estimation, and any sub-network in the super-network is also used for monocular depth estimation. In some embodiments, the accuracy of different sub-networks in the super-network can be the same or different.

[0086] 302. The server 10 determines a first subnetwork from at least one subnetwork, wherein the spatial degree of freedom of the first subnetwork satisfies preset spatial degree of freedom information and the number of blocks in the first subnetwork is the same as the number of blocks in the initial network corresponding to the basic network.

[0087] It should be noted that the spatial degrees of freedom of a certain subnetwork (such as the first subnetwork, etc.) mentioned in this application may include at least one of the following: the depth of the convolution kernel in the subnetwork, the width of the convolution kernel in the subnetwork, the depth of the subnetwork, and the width of the subnetwork. Generally speaking, the convolution kernel can be regarded as a weight (or weight) matrix, the depth of the convolution kernel refers to the number of rows of the weight matrix, and the width of the convolution kernel refers to the number of columns of the weight matrix. The depth of the subnetwork refers to the number of blocks of the subnetwork. The width of the subnetwork may refer to the number of channels of the block. For example, the preset spatial degrees of freedom include the preset network depth (7) and the preset network width (9), and the super network includes 4 subnetworks, namely, subnetwork 1 to subnetwork 4, the depths of subnetwork 1 to subnetwork 4 are 5, 6, 7 and 8 respectively, and the widths of subnetwork 1 to subnetwork 4 are 3, 5, 7 and 9 respectively. The depths of subnetwork 3 and subnetwork 4 meet the preset network depths of the subnetworks, and the width of subnetwork 4 meets the preset network width, that is, it is subnetwork 4 that meets the preset spatial degrees of freedom. Therefore, subnetwork 4 can be used as the first subnetwork.

[0088] In some embodiments, when the first subnetwork is used for depth estimation, the input data of the first subnetwork is an image (the image can be an image of a single perspective or any one of the images of the left and right viewpoints in the same scene taken by a binocular camera), and the output data of the first subnetwork is a depth map obtained by depth estimation based on the input data of the first subnetwork, and the ratio between the pixels with incorrect estimation in the depth map and all the pixels in the depth map (the ratio can be understood as a bad indicator) is less than or equal to a preset threshold.

[0089] In some embodiments, performance parameters (including latency and / or power consumption) of the first sub-network running on the terminal device meet preset performance requirements.

[0090] Among them, the above-mentioned initial network can be understood as a network to be trained, that is, the basic network can be obtained after the initial network is trained using the training sample set. The training sample set mentioned in this application can refer to at least one image, for example. For example, the images in the training sample set are all images of a single perspective; for another example, the images in the training sample set are all images of the left and right viewpoints of the same scene taken by a binocular camera; for another example, the images in the training sample set are all facial images of at least one user, etc.

[0091] 303. The server 10 constructs a second subnetwork based on the first subnetwork and the initial network. The number of blocks in the second subnetwork is the same as the number of blocks in the first subnetwork. The network structure of the second subnetwork is different from the network structure of the first subnetwork and the network structure of the initial network.

[0092] In some embodiments, the number of channels of the same block in the first sub-network and the initial network is different, that is, the number of channels of a block in the first sub-network is greater than or equal to the number of channels of the corresponding block in the initial network.

[0093] In some embodiments, the network structure of the second subnetwork is different from the network structure of the first subnetwork and the network structure of the initial network, including: the number of channels of the same block in the second subnetwork, the first subnetwork and the initial network are different. Exemplarily, the second subnetwork, the first subnetwork and the initial network all include three blocks, namely block1 to block3, the number of channels of block1 in the second subnetwork is greater than the number of channels of block1 in the initial network, and the number of channels of block1 in the second subnetwork is less than the number of channels of block1 in the first subnetwork; the number of channels of block2 in the second subnetwork is greater than the number of channels of block2 in the initial network, and the number of channels of block2 in the second subnetwork is less than the number of channels of block2 in the first subnetwork; the number of channels of block3 in the second subnetwork is greater than the number of channels of block3 in the initial network, and the number of channels of block3 in the second subnetwork is less than the number of channels of block3 in the first subnetwork.

[0094] In other embodiments, the network structure of the second subnetwork is different from the network structure of the first subnetwork and the network structure of the initial network, including: the number of channels of at least one block in the second subnetwork is different from the number of channels of the corresponding block in the first subnetwork and the initial network. Exemplarily, the second subnetwork, the first subnetwork and the initial network all include three blocks, namely block1 to block3, the number of channels of block1 in the second subnetwork is greater than the number of channels of block1 in the initial network, and the number of channels of block1 in the second subnetwork is less than the number of channels of block1 in the first subnetwork; the number of channels of block2 in the second subnetwork is equal to the number of channels of block2 in the initial network; the number of channels of block3 in the second subnetwork is equal to the number of channels of block3 in the first subnetwork.

[0095] The number of the second subnetworks may be one or more. The method for determining the number of channels of a block in each of the plurality of second subnetworks is similar to the above, and is not repeated here. In some embodiments, the number of channels of at least one block in the plurality of second subnetworks is different. Exemplarily, the multiple second subnetworks include second subnetworks 1 to second subnetworks 3, and the second subnetworks 1 to second subnetworks 3, the first subnetwork and the initial network all include three blocks, namely block1 to block3, the number of channels of block1 in second subnetwork 1 is 100, the number of channels of block1 in second subnetwork 2 is 112, the number of channels of block1 in second subnetwork 3 is 89, the number of channels of block1 in the first subnetwork is 128, and the number of channels of block1 in the initial network is 64, that is, the number of channels of block1 of any second subnetwork from second subnetwork 1 to second subnetwork 3 is greater than the number of channels of block1 in the initial network, and less than the number of channels of block1 in the first subnetwork; the number of channels of block2 in second subnetworks 1 to second subnetworks 3 is equal to the number of channels of block2 in the initial network; the number of channels of block3 in second subnetworks 1 to second subnetworks 3 is equal to the number of channels of block3 in the first subnetwork.

[0096] 304. The server 10 adjusts the second sub-network according to the loss value of the first sub-network, the loss value of the second sub-network, and the loss value of the initial network to obtain a target network corresponding to the basic network.

[0097] In some embodiments, step 304 may include: the server 10 calculates the total loss value according to the loss value of the first sub-network, the loss value of the second sub-network and the loss value of the initial network; the server 10 adjusts the second sub-network according to the total loss value to obtain the target network corresponding to the basic network. It should be noted that the loss value of the first sub-network, the loss value of the second sub-network and the loss value of the initial network are obtained by training the first sub-network, the second sub-network and the initial network respectively according to the same training sample set, and this application does not limit the specific process.

[0098] Among them, the performance parameters (including latency and / or power consumption) of the target network running on the terminal device meet the preset performance requirements.

[0099] In some embodiments, after step 304 and before step 305, the method further includes: the server 10 updates the parameters of the target network using the training sample set, which can further improve the accuracy of the target network.

[0100] Optionally, the method may further include step 305 and step 306.

[0101] 305 . The server 10 sends the parameters of the target network to the terminal device 20 .

[0102] Accordingly, the terminal device receives the parameters of the target network from the server.

[0103] The parameters of the target network may include at least one of the following: the number of blocks in the target network, the number of nodes in each block, the operations on each node in the block and the ratio of the number of channels, the connection relationship between each block, the size of the convolution kernel, the number of convolution kernels, etc.

[0104] In some embodiments, before the server 10 sends the parameters of the target network to the terminal device 20, the method further includes: the server 10 receives indication information from the terminal device 20, where the indication information is used to request the parameters of the target network.

[0105] 306. The terminal device 20 operates the target network according to the parameters of the target network.

[0106] It can be seen from the above process that by implementing a network acquisition method provided by the present application, the server 10 combines the loss values ​​of multiple networks related to the basic network to adjust the second sub-network, which can not only reduce the power consumption and delay of the target network, but also improve the accuracy of the target network. In this way, when the server 10 sends the parameters of the target network to the terminal device 20, the terminal device 20 can successfully operate the target network according to the parameters of the target network, that is, the terminal device can support the operation of the target network.

[0107] Figure 6 The hardware structure diagram of the terminal device 20 is exemplarily shown.

[0108] The terminal device 20 may include a processor 21, an external memory interface 22, an internal memory 221, a universal serial bus (USB) interface 23, a charging management module 24, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 25, a wireless communication module 26, an audio module 27, a speaker 27A, a receiver 27B, a microphone 27C, an earphone interface 27D, a sensor module 28, a button 29, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc. The sensor module 28 may include a pressure sensor 28A, a gyroscope sensor 28B, an air pressure sensor 28C, a magnetic sensor 28D, an acceleration sensor 28E, a distance sensor 28F, a proximity light sensor 28G, a fingerprint sensor 28H, a temperature sensor 28J, a touch sensor 28K, an ambient light sensor 28L, a bone conduction sensor 28M, etc.

[0109] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal device 20. In other embodiments of the present application, the terminal device 20 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0110] The processor 21 may include one or more processing units, for example, the processor 21 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0111] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0112] The processor 21 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 21 is a cache memory. The memory may store instructions or data that the processor 21 has just used or cyclically used. If the processor 21 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 21, and thus improves the efficiency of the system.

[0113] In the embodiment of the present application, the memory may be used to store parameters of the target network so that the processor 21 operates the target network based on the parameters of the target network.

[0114] In some embodiments, the processor 21 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0115] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 21 may include multiple groups of I2C buses. The processor 21 may be coupled to the touch sensor 28K, the charger, the flash, the camera 293, etc. through different I2C bus interfaces. For example, the processor 21 may be coupled to the touch sensor 28K through the I2C interface, so that the processor 21 communicates with the touch sensor 28K through the I2C bus interface to realize the touch function of the terminal device 20.

[0116] The I2S interface can be used for audio communication. In some embodiments, the processor 21 can include multiple I2S buses. The processor 21 can be coupled to the audio module 27 via the I2S bus to achieve communication between the processor 21 and the audio module 27. In some embodiments, the audio module 27 can transmit an audio signal to the wireless communication module 26 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

[0117] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 27 and the wireless communication module 26 can be coupled via a PCM bus interface. In some embodiments, the audio module 27 can also transmit audio signals to the wireless communication module 26 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0118] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 21 and the wireless communication module 26. For example, the processor 21 communicates with the Bluetooth module in the wireless communication module 26 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 27 can transmit an audio signal to the wireless communication module 26 through the UART interface to realize the function of playing music through a Bluetooth headset.

[0119] The MIPI interface can be used to connect the processor 21 with peripheral devices such as the display screen 294 and the camera 293. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 21 and the camera 293 communicate via the CSI interface to realize the shooting function of the terminal device 20. The processor 21 and the display screen 294 communicate via the DSI interface to realize the display function of the terminal device 20.

[0120] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 21 with the camera 293, the display screen 294, the wireless communication module 26, the audio module 27, the sensor module 28, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0121] The USB interface 23 is an interface that complies with the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 23 can be used to connect a charger to charge the terminal device 20, and can also be used to transmit data between the terminal device 20 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0122] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the terminal device 20. In other embodiments of the present application, the terminal device 20 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0123] The charging management module 24 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 24 may receive charging input from a wired charger through the USB interface 23. In some wireless charging embodiments, the charging management module 24 may receive wireless charging input through a wireless charging coil of the terminal device 20. While the charging management module 24 is charging the battery 242, it may also power the electronic device through the power management module 241.

[0124] The power management module 241 is used to connect the battery 242, the charging management module 24 and the processor 21. The power management module 241 receives input from the battery 242 and / or the charging management module 24, and supplies power to the processor 21, the internal memory 221, the display screen 294, the camera 293, and the wireless communication module 26. The power management module 241 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 241 can also be set in the processor 21. In other embodiments, the power management module 241 and the charging management module 24 can also be set in the same device.

[0125] The wireless communication function of the terminal device 20 can be implemented through the antenna 1, the antenna 2, the mobile communication module 25, the wireless communication module 26, the modem processor and the baseband processor.

[0126] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 20 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0127] The mobile communication module 25 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the terminal device 20. The mobile communication module 25 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 25 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 25 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 25 can be set in the processor 21. In some embodiments, at least some of the functional modules of the mobile communication module 25 can be set in the same device as at least some of the modules of the processor 21.

[0128] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 27A, a receiver 27B, etc.), or displays an image or video through a display screen 294. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 21 and be set in the same device as the mobile communication module 25 or other functional modules.

[0129] The wireless communication module 26 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the terminal device 20. The wireless communication module 26 can be one or more devices integrating at least one communication processing module. The wireless communication module 26 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 21. The wireless communication module 26 can also receive the signal to be sent from the processor 21, modulate the frequency of the signal, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0130] In the embodiment of the present application, the parameters of the target network may be obtained by the terminal device 20 from the server through the wireless communication function.

[0131] In some embodiments, the antenna 1 of the terminal device 20 is coupled to the mobile communication module 25, and the antenna 2 is coupled to the wireless communication module 26, so that the terminal device 20 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0132] The terminal device 20 implements the display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 21 may include one or more GPUs, which execute program instructions to generate or change display information.

[0133] The display screen 294 is used to display images, videos, etc. The display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 20 may include 1 or N display screens 294, where N is a positive integer greater than 1.

[0134] The terminal device 20 can realize the shooting function through ISP, camera 293, video codec, GPU, display screen 294 and application processor.

[0135] The ISP is used to process the data fed back by the camera 293. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise and brightness of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 293.

[0136] The camera 293 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 20 may include 1 or N cameras 293, where N is a positive integer greater than 1. For example, two cameras can capture images of the left and right viewpoints in the same scene.

[0137] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 20 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0138] Video codecs are used to compress or decompress digital video.

[0139] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal device 20 can be realized, such as depth estimation, target detection, target recognition, semantic segmentation or intent recognition, etc.

[0140] In an embodiment of the present application, the NPU may run the target network according to the parameters of the target network. For example, when the target network is used for binocular depth estimation, the NPU may obtain an image corresponding to the user through the camera 293 and input the image into the target network to estimate a depth map corresponding to the image.

[0141] The internal memory 221 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0142] Random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc. Non-volatile memory may include disk storage devices and flash memory.

[0143] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle; single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level; universal flash storage (UFS), embedded multi media card (eMMC), etc. according to the storage specification.

[0144] The random access memory can be directly read and written by the processor 21, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0145] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 21 .

[0146] In an embodiment of the present application, the internal memory 221 can support the terminal device 20 to apply for Surface, BufferQueue, etc. from the memory.

[0147] The external memory interface 22 can be used to connect an external non-volatile memory to expand the storage capacity of the terminal device 20. The external non-volatile memory communicates with the processor 21 through the external memory interface 22 to realize the data storage function. For example, music, video and other files are saved in the external non-volatile memory. In the embodiment of the present application, the terminal device 20 can collect sound through the microphone 27C when shooting HDR video. During the video playback process, the speaker 27A or the speaker connected to the headphone interface 27D can support the audio in the video.

[0148] The terminal device 20 can implement audio functions such as music playing and recording through the audio module 27, the speaker 27A, the receiver 27B, the microphone 27C, the earphone interface 27D, and the application processor.

[0149] The audio module 27 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 27 can also be used to encode and decode audio signals. In some embodiments, the audio module 27 can be arranged in the processor 21, or some functional modules of the audio module 27 can be arranged in the processor 21.

[0150] The speaker 27A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The terminal device 20 can listen to music or listen to a hands-free call through the speaker 27A.

[0151] The receiver 27B, also called a "handset", is used to convert audio electrical signals into sound signals. When the terminal device 20 receives a call or voice message, the voice can be received by placing the receiver 27B close to the ear.

[0152] Microphone 27C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by putting his mouth close to microphone 27C to input the sound signal into microphone 27C. The terminal device 20 can be provided with at least one microphone 27C. In other embodiments, the terminal device 20 can be provided with two microphones 27C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the terminal device 20 can also be provided with three, four or more microphones 27C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0153] The earphone interface 27D is used to connect a wired earphone and can be a USB interface 23, or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0154] The pressure sensor 28A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 28A can be set on the display screen 294. There are many types of pressure sensors 28A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 28A, the capacitance between the electrodes changes. The terminal device 20 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 294, the terminal device 20 detects the touch operation intensity according to the pressure sensor 28A. The terminal device 20 can also calculate the touch position according to the detection signal of the pressure sensor 28A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0155] The gyroscope sensor 28B can be used to determine the motion posture of the terminal device 20. In some embodiments, the angular velocity of the terminal device 20 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 28B. The gyroscope sensor 28B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 28B detects the angle of the terminal device 20 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the terminal device 20 through reverse movement to achieve anti-shake. The gyroscope sensor 28B can also be used for navigation and somatosensory game scenes.

[0156] The air pressure sensor 28C is used to measure air pressure. In some embodiments, the terminal device 20 calculates the altitude through the air pressure value measured by the air pressure sensor 28C to assist positioning and navigation.

[0157] The magnetic sensor 28D includes a Hall sensor. The terminal device 20 can use the magnetic sensor 28D to detect the opening and closing of the flip leather case. In some embodiments, when the terminal device 20 is a flip phone, the terminal device 20 can detect the opening and closing of the flip cover according to the magnetic sensor 28D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover automatic unlocking and other features are set.

[0158] The acceleration sensor 28E can detect the magnitude of the acceleration of the terminal device 20 in various directions (generally three axes). When the terminal device 20 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0159] The distance sensor 28F is used to measure the distance. The terminal device 20 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the terminal device 20 can use the distance sensor 28F to measure the distance to achieve fast focusing.

[0160] The proximity light sensor 28G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 20 emits infrared light outward through the light emitting diode. The terminal device 20 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 20. When insufficient reflected light is detected, the terminal device 20 can determine that there is no object near the terminal device 20. The terminal device 20 can use the proximity light sensor 28G to detect that the user holds the terminal device 20 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 28G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0161] The ambient light sensor 28L is used to sense the ambient light brightness. The terminal device 20 can adaptively adjust the brightness of the display screen 294 according to the perceived ambient light brightness. The ambient light sensor 28L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 28L can also cooperate with the proximity light sensor 28G to detect whether the terminal device 20 is in a pocket to prevent accidental touch.

[0162] The fingerprint sensor 28H is used to collect fingerprints. The terminal device 20 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0163] The temperature sensor 28J is used to detect temperature. In some embodiments, the terminal device 20 uses the temperature detected by the temperature sensor 28J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 28J exceeds a threshold, the terminal device 20 reduces the performance of the processor located near the temperature sensor 28J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the terminal device 20 heats the battery 242 to avoid abnormal shutdown of the terminal device 20 due to low temperature. In some other embodiments, when the temperature is lower than another threshold, the terminal device 20 performs a boost on the output voltage of the battery 242 to avoid abnormal shutdown caused by low temperature.

[0164] The touch sensor 28K is also called a "touch control device". The touch sensor 28K can be set on the display screen 294. The touch sensor 28K and the display screen 294 form a touch screen, also called a "touch control screen". The touch sensor 28K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 294. In other embodiments, the touch sensor 28K can also be set on the surface of the terminal device 20, which is different from the position of the display screen 294.

[0165] The bone conduction sensor 28M can obtain a vibration signal. In some embodiments, the bone conduction sensor 28M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 28M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 28M can also be set in an earphone to form a bone conduction earphone. The audio module 27 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 28M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 28M to realize a heart rate detection function.

[0166] The key 29 includes a power key, a volume key, etc. The key 29 can be a mechanical key or a touch key. The terminal device 20 can receive key input and generate key signal input related to user settings and function control of the terminal device 20.

[0167] Motor 291 can generate vibration prompts. Motor 291 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 294, motor 291 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0168] Indicator 292 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0169] The SIM card interface 295 is used to connect a SIM card. The SIM card can be connected to and separated from the terminal device 20 by inserting the SIM card interface 295 or pulling it out from the SIM card interface 295. The terminal device 20 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 295 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 295 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 295 can also be compatible with different types of SIM cards. The SIM card interface 295 can also be compatible with external memory cards. The terminal device 20 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the terminal device 20 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the terminal device 20 and cannot be separated from the terminal device 20.

[0170] The " / " used in the specification and the appended claims of this application indicates that the objects associated before and after are in an "or" relationship. For example, A / B can represent A or B; the "and / or" in this application is only a kind of association relationship describing the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, in the description of this application, unless otherwise specified, "multiple" means two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be one or more. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish the network elements and the same or similar items with basically the same functions. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the differences.

[0171] References to "one embodiment" or "some embodiments" etc. described in the embodiments of the present application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0172] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0173] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A method for acquiring a network, characterized in that: include: Building a super network based on a basic network, the super network includes at least one sub network, each sub network in the at least one sub network includes at least one block; the performance parameters of the basic network running on the terminal device do not meet the preset performance requirements; Determine a first subnetwork from the at least one subnetwork, wherein the spatial degree of freedom of the first subnetwork satisfies the preset spatial degree of freedom information, the number of blocks in the first subnetwork is the same as the number of blocks in the initial network corresponding to the basic network, the number of channels of the same block in the first subnetwork and the initial network is different, and the performance parameters of the first subnetwork running on the terminal device meet the preset performance requirements; The basic network is a network obtained by training the initial network using a training sample set; The spatial degrees of freedom of the first sub-network include: the depth of the convolution kernel in the first sub-network, the width of the convolution kernel in the first sub-network, the depth of the first sub-network, and the width of the first sub-network; Based on the first subnetwork and the initial network, construct a second subnetwork, the number of blocks in the second subnetwork is the same as the number of blocks in the first subnetwork, and the network structure of the second subnetwork is different from the network structure of the first subnetwork and the network structure of the initial network; According to the loss value of the first subnetwork, the loss value of the second subnetwork and the loss value of the initial network, the second subnetwork is adjusted to obtain a target network corresponding to the basic network, and the performance parameters of the target network when running on the terminal device meet the preset performance requirements; the loss value of the first subnetwork, the loss value of the second subnetwork and the loss value of the initial network are obtained by training based on the same training sample set.

2. The method according to claim 1, characterized in that The method further comprises: The parameters of the target network are sent to the terminal device, where the parameters of the target network are used by the terminal device to run the target network.

3. The method according to claim 2, characterized in that Before sending the parameters of the target network to the terminal device, the method further includes: Receive indication information from the terminal device, where the indication information is used to request parameters of the target network.

4. The method according to claim 1, characterized in that: The step of constructing a super network based on the basic network includes: Establishing a search space according to the basic network, wherein the search space is used to indicate a range of changes in the network structure of the basic network; The super network is constructed according to the search space.

5. The method according to claim 1, characterized in that The input data of the first subnetwork is an image, and the output data of the first subnetwork is a depth map obtained by depth estimation based on the input data of the first subnetwork, and the ratio between the estimated incorrect pixels in the depth map and all the pixels in the depth map is less than or equal to a preset threshold.

6. The method according to claim 1, characterized in that The network structure of the second sub-network is different from the network structure of the first sub-network and the network structure of the initial network, including: The number of channels of a same block in the second sub-network, the first sub-network and the initial network are all different.

7. A method for acquiring a network, characterized in that: include: receiving parameters of the target network from the server; Running the target network according to the parameters of the target network; The performance parameters of the target network running on the terminal device meet the preset performance requirements; The target network is obtained by adjusting the second sub-network according to the loss value of the first sub-network, the loss value of the second sub-network and the loss value of the initial network corresponding to the basic network, and the loss value of the first sub-network, the loss value of the second sub-network and the loss value of the initial network are obtained by training based on the same training sample set; the performance parameters of the basic network running on the terminal device do not meet the preset performance requirements, and the basic network is a network obtained by training the initial network using the training sample set; the first sub-network is determined from at least one sub-network included in the super network, the spatial degree of freedom of the first sub-network satisfies the preset spatial degree of freedom information and the number of blocks in the first sub-network The number of blocks is the same as that in the initial network, the number of channels in the same block in the first sub-network and the initial network is different, and the performance parameters of the first sub-network running on the terminal device meet the preset performance requirements, the second sub-network is constructed according to the first sub-network and the initial network, the number of blocks in the second sub-network is the same as the number of blocks in the first sub-network, and the network structure of the second sub-network is different from the network structure of the first sub-network and the network structure of the initial network; the spatial degrees of freedom of the first sub-network include: the depth of the convolution kernel in the first sub-network, the width of the convolution kernel in the first sub-network, the depth of the first sub-network, and the width of the first sub-network.

8. The method according to claim 7, characterized in that Before receiving the parameters of the target network from the server, the method further includes: Sending indication information to a server, where the indication information is used to request parameters of the target network.

9. The method according to claim 8, characterized in that The input data of the first subnetwork is an image, and the output data of the first subnetwork is a depth map obtained by depth estimation based on the input data of the first subnetwork, and the ratio between the estimated incorrect pixels in the depth map and all the pixels in the depth map is less than or equal to a preset threshold.

10. The method according to claim 7, characterized in that The network structure of the second sub-network is different from the network structure of the first sub-network and the network structure of the initial network, including: The number of channels of a same block in the second sub-network, the first sub-network and the initial network are all different.

11. A server, characterized in that: The method comprises one or more processors and one or more memories; wherein the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, wherein the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the method according to any one of claims 1 to 6 is executed.

12. A terminal device, characterized in that: It comprises one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the method as described in any one of claims 7 to 10 is executed.

13. A computer program product comprising instructions, characterized in that When the computer program product runs on a server, the server is caused to execute the method according to any one of claims 1 to 6.

14. A computer program product comprising instructions, characterized in that When the computer program product runs on a terminal device, the terminal device is caused to execute the method according to any one of claims 7 to 10.

15. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on the server, the method according to any one of claims 1 to 6 is executed.

16. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on the terminal device, the method according to any one of claims 7 to 10 is executed.

Images