Optimization methods and apparatus for neural networks, data processing methods, equipment and media
By optimizing the neural network through filtering and splitting convolutional layers, the problem of excessive memory consumption was solved, and the operating efficiency and data processing capabilities of terminal devices were improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PING AN TECH (SHENZHEN) CO LTD
- Filing Date
- 2023-05-31
- Publication Date
- 2026-05-26
Smart Images
Figure CN116596039B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of financial technology, and in particular to a method and apparatus for optimizing neural networks, a data processing method, equipment and medium. Background Technology
[0002] With the development of science and technology, acquiring data and using data modeling to solve corresponding problems has become a common technical approach. For example, in the Internet of Things (IoT) within supply chain finance, neural networks are run on terminal devices to improve operational efficiency, enabling functions such as report data processing and monitoring alerts.
[0003] In related technologies, neural networks are deployed on the pre-set memory of terminal devices. During network operation, this pre-set memory requires a dedicated space to store the data matrix input to the neural network and the data matrix output by the convolutional neural network. Due to the massive amounts of data involved in the financial sector, the network size and data matrix sizes are constantly increasing, leading to a significant increase in the memory space occupied by the data matrix and the memory consumption of the pre-set memory. Therefore, there is an urgent need to optimize the memory usage of convolutional neural networks. Summary of the Invention
[0004] The main objective of this application is to propose an optimization method and apparatus for neural networks, a data processing method, a device and a medium, which aims to save the memory space occupied by neural networks running in preset memory, improve the efficiency of neural networks used in the Internet of Things, and thus improve the data processing efficiency in the financial field.
[0005] To achieve the above objectives, a first aspect of this application proposes a method for optimizing a neural network, the method comprising:
[0006] Obtain memory usage information of the original neural network running in a preset memory; wherein, the original neural network includes at least two candidate convolutional layers, and the memory usage information includes the input memory value and output memory value of each candidate convolutional layer;
[0007] Selected convolutional layers are selected from the candidate convolutional layers based on the input memory value and the output memory value;
[0008] Split layers are selected based on the selected convolutional layer and the candidate convolutional layers to obtain layer splitting parameters; wherein, the layer splitting parameters include the number of splits and the splitting number of each candidate convolutional layer;
[0009] The candidate convolutional layer is split according to the number of splits and the number of splits to obtain a candidate split convolutional layer;
[0010] Obtain memory optimization information for each of the candidate split convolutional layers;
[0011] The target split convolutional layer is selected from the candidate split convolutional layers based on the memory optimization information;
[0012] The target split convolutional layer and the candidate convolutional layer are concatenated to obtain the target neural network.
[0013] In some embodiments, the step of splitting the candidate convolutional layer according to the number of splits and the number of splits to obtain a candidate split convolutional layer includes:
[0014] Selected convolutional layers are selected from the candidate convolutional layers based on the split number;
[0015] The selected convolutional layer is copied according to the number of splits to obtain an updated convolutional layer;
[0016] The updated convolutional layer and the selected convolutional layer are concatenated to obtain the candidate split convolutional layer.
[0017] In some embodiments, concatenating the target split convolutional layer and the candidate convolutional layer to obtain the target neural network includes:
[0018] Based on the target split convolutional layer, a fused convolutional layer is selected from the candidate convolutional layers;
[0019] The fused convolutional layer and the target split convolutional layer are concatenated to obtain the target neural network.
[0020] In some embodiments, selecting a chosen convolutional layer from the candidate convolutional layers based on the input memory value and the output memory value includes:
[0021] The memory usage of the input memory value and the output memory value is calculated to obtain the memory usage value of each candidate convolutional layer.
[0022] Obtain the maximum value of the occupied memory to get the maximum memory value;
[0023] The candidate convolutional layer with the largest memory value is selected as the chosen convolutional layer.
[0024] To achieve the above objectives, a second aspect of this application provides a data processing method, the method comprising:
[0025] Obtain the input data matrix to be processed;
[0026] The input data matrix is split to obtain a split data matrix;
[0027] The split data matrix is input into the target neural network for convolution processing to obtain the target data; wherein the target neural network is obtained by the neural network optimization method described in the first aspect.
[0028] In some embodiments, the target neural network includes: a target splitting convolutional layer and a fusion convolutional layer; the step of inputting the split data matrix into the target neural network for convolution processing to obtain target data includes:
[0029] The split data matrix is input into the target split convolutional layer for convolution processing to obtain the activation data matrix;
[0030] Each of the activation data matrices is merged to obtain the target data matrix;
[0031] The target data matrix is input into the fusion convolutional layer for convolution processing to obtain the target data.
[0032] In some embodiments, the step of inputting the target data matrix into the fusion convolutional layer for convolution processing to obtain the target data includes:
[0033] Obtain the receptive field data of the target data matrix;
[0034] Each receptive field data is input into the fusion convolutional layer for convolution processing to obtain preliminary data;
[0035] Each of the preliminary data is collected and processed to obtain the target data.
[0036] To achieve the above objectives, a second aspect of this application provides an optimization apparatus for a neural network, the apparatus comprising:
[0037] A memory acquisition module is used to acquire memory usage information of the original neural network running in a preset memory; wherein the original neural network includes at least two candidate convolutional layers, and the memory usage information includes the input memory value and output memory value of each candidate convolutional layer;
[0038] A selection filtering module is used to filter out selected convolutional layers from the candidate convolutional layers based on the input memory value and the output memory value;
[0039] A split layer selection module is used to select a split layer based on the selected convolutional layer and the candidate convolutional layers to obtain layer split parameters; wherein, the layer split parameters include the number of splits and the split number of each candidate convolutional layer;
[0040] A layer splitting module is used to perform layer splitting processing on the candidate convolutional layer according to the number of splits and the number of splits, so as to obtain a candidate split convolutional layer;
[0041] An optimization acquisition module is used to acquire memory optimization information for each of the candidate split convolutional layers.
[0042] The target selection module is used to select a target split convolutional layer from the candidate split convolutional layers based on the memory optimization information.
[0043] The splicing module is used to splice the target split convolutional layer and the candidate convolutional layer to obtain the target neural network.
[0044] To achieve the above objectives, a third aspect of the present application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect or the method described in the second aspect.
[0045] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect or the method described in the second aspect.
[0046] The neural network optimization method, apparatus, data processing method, device, and medium proposed in this application obtain candidate split convolutional layers by splitting candidate convolutional layers, then obtains candidate split convolutional layers that meet preset requirements for memory optimization as target split convolutional layers, and concatenates the target split convolutional layers and candidate convolutional layers to form a target neural network. This not only ensures that the accuracy of convolution processing of the target neural network is not affected, but also reduces the space occupied by the target neural network when running in preset memory, thereby improving the efficiency of the target neural network running in preset memory. It also saves memory space to deploy higher-level networks, thereby improving the operating efficiency of neural networks corresponding to the Internet of Things in the supply chain, and thus improving transaction efficiency. Attached Figure Description
[0047] Figure 1 This is a schematic diagram of the original neural network structure provided in the embodiments of this application;
[0048] Figure 2 This is a flowchart of the neural network optimization method provided in the embodiments of this application;
[0049] Figure 3 yes Figure 2 The flowchart of step S202 in the text;
[0050] Figure 4 yes Figure 2 The flowchart of step S204 in the process;
[0051] Figure 5 This is a schematic diagram of the structure of the target neural network provided in the embodiments of this application;
[0052] Figure 6 yes Figure 2 The flowchart of step S207 in the document;
[0053] Figure 7 This is a flowchart of the data processing method provided in the embodiments of this application;
[0054] Figure 8 yes Figure 7 The flowchart of step S703 in the process;
[0055] Figure 9 yes Figure 8 The flowchart of step S803 in the process;
[0056] Figure 10 This is a flowchart of the convolution processing of the fused convolutional layer provided in an embodiment of this application;
[0057] Figure 11 This is a schematic diagram of the structure of the neural network optimization device provided in the embodiments of this application;
[0058] Figure 12 This is a schematic diagram of the structure of the data processing device provided in the embodiments of this application.
[0059] Figure 13 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0061] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.
[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0063] First, let's analyze some of the terms used in this application:
[0064] Artificial Intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.
[0065] Artificial Neural Networks (ANNs): A neural network (NN) is a mathematical or computational model that mimics the structure and function of biological neural networks. Neural networks consist of a large number of interconnected artificial neurons performing computations. In most cases, artificial neural networks can change their internal structure based on external information, making them adaptive systems. Modern neural networks are a nonlinear statistical data modeling tool, often used to model complex relationships between inputs and outputs or to explore patterns in data. A neural network is a computational model composed of a large number of nodes (or "neurons") and the connections between them. Each node represents a specific output function, called the activation function. Each connection between two nodes represents a weighted value for the signal passing through that connection, called a weight, which is equivalent to the memory of the artificial neural network. The network's output varies depending on the network's connection methods, weight values, and activation functions.
[0066] Convolutional layer: In a convolutional neural network, each convolutional layer consists of several convolutional units. The parameters of each convolutional unit are optimized through backpropagation. The purpose of convolution is to extract different features from the input. The first convolutional layer may only extract some low-level features such as edges, lines, and corners. More layers in the network can iteratively extract more complex features from low-level features.
[0067] Microcontroller Unit (MCU): MCU, also known as a microcontroller, is a chip-level computer microcontroller. It's essentially a chip that reduces the frequency and specifications of a CPU and integrates peripheral interfaces such as memory, counters, USB, A / D converters, UART, PLC, DMA, and even LCD driver circuitry onto a single chip. Simply put, it's a chip that can be used to make smart electronic products.
[0068] Receptive Field: In convolutional neural networks (CNNs), the receptive field is defined as the area on the input image that corresponds to a pixel in the feature map output by each layer of the CNN. In simpler terms, a pixel in the feature map corresponds to a region in the input image. In a typical CNN architecture, the value of each output node in a fully connected (FC) layer depends on all inputs from the FC layer, while the value of each output node in a convolutionally connected (CONV) layer depends only on a region of inputs from the CONV layer. Input values outside this region do not affect the output value; this region is the receptive field.
[0069] Neural Architecture Search (NAS) is one of the hottest research topics in the field of artificial intelligence in recent years. The significance of NAS lies in solving the hyperparameter tuning problem of deep learning models, representing an interdisciplinary study combining optimization and machine learning. Before deep learning, traditional machine learning models also encountered hyperparameter tuning problems. Because shallow models have relatively simple structures, most research unified the model structure as hyperparameters for searching, such as the number of hidden neurons in a three-layer neural network. Methods for optimizing these hyperparameters are mainly black-box optimization methods, such as evolutionary optimization, Bayesian optimization, and reinforcement learning.
[0070] With the development of science and technology, acquiring data and using data modeling to solve corresponding problems has become a common technical approach. For example, in the Internet of Things (IoT) within supply chain finance, neural networks are run on terminal devices to improve the efficiency of IoT operations, serving functions such as generating reports, monitoring, and issuing alarms.
[0071] A preset memory is allocated on the terminal device to deploy the convolutional neural network (CNN). This allows the CNN to run on the preset memory, eliminating the need for a backend server. AI operations are then performed directly on the terminal device via CNN computation, making AI control more convenient. For example, CNNs can be used on the terminal device to efficiently process financial data such as enterprise transaction data, product data, profit data, interest coverage ratios, and input / output status, or to ensure risk control in supply chain finance.
[0072] With the miniaturization of terminal devices, the pre-installed memory is now typically provided by an MCU (Microcontroller Unit), which is also trending towards miniaturization, with some MCUs having only a few hundred KB of memory. When a convolutional neural network (CNN) runs, the MCU needs to allocate memory to store the input and output data matrices of the CNN. However, with the rapid development of CNNs, the network size, parameters, and data matrices are increasing, leading to a significant increase in the memory required for these matrices. Therefore, optimizing CNNs is crucial to reducing their memory footprint.
[0073] Based on this, embodiments of this application provide a method and apparatus for optimizing a neural network, a data processing method, device, and medium. This involves obtaining the input and output memory values of each candidate convolutional layer in the original neural network, selecting a chosen convolutional layer from the candidate convolutional layers based on these values, and then selecting a splitting layer based on the selected and candidate convolutional layers to obtain splitting parameters. The candidate convolutional layers are then split according to these parameters to obtain candidate split convolutional layers. Memory optimization information for the candidate split convolutional layers is obtained, and a target split convolutional layer is selected from the candidate split convolutional layers based on this information. Finally, the target split convolutional layer and the candidate convolutional layers are concatenated to obtain the target neural network. Therefore, by selecting a target split convolutional layer from the candidate split convolutional layers based on memory optimization information, and then concatenating the target split convolutional layer and the candidate convolutional layers to form a new target neural network, data processing through the target neural network involves separate inputs to the target split convolutional layer and the candidate convolutional layers. This reduces the memory footprint of the target neural network during operation, ensuring that the data accuracy of the target neural network is not compromised while saving memory space. By saving memory space occupied by the target neural network, the efficiency of the target neural network running on the terminal device can be improved, so as to efficiently process financial data in supply chain finance.
[0074] The neural network optimization method, apparatus, data processing method, device, and medium provided in this application are specifically described through the following embodiments. First, the neural network optimization method in this application embodiment is described.
[0075] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.
[0076] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.
[0077] The neural network optimization method provided in this application relates to the field of artificial intelligence technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the neural network optimization method, but is not limited to the above forms.
[0078] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0079] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data required for the proper functioning of these embodiments acquired.
[0080] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the original neural network structure provided in the embodiments of this application. Figure 1 It is known that the original neural network includes at least two candidate convolutional layers, which are stacked one on top of the other. After the input data matrix passes through each candidate convolutional layer, an activation data matrix, also known as a feature map, is generated. The original neural network runs in a pre-allocated memory. This pre-allocated memory includes a memory space, which is then loaded with the input data matrix and convolutional parameters of the currently running candidate convolutional layer. An additional memory space is allocated to store the activation data matrix generated by the current candidate convolutional layer. This process continues until all activation data matrices have been computed, at which point the space occupied by the input data matrix is released. The size of the convolutional layer parameters is relatively small compared to the size of the input data matrix, so the space occupied by the convolutional layer parameters is negligible. Therefore, in the original neural network running in the pre-allocated memory, the input data matrix and activation data matrix occupy the largest amount of memory. For example, through... Figure 1 Five candidate convolutional layers are identified: CONV1, CONV2, CONV3, CONV4, and CONV5. CONV1 occupies 2 units for its input data matrix and 4 units for its activation data matrix; CONV2 occupies 2 units for its input data matrix and 4 units for its activation data matrix; CONV3 occupies 4 units for its input data matrix and 8 units for its activation data matrix; CONV4 occupies 8 units for its input data matrix and 2 units for its activation data matrix; and CONV5 occupies 2 units for both its input data matrix and activation data matrix. It is evident that CONV3 requires the largest memory space for convolutional computation, requiring 12 units. Therefore, it is crucial to reduce the memory footprint of the neural network when operating within the preset memory limit, so that more advanced networks can be deployed on terminal devices, or the neural network can run more efficiently on terminal devices. For example, more efficient operation of the neural network on terminal devices would lead to more efficient processing of financial data in supply chain finance, or more efficient risk control in supply chain finance.
[0081] Figure 2 This is an optional flowchart of the neural network optimization method provided in the embodiments of this application. Figure 2 The method may include, but is not limited to, steps S201 to S207.
[0082] Step S201: Obtain memory usage information of the original neural network running in a preset memory; wherein, the original neural network includes at least two candidate convolutional layers, and the memory usage information includes the input memory value and output memory value of each candidate convolutional layer;
[0083] Step S202: Select the convolutional layer from the candidate convolutional layers based on the input memory value and the output memory value;
[0084] Step S203: Select a split layer based on the selected convolutional layer and candidate convolutional layers to obtain the layer splitting parameters; wherein, the layer splitting parameters include the number of splits and the splitting number of each candidate convolutional layer.
[0085] Step S204: Perform layer splitting on the candidate convolutional layers according to the number of splits and the split number to obtain candidate split convolutional layers;
[0086] Step S205: Obtain memory optimization information for each candidate split convolutional layer;
[0087] Step S206: Select the target split convolutional layer from the candidate split convolutional layers based on memory optimization information;
[0088] Step S207: The target split convolutional layer and the candidate convolutional layer are spliced together to obtain the target neural network.
[0089] Steps S201 to S207 of this embodiment involve obtaining memory usage information of the original neural network running in a preset memory, including the input and output memory values of each candidate convolutional layer. Then, a selected convolutional layer is selected from the candidate convolutional layers based on the input and output memory values. Next, a splitting layer selection is performed based on the selected and candidate convolutional layers to obtain the number of splits and the splitting number of the candidate convolutional layers to be split, using the selected convolutional layer as a reference layer for splitting. Candidate splitting convolutional layers are obtained by performing layer splitting processing on the candidate convolutional layers according to the number of splits and the splitting number. The memory usage reduction rate of each candidate splitting convolutional layer during data processing is tested to obtain memory optimization information. A target splitting convolutional layer is then selected from the candidate splitting convolutional layers based on the memory optimization information. Finally, the target splitting convolutional layer and the candidate convolutional layers are concatenated to form the target neural network. Therefore, by splitting candidate convolutional layers to obtain candidate split convolutional layers, and then obtaining candidate split convolutional layers that meet the preset requirements for memory optimization as target split convolutional layers, the target split convolutional layers and candidate convolutional layers are spliced together to form the target neural network. This not only ensures that the accuracy of the target neural network in convolution processing is not affected, but also reduces the space occupied by the target neural network when running in the preset memory, thereby improving the efficiency of the target neural network running in the preset memory, and also saves memory space to deploy higher-level networks.
[0090] In step S201 of some embodiments, after the original neural network is run in a preset memory, each candidate convolutional layer of the original neural network performs convolution calculations. Then, the memory space occupied by the data matrix input to each candidate convolutional layer is obtained to obtain the input memory value, and the memory space occupied by the data matrix output by each candidate convolutional layer is also obtained to obtain the output memory value. Specifically, when performing convolution calculations on each candidate convolutional layer, the memory space occupied by the data matrix input to the currently calculated candidate convolutional layer is obtained to obtain the input memory value, and simultaneously, the memory space reserved in the preset memory for storing the output data matrix of the currently calculated candidate convolutional layer is obtained to obtain the output memory value. For example, through... Figure 1 It can be seen that CONV1 requires 2 units of input memory and 2 units of output memory, while CONV3 requires 4 units of input memory and 8 units of output memory. Therefore, by obtaining the input and output memory values of each candidate convolutional layer, we can analyze the amount of memory space required for the original neural network to run in the preset memory. This can serve as a basis for deciding whether to deploy other networks and also as a basis for judging the running efficiency of the original neural network.
[0091] It should be noted that the scenario applied in this embodiment is supply chain finance. Since the neural network involved in supply chain finance involves multiple convolutional layers, the memory requirements of the neural network involved in supply chain finance are calculated in advance, and then the reserved memory space is determined based on the memory requirements to ensure that the neural network of supply chain finance can run efficiently on the terminal device.
[0092] Please see Figure 3 In some embodiments, step S202 may include, but is not limited to, steps S301 to S303:
[0093] Step S301: Calculate the memory usage of the input memory value and the output memory value to obtain the memory usage value of each candidate convolutional layer;
[0094] Step S302: Obtain the maximum value of the memory occupied, and get the maximum memory value;
[0095] Step S303: Select the candidate convolutional layer with the largest memory value as the selected convolutional layer.
[0096] In step S301 of some embodiments, the input memory value and the output memory value are added together to calculate the memory occupied value of each candidate convolutional layer. Based on the memory occupied value, it is determined how much memory space the preset memory needs to allocate to store the data matrix of the input and output of each candidate convolutional layer when performing convolution calculation. For example, refer to... Figure 1 It can be seen that CONV1 occupies 4 memory units, CONV2 occupies 6 memory units, CONV3 occupies 12 memory units, CONV4 occupies 10 memory units, and CONV5 occupies 4 memory units. Therefore, by calculating the memory usage, the preset memory space allocation required for each candidate convolutional layer during convolution calculation can be determined.
[0097] In step S302 of some embodiments, the memory usage values are sorted in descending order, and the memory usage value that ranks first in the sorted list, i.e., the largest memory usage value, is obtained to obtain the maximum memory value. Therefore, by obtaining the maximum memory value, the candidate convolutional layer is determined as the selected convolutional layer based on the maximum memory value. For example, refer to... Figure 1 It can be seen that CONV3 has the largest memory usage, which means the maximum memory usage is 12 units.
[0098] In step S303 of some embodiments, the candidate convolutional layer with the largest memory value is obtained as the selected convolutional layer. The splitting layer is then selected based on the selected convolutional layer, enabling the selection of candidate convolutional layers for splitting, thereby reducing the memory space occupied by the original neural network when running within a preset memory. For example, refer to... Figure 1It is evident that CONV3, as the selected convolutional layer, requires the most memory. Therefore, splitting the candidate convolutional layers based on this selected layer is necessary to reduce the space occupied by the neural network when running within the preset memory. This allows for the deployment of other networks within the preset memory or improves the efficiency of the neural network, thereby enhancing supply chain finance. Correspondingly, financial data neural networks or risk control neural networks can operate efficiently, enabling efficient processing of financial data in supply chain finance and effectively ensuring risk control.
[0099] In steps S301 to S303 of this embodiment, the occupied memory value is calculated by adding the input memory value and the output memory value, and the maximum memory value among the occupied memory values is obtained. Then, the candidate convolutional layer with the maximum memory value is selected as the selected convolutional layer. Therefore, by selecting the candidate convolutional layer corresponding to the maximum memory value, it is possible to determine the candidate convolutional layers that can be split based on the selected convolutional layer. This allows for saving space occupied by the neural network when running in the preset memory after splitting the candidate convolutional layers, thereby improving the operating efficiency of the neural network, or facilitating the redeployment of other networks in the preset memory.
[0100] In step S203 of some embodiments, splitting layers are selected based on the selected convolutional layer and candidate convolutional layers to obtain the splitting number and number of candidate convolutional layers to be split. Where the selected convolutional layer occupies the most memory during convolution calculation, that is, the activation data matrix output by the selected convolutional layer occupies the largest space, then the candidate convolutional layers before the selected convolutional layer are selected as the candidate splitting layers. Since methods to reduce memory space require splitting layers before the activation data matrix occupies the largest space, splitting the candidate convolutional layers before the selected convolutional layer can save memory space, but splitting the candidate convolutional layers after the selected convolutional layer cannot save memory space. To specifically select several candidate convolutional layers as splitting layers, each needs to be selected individually to form multiple layer splitting parameters to be split, so as to construct different layer splitting schemes based on the layer splitting parameters.
[0101] It should be noted that the selection of candidate convolutional layers for layer splitting based on the selected convolutional layer is determined by a neural architecture search algorithm. The selected convolutional layer serves as the reference layer, and the candidate convolutional layers preceding the selected layer are randomly combined to obtain multiple layer splitting parameters. The memory optimization effect of each splitting scheme is then tested to obtain the optimal splitting scheme with the best memory optimization effect. This optimal scheme is then used to construct a new target neural network. Therefore, the target neural network operates with optimal memory optimization within a preset memory environment, effectively reducing the memory usage during neural network operation.
[0102] Please see Figure 4 In some embodiments, step S204 may include, but is not limited to, steps S401 to S403:
[0103] Step S401: Select the convolutional layer from the candidate convolutional layers according to the split number;
[0104] Step S402: Copy the selected convolutional layer according to the number of splits to obtain the updated convolutional layer;
[0105] Step S403: The updated convolutional layer and the selected convolutional layer are spliced together to obtain a candidate split convolutional layer.
[0106] In step S401 of some embodiments, the layer splitting parameters include the number of splits and the splitting number of the candidate convolutional layers. To test the optimization effect of the layer splitting scheme corresponding to each splitting parameter, it is necessary to construct candidate splitting convolutional layers corresponding to each splitting parameter. Only by testing the optimization effect of each candidate splitting convolutional layer can the layer splitting parameter corresponding to the optimal effect be selected. Therefore, the selected convolutional layer is first selected from the candidate convolutional layers according to the splitting number. A convolutional layer number is set on each candidate convolutional layer, and the candidate convolutional layer with the same convolutional layer number and splitting number is selected as the selected convolutional layer. The number of selected convolutional layers is at least one. For example, please refer to... Figure 1 It can be seen that if convolutional layer CONV3 is selected, then the candidate convolutional layers that can be used as layers are CONV1, CONV2, and CONV3. There are 7 possible combinations of the three candidate convolutional layers. If the split number of the first group is C1, and C1 corresponds to CONV1, then CONV1 is selected as the convolutional layer. If the numbers of the second group are C1, C2, and C3, then CONV1, CONV2, and CONV3 are selected as convolutional layers.
[0107] In step S402 of some embodiments, after determining the selected convolutional layer that can be split, the selected convolutional layer is copied to obtain an updated convolutional layer consistent with the selected convolutional layer. The number of times the selected convolutional layer needs to be copied is also determined based on the number of splits, so that the number of layers in the updated convolutional layer is consistent with the number of splits. Specifically, the weights of the selected convolutional layer are copied in equal amounts to form the updated convolutional layer. For example, if the number of splits is 1, and the selected convolutional layers are CONV1, CONV2, and CONV3, then the weights of the selected convolutional layers are copied in equal amounts to form the updated convolutional layer, which is CONV1*, CONV2*, and CONV3*. Since the updated convolutional layer and the selected convolutional layer perform convolution calculations with the same effect, it does not affect the data processing effect and can also save memory space in the runtime memory.
[0108] In step S403 of some embodiments, a candidate split convolutional layer is obtained by concatenating the updated convolutional layer and the selected convolutional layer, that is, the input ends of the updated convolutional layer and the selected convolutional layer are connected, and the output ends are connected. If there are multiple updated convolutional layers, the multiple updated convolutional layers and the selected convolutional layers are concatenated in parallel, that is, the input ends are connected, and the output ends are connected. By concatenating the updated convolutional layer and the selected convolutional layer to obtain the candidate split convolutional layer, the original data is divided into multiple parts during data processing, and each part is input into the updated convolutional layer and the selected convolutional layer for convolution calculation. Since the data processing has an order, the updated convolutional layer processes the data after the selected convolutional layer completes the data processing. Therefore, the output activation data matrix is half of the original, and the memory space occupied by the output data matrix is also half of the original, thereby saving the preset memory space. Therefore, it does not affect the data processing effect and can save the memory space of the preset memory, so that the neural network corresponding to the supply chain finance field can run efficiently on the terminal device, improving the processing efficiency of financial data such as enterprise transaction data, goods data, profit data, interest coverage ratio, and input and output status in supply chain finance.
[0109] For example, if the updated convolutional layers are CONV1*, CONV2*, and CONV3*, and the selected convolutional layers are CONV1, CONV2, and CONV3, then the selected convolutional layers and the updated convolutional layers will be concatenated to form a candidate split convolutional layer. Please refer to [reference needed]. Figure 5 , Figure 5 This is a schematic diagram of the target neural network structure. If the connection structure of the candidate split convolutional layer is as follows... Figure 5 As shown. (Through) Figure 5 As can be seen, by copying the weights of each selected convolutional layer to form an updated convolutional layer, the input data matrix is split into two parts during data processing. These two parts are then input into CONV1 and CONV1* respectively. CONV1 performs convolution calculations first and outputs its data matrix to CONV2 for calculation. CONV2 performs convolution calculations and outputs its data matrix to CONV3. After completing one side of the convolution calculation, CONV1* performs convolution calculations and outputs its data matrix to CONV2* for calculation. Then, the data matrix output by CONV2* is input into CONV3*. Therefore, when each convolutional layer performs a convolution calculation, the output activation data matrix is half of the previous one, so the memory space occupied by the output activation data matrix is also half, thus saving the preset memory space.
[0110] In steps S401 to S403 of this embodiment, a selected convolutional layer is selected from the candidate convolutional layers according to the split number, i.e., a candidate convolutional layer for layer splitting is selected. Then, the weights of the selected convolutional layer are copied in equal proportions to obtain an updated convolutional layer. Finally, the updated convolutional layer and the selected convolutional layer are concatenated to obtain a candidate split convolutional layer. Therefore, by copying the weights of the selected convolutional layer to form an updated convolutional layer, and then concatenating the selected convolutional layer and the updated convolutional layer to form a candidate split convolutional layer, the data processing effect can be maintained, while reducing the space occupied by the output activation data matrix, thereby optimizing the memory usage of the neural network during runtime.
[0111] In step S205 of some embodiments, memory optimization information is obtained by testing the memory usage reduction rate during data processing for each candidate split convolutional layer. This memory optimization information is then used to determine the memory optimization effect of each candidate split convolutional layer, facilitating the selection of the target split convolutional layer with the best memory optimization effect. If the connection structure of the candidate split convolutional layer is as follows... Figure 5 As shown, the memory usage reduction rate of the candidate split convolutional layer during data processing is tested to obtain memory optimization information. Through... Figure 1 The maximum memory value shown is 12 units, and after splitting, it forms as follows: Figure 5 The candidate split convolutional layer shown first divides the input data matrix into two parts during data processing, with each part occupying one memory unit. One part of the data matrix passes through CONV1, CONV2, and CONV3, outputting an activation data matrix occupying four memory units. Then, the other part of the data matrix passes through CONV1*, CONV2*, and CONV3*, outputting another activation data matrix occupying four memory units. Therefore, the time with the highest memory usage in the entire data processing is when calculating the CONV3 convolutional layer. Thus, CONV3 requires four units to store the activation data matrix during convolution calculation, two units to store the input data matrix, and four units to store the activation data matrix output by CONV3*. Therefore, Figure 5 The memory optimization information for the candidate split convolutional layers is 16.6%, meaning the memory usage reduction rate is 16.6%. Therefore, by obtaining the memory optimization information, we can know how much memory space each candidate split convolutional layer reduces during data processing, which is helpful in selecting the candidate split convolutional layer with the best optimization effect.
[0112] In step S206 of some embodiments, the memory optimization information, i.e., the memory usage reduction rate, is sorted in order, and then the candidate split convolutional layer corresponding to the highest memory usage reduction rate is selected as the target split convolutional layer. Therefore, data processing using the convolutional layer connection structure of the target split convolutional layer can achieve the optimal effect of memory space optimization, so that the preset memory can be used to deploy other networks, and the terminal device configured with the preset memory can realize richer and more powerful functions.
[0113] Please see Figure 6 In some embodiments, step S207 may include, but is not limited to, steps S601 to S602:
[0114] Step S601: Select the fused convolutional layer from the candidate convolutional layers according to the target split convolutional layer;
[0115] Step S602: The fused convolutional layer and the target split convolutional layer are spliced together to obtain the target neural network.
[0116] In step S601 of some embodiments, since the target split convolutional layer includes a selected convolutional layer and an updated convolutional layer, a fusion convolutional layer is selected from candidate convolutional layers based on the selected convolutional layer. Candidate convolutional layers that do not belong to the selected convolutional layer are determined as fusion convolutional layers to determine in which convolutional layer the multiple activation data matrices output by the target split convolutional layer will be fused. For example, please refer to... Figure 5 It can be seen that CONV4 and CONV5 are selected from the candidate convolutional layers as fusion convolutional layers, so that the activation data matrix output by the target split convolutional layer is fused by CONV4 and CONV5 and then processed.
[0117] In step S602 of some embodiments, after selecting the fused convolutional layer, the target split convolutional layer and the fused convolutional layer are concatenated to form a target neural network. By concatenating the output of the target split convolutional layer and the input of the fused convolutional layer to obtain the target neural network, a target neural network running in a preset memory is constructed. Since the target neural network occupies less memory in the preset memory compared to the original neural network, the saved memory space can be used to deploy more advanced networks, enabling the terminal device configured with the preset memory to achieve richer and more powerful functions. Therefore, this provides stronger technical support for supply chain finance, enabling terminal devices in supply chain finance to have more advanced functions.
[0118] In steps S601 to S602 of this embodiment, candidate convolutional layers are screened using a target split convolutional layer. Specifically, candidate convolutional layers different from the selected convolutional layer are selected to obtain a fused convolutional layer. Then, the output of the target split convolutional layer and the input of the fused convolutional layer are concatenated to form the target neural network. Therefore, by constructing a target neural network including a target split convolutional layer and a fused convolutional layer, the target neural network can reduce memory usage when running within a preset memory, enabling efficient operation. The saved memory space can also be used to deploy higher-level networks, enriching the functionality of the terminal device configured with the preset memory. This allows the terminal device to be equipped with models for processing various report data in supply chain finance, reducing terminal device setup and thus lowering the cost of report data processing in supply chain finance.
[0119] Please see Figure 7 This application also provides a data processing method, which may include, but is not limited to, steps S701 to S703:
[0120] Step S701: Obtain the input data matrix to be processed;
[0121] Step S702: Split the input data matrix to obtain a split data matrix;
[0122] Step S703: Input the split data matrix into the target neural network for convolution processing to obtain the target data; wherein, the target neural network is obtained by the above-mentioned neural network optimization method.
[0123] In steps S701 to S703 of this embodiment, the input data matrix to be processed is obtained, then split into at least two split data matrices, and then the split data matrices are input into the target neural network. Since the target neural network has split convolutional layers, the split data matrices are input into the split convolutional layers of the target neural network for convolution processing, thus reducing the memory space occupied by the activation data matrix output by the split convolutional layers. Therefore, by inputting the split data matrix into the target neural network and performing convolution calculations through the split convolutional layers of the target neural network, the output activation data matrix is reduced, and the memory space occupied by the activation data matrix is also reduced. This allows the target neural network to reduce the memory space occupied by the preset memory during data processing, which not only improves the efficiency of the target neural network but also allows the saved memory space to be deployed to other more advanced networks, thereby enabling the terminal device with the preset memory to achieve richer functions.
[0124] In step S701 of some embodiments, target data to be processed is obtained, and the target data includes any one of the following: target image, target text, target speech data. If the target data is a target image, the target image is represented in the form of a data matrix to obtain an input data matrix, that is, the input data matrix to be processed is obtained.
[0125] In step S702 of some embodiments, since the target neural network includes split convolutional layers, in order to reduce the memory space occupied by the target neural network during data processing, it is necessary to split the input data matrix to obtain a split data matrix. Specifically, the input data matrix is split according to the number of target split convolutional layers in the target neural network to obtain a split data matrix with the same number of layers as the target split convolutional layers. This allows each split data matrix to be convolved by its corresponding convolutional layer, thereby saving memory space during convolutional calculations.
[0126] In step S703 of some embodiments, the split data matrix is input to the target neural network, which includes a target split convolutional layer. The split data matrix is input to each convolutional layer of the target split convolutional layer, thus reducing the input data matrix of the convolutional layer. Consequently, the activation data matrix output by the convolutional layer is also reduced, and the memory space consumed by the target neural network during operation is correspondingly reduced, improving the operating efficiency of the target neural network and saving memory space for deploying other networks. Specifically, if the input data matrix is a target image, the target neural network performs convolution processing on the split data matrix to obtain target data, which is image features, to achieve image feature extraction. Furthermore, image feature extraction saves the memory space occupied by the preset memory.
[0127] Please see Figure 8 In some embodiments, the target neural network includes: a target splitting convolutional layer and a fusion convolutional layer, and step S703 may include, but is not limited to, steps S801 to S803:
[0128] Step S801: Input the split data matrix into the target split convolutional layer for convolution processing to obtain the activation data matrix;
[0129] Step S802: Merge each activated data matrix to obtain the target data matrix;
[0130] Step S803: Input the target data matrix into the fusion convolutional layer for convolution processing to obtain the target data.
[0131] In step S801 of some embodiments, the target neural network includes a target split convolutional layer and a fusion convolutional layer. The target split convolutional layer includes an update convolutional layer and a selection convolutional layer. Therefore, the split data matrix is input to the target split convolutional layer for convolution processing, that is, the split data matrix is input to the update convolutional layer and the selection convolutional layer for convolution processing respectively. Since the split data matrix is a part of the input data matrix, only the split data matrix and the activation data matrix output by the target split convolutional layer need to be stored during convolution processing, thus reducing memory usage.
[0132] For example, if the network structure of the target neural network is Figure 5 As shown, one split data matrix is input into CONV1, CONV2, and CONV3, and another split data matrix is input into CONV1*, CONV2*, and CONV3*. CONV1, CONV2, and CONV3 output activation data matrices occupying 4 units, and CONV1*, CONV2*, and CONV3* output activation data matrices occupying 4 units each. Therefore, the two activation data matrices occupy 8 units. During the convolution calculation process of the target split convolutional layer, CONV3 occupies the largest amount of preset memory space during convolution calculation; that is, it requires 4 units to store the activation data matrix of CONV3, 2 units to store the split data matrix of CONV3, and 4 units to store the activation data matrix output by CONV3*. Thus, the maximum memory occupied during the entire operation of the target neural network is 10 units, while the original neural network occupies a maximum of 12 units. Therefore, using the target split convolutional layer to perform convolution processing on the split data matrix saves memory space. Since the updated convolutional layer and the selected convolutional layer are convolutional layers with the same weights, the effect of convolution processing can be guaranteed, while saving the memory space occupied by the convolution processing.
[0133] In step S802 of some embodiments, after the target split convolutional layer outputs the activation data matrix, the activation data matrices need to be merged to obtain the target data matrix, so that the target data matrix can be centrally input into the fusion convolutional layer. For example, please refer to... Figure 5 It can be seen that the activation data matrix output by CONV3 and the activation data matrix output by CONV3* are merged to form a target data matrix occupying 8 units.
[0134] In step S803 of some embodiments, the target data matrix is input to the fusion convolutional layer so that the fusion convolutional layer performs fusion convolution calculations on the target data matrix to obtain the target data. The fusion convolutional layer includes multiple convolutional layers. To save memory during the convolution processing of the target data matrix, the fusion convolutional layer does not directly calculate all the target data matrix. Instead, it obtains a portion of the target data matrix, calculates it, and then calculates the remaining portion, thus saving memory space occupied by the convolution processing.
[0135] In steps S801 to S803 of this embodiment, the split data matrix is input into the target split convolutional layer for convolutional processing to obtain the activation data matrix. The activation data matrices of each target split convolutional layer are merged to obtain the target data matrix. The target data matrix is then input into the fusion convolutional layer for convolutional processing to obtain the target data. Therefore, by calculating the split data matrix separately through each convolutional layer of the target split convolutional layer, and since the split data matrix is a part of the input data matrix, the memory space required to store the split data matrix and the target data matrix in the preset memory allocation is reduced, thereby saving the preset memory space and allowing terminal devices configured with preset memory to deploy more networks.
[0136] Please see Figure 9 In some embodiments, step S803 may include, but is not limited to, steps S901 to S903:
[0137] Step S901: Obtain the receptive field data of the target data matrix;
[0138] Step S902: Input the data of each receptive field into the fusion convolutional layer for convolution processing to obtain preliminary data;
[0139] Step S903: Collect and process each preliminary data to obtain the target data.
[0140] In step S901 of some embodiments, the receptive field data of the target data matrix is obtained. The receptive field data is a portion of the data in the target data matrix. For example, if the target data matrix is an image feature, then in a convolutional neural network, the receptive field is the size of the region mapped by the pixel of the image feature on the input image. That is, each image feature corresponds to a region of the image. Therefore, obtaining the receptive field data is also obtaining the data matrix region corresponding to one image feature in the target data matrix. For example, please refer to... Figure 10 , Figure 10This is a flowchart of the convolutional processing of the fused convolutional layers. If the target data matrix is a 7*7 data matrix, after passing through CONV4, it outputs a 5*5 data matrix, and after passing through CONV5, it outputs a 3*3 data matrix. Among them, regions A and B in the target data matrix are 5*5 data matrices, and regions A and B are a receptive field data. Therefore, the receptive field data of the target data matrix is obtained first, which is to obtain the data matrix of region A.
[0141] In step S902 of some embodiments, each receptive field data is input to the fusion convolutional layer for convolution processing to obtain preliminary data. Instead of inputting the entire target data matrix into the fusion convolutional layer one by one, this process is repeated. The fusion convolutional layer processes the data and outputs preliminary data before processing the next receptive field data. Although this increases the data computation, it saves the memory space occupied by the fusion convolutional layer during execution. Because data processing is rapid in the era of big data, the computational overhead does not affect the overall operating efficiency of the target neural network. Therefore, the saved memory space can be used to deploy other more advanced networks, thereby improving the functionality of the terminal device configured with the preset memory.
[0142] In step S903 of some embodiments, each preliminary data output by the fused convolutional layer is aggregated to obtain the target data, which not only ensures the complete generation of the target data, but also saves the memory space occupied by the fused convolutional layer when performing convolution processing.
[0143] For example, please refer to Figure 10 Each receptive field data point is input into the fusion convolutional layer, specifically CONV4 and CONV5. The receptive field data is a 5x5 matrix. CONV4 performs convolution on the receptive field data, outputting a 3x3 matrix. CONV5 then performs convolution on the 3x3 matrix to obtain a 1x1 preliminary data point. After processing the receptive field data for region A, the next receptive field data for region B is input into CONV4 and CONV5, until all receptive field data undergoes convolution processing to obtain nine preliminary data points. These nine preliminary data points are then combined to obtain the 3x3 target data. Therefore, the maximum memory space occupied during convolution processing in the fusion convolutional layer is the size of one receptive field data point, thus reducing the memory usage of the target neural network during execution.
[0144] In steps S901 to S903 of this embodiment, receptive field data of the target data matrix is acquired, and the receptive field data is a part of the target data matrix. Each receptive field data is input into a fusion convolutional layer for convolution processing to obtain preliminary data. That is, after one receptive field data value is input into the fusion convolutional layer for convolution processing, the next receptive field data value is input into the fusion convolutional layer for convolution processing, until the entire target data matrix is calculated to obtain multiple preliminary data. Then, the multiple preliminary data are combined to obtain the target data. Therefore, by using a convolution processing method that inputs receptive field data one by one for convolution processing, the memory space occupied during each convolution processing is saved, so that the saved memory space can be used to deploy other networks and increase the functionality of terminal devices configured with preset memory.
[0145] This application embodiment obtains the memory usage information of the original neural network running in a preset memory. The memory usage information includes the input memory value and output memory value of each candidate convolutional layer. The input and output memory values are added together to calculate the memory usage value of each candidate convolutional layer, and the maximum memory value is obtained. The candidate convolutional layer with the maximum memory value is then selected as the chosen convolutional layer. The candidate convolutional layers before the selected convolutional layer are used as potential splitting layers. A splitting layer selection process is then performed to obtain the splitting number and number of splits for the candidate convolutional layers to be split. The selected convolutional layer is selected from the candidate convolutional layers based on the splitting number. The weights of the selected convolutional layer are then copied in equal proportions to obtain an updated convolutional layer. Finally, the updated convolutional layer and the selected convolutional layer are concatenated to obtain the candidate splitting convolutional layer. The memory usage reduction rate during data processing is tested for each candidate splitting convolutional layer to obtain memory optimization information. The memory usage reduction rates are sorted in order, and the candidate splitting convolutional layer with the highest memory usage reduction rate is selected as the target splitting convolutional layer. Candidate convolutional layers different from the selected convolutional layer are selected to obtain a fused convolutional layer. The output of the target split convolutional layer and the input of the fused convolutional layer are concatenated to form the target neural network. During data processing, the input data matrix is split into at least two split data matrices, which are then input into the target neural network. Since the target neural network has split convolutional layers, the split data matrices are input into the split convolutional layers of the target neural network for convolution processing. This reduces the memory space occupied by the activation data matrix output by the split convolutional layers, allowing the saved memory space to be used to deploy other networks and enhance the functionality of terminal devices with preset memory configurations.
[0146] Please see Figure 11 This application also provides a neural network optimization apparatus that can implement the above-described neural network optimization method. The apparatus includes:
[0147] The memory acquisition module 1101 is used to acquire memory usage information of the original neural network running in a preset memory; wherein the original neural network includes at least two candidate convolutional layers, and the memory usage information includes the input memory value and output memory value of each candidate convolutional layer;
[0148] The selection filtering module 1102 is used to filter out the selected convolutional layer from the candidate convolutional layers based on the input memory value and the output memory value.
[0149] The split layer selection module 1103 is used to select a split layer based on the selected convolutional layer and candidate convolutional layers to obtain layer split parameters; wherein, the layer split parameters include the number of splits and the split number of each candidate convolutional layer.
[0150] The layer splitting processing module 1104 is used to perform layer splitting processing on the candidate convolutional layer according to the number of splits and the split number to obtain the candidate split convolutional layer.
[0151] The optimization acquisition module 1105 is used to acquire memory optimization information for each candidate split convolutional layer;
[0152] The target selection module 1106 is used to select the target split convolutional layer from the candidate split convolutional layers based on memory optimization information.
[0153] The splicing module 1107 is used to splice the target split convolutional layer and the candidate convolutional layer to obtain the target neural network.
[0154] The specific implementation of the optimization device for this neural network is basically the same as the specific implementation of the optimization method for the neural network described above, and will not be repeated here.
[0155] Please see Figure 12 This application also provides a data processing apparatus that can implement the above-described data processing method. The apparatus includes:
[0156] The matrix acquisition module 1201 is used to acquire the input data matrix to be processed.
[0157] The splitting processing module 1202 is used to split the input data matrix to obtain a split data matrix.
[0158] The convolution processing module 1203 is used to input the split data matrix into the target neural network for convolution processing to obtain the target data; wherein, the target neural network is obtained by the above-mentioned neural network optimization method.
[0159] The specific implementation of this data processing device is basically the same as the specific embodiment of the data processing method described above, and will not be repeated here.
[0160] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the aforementioned neural network optimization method or data processing method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.
[0161] Please see Figure 13 , Figure 13 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:
[0162] The processor 1301 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.
[0163] The memory 1302 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1302 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1302 and is called and executed by the processor 1301 to execute the neural network optimization method or data processing method of the embodiments of this application.
[0164] The input / output interface 1303 is used to implement information input and output;
[0165] The communication interface 1304 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0166] Bus 1305 transmits information between various components of the device (e.g., processor 1301, memory 1302, input / output interface 1303, and communication interface 1304);
[0167] The processor 1301, memory 1302, input / output interface 1303 and communication interface 1304 are connected to each other within the device via bus 1305.
[0168] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described neural network optimization method or data processing method.
[0169] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0170] The neural network optimization method, apparatus, data processing method, device, and medium provided in this application embodiment acquire memory usage information of the original neural network running in a preset memory, including the input and output memory values of each candidate convolutional layer. Then, based on the input and output memory values, a selected convolutional layer is selected from the candidate convolutional layers. Next, the candidate convolutional layers before the selected convolutional layer are split to obtain the split number and number of splits for each candidate convolutional layer. Then, the candidate convolutional layers are split according to the split number and split number to obtain candidate split convolutional layers. The memory usage reduction rate of each candidate split convolutional layer during data processing is tested to obtain memory optimization information. Based on the memory optimization information, a target split convolutional layer is selected from the candidate split convolutional layers. Finally, the target split convolutional layer and the candidate convolutional layers are concatenated to obtain the target neural network. Therefore, by selecting candidate convolutional layers that can be split based on the selected convolutional layer to obtain the layer splitting parameters, and then splitting the corresponding candidate convolutional layers according to the layer splitting parameters to obtain candidate split convolutional layers, the target split convolutional layer is selected from the candidate split convolutional layers using memory optimization information. That is, it is determined that the target split convolutional layer can reduce the optimal amount of running memory. Thus, the target neural network constructed by the target split layer and the candidate convolutional layers can reduce the memory space occupied when running in the preset memory, so that other more advanced networks can be deployed in the preset memory. The terminal device configured with the preset memory can realize richer and more powerful functions, improve the efficiency of processing various report data in supply chain finance, and effectively protect the risk control of supply chain finance.
[0171] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.
[0172] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.
[0173] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0174] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.
[0175] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0176] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.
[0177] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0178] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0179] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0180] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0181] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.
Claims
1. A data processing method, characterized in that, The method includes: Obtain memory usage information of the original neural network running in a preset memory; wherein, the original neural network includes at least two candidate convolutional layers, and the memory usage information includes the input memory value and output memory value of each candidate convolutional layer; Selected convolutional layers are selected from the candidate convolutional layers based on the input memory value and the output memory value; Split layers are selected based on the selected convolutional layer and the candidate convolutional layers to obtain layer splitting parameters; wherein, the layer splitting parameters include the number of splits and the splitting number of each candidate convolutional layer; The candidate convolutional layer is split according to the number of splits and the number of splits to obtain a candidate split convolutional layer; Obtain memory optimization information for each of the candidate split convolutional layers; The target split convolutional layer is selected from the candidate split convolutional layers based on the memory optimization information; The target split convolutional layer and the candidate convolutional layer are concatenated to obtain the target neural network; Obtain the input data matrix to be processed; wherein, the input data matrix to be processed is the target data represented in the form of a data matrix, and the target data includes any one of the following: target image, target text, and target speech data; The input data matrix is split to obtain a split data matrix; The split data matrix is input into the target neural network for convolution processing to obtain the target data.
2. The method according to claim 1, characterized in that, The step of splitting the candidate convolutional layer according to the number of splits and the number of splits to obtain a candidate split convolutional layer includes: Selected convolutional layers are selected from the candidate convolutional layers based on the split number; The selected convolutional layer is copied according to the number of splits to obtain an updated convolutional layer; The updated convolutional layer and the selected convolutional layer are concatenated to obtain the candidate split convolutional layer.
3. The method according to claim 1, characterized in that, The step of concatenating the target split convolutional layer and the candidate convolutional layer to obtain the target neural network includes: Based on the target split convolutional layer, a fused convolutional layer is selected from the candidate convolutional layers; The fused convolutional layer and the target split convolutional layer are concatenated to obtain the target neural network.
4. The method according to any one of claims 1 to 3, characterized in that, The step of selecting a convolutional layer from the candidate convolutional layers based on the input memory value and the output memory value includes: The memory usage of the input memory value and the output memory value is calculated to obtain the memory usage value of each candidate convolutional layer. Obtain the maximum value of the occupied memory to get the maximum memory value; The candidate convolutional layer with the largest memory value is selected as the chosen convolutional layer.
5. The method according to claim 1, characterized in that, The target neural network includes: a target splitting convolutional layer and a fusion convolutional layer; the step of inputting the splitting data matrix into the target neural network for convolution processing to obtain target data includes: The split data matrix is input into the target split convolutional layer for convolution processing to obtain the activation data matrix; Each of the activation data matrices is merged to obtain the target data matrix; The target data matrix is input into the fusion convolutional layer for convolution processing to obtain the target data.
6. The method according to claim 5, characterized in that, The step of inputting the target data matrix into the fusion convolutional layer for convolution processing to obtain the target data includes: Obtain the receptive field data of the target data matrix; Each receptive field data is input into the fusion convolutional layer for convolution processing to obtain preliminary data; Each of the preliminary data is collected and processed to obtain the target data.
7. A data processing apparatus, characterized in that, The device includes: A memory acquisition module is used to acquire memory usage information of the original neural network running in a preset memory; wherein the original neural network includes at least two candidate convolutional layers, and the memory usage information includes the input memory value and output memory value of each candidate convolutional layer; A selection filtering module is used to filter out selected convolutional layers from the candidate convolutional layers based on the input memory value and the output memory value; A split layer selection module is used to select a split layer based on the selected convolutional layer and the candidate convolutional layers to obtain layer split parameters; wherein, the layer split parameters include the number of splits and the split number of each candidate convolutional layer; A layer splitting module is used to perform layer splitting processing on the candidate convolutional layer according to the number of splits and the number of splits, so as to obtain a candidate split convolutional layer; An optimization acquisition module is used to acquire memory optimization information for each of the candidate split convolutional layers. The target selection module is used to select a target split convolutional layer from the candidate split convolutional layers based on the memory optimization information. The splicing module is used to splice the target split convolutional layer and the candidate convolutional layer to obtain the target neural network; The matrix acquisition module is used to acquire the input data matrix to be processed. A splitting processing module is used to split the input data matrix to obtain a split data matrix; The convolution processing module is used to input the split data matrix into the target neural network for convolution processing to obtain target data; wherein, the input data matrix to be processed is the target data represented in the form of a data matrix, and the target data includes any one of the following: target image, target text, and target speech data.
8. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the data processing method according to any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the data processing method according to any one of claims 1 to 6.