Vector fusion calculation method and device applied in neural network data processing

Abstract

The application relates to the field of neural networks, in particular to a vector fusion calculation method and device applied to neural network data processing. The method and device adopt an instruction set control mode, encode two different Route operators and Shortcut operators into different parameters in an instruction set, control the operation of the system in a state machine mode, store function information of each layer of the network in an SRAM of the instruction set, then decode the function information in a decoding state of the state machine, and when the operator type of the instruction set is decoded as Route or Shortcut, corresponding operator processing is carried out. Because the two operators of Route and Shortcut are both used for data processing on two intermediate layers in the network layer, whether splicing or adding corresponding position data is carried out, the characteristic information of the output feature map is increased, and the subsequent processing precision of the picture is improved.

Description

Technical Field

[0001] The present invention relates to the field of neural networks, and in particular to a vector fusion calculation method and device applied in neural network data processing. Background Art

[0002] As neural network structures become increasingly complex and the number of layers increases, network accuracy begins to saturate or even degrade. Therefore, as the network progresses, it becomes increasingly necessary to perform necessary processing on intermediate data to ensure the accuracy and precision of the network data. This is because in convolutional neural networks, as the network layers grow deeper, operations like convolution and pooling on an image remove some of the previous data, causing a partial loss of the image's feature information, which can lead to biased or inaccurate image recognition results.

[0003] The existing technology also has some solutions for the above defects, but still has the following shortcomings:

[0004] 1. The images are processed page by page without parallel processing, which results in low processing efficiency.

[0005] 2. The image interacts multiple times from the off-chip DDR to the SRAM, resulting in large data storage delays.

[0006] 3. It is highly targeted, but the design process is complex and cannot be well integrated with other operator modules of the system, resulting in complicated system operation, high energy consumption and resource overhead. Summary of the Invention

[0007] The embodiments of the present invention provide a vector fusion calculation method and device for use in neural network data processing, so as to at least solve the technical problem of low efficiency in processing input feature maps in the prior art.

[0008] According to one embodiment of the present invention, a vector fusion calculation method for use in neural network data processing is provided, comprising the following steps:

[0009] Using instruction set control, two different Route operators and Shortcut operators are encoded into different parameters in the instruction set;

[0010] The operation of the system is controlled by a state machine. The SRAM of the instruction set stores the functional information of each layer of the network, which is then decoded in the decoding state of the state machine.

[0011] When the operator type of the decoded instruction set is Route or Shortcut, the corresponding operator processing is performed.

[0012] Furthermore, the method initially integrates the processes of two different Route operators and Shortcut operators into the same state, and then separates them into their respective different processing states when the specific vector data are processed.

[0013] Furthermore, the Route operator and Shortcut operator in the method are both implemented under the entire state machine framework, and adopt the instruction set configuration and decoding method. The Route operator and Shortcut operator have the same configuration and decoding process in the early stage of the system, but different parameter codes in the instruction set.

[0014] Furthermore, the method specifically includes:

[0015] First, the system performs the early control and configuration process and stores the instruction set information in SRAM through STATE_CFG_RAM;

[0016] After the instruction set information is configured, the state machine decodes it in the STATE_DECODE_INST state to analyze what action to take next;

[0017] When the state machine decodes the network layer type into the instruction set as Route or Shortcut, it will jump to the STATE_SHORT_ROUTE state; the process of the Route operator or the Shortcut operator in the early stage of indexing the starting position of the target network layer from the off-chip DDR is the same, except that the specific parameters are different. Route is to transfer the data of the two feature maps into the SRAM for splicing, obtain the target output data and then write it back to the DDR, while Shortcut is to transfer the data of the two layers of feature maps into the system's computing unit PE for corresponding addition, obtain the data of the new output feature map and then write it back to the DDR.

[0018] Furthermore, the Route operator splices network data from different layers so that the output network layer takes into account the initial feature information. The Route operator combines the feature information of multiple layers of feature graphs and splices their data into a new feature graph. The new feature graph not only increases the amount of feature information data, but also combines feature information from different network layers.

[0019] Furthermore, the Shortcut operator obtains the image data of the new output feature map by adding the network data of the two middle layers; the data of the corresponding pixel points of the feature map in the Shortcut operator changes, and the data of the corresponding pixel points of the new output feature map is obtained by adding the data of the corresponding pixel points of the two middle network layers.

[0020] Furthermore, the SRAM of the instruction set stores functional information of each layer of the network, including the type of functional layer and operator, and the size of the input feature map.

[0021] According to another embodiment of the present invention, a vector fusion computing device for use in neural network data processing is provided, comprising:

[0022] The encoding unit is used to encode two different route operators and shortcut operators into different parameters in the instruction set using instruction set control;

[0023] The decoding unit is used to control the operation of the system through the state machine. The SRAM of the instruction set stores the functional information of each layer of the network, and then decodes it in the decoding state of the state machine;

[0024] The operator corresponding processing unit is used to perform corresponding operator processing when the operator type of the decoded instruction set is Route or Shortcut.

[0025] A storage medium stores a program file capable of implementing any one of the above-mentioned vector fusion calculation methods applied in neural network data processing.

[0026] A processor is used to run a program, wherein when the program is run, any one of the above-mentioned vector fusion calculation methods applied in neural network data processing is executed.

[0027] The vector fusion calculation method and device used in the neural network data processing in the embodiment of the present invention adopts the instruction set control method to encode two different Route operators and Shortcut operators into different parameters in the instruction set, and controls the operation of the system through the state machine. The SRAM of the instruction set stores the functional information of each layer of the network, and then decodes it in the decoding state of the state machine. When the operator type of the decoded instruction set is Route or Shortcut, the corresponding operator processing is then performed. Because both the Route and Shortcut operators process data of the two middle layers in the network layer, whether it is splicing or adding corresponding position data, the feature information of the output feature map is increased, and the subsequent processing accuracy of the image will be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0029] Figure 1This is a working process diagram of the Route operator in the vector fusion calculation method used in neural network data processing according to the present invention;

[0030] Figure 2 This is a working process diagram of the Shortcut operator in the vector fusion calculation method used in neural network data processing according to the present invention;

[0031] Figure 3 This is a working process diagram of a state machine in a vector fusion calculation method used in neural network data processing according to the present invention;

[0032] Figure 4 This is a partial code diagram of the state machine in the vector fusion calculation method used in neural network data processing according to the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Example 1

[0036] According to an embodiment of the present invention, a vector fusion calculation method for use in neural network data processing is provided, comprising the following steps:

[0037] Using instruction set control, two different Route operators and Shortcut operators are encoded into different parameters in the instruction set;

[0038] The operation of the system is controlled by a state machine. The SRAM of the instruction set stores the functional information of each layer of the network, which is then decoded in the decoding state of the state machine.

[0039] When the operator type of the decoded instruction set is Route or Shortcut, the corresponding operator processing is performed.

[0040] The vector fusion calculation method used in the neural network data processing in the embodiment of the present invention adopts the instruction set control method to encode two different Route operators and Shortcut operators into different parameters in the instruction set, and controls the operation of the system through the state machine. The SRAM of the instruction set stores the functional information of each layer of the network, and then decodes it in the decoding state of the state machine. When the operator type of the decoded instruction set is Route or Shortcut, the corresponding operator processing is then performed. Because both the Route and Shortcut operators process data of the two middle layers in the network layer, whether it is splicing or adding corresponding position data, the feature information of the output feature map is increased, and the subsequent processing accuracy of the image will be improved.

[0041] The method initially integrates the processes of two different Route operators and Shortcut operators into the same state, and then separates them into their own different processing states when processing specific vector data.

[0042] Among them, the Route operator and Shortcut operator in the method are both implemented under the entire state machine framework, and adopt the instruction set configuration and decoding method. The Route operator and Shortcut operator have the same configuration and decoding process in the early stage of the system, but different parameter codes in the instruction set.

[0043] The method specifically includes:

[0044] First, the system performs the early control and configuration process and stores the instruction set information in SRAM through STATE_CFG_RAM;

[0045] After the instruction set information is configured, the state machine decodes it in the STATE_DECODE_INST state to analyze what action to take next;

[0046] When the state machine decodes the network layer type into the instruction set as Route or Shortcut, it will jump to the STATE_SHORT_ROUTE state; the process of the Route operator or the Shortcut operator in the early stage of indexing the starting position of the target network layer from the off-chip DDR is the same, except that the specific parameters are different. Route is to transfer the data of the two feature maps into the SRAM for splicing, obtain the target output data and then write it back to the DDR, while Shortcut is to transfer the data of the two layers of feature maps into the system's computing unit PE for corresponding addition, obtain the data of the new output feature map and then write it back to the DDR.

[0047] Among them, the Route operator splices network data from different layers so that the output network layer takes into account the initial feature information; the Route operator combines the feature information of multiple layers of feature graphs and splices their data into a new feature graph. The new feature graph not only increases the amount of feature information data, but also combines feature information from different network layers.

[0048] Among them, the Shortcut operator obtains the image data of the new output feature map by adding the network data of the two middle layers; the data of the corresponding pixel points of the feature map in the Shortcut operator changes, and the data of the corresponding pixel points of the new output feature map is obtained by adding the data of the corresponding pixel points of the two middle network layers.

[0049] Among them, the SRAM of the instruction set stores the functional information of each layer of the network, including the type of functional layer and operator, and the size of the input feature map.

[0050] The vector fusion calculation method applied in neural network data processing of the present invention is described in detail below with reference to a specific embodiment:

[0051] In order to solve the above technical problems, the present invention designs two different targeted operators, which use different operator processing according to different situations or different network structures. They are Route and Shortcut operators. The former splices network data from different layers so that the output network layer can take into account the initial feature information. The latter adds the network data from the two middle layers to obtain the new output feature map image data. The most similar to the present invention is the current residual network implementation method, which performs calculations by doping different residual blocks in the network.

[0052] The present invention adopts an instruction set control method, encoding two different operators into different parameters in the instruction set, and then controlling the operation of the system through a state machine. When the chip is working, the SRAM of the instruction set will store the functional information of each layer of the network, including the type of functional layer and operator, the size of the input feature map, etc., and then decode it in the decoding state of the state machine. When the operator type of the decoded instruction set is Route or Shortcut, the corresponding operator processing is then performed. Moreover, because both the Route and Shortcut operators process data of the two middle layers in the network layer, whether it is splicing or adding corresponding position data, the feature information of the output feature map is increased, and the subsequent processing accuracy of the image will be improved.

[0053] 1. Design principles of the two operators

[0054] The difference of the Route operator is that it combines the feature information of multiple layers of feature maps, and splices their data into a new feature map. The feature map not only increases the amount of feature information data, but also combines feature information from different network layers. For example, if you want to process the output of the 15th layer, then if you only use the output of the 15th layer, it is inevitable that some information from the first 14 layers will be lost, resulting in insufficient image accuracy or even errors. At this time, you need to find a way to extract some of the previous information, so the present invention uses a splicing method. For example, the present invention can select the output of the fourth layer or the output of the tenth layer and the output of the fifteenth layer to splice them together, and the pixel data corresponding to the image are spliced ​​according to the depth. The length and width of the image remain unchanged, but the depth becomes the sum of the original two layers. The resulting output feature map will contain more feature information. The specific working principle process is as follows. Figure 1 shown.

[0055] Figure 1 The left side of the figure is the middle layer of the neural network. When performing the Route operation, the two middle layers are regarded as input feature maps, and then the image data of the two layers are spliced ​​according to the corresponding positions to obtain the output feature map on the far right. Figure 1 For illustrative purposes, the two input feature maps on the left are both sized relatively small, 5x5x6. This means that the input feature maps have a height and width of 5 and six channels. Finally, after the Route operation, the resulting output feature map is sized 5x5x12. Of course, in real-world scenarios, images are unlikely to be this small; they may be larger and have more channels. However, the principle remains the same. This is a simple example to illustrate the process.

[0056] The Shortcut operator also processes the two intermediate network layers, but unlike the Route operator, the most obvious difference between the Shortcut operator and the Route operator is that the data of the corresponding pixels of the feature map has changed. That is, the data of the corresponding pixels of the new output feature map is obtained by adding the data of the corresponding pixels of the two intermediate network layers. The working principle is as follows: Figure 2 As shown:

[0057] like Figure 2 As shown in the figure, on the left are two 6x6x7 input feature maps. After adding the data of the corresponding pixel positions of the two feature maps, the output feature map is still 6x6x7. The size of the output feature map remains unchanged, but the corresponding pixel image data of the output feature map is different from the original two intermediate layers. It better incorporates the feature information of the two intermediate layers, making the network feature information more reasonable and higher in accuracy. Therefore, these are two different methods to enhance the feature information of the feature map.

[0058] 2. Route and Shortcut Operator State Machine and Code Implementation

[0059] Both Route and Shortcut operators are implemented under the entire state machine framework, and use the instruction set configuration and decoding method. The only difference is that different operators have different parameter codes in the instruction set, and the early configuration and decoding processes of the system are similar. Because both the Route operator and the Shortcut operator process two layers of input feature maps, in order to simplify the code and program operation, and different operators can be used for different networks, the two operators are integrated together. It is equivalent to integrating the processes of both operators into the same state at the beginning, and then separating them into their own different processing states when processing specific vector data. The specific state machine working process is as follows. Figure 3 shown.

[0060] Figure 3The state machine in the upper middle is the complete state machine of the Route operator, and the one below is the complete state machine of the Shortcut operator. The states on the left in the state diagram show the control and configuration process of the early system. STATE_CFG_RAM is the process of storing instruction set information in an SRAM, which includes various information of the input feature map, such as which two layers are used for Route or Shortcut, the size of the feature maps of the two target layers, etc. After the instruction set information is configured, the state machine will decode it in the STATE_DECODE_INST state to analyze what action to take next. Because there are many functional layers in the convolutional neural network, including convolution, pooling, full connection, etc., Route and Shortcut in this design are two of them, so this instruction set method can flexibly add or delete operator functions. Figure 3 As shown in , when the state machine decodes the network layer type into the instruction set as Route or Shortcut, it will jump to the STATE_SHORT_ROUTE state. It can be noticed that there is only one state that is different between the two state machine diagrams, that is, STATE_ROUTE_READ and STATE_SHORT_READ. This is because both operators are processing the other two feature maps, so the early process of indexing the starting position of the target network layer from the off-chip DDR is the same, except that the specific parameters are different. However, the two processes of reading feature maps are also different. Route is to transfer the data of the two feature maps into the SRAM for splicing, and then write back to the DDR after obtaining the target output data. Shortcut is to transfer the data of the two layers of feature maps into the system's computing unit PE (Processing Element, PE) for corresponding addition, and then write back to the DDR after obtaining the data of the new output feature map. Therefore, the operations performed in the reading process states of the two state machines are different. The early extraction configuration and the later submission and write back states are the same. Part of the code is as follows Figure 4 shown.

[0061] Both operators, Route and Shortcut, employ batch processing, enabling data processing for multiple images simultaneously. Furthermore, the use of instruction-based control allows for the addition of different operator modules, increasing system integration flexibility. This approach leverages the similarities between the two operators, enabling resource reuse in both early system configuration and target addressing, and later submission modules. This simplifies the system architecture, reduces resource overhead, and improves performance. The front-end simulation of the present invention passed, and back-end FPGA verification also confirmed correct results.

[0062] Example 2

[0063] According to another embodiment of the present invention, a vector fusion computing device for use in neural network data processing is provided, comprising:

[0064] The encoding unit is used to encode two different route operators and shortcut operators into different parameters in the instruction set using instruction set control;

[0065] The decoding unit is used to control the operation of the system through the state machine. The SRAM of the instruction set stores the functional information of each layer of the network, and then decodes it in the decoding state of the state machine;

[0066] The operator corresponding processing unit is used to perform corresponding operator processing when the operator type of the decoded instruction set is Route or Shortcut.

[0067] The vector fusion computing device used in the neural network data processing in the embodiment of the present invention adopts the instruction set control method to encode two different Route operators and Shortcut operators into different parameters in the instruction set, and controls the operation of the system through the state machine method. The SRAM of the instruction set stores the functional information of each layer of the network, and then decodes it in the decoding state of the state machine. When the operator type of the decoded instruction set is Route or Shortcut, the corresponding operator processing is then performed. Because both the Route and Shortcut operators process data of the two middle layers in the network layer, whether it is splicing or adding corresponding position data, the feature information of the output feature map is increased, and the subsequent processing accuracy of the image will be improved.

[0068] The vector fusion computing device for neural network data processing according to the present invention is described in detail below with reference to a specific embodiment.

[0069] In order to solve the above technical problems, the present invention designs two different targeted operators, which use different operator processing according to different situations or different network structures. They are Route and Shortcut operators. The former splices network data from different layers so that the output network layer can take into account the initial feature information. The latter adds the network data from the two middle layers to obtain the new output feature map image data. The most similar to the present invention is the current residual network implementation method, which performs calculations by doping different residual blocks in the network.

[0070] The present invention adopts an instruction set control method, encoding two different operators into different parameters in the instruction set, and then controlling the operation of the system through a state machine. When the chip is working, the SRAM of the instruction set will store the functional information of each layer of the network, including the type of functional layer and operator, the size of the input feature map, etc., and then decode it in the decoding state of the state machine. When the operator type of the decoded instruction set is Route or Shortcut, the corresponding operator processing is then performed. Moreover, because both the Route and Shortcut operators process data of the two middle layers in the network layer, whether it is splicing or adding corresponding position data, the feature information of the output feature map is increased, and the subsequent processing accuracy of the image will be improved.

[0071] 1. Design principles of the two operators

[0072] The difference of the Route operator is that it combines the feature information of multiple layers of feature maps, and splices their data into a new feature map. The feature map not only increases the amount of feature information data, but also combines feature information from different network layers. For example, if you want to process the output of the 15th layer, then if you only use the output of the 15th layer, it is inevitable that some information from the first 14 layers will be lost, resulting in insufficient image accuracy or even errors. At this time, you need to find a way to extract some of the previous information, so the present invention uses a splicing method. For example, the present invention can select the output of the fourth layer or the output of the tenth layer and the output of the fifteenth layer to splice them together, and the pixel data corresponding to the image are spliced ​​according to the depth. The length and width of the image remain unchanged, but the depth becomes the sum of the original two layers. The resulting output feature map will contain more feature information. The specific working principle process is as follows. Figure 1 shown.

[0073] Figure 1 The left side of the figure is the middle layer of the neural network. When performing the Route operation, the two middle layers are regarded as input feature maps, and then the image data of the two layers are spliced ​​according to the corresponding positions to obtain the output feature map on the far right. Figure 1 For illustrative purposes, the two input feature maps on the left are both sized relatively small, 5x5x6. This means that the input feature maps have a height and width of 5 and six channels. Finally, after the Route operation, the resulting output feature map is sized 5x5x12. Of course, in real-world scenarios, images are unlikely to be this small; they may be larger and have more channels. However, the principle remains the same. This is a simple example to illustrate the process.

[0074] The Shortcut operator also processes the two intermediate network layers, but unlike the Route operator, the most obvious difference between the Shortcut operator and the Route operator is that the data of the corresponding pixels of the feature map has changed. That is, the data of the corresponding pixels of the new output feature map is obtained by adding the data of the corresponding pixels of the two intermediate network layers. The working principle is as follows: Figure 2 As shown:

[0075] like Figure 2 As shown in the figure, on the left are two 6x6x7 input feature maps. After adding the data of the corresponding pixel positions of the two feature maps, the output feature map is still 6x6x7. The size of the output feature map remains unchanged, but the corresponding pixel image data of the output feature map is different from the original two intermediate layers. It better incorporates the feature information of the two intermediate layers, making the network feature information more reasonable and higher in accuracy. Therefore, two different methods are used to enhance the feature information of the feature map.

[0076] 2. Route and Shortcut Operator State Machine and Code Implementation

[0077] Both Route and Shortcut operators are implemented under the entire state machine framework, and use the instruction set configuration and decoding method. The only difference is that different operators have different parameter codes in the instruction set, and the early configuration and decoding processes of the system are similar. Because both the Route operator and the Shortcut operator process two layers of input feature maps, in order to simplify the code and program operation, and different operators can be used for different networks, the two operators are integrated together. It is equivalent to integrating the processes of both operators into the same state at the beginning, and then separating them into their own different processing states when processing specific vector data. The specific state machine working process is as follows. Figure 3 shown.

[0078] Figure 3The state machine in the upper middle is the complete state machine of the Route operator, and the one below is the complete state machine of the Shortcut operator. The states on the left in the state diagram show the control and configuration process of the early system. STATE_CFG_RAM is the process of storing instruction set information in an SRAM, which includes various information of the input feature map, such as which two layers are used for Route or Shortcut, the size of the feature maps of the two target layers, etc. After the instruction set information is configured, the state machine will decode it in the STATE_DECODE_INST state to analyze what action to take next. Because there are many functional layers in the convolutional neural network, including convolution, pooling, full connection, etc., Route and Shortcut in this design are two of them, so this instruction set method can flexibly add or delete operator functions. Figure 3 As shown in , when the state machine decodes the network layer type into the instruction set as Route or Shortcut, it will jump to the STATE_SHORT_ROUTE state. It can be noticed that there is only one state that is different between the two state machine diagrams, that is, STATE_ROUTE_READ and STATE_SHORT_READ. This is because both operators are processing the other two feature maps, so the early process of indexing the starting position of the target network layer from the off-chip DDR is the same, except that the specific parameters are different. However, the two processes of reading feature maps are also different. Route is to transfer the data of the two feature maps into the SRAM for splicing, and then write back to the DDR after obtaining the target output data. Shortcut is to transfer the data of the two layers of feature maps into the system's computing unit PE (Processing Element, PE) for corresponding addition, and then write back to the DDR after obtaining the data of the new output feature map. Therefore, the operations performed in the reading process states of the two state machines are different. The early extraction configuration and the later submission and write back states are the same. Part of the code is as follows Figure 4 shown.

[0079] Both operators, Route and Shortcut, employ batch processing, enabling data processing for multiple images simultaneously. Furthermore, the use of instruction-based control allows for the addition of different operator modules, increasing system integration flexibility. This approach leverages the similarities between the two operators, enabling resource reuse in both early system configuration and target addressing, and later submission modules. This simplifies the system architecture, reduces resource overhead, and improves performance. The front-end simulation of the present invention passed, and back-end FPGA verification also confirmed correct results.

[0080] Example 3

[0081] A storage medium stores a program file capable of implementing any one of the above-mentioned vector fusion calculation methods applied in neural network data processing.

[0082] Example 4

[0083] A processor is used to run a program, wherein when the program is run, any one of the above-mentioned vector fusion calculation methods applied in neural network data processing is executed.

[0084] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0085] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the system embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0087] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0088] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0089] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0090] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A vector fusion calculation method applied in neural network data processing, characterized in that: The following steps are involved: Using instruction set control, two different Route operators and Shortcut operators are encoded into different parameters in the instruction set; The operation of the system is controlled by a state machine. The SRAM of the instruction set stores the functional information of each layer of the network, which is then decoded in the decoding state of the state machine. When the operator type of the decoded instruction set is Route or Shortcut, the corresponding operator processing is performed; among them: The Route operator splices network data from different layers so that the output network layer takes into account the initial feature information. The Route operator combines the feature information of multiple layers of feature graphs and splices their data into a new feature graph. The new feature graph not only increases the amount of feature information data, but also combines feature information from different network layers. The Shortcut operator obtains the image data of the new output feature map by adding the network data of the two middle layers; the data of the corresponding pixel points of the feature map in the Shortcut operator changes, and the data of the corresponding pixel points of the new output feature map is obtained by adding the data of the corresponding pixel points of the two middle network layers.

2. The vector fusion calculation method for application in neural network data processing according to claim 1, characterized in that: In the method, the processes of two different Route operators and Shortcut operators are initially integrated into the same state, and then separated into different processing states when the specific vector data are processed.

3. The vector fusion calculation method for application in neural network data processing according to claim 2, characterized in that: In the method, both the Route operator and the Shortcut operator are implemented under the entire state machine framework, and adopt the instruction set configuration and decoding method. The Route operator and the Shortcut operator have the same configuration and decoding process in the early stage of the system, but different parameter codes in the instruction set.

4. The vector fusion calculation method for application in neural network data processing according to claim 1, characterized in that: The method specifically includes: First, the system performs the early control and configuration process and stores the instruction set information in SRAM through STATE_CFG_RAM; After the instruction set information is configured, the state machine decodes it in the STATE_DECODE_INST state to analyze what action to take next; When the state machine decodes the network layer type into the instruction set as Route or Shortcut, it will jump to the STATE_SHORT_ROUTE state; the process of the Route operator or the Shortcut operator in the early stage of indexing the starting position of the target network layer from the off-chip DDR is the same, except that the specific parameters are different. Route is to transfer the data of the two feature maps into the SRAM for splicing, obtain the target output data and then write it back to the DDR, while Shortcut is to transfer the data of the two layers of feature maps into the system's computing unit PE for corresponding addition, obtain the data of the new output feature map and then write it back to the DDR.

5. The vector fusion calculation method for application in neural network data processing according to claim 1, characterized in that: The SRAM of the instruction set stores functional information of each layer of the network, including the type of functional layer and operator, and the size of the input feature map.

6. A vector fusion computing device for use in neural network data processing using the vector fusion computing method for use in neural network data processing according to claim 1, characterized in that: include: The encoding unit is used to encode two different route operators and shortcut operators into different parameters in the instruction set using instruction set control; The decoding unit is used to control the operation of the system through the state machine. The SRAM of the instruction set stores the functional information of each layer of the network, and then decodes it in the decoding state of the state machine; The operator corresponding processing unit is used to perform corresponding operator processing when the operator type of the decoded instruction set is Route or Shortcut.

7. A storage medium, characterized in that: The storage medium stores a program file capable of implementing the vector fusion calculation method applied in neural network data processing as described in any one of claims 1 to 5.

8. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the vector fusion calculation method applied in neural network data processing as described in any one of claims 1 to 5.

Images