Method and apparatus for multi-scale convolution, terminal device and storage medium

Abstract

The application is suitable for the technical field of convolution processing, and provides a multi-scale convolution method, device, terminal equipment and storage medium, the method comprising: inputting a to-be-processed matrix into a preset channel convolutional neural network, the convolutional neural network comprising a convolution group corresponding to the preset channel; performing multi-scale convolution calculation on the to-be-processed matrix through the convolution group, and outputting a target feature matrix. The above scheme improves the extraction capability of the convolutional neural network for the multi-scale features of the to-be-processed matrix through multiple group convolutions. Without additional increase of convolution layers of different scales, the processing effect of the model can be improved without increasing the calculation amount.

Description

Technical Field

[0001] This application belongs to the technical field of convolution processing, and particularly relates to a method, apparatus, terminal device and storage medium for multi-scale convolution. Background Technology

[0002] Convolutional Neural Networks (CNNs) are a type of feedforward neural network. CNNs have been widely applied in all aspects of computer vision, encompassing techniques such as object detection, semantic segmentation, image classification, and image restoration. With continuous technological advancements, the demands on the processing performance of CNNs are increasing. To improve the image processing performance of CNNs, traditional CNNs often employ multi-scale convolutions for feature extraction. While this improves the processing performance, multi-scale convolutions require adding convolutional layers of different scales, resulting in a significant computational burden. This is a problem that urgently needs to be addressed. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method, apparatus, terminal device and storage medium for multi-scale convolution, which can solve the technical problem that multi-scale convolution requires adding convolutional layers of different scales, resulting in a large amount of computation.

[0004] A first aspect of this application provides a method for multi-scale convolution, including:

[0005] The matrix to be processed is input into a convolutional neural network according to a preset channel. The convolutional neural network includes convolutional groups corresponding to the preset channels.

[0006] The target feature matrix is ​​output by performing multi-scale convolution calculations on the matrix to be processed through convolution groups.

[0007] A second aspect of this application provides an apparatus for multi-scale convolution, comprising:

[0008] The first processing unit is used to input the matrix to be processed into the convolutional neural network according to the preset channels. The convolutional neural network includes convolutional groups corresponding to the preset channels.

[0009] The second processing unit is used to perform multi-scale convolution calculations on the matrix to be processed through convolution groups and output the target feature matrix.

[0010] A third aspect of this application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method described in the first aspect.

[0011] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0012] The beneficial effects of this application's embodiments compared to existing technologies are as follows: This application inputs the matrix to be processed into a convolutional neural network according to preset channels, and the convolutional neural network includes convolutional groups corresponding to the preset channels; the convolutional groups perform multi-scale convolution calculations on the matrix to be processed, and output the target feature matrix. The above scheme improves the multi-scale feature extraction capability of the convolutional neural network of the matrix to be processed by using multiple groups of convolutions. No additional convolutional layers of different scales are needed, thus improving the model's processing performance without increasing computational load. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A schematic flowchart of a multi-scale convolution method provided in this application is shown;

[0015] Figure 2 A schematic diagram illustrating the connection relationships between conventional convolutions provided in this application is shown;

[0016] Figure 3 A schematic diagram of the group convolution provided in this application is shown;

[0017] Figure 4 A schematic diagram of the group convolution provided in this application is shown;

[0018] Figure 5 A schematic diagram of the group convolution provided in this application is shown;

[0019] Figure 6 A schematic diagram of the first arrangement of group convolutions provided in this application is shown;

[0020] Figure 7 A schematic diagram of the second arrangement of group convolutions provided in this application is shown;

[0021] Figure 8 A schematic diagram of the third convolutional group arrangement provided in this application is shown;

[0022] Figure 9 A schematic diagram of the uniform sparse group convolution provided in this application is shown;

[0023] Figure 10 A schematic diagram of the non-uniform sparse group convolution provided in this application is shown;

[0024] Figure 11 A schematic diagram of the reverse sparse group convolution provided in this application is shown;

[0025] Figure 12 A schematic diagram of the staggered sparse group convolution provided in this application is shown;

[0026] Figure 13 A schematic diagram of the single-channel sparse group convolution provided in this application is shown;

[0027] Figure 14 A schematic diagram of a multi-scale convolution apparatus provided in this application is shown;

[0028] Figure 15 A schematic diagram of a terminal device provided in one embodiment of this application is shown. Detailed Implementation

[0029] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0030] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0031] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0032] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0033] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0034] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0035] To improve the image processing performance of convolutional neural networks (CNNs), traditional CNNs often employ multi-scale convolution for feature extraction. While this improves the processing performance, multi-scale convolution requires adding convolutional layers of different scales, resulting in a significant computational burden.

[0036] This application provides a method, apparatus, terminal device, and computer-readable storage medium for multi-scale convolution, thereby solving the aforementioned problems.

[0037] Please see Figure 1 , Figure 1 A schematic flowchart of a multi-scale convolution method provided in this application is shown.

[0038] like Figure 1 As shown, the method may include the following steps:

[0039] Step 101: The terminal device inputs the matrix to be processed into the convolutional neural network according to the preset channels. The convolutional neural network includes convolutional groups corresponding to the preset channels.

[0040] The matrix to be processed refers to a matrix composed of multiple data channels. The matrix to be processed includes, but is not limited to, images (images are composed of multiple image channels), etc.

[0041] Specifically, the matrix to be processed includes the image to be processed. To better explain the technical solution of this application, this application uses the image to be processed as an example to illustrate the technical solution of this application.

[0042] The image to be processed is an image with multiple image channels. The number of image channels in the image to be processed is the same as the number of input neurons in the convolutional neural network, so that the convolutional neural network can process the image to be processed.

[0043] Step 102: The terminal device performs multi-scale convolution calculations on the matrix to be processed through convolution groups and outputs the target feature matrix.

[0044] To better explain the convolutional neural network in this embodiment, after briefly explaining conventional convolution, this embodiment focuses on the improvements made to conventional convolution.

[0045] First, please see Figure 2 , Figure 2 A schematic diagram illustrating the connection relationships between conventional convolutions provided in this application is shown. For example... Figure 2 As shown, Figure 2 The diagram includes three block groups: A, B, and C. Block group A represents the input neurons for 16 image channels. Block group C represents the output neurons for 16 image channels. Block group B represents interconnected convolutional layers. Each block in block group B represents a convolutional layer, and each convolutional layer has the same receptive field.

[0046] Because conventional convolution uses the same receptive field, it cannot capture comprehensive features, resulting in poor image processing performance. Therefore, this embodiment improves upon conventional convolution by using group convolution to acquire image features at different scales, thereby enhancing image processing performance. Specifically, group convolutions along different image channels form a convolutional neural network. For example, 16 image channels are divided into 4 groups, with each group responsible for convolution calculations across four channels. The receptive fields differ between different convolution groups, but the receptive fields of convolutional layers within the same group are identical. Convolution groups with different receptive fields are evenly distributed between the input neurons and their corresponding output neurons within the convolutional neural network.

[0047] Specifically, in a convolutional neural network, convolutional groups corresponding to different receptive fields are arranged in different orders among multiple sets of input neurons and multiple sets of output neurons.

[0048] Because existing deep learning frameworks can only handle groups of convolutions arranged sequentially along the image channel direction. See also... Figure 3 , Figure 3 A schematic diagram of the group convolution provided in this application is shown. Please refer to... Figure 4 , Figure 4 A schematic diagram of the group convolution provided in this application is shown. Wherein, Figure 3 and Figure 4 This is merely an example; for Figure 3 and Figure 4 There are no limitations on the number of image channels or the number of group convolutions. For example... Figure 3 and Figure 4As shown, when the convolutional groups are arranged sequentially along the image channel direction, they conform to the form of group convolution, and therefore group convolution calculation can be performed directly. However, when the convolutional groups are not arranged sequentially along the image channel direction, please refer to [the relevant documentation / reference needed]. Figure 5 , Figure 5 A schematic diagram of the group convolution provided in this application is shown. Wherein, Figure 5 This is merely an example; for Figure 5 There are no limitations on the number of image channels or the number of group convolutions. For example... Figure 5 As shown, when the convolutional groups are not arranged sequentially along the channel directions, group convolution cannot be performed directly. It is necessary to convolve the four groups separately to form feature maps along different channel directions, and then perform the next group convolution at different scales. Before performing the next group convolution at different scales, the feature maps along different channel directions need to be matched sequentially with different convolutional groups. This matching process involves feature map transfer, resulting in inference time. Therefore, the convolutional groups corresponding to different receptive fields need to be arranged in a variable order among multiple input neurons and multiple output neurons. This maximizes the arrangement of group convolutions to avoid the inference time caused by feature map transfer. This embodiment provides two arrangement methods:

[0049] First arrangement method:

[0050] Specifically, in a convolutional neural network, the order of convolutional groups between the first group of input neurons and the corresponding first group of output neurons is: first convolutional group, second convolutional group, third convolutional group, and fourth convolutional group. Similarly, the order of convolutional groups between the second group of input neurons and the corresponding second group of output neurons is: fourth convolutional group, first convolutional group, second convolutional group, and third convolutional group. The order of convolutional groups between the third group of input neurons and the corresponding third group of output neurons is: third convolutional group, fourth convolutional group, first convolutional group, and second convolutional group. Finally, the order of convolutional groups between the fourth group of input neurons and the corresponding fourth group of output neurons is: second convolutional group, third convolutional group, fourth convolutional group, and first convolutional group.

[0051] To better illustrate the first arrangement, please refer to [link / reference]. Figure 6 , Figure 6 This diagram illustrates the first arrangement of group convolutions provided in this application. Wherein, Figure 6 This is merely an example; for Figure 6 There are no limitations on the number of image channels or the number of group convolutions. For example... Figure 6As shown, the convolutional neural network is divided into four groups along the image channel direction. The 4x4 convolutional group formed by square 1 is the first convolutional group, the 4x4 convolutional group formed by square 2 is the second convolutional group, the 4x4 convolutional group formed by square 3 is the third convolutional group, and the 4x4 convolutional group formed by square 4 is the fourth convolutional group. From top to bottom, the convolutional groups between the first group of input neurons and the first group of output neurons are arranged in the following order: first convolutional group, second convolutional group, third convolutional group, and fourth convolutional group. The convolutional groups between the second group of input neurons and the second group of output neurons are arranged in the following order: fourth convolutional group, first convolutional group, second convolutional group, and third convolutional group. The convolutional groups between the third group of input neurons and the third group of output neurons are arranged in the following order: third convolutional group, fourth convolutional group, first convolutional group, and second convolutional group. The convolutional groups between the fourth group of input neurons and the fourth group of output neurons are arranged in the following order: fourth convolutional group, first convolutional group, second convolutional group, and third convolutional group. That is, the first arrangement is a sorting method where the diagonal lines are the same convolutional groups.

[0052] The second arrangement method:

[0053] Specifically, in a convolutional neural network, the order of convolutional groups between the first group of input neurons and the corresponding first group of output neurons is: first convolutional group, second convolutional group, third convolutional group, and fourth convolutional group. Similarly, the order of convolutional groups between the second group of input neurons and the corresponding second group of output neurons is: second convolutional group, first convolutional group, fourth convolutional group, and third convolutional group. The order of convolutional groups between the third group of input neurons and the corresponding third group of output neurons is: third convolutional group, fourth convolutional group, first convolutional group, and second convolutional group. Finally, the order of convolutional groups between the fourth group of input neurons and the corresponding fourth group of output neurons is: fourth convolutional group, third convolutional group, second convolutional group, and first convolutional group.

[0054] To better illustrate the second arrangement, please refer to [link / reference]. Figure 7 , Figure 7 A schematic diagram of the second arrangement of group convolutions provided in this application is shown. Wherein, Figure 7 This is merely an example; for Figure 7 There are no limitations on the number of image channels or the number of group convolutions. For example... Figure 7As shown, the convolutional neural network is divided into four groups along the image channel direction. The 4x4 convolutional group formed by square 1 is the first convolutional group, the 4x4 convolutional group formed by square 2 is the second convolutional group, the 4x4 convolutional group formed by square 3 is the third convolutional group, and the 4x4 convolutional group formed by square 4 is the fourth convolutional group. From top to bottom, the convolutional groups between the first group of input neurons and the first group of output neurons are arranged in the following order: first convolutional group, second convolutional group, third convolutional group, and fourth convolutional group. The convolutional groups between the second group of input neurons and the second group of output neurons are arranged in the following order: second convolutional group, first convolutional group, fourth convolutional group, and third convolutional group. The convolutional groups between the third group of input neurons and the third group of output neurons are arranged in the following order: third convolutional group, fourth convolutional group, first convolutional group, and second convolutional group. The convolutional groups between the fourth group of input neurons and the fourth group of output neurons are arranged in the following order: fourth convolutional group, third convolutional group, second convolutional group, and first convolutional group. That is, the second arrangement is a sorting method with orthogonal diagonals as the same convolution group.

[0055] It is understandable that the first and fourth convolutional groups in the second arrangement can be directly implemented using group convolution. The arrangement of the third convolutional group in the second arrangement is as follows: Figure 8 As shown, please refer to Figure 8 , Figure 8 A schematic diagram of the third convolutional group arrangement provided in this application is shown. Wherein, Figure 8 This is merely an example; for Figure 8 There are no limitations on the number of image channels or the number of group convolutions. For example... Figure 8 As shown, the two third convolutional groups in the lower left corner and the two third convolutional groups in the upper right corner each conform to the arrangement of group convolutions. Therefore, in this embodiment, the two third convolutional groups in the lower left corner and the two third convolutional groups in the upper right corner are grouped to conform to the arrangement of group convolutions. That is, the two third convolutional groups in the lower left corner undergo separate group convolution processing, and the two third convolutional groups in the upper right corner undergo separate group convolution processing. In this way, the convolutional groups conform to the arrangement of group convolutions, eliminating the need for data transfer of feature maps, reducing transfer time, and improving the processing efficiency of the model. Similarly, the second convolutional group in the second arrangement can also adopt the above method.

[0056] Therefore, the second arrangement method is more efficient than the first arrangement method because it does not require data transfer.

[0057] To enable different convolutional groups to have different receptive fields, this implementation employs the following two methods to achieve different receptive fields:

[0058] The first method:

[0059] Specifically, the first convolutional group uses a convolutional kernel of a first size; the second convolutional group uses a convolutional kernel of a second size; the third convolutional group uses a convolutional kernel of a third size; and the fourth convolutional group uses a convolutional kernel of a fourth size. The first, second, third, and fourth sizes are all different from each other. The convolutional group includes a preset number of convolutional kernels. The receptive fields within the same convolutional group are the same, while the receptive fields between different convolutional groups are different.

[0060] That is, the first method is to set the first convolutional group, the second convolutional group, the third convolutional group, and the fourth convolutional group to different convolutional kernels, so that different convolutional groups have different receptive fields.

[0061] The second method:

[0062] Specifically, the first, second, third, and fourth convolutional groups all use fifth-size convolutional kernels, and each of them uses a different kernel dilation factor. Each convolutional group includes a preset number of convolutional kernels, with the same receptive field within the same convolutional group and different receptive fields between different convolutional groups.

[0063] That is, the second method is to set the first, second, third and fourth convolutional groups to the same convolutional kernel, and use different convolutional kernel dilation factors for different convolutional groups, so that different convolutional groups have different receptive fields.

[0064] This embodiment involves inputting the matrix to be processed into a convolutional neural network (CNN) according to preset channels. The CNN includes convolutional groups corresponding to the preset channels. The convolutional groups perform multi-scale convolution calculations on the matrix to be processed, outputting a target feature matrix. This approach improves the CNN's ability to extract multi-scale features from the matrix to be processed through multiple groups of convolutions. Since no additional convolutional layers of different scales are needed, the model's processing performance can be improved without increasing computational cost.

[0065] Optionally, this application also provides various sparse group convolutions. Sparse group convolutions include, but are not limited to, uniform sparse group convolutions, non-uniform sparse group convolutions, reverse sparse group convolutions, staggered sparse group convolutions, and single-channel sparse group convolutions.

[0066] Please see Figure 9 , Figure 9 A schematic diagram of the uniform sparse group convolution provided in this application is shown. Figure 9 As shown, uniformly sparse group convolutions contain only four groups of convolutions (i.e., convolutional layers other than the four groups of convolutions are removed). Each group of convolutions corresponds to a different image channel and uses a different receptive field to achieve the effect of multi-scale convolution. Figure 9 This is merely an example; for Figure 9 There are no restrictions on the number of image channels or the number of group convolutions.

[0067] Please see Figure 10 , Figure 10 A schematic diagram of the non-uniform sparse group convolution provided in this application is shown. Figure 10 As shown, non-uniform sparse group convolutions contain only three groups of convolutions (i.e., convolutional layers other than these three groups are removed). Each group of convolutions corresponds to a different image channel and uses a different receptive field to achieve the effect of multi-scale convolution. Figure 10 This is merely an example; for Figure 10 There are no restrictions on the number of image channels or the number of group convolutions.

[0068] Please see Figure 11 , Figure 11 A schematic diagram of the inverse sparse group convolution provided in this application is shown. For example... Figure 11 As shown, the inverse sparse group convolution contains only three groups of convolutions (i.e., convolutional layers other than these three groups are removed). Each group of convolutions corresponds to a different image channel and uses a different receptive field to achieve the effect of multi-scale convolution. Figure 11 This is merely an example; for Figure 11 There are no restrictions on the number of image channels or the number of group convolutions.

[0069] Please see Figure 12 , Figure 12 A schematic diagram of the staggered sparse group convolution provided in this application is shown. Figure 12 As shown, the interleaved sparse group convolution contains only four groups of convolutions (i.e., convolutional layers other than the four groups of convolutions are removed). Each group of convolutions corresponds to a different image channel and uses a different receptive field to achieve the effect of multi-scale convolution. Figure 12 This is merely an example; for Figure 12 There are no restrictions on the number of image channels or the number of group convolutions.

[0070] Please see Figure 13 , Figure 13 A schematic diagram of the single-channel sparse group convolution provided in this application is shown. For example... Figure 12 As shown, the single-channel sparse group convolution contains only sixteen groups of convolutions (i.e., convolutional layers other than the sixteen groups of convolutions are removed). Each group of convolutions corresponds to a different image channel and uses a different receptive field to achieve the effect of multi-scale convolution. Figure 13 This is merely an example; for Figure 13 There are no restrictions on the number of image channels or the number of group convolutions.

[0071] The sparse group convolution described above reduces the number of processing parameters in the model by eliminating redundant convolutional layers, thus greatly reducing the computational load.

[0072] like Figure 14 This application provides a device 14 for multi-scale convolution, see [link to device]. Figure 14 , Figure 14 A schematic diagram of a multi-scale convolution apparatus provided in this application is shown, such as... Figure 14 The apparatus shown includes a multi-scale convolution device:

[0073] The first processing unit 141 is used to input the matrix to be processed into a convolutional neural network according to a preset channel. The convolutional neural network includes convolutional groups corresponding to the preset channels.

[0074] The second processing unit 142 is used to perform multi-scale convolution calculations on the matrix to be processed through convolution groups and output the target feature matrix.

[0075] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0076] Figure 15 A schematic diagram of a terminal device according to an embodiment of this application is shown. Figure 15 As shown, a terminal device 15 in this embodiment includes: a processor 150, a memory 151, and a computer program 152 stored in the memory 151 and executable on the processor 150, such as a multi-scale convolution program. When the processor 150 executes the computer program 152, it implements the steps in the various multi-scale convolution method embodiments described above, for example... Figure 1 Steps 101 to 102 are shown. Alternatively, when processor 150 executes computer program 152, it implements the functions of each unit in the above-described device embodiments, for example... Figure 14 Units 141 to 142 are shown.

[0077] For example, computer program 152 can be divided into one or more units, one or more of which are stored in memory 151 and executed by processor 150 to complete this application. The one or more units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 152 in a terminal device 15. For example, computer program 152 can be divided into an acquisition unit and a calculation unit. The specific functions of each unit are as follows:

[0078] The first processing unit is used to input the matrix to be processed into the convolutional neural network according to the preset channels. The convolutional neural network includes convolutional groups corresponding to the preset channels.

[0079] The second processing unit is used to perform multi-scale convolution calculations on the matrix to be processed through convolution groups and output the target feature matrix.

[0080] A terminal device 15 may be a computing device such as a mobile terminal, desktop computer, laptop, handheld computer, or cloud server. A terminal device may include, but is not limited to, a processor 150 and a memory 151. Those skilled in the art will understand that... Figure 15 This is merely an example of a terminal device 15 and does not constitute a limitation on a terminal device 15. It may include more or fewer components than shown, or combine certain components, or different components. For example, a terminal device may also include input / output devices, network access devices, buses, etc.

[0081] The processor 150 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0082] The memory 151 can be an internal storage unit of a terminal device 15, such as a hard disk or RAM of the terminal device 15. The memory 151 can also be an external storage device of the terminal device 15, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 15. Furthermore, the memory 151 can include both internal and external storage units of the terminal device 15. The memory 151 is used to store computer programs and other programs and data required by the terminal device. The memory 151 can also be used to temporarily store data that has been output or will be output.

[0083] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0084] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0085] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] In the embodiments provided in this application, it should be understood that the disclosed terminal devices and methods can be implemented in other ways. For example, the terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0089] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0090] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for multi-scale convolution, characterized in that, include: The matrix to be processed is input into a convolutional neural network according to a preset channel. The convolutional neural network includes convolutional groups corresponding to the preset channels. The matrix to be processed includes an image to be processed. The image to be processed is an image with multiple image channels, and the number of image channels of the image to be processed is the same as the number of input neurons of the convolutional neural network. The convolutional groups corresponding to different receptive fields in the convolutional neural network are arranged in different orders among multiple sets of input neurons and multiple sets of output neurons. The receptive fields within the same convolutional group are the same, and the receptive fields between different convolutional groups are different. The matrix to be processed is subjected to multi-scale convolution calculations by the convolution group to output the target feature matrix. In the convolutional neural network, the order of convolutional groups between the first group of input neurons and the first group of output neurons corresponding to the first group of input neurons is as follows: first convolutional group, second convolutional group, third convolutional group, and fourth convolutional group. Similarly, the order of convolutional groups between the second group of input neurons and the second group of output neurons corresponding to the second group of input neurons is as follows: fourth convolutional group, first convolutional group, second convolutional group, and third convolutional group. Furthermore, the order of convolutional groups between the third group of input neurons and the third group of output neurons corresponding to the third group of input neurons is as follows: third convolutional group, fourth convolutional group, first convolutional group, and second convolutional group. Finally, the order of convolutional groups between the fourth group of input neurons and the fourth group of output neurons corresponding to the fourth group of input neurons is as follows: second convolutional group, third convolutional group, fourth convolutional group, and first convolutional group. Alternatively, in the convolutional neural network, the order of the convolutional groups between the first group of input neurons and the first group of output neurons corresponding to the first group of input neurons is, in sequence, the first convolutional group, the second convolutional group, the third convolutional group, and the fourth convolutional group; the order of the convolutional groups between the second group of input neurons and the second group of output neurons corresponding to the second group of input neurons is, in sequence, the second convolutional group, the first convolutional group, the fourth convolutional group, and the third convolutional group; the order of the convolutional groups between the third group of input neurons and the third group of output neurons corresponding to the third group of input neurons is, in sequence, the third convolutional group, the first convolutional group, and the second convolutional group; and the order of the convolutional groups between the fourth group of input neurons and the fourth group of output neurons corresponding to the fourth group of input neurons is, in sequence, the fourth convolutional group, the third convolutional group, the second convolutional group, and the first convolutional group.

2. The method as described in claim 1, characterized in that, The first convolutional group uses a convolutional kernel of a first size; the second convolutional group uses a convolutional kernel of a second size; the third convolutional group uses a convolutional kernel of a third size; and the fourth convolutional group uses a convolutional kernel of a fourth size; wherein the first size, the second size, the third size, and the fourth size are all different from each other; and the convolutional group includes a preset number of convolutional kernels.

3. The method as described in claim 1, characterized in that, The first convolutional group, the second convolutional group, the third convolutional group, and the fourth convolutional group all use convolutional kernels of the fifth size, and the first convolutional group, the second convolutional group, the third convolutional group, and the fourth convolutional group all use different convolutional kernel dilation coefficients; the convolutional group includes a preset number of convolutional kernels.

4. A device for multi-scale convolution, characterized in that, include: The first processing unit is used to input the matrix to be processed into a convolutional neural network according to a preset channel. The convolutional neural network includes convolutional groups corresponding to the preset channels. The matrix to be processed includes an image to be processed. The image to be processed is an image with multiple image channels, and the number of image channels of the image to be processed is the same as the number of input neurons of the convolutional neural network. The convolutional groups corresponding to different receptive fields in the convolutional neural network are arranged in different orders among multiple sets of input neurons and multiple sets of output neurons. The receptive fields within the same convolutional group are the same, and the receptive fields between different convolutional groups are different. The second processing unit is used to perform multi-scale convolution calculations on the matrix to be processed through the convolution group and output the target feature matrix. In the convolutional neural network, the order of convolutional groups between the first group of input neurons and the first group of output neurons corresponding to the first group of input neurons is as follows: first convolutional group, second convolutional group, third convolutional group, and fourth convolutional group. Similarly, the order of convolutional groups between the second group of input neurons and the second group of output neurons corresponding to the second group of input neurons is as follows: fourth convolutional group, first convolutional group, second convolutional group, and third convolutional group. Furthermore, the order of convolutional groups between the third group of input neurons and the third group of output neurons corresponding to the third group of input neurons is as follows: third convolutional group, fourth convolutional group, first convolutional group, and second convolutional group. Finally, the order of convolutional groups between the fourth group of input neurons and the fourth group of output neurons corresponding to the fourth group of input neurons is as follows: second convolutional group, third convolutional group, fourth convolutional group, and first convolutional group. Alternatively, in the convolutional neural network, the order of the convolutional groups between the first group of input neurons and the first group of output neurons corresponding to the first group of input neurons is, in sequence, the first convolutional group, the second convolutional group, the third convolutional group, and the fourth convolutional group; the order of the convolutional groups between the second group of input neurons and the second group of output neurons corresponding to the second group of input neurons is, in sequence, the second convolutional group, the first convolutional group, the fourth convolutional group, and the third convolutional group; the order of the convolutional groups between the third group of input neurons and the third group of output neurons corresponding to the third group of input neurons is, in sequence, the third convolutional group, the first convolutional group, and the second convolutional group; and the order of the convolutional groups between the fourth group of input neurons and the fourth group of output neurons corresponding to the fourth group of input neurons is, in sequence, the fourth convolutional group, the third convolutional group, the second convolutional group, and the first convolutional group.

5. A terminal device, characterized in that, The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1 to 3.

Images