Image magnification method, device, electronic device and storage medium

Abstract

The present invention discloses an image magnification method, device, electronic device and storage medium, wherein the image magnification method may include: obtaining a first image to be magnified, receiving a magnification corresponding to the first image, determining a convolution kernel for performing at least one convolution process on the first image based on the magnification, performing convolution process on the first image using the convolution kernel to obtain a convolution result; and then generating a second image according to the convolution result, the second image being an image generated by magnifying the first image. The present invention determines the corresponding convolution kernel according to the magnification to realize the conversion of the image magnification process into the operation of a convolutional neural network, thereby avoiding the problems of increased chip area, increased cost and low equipment working efficiency caused by the separate setting of a hardware acceleration module, and avoiding the problem of low processing efficiency existing in conventional programming strategies, and can meet the requirements of occasions such as real-time processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology. More specifically, the present invention can provide an image magnification method, device, electronic equipment and storage medium. Background Art

[0002] At present, the application fields of image processing are becoming more and more extensive, and image enlargement processing is used very frequently in image processing. Image enlargement processing often refers to increasing the image size to achieve the purpose of adapting to the image size requirements of related image processing algorithms. Traditional image enlargement processing solutions mainly include the following two types: (1) dedicated hardware acceleration solution, (2) programming through the main control CPU or DSP. For the enlargement processing solution (1), the dedicated hardware acceleration module needs to occupy the chip area, which increases the implementation cost, and the data to be processed needs to be moved back and forth between different hardware, which not only increases the power consumption of the device, but also reduces the working efficiency of the device; for the enlargement processing solution (2), the conventional programming strategy has the problem of low processing efficiency and cannot be applied to occasions with high requirements for real-time processing. Summary of the invention

[0003] In order to solve the problems of high implementation cost, high device power consumption or low processing efficiency in conventional image magnification processing technology, one or more embodiments of the present invention may provide an image magnification method, device, electronic device and storage medium to achieve technical purposes such as reducing implementation cost and improving processing efficiency.

[0004] To achieve the above technical objectives, the present invention can provide an image magnification method, which may include but is not limited to one or more of the following steps. First, a first image to be magnified is acquired, and then a magnification factor corresponding to the first image is received, and a convolution kernel for performing at least one convolution process on the first image is determined based on the magnification factor, and the first image is convoluted using the convolution kernel to obtain a convolution result; the present invention can generate a second image based on the convolution result, and the second image is an image generated by magnifying the first image.

[0005] To achieve the above technical objectives, the present invention may also provide an image magnification device, which may include but is not limited to an image acquisition module, a multiple receiving module, a convolution kernel determination module, a convolution processing module and an image generation module.

[0006] The image acquisition module is used to acquire a first image to be magnified.

[0007] The magnification receiving module is used to receive the magnification corresponding to the first image.

[0008] A convolution kernel determination module is used to determine a convolution kernel used to perform at least one convolution process on the first image based on the magnification factor.

[0009] The convolution processing module is used to perform convolution processing on the first image using the convolution kernel to obtain a convolution result.

[0010] An image generation module is used to generate a second image according to the convolution result; wherein the second image is an image generated by enlarging the first image.

[0011] To achieve the above technical objectives, the present invention may provide an electronic device, including a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the image magnification method described in any embodiment of the present invention.

[0012] To achieve the above technical objectives, the present invention may also provide a storage medium storing computer-readable instructions, which, when executed by one or more processors, enables the one or more processors to perform the steps of the image magnification method described in any embodiment of the present invention.

[0013] To achieve the above technical objectives, the present invention can also provide a computer program product. When the instructions in the computer program product are executed by a processor, the steps of the image magnification method in any embodiment of the present invention are executed.

[0014] The beneficial effects of the present invention are as follows: the present invention determines the corresponding convolution kernel according to the magnification factor to realize the conversion of image magnification processing into convolutional neural network operations, thereby avoiding the problems of increased chip area, increased cost and low equipment efficiency caused by the separate setting of a hardware acceleration module, and avoiding the problem of low processing efficiency in conventional programming strategies. The present invention can utilize the powerful computing power provided by neural network technology to better realize image magnification processing, and has the advantages of high processing efficiency, low power consumption and low cost. It can meet the requirements of special occasions such as real-time processing without adding a separate hardware module, and has a good user experience and high user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic flow chart of an image magnification method in one or more embodiments of the present invention is shown.

[0016] Figure 2 A schematic diagram of a process for determining a convolution kernel for performing at least one convolution process on a first image based on a magnification in one or more embodiments of the present invention is shown.

[0017] Figure 3A schematic flow chart of a method for implementing image magnification by offline calculation in one or more embodiments of the present invention is shown.

[0018] Figure 4 A schematic diagram showing the principle of the bilinear interpolation method used in one or more embodiments of the present invention.

[0019] Figure 5 A schematic diagram showing the target point position of 2x interpolation in one or more embodiments of the present invention.

[0020] Figure 6 A schematic diagram showing the structural composition of an image magnifying device in one or more embodiments of the present invention is shown.

[0021] Figure 7 A schematic diagram of the internal structure of an electronic device in one or more embodiments of the present invention is shown. DETAILED DESCRIPTION

[0022] The image magnification method, device, electronic device and storage medium provided by the present invention are explained and illustrated in detail below in conjunction with the drawings in the specification.

[0023] like Figure 1 As shown, one or more embodiments of the present invention can provide an image magnification method, which may include but is not limited to one or more of the following steps.

[0024] Step 100, obtaining a first image to be magnified. In some embodiments of the present invention, the first image may be a source image, that is, an unprocessed image, for example, an image captured by a camera for face recognition; the first image may also be an image that needs to be further magnified after undergoing a preprocessing process. The magnification process involved in the present invention may include one or more image magnification processes.

[0025] Step 200, receiving a magnification corresponding to the first image; the magnification may be a magnification input or selected by a user. It should be understood that the magnification involved in the present invention may be an integer, in which case the present invention can provide an integer multiple image magnification method; of course, the magnification involved in the present invention may also be a non-integer, in which case the non-integer may be split into multiple integer combinations and then magnified multiple times to achieve non-integer multiple image magnification.

[0026] Step 300, based on the magnification, determine a convolution kernel for performing at least one convolution process on the first image to be magnified. Specifically, in the embodiment of the present invention, receiving the magnification corresponding to the first image includes the following steps: determining whether the magnification is an integer: when the magnification is an integer, executing the subsequent step of determining the convolution kernel; when the magnification is a non-integer, the present invention splits the magnification into a combination of multiple integers, and the multiple integers obtained by splitting can be used as magnifications and correspondingly execute the subsequent step of determining the convolution kernel. For example, for the case of 1.5 times magnification, the present invention can split it into 3÷2. In specific implementation, the step of magnification by three times can be performed first and then the step of reduction by two times. Of course, the present invention can also perform the step of reduction by two times and then the step of magnification by three times, so as to achieve the technical purpose of the present invention. It can be understood that any non-integer can be split into the form of multiple integer combinations, such as 0.33=33÷100, which includes two cases of magnification by 33 times and reduction by 100 times, wherein the reduction or magnification process can also be implemented by executing the technical solution of the present invention multiple times, for example, 50 times of the two-fold reduction process is equivalent to reduction by 100 times. The reduction process can be selected from conventional technologies, such as uniformly selecting a number of target points from the target points of the first image for pixel replication, and using the replicated target points to form a second image to achieve image reduction; or the reduction process can be reasonably modified based on the present invention, such as forming a second image by determining a certain number of interpolation points from the first image to achieve image reduction. Of course, the reduction scheme used in the present invention is not limited to the above examples.

[0027] like Figure 2 As shown, in the embodiment of the present invention, determining a convolution kernel for performing at least one convolution process on the first image based on the magnification includes but is not limited to the following steps 301 to 303.

[0028] Step 301, determining a first number of interpolation points formed in the second image by the magnification factor, and determining a second number of interpolation coefficients by the number of neighboring points of the target point in the first image. The neighboring points in the embodiment of the present invention may be a number or all of the pixel points in a preset area centered on the target point position, or the neighboring points in the embodiment of the present invention may also be the pixel points closest to the target point. In one or more embodiments of the present invention, when the image is magnified, if the magnification factor is n, then the first number of interpolation points used for image magnification is n. 2 ; If the number of neighboring points of the target point is m, the second number of interpolation coefficients used for image enlargement is m 2 .

[0029] like Figure 4 As shown, the embodiment of the present invention is described in detail by taking the case of image magnification as an example. Figure 4Point D in the middle represents the position of a target point in the original image, and the coordinates of point D are (x, y); 11 , S 12 , S 21 , S 22 They are the four original image pixels adjacent to point D, i.e. the four adjacent points of the target point, S 11 The coordinates of S are (x1, y1), 12 The coordinates of S are (x1, y2), 21 The coordinates of S are (x2, y1), 22 The coordinates are (x2, y2); the value of point D in this embodiment can be obtained by the following two interpolation methods.

[0030] (1) This embodiment uses adjacent points S 11 , S 21 Linear interpolation is performed in the horizontal direction to obtain M1, and the adjacent point S is used 12 , S 22 M2 is obtained by linear interpolation in the horizontal direction. The present invention relates to c i (x) can represent the value of the pixel corresponding to point x in the i-th image / attribute channel.

[0031]

[0032]

[0033] (2) The embodiment of the present invention uses points M1 and M2 to perform interpolation in the vertical direction to obtain the image value of point D.

[0034]

[0035] like Figure 5 As shown, the embodiment of the present invention is described by taking 2 times interpolation as an example. Figure 5 Midpoint D 2k,2k Fixed at point S k,k 0.25 pixels to the right and bottom, point D 2k+1,2k Fixed at point S k,k 0.75 pixels to the right and 0.25 pixels below, point D 2k,2k+1 Fixed at point S k,k+1 0.25 pixels to the right and above, point D 2k+1,2k+1 Fixed at point S k,k+1 0.75 pixel units to the right and 0.25 pixel units above, etc. Based on this rule, the embodiment of the present invention uses a fixed interpolation coefficient to determine the four points D 2k,2k , D 2k+1,2k , D 2k,2k+1 , D 2k+1,2k+1 The coordinates of .

[0036]

[0037]

[0038]

[0039]

[0040] Based on the above results, the embodiment of the present invention implements the interpolation coefficients of the interpolation points at four positions through four 2×2 convolution kernels:

[0041]

[0042] After determining the first number of interpolation points and the second number of interpolation coefficients, an embodiment of the present invention can generate a convolution kernel based on the first number and the second number, and the generated convolution kernel is used for corresponding convolution calculations, which is specifically described as follows.

[0043] The step of generating a convolution kernel based on the first quantity and the second quantity in the embodiment of the present invention may include but is not limited to the following step 302 and step 303.

[0044] Step 302: The first number is used as the number of convolution kernels. Taking image magnification as an example, in this embodiment, when the magnification factor is n, the first number of interpolation points used for image magnification is n. 2 , that is, the number of convolution kernels is n 2 For example, when the magnification is 2, the first number of interpolation points is 2 2 =4, four convolution kernels, such as the four convolution kernels shown in the above two-fold magnification case of the embodiment of the present invention, are k 11 , k 21 , k 12 , k 22 .

[0045] Step 303, determine the elements in each convolution kernel based on the second number and the distance between the interpolation point and the adjacent point in the second image. Taking the image magnification as an example, when the number of adjacent points of the target point is m, the second number of interpolation coefficients used for image magnification is m 2 For example, when the number of neighboring points of the target point is 4, the second number of interpolation coefficients used for image enlargement is 4. 2 =16. Combined Figure 5 As shown, the elements in the convolution kernel (i.e., the specific values ​​of the interpolation coefficients) can be specifically calculated according to the distance between the interpolation point and the adjacent points. In the embodiment of the present invention, n 2 The convolution kernel can be calculated by the following formula.

[0046]

[0047] w(x) is a function for calculating the interpolation coefficient according to the distance between the target point and the adjacent point of the original image. In this embodiment, w(x) in the case of bilinear interpolation and bicubic interpolation are respectively as follows: bilinear (x) represents w(x) in the case of bilinear interpolation, w bicubic (x) represents w(x) in the case of bicubic interpolation.

[0048] w bilinear (x) = 1-|x|

[0049]

[0050] For n-fold magnification of m×m adjacent point interpolation, the interpolation points in the embodiment of the present invention can be calculated by the following formula.

[0051]

[0052] Among them, x d,p ,y d,q Denotes the interpolation point D nk+p,nk+q The coordinates in the original image, p and q represent the point D nk+p,nk+q The horizontal and vertical relative positions in the original image (there are n relative positions for each, a total of n×n combinations), S k+i,k+j represents the i-th horizontal and j-th vertical adjacent point, x s,i and s,j Represents its horizontal and vertical coordinates respectively.

[0053] For example, the 16 interpolation coefficients shown in the above embodiment of the present invention in the case of two-fold magnification based on bilinear interpolation are respectively Of course, it is not limited to this.

[0054] The present invention may use an interpolation method to determine interpolation points during image magnification. The interpolation method involved in the present invention may specifically include but is not limited to bilinear interpolation, cubic interpolation, Lanczos interpolation, etc. For example, the bilinear interpolation in the embodiment of the present invention may include 2×2 adjacent points, that is, the four adjacent points closest to the target point near the original image position are used for interpolation; the bicubic interpolation in this embodiment may include 4×4 adjacent points, and the Lanczos interpolation may include 8×8 interpolation points, etc. It can be seen that the technical solution provided by the present invention can be applied to bilinear interpolation, bicubic interpolation, Lanczos interpolation and other interpolation methods.

[0055] Step 400, convolution processing is performed on the first image using a convolution kernel to obtain a convolution result. The present invention performs convolution processing on the first image using a convolutional neural network (CNN), and converts the image magnification processing process into a convolution calculation process of the convolutional neural network. The embodiment of the present invention uses a convolution kernel to perform convolution processing on the first image, including: obtaining the number of channels of the first image, using the number of channels as the number of convolution processing times, and using the convolution kernel to perform convolution processing on the first images of different channels. For the c×h×w (feature tensor) image to be magnified n times, wherein c represents the feature channel, h represents the feature height, w represents the feature width, and h×w represents the feature size, the embodiment of the present invention can be divided into c groups (groups) according to the feature channel c, so as to perform convolution processing on each group separately, which is equivalent to repeating the obtained interpolation coefficients c times and organizing them as cn 2 An m×m convolution kernel cn 2 ×1×m×m, The convolution operation is performed and c×n is obtained. 2 ×h×w. The magnification (specifically, the magnification) of the present invention in this embodiment is 2, then the first number of interpolation points formed in the second image is 4, and the number of interpolation coefficients is 16. For example, for a three-channel color image of size 3×h×w, the embodiment of the present invention uses a certain convolution kernel (kernel) on each channel to perform convolution to obtain images of the same size. For example, after convolution calculations are performed on each channel using the four convolution kernels in the above embodiment, images of the same size are obtained. In the embodiment of the present invention, the same size can be maintained by filling the edges with 0 or copying the padding (padding) to obtain an output of 3×4×h×w.

[0056] Of course, in the specific implementation of the image magnification process of the present invention, not only the method of interpolating the pixels in the original image can be adopted, but the present invention may also include a pixel copying process, such as nearest point copying, etc., so as to achieve the technical purpose of the present invention, and this embodiment will not be repeated.

[0057] Based on converting the core processing process of image magnification into the operation process of convolutional neural network, the present invention can implement the method of image magnification on a neural network processor, wherein the neural network processor can be integrated into the device DSP (Digital Signal Process) or other processors. By utilizing the powerful computing power of the neural network processor, the present invention can not only reduce the consumption of DSP computing power, but also avoid the consumption of image data moving back and forth between different hardware accelerators in traditional hardware acceleration solutions, effectively reducing device power consumption and improving image processing efficiency.

[0058] Step 500, generating a second image according to the convolution result, the second image being an image generated by enlarging the first image. Specifically, in one or more embodiments of the present invention, generating the second image according to the convolution result includes: performing data rearrangement processing on the convolution result to obtain the second image. The present invention can 2 The convolution result of ×h×w is processed by data rearrangement to obtain the result of c×nh×nw, that is, the image is upsampled n times by data rearrangement. The embodiment of the present invention performs data rearrangement processing based on the convolution result, has outstanding advantages such as high execution efficiency and rapid acquisition of the enlarged image, and is suitable for occasions with high real-time requirements for image processing.

[0059] In this embodiment, the output of 3×4×h×w is taken as an example, and the output of the four convolution kernels of each channel is rearranged to obtain an output of size 2h×2w, so that the three channels obtain a 2-fold image magnification of 3×2h×2w. The present invention can realize efficient data rearrangement in various scenarios through the DMA (Direct Memory Access) module configured in the neural network processor. One or more embodiments of the present invention can achieve c×n by data rearrangement. 2 ×h×w→c×nh×nw, for example, the above-mentioned 3×4×h×w→3×2h×2w data rearrangement process can be realized by the DMA module. Of course, some embodiments of the present invention can also realize data rearrangement on the CPU or DSP, and can also realize c×n 2 The purpose of ×h×w→c×nh×nw is certainly not limited to this. After obtaining the image magnification result, that is, after obtaining the second image, the present invention can continue to perform subsequent image signal processing.

[0060] It should be understood that some embodiments of the present invention may adopt a method of online generation of a convolution kernel corresponding to the current magnification factor to convert the image magnification process into a convolution operation; of course, embodiments of the present invention can also generate and store the convolution kernel by prior offline calculation, as described below.

[0061] like Figure 3 As shown, in an embodiment of the present invention, determining a convolution kernel for performing at least one convolution process on a first image based on a magnification includes: directly calling a convolution kernel that matches the magnification; wherein multiple convolution kernels are determined by offline calculation. The present invention can save the convolution kernel obtained by offline calculation in advance, so that when using the convolution function of the neural network processor, the convolution kernel that matches the current magnification can be directly called. The present invention can greatly reduce the process of online calculation, thereby further improving the execution efficiency of the present invention, and further improving the efficiency of image magnification processing.

[0062] The present invention can provide an image magnification technical solution for image processing based on a convolutional neural network, which is very suitable for use in the hardware environment of a neural network processor; through the support of the neural network processor for the convolution operation of the CNN network, the present invention can convert integer multiple image magnification into the convolution operation of the convolutional neural network, and can convert non-integer multiple image magnification into multiple integer multiple image magnifications. The present invention can implement a method for integer multiple image magnification through a neural network processor, effectively utilizing the high-speed computing power of the neural network processor, thereby avoiding excessive consumption of computing power of other processors on the chip, and avoiding the problem of developing a dedicated image magnification acceleration module, so as to reduce the chip area and reduce the cost of the neural network processing chip, so the present invention has a wide range of applications and has a very broad market value.

[0063] The technical solution provided by the present invention can be used in scenarios such as image classification and face recognition. Taking the face recognition scenario as an example, when the image needs to be magnified, the present invention can receive the current magnification factor input or selected by the user, and call the pre-stored convolution kernel or generate the required convolution kernel based on the current magnification factor, thereby effectively converting the core process of image magnification processing into a convolution operation, so as to effectively improve the execution efficiency, avoid the problems of occupying chip area, high implementation cost and high equipment power consumption existing in conventional dedicated hardware acceleration solutions, and avoid the problems of low processing efficiency existing in conventional software programming strategies, so as to meet the occasions with high real-time requirements such as face recognition.

[0064] like Figure 6 As shown, based on the same inventive technical concept as the image magnification method, one or more embodiments of the present invention can also provide an image magnification device.

[0065] The image magnification device in the embodiment of the present invention may specifically include but is not limited to at least one of an image acquisition module 601, a multiple receiving module 602, a convolution kernel determination module 603, a convolution processing module 604, and an image generation module 605, which are specifically described as follows.

[0066] The image acquisition module 601 is used to acquire a first image to be enlarged.

[0067] The magnification receiving module 602 is used to receive the magnification corresponding to the first image.

[0068] The convolution kernel determination module 603 may be used to determine a convolution kernel for performing at least one convolution process on the first image based on the magnification. Optionally, the convolution kernel determination module 603 may be used to determine whether the magnification is an integer; the convolution kernel determination module 603 may be used to directly execute the subsequent steps of determining the convolution kernel when the magnification is an integer; the convolution kernel determination module 603 may be used to split the magnification into a combination of multiple integers when the magnification is a non-integer, and to use the multiple integers as magnifications, respectively, and to execute the subsequent steps of determining the convolution kernel accordingly.

[0069] The convolution kernel determination module 603 may be used to determine a first number of interpolation points formed in the second image by the magnification, and to determine a second number of interpolation coefficients by the number of neighboring points of the target point in the first image; the convolution kernel determination module 603 is used to generate a convolution kernel based on the first number and the second number. In this embodiment, the convolution kernel determination module 603 may be used to use the first number as the number of convolution kernels, and to determine the elements in each convolution kernel based on the second number and the distance between the interpolation point and the neighboring point in the second image.

[0070] Optionally, the convolution kernel determination module 603 in some embodiments of the present invention is used to directly call a convolution kernel that matches the magnification factor; wherein multiple convolution kernels are determined by offline calculation.

[0071] The neighboring points in one or more embodiments of the present invention are all pixel points in a preset area centered on the target point position; or, the neighboring points may be the pixel points that are closest to the target point.

[0072] The convolution processing module 604 is used to perform convolution processing on the first image using a convolution kernel to obtain a convolution result. The convolution processing module 604 can be specifically used to obtain the number of channels of the first image, and to use the number of channels as the number of convolution processing times and to perform convolution processing on the first images of different channels using a convolution kernel.

[0073] The image generation module 605 is used to generate a second image according to the convolution result; wherein the second image in the present invention is an image generated by enlarging the first image.

[0074] The image generation module 605 may be specifically configured to perform data rearrangement processing on the convolution result to obtain a second image.

[0075] like Figure 7As shown, based on the same technical concept as the image magnification method, one or more embodiments of the present invention can provide an electronic device, the electronic device includes a memory and a processor, the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the image magnification method in any embodiment of the present invention. It is understandable that the specific implementation process of the image magnification method has been described in detail in this specification and will not be repeated here.

[0076] like Figure 7 As shown, based on the same inventive technical concept as the image magnification method, one or more embodiments of the present invention can also provide a storage medium storing computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the image magnification method in any embodiment of the present invention. It should be understood that the specific implementation process of the image magnification method has been described in detail in this specification and will not be repeated here.

[0077] Based on the same technical concept as the image magnification method, the embodiment of the present invention may also specifically provide a computer program product. When the instructions in the computer program product are executed by a processor, the steps of the image magnification method in any embodiment of the present invention are executed. It should be understood that the specific implementation process of the image magnification method of the present invention has been described in detail in this specification and will not be repeated here.

[0078] The logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, apparatus, or device and execute instructions), or in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, a "computer-readable storage medium" may be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device. More specific examples of computer-readable storage media (a non-exhaustive list) include the following: an electrical connection with one or more wirings (electronic devices), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, Erasable Programmable Read-Only Memory, or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM, Compact Disc Read-Only Memory). In addition, the computer-readable storage medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0079] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA, Programmable Gate Array), a field programmable gate array (FPGA, Field Programmable Gate Array), etc.

[0080] In the description of this specification, the description with reference to the terms "this embodiment", "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0081] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and simple improvements made to the essential contents of the present invention should be included in the protection scope of the present invention.

Claims

1. An image magnification method, characterized in that: include: Acquire a first image to be magnified; receiving a magnification corresponding to the first image; Determining a convolution kernel for performing at least one convolution process on the first image based on the magnification, comprising: determining a first number of interpolation points formed in the second image by the magnification, and determining a second number of interpolation coefficients by the number of neighboring points of the target point in the first image, if the number of neighboring points of the target point is m, then the second number is m 2 ; using the first number as the number of convolution kernels; determining the elements in each convolution kernel based on the second number and the distance between the interpolation point and the adjacent point in the second image; Using the convolution kernel to perform convolution processing on the first image to obtain a convolution result; A second image is generated according to the convolution result, where the second image is an image generated by enlarging the first image.

2. The image magnification method according to claim 1, characterized in that: Generating a second image according to the convolution result comprises: Perform data rearrangement processing on the convolution result to obtain the second image.

3. The image magnification method according to claim 1 or 2, characterized in that: The performing convolution processing on the first image by using the convolution kernel comprises: Get the number of channels of the first image; The number of channels is used as the number of convolution processing times, and the convolution kernel is used to perform convolution processing on the first images of different channels.

4. The image magnification method according to claim 1, characterized in that: The receiving the magnification corresponding to the first image comprises: Determining whether the magnification is an integer; When the magnification is an integer, the subsequent step of determining the convolution kernel is performed; When the magnification factor is a non-integer, the magnification factor is split into a combination of multiple integers, and the multiple integers are used as magnification factors and corresponding steps for performing subsequent convolution kernel determination.

5. An image magnification device, characterized in that: include: An image acquisition module, used for acquiring a first image to be magnified; A magnification receiving module, used for receiving a magnification corresponding to the first image; A convolution kernel determination module, used to determine a convolution kernel for performing at least one convolution process on the first image based on the magnification; The convolution kernel determination module is used to determine a first number of interpolation points formed in the second image according to the magnification, and to determine a second number of interpolation coefficients according to the number of neighboring points of the target point in the first image, if the number of neighboring points of the target point is m, then the second number is m 2 ; A convolution kernel determination module is used to use the first number as the number of convolution kernels, and to determine the elements in each convolution kernel based on the second number and the distance between the interpolation point and the adjacent point in the second image; A convolution processing module, used for performing convolution processing on the first image using the convolution kernel to obtain a convolution result; An image generation module is used to generate a second image according to the convolution result; wherein the second image is an image generated by enlarging the first image.

6. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the image magnification method according to any one of claims 1 to 4.

7. A storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by one or more processors, the one or more processors are enabled to perform the steps of the image magnification method according to any one of claims 1 to 4.

Images