Image processing method and electronic device

Abstract

The present invention relates to the field of image processing technology, and specifically provides an image processing method, an electronic device, an image acquisition device and a storage medium, aiming to solve the problem of a single convolution effect in image processing. To this end, the method of the present invention includes: acquiring an image to be processed based on an image acquisition device, acquiring posture information of the image acquisition device, determining a convolution form based on the posture information, and processing the image to be processed based on the convolution form. Through the above-mentioned implementation, a corresponding convolution form can be selected according to the motion state of the image acquisition device, which increases the flexibility of image processing, can more accurately express the current shooting scene and the shooting semantics that the user wants to convey, expands the shooting effect, and improves the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method, electronic equipment, image acquisition equipment and storage medium. Background Art

[0002] At present, the technology for convolution of captured images is usually relatively fixed, the shooting time cannot be changed, and the convolution effect achieved is relatively simple.

[0003] Accordingly, the art needs a new technical solution to solve the above problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects, the present invention is proposed to provide an image processing method, electronic device, image acquisition device and storage medium that solve or at least partially solve the technical problem of a single convolution effect in image processing.

[0005] In a first aspect, an image processing method is provided, the method comprising:

[0006] Acquire an image to be processed based on an image acquisition device;

[0007] Acquiring position and posture information of the image acquisition device;

[0008] Determine a convolution form based on the posture information;

[0009] The image to be processed is processed based on the convolution form.

[0010] In a technical solution of the above-mentioned image processing method, the posture information includes angular velocity data and acceleration data; and obtaining the posture information of the image acquisition device includes:

[0011] Acquiring the angular velocity data based on a gyro sensor of the image acquisition device;

[0012] The acceleration data is acquired based on an acceleration sensor of the image acquisition device.

[0013] In a technical solution of the above-mentioned image processing method, determining the convolution form based on the posture information includes:

[0014] Processing the angular velocity data and the acceleration data;

[0015] comparing the processed angular velocity data and the acceleration data;

[0016] If the angular velocity data is greater than or equal to the acceleration data, it is determined that the convolution form is a radial convolution; otherwise, it is determined that the convolution form is a rotational convolution.

[0017] In a technical solution of the above-mentioned image processing method, processing the image to be processed based on the convolution form includes:

[0018] Obtaining a current convolution point in the image to be processed;

[0019] Determining an offset step size of convolution sampling based on the convolution form;

[0020] Acquire a current offset sampling point based on the current convolution point and the offset step size, and determine whether the current offset sampling point exceeds a boundary of the image to be processed;

[0021] If so, the position of the current offset sampling point is adjusted according to a preset rule; otherwise, the convolution sampling is performed based on the current offset sampling point.

[0022] In a technical solution of the above-mentioned image processing method, the determining the offset step size of the convolution sampling based on the convolution form includes:

[0023] Acquire a reference point of the convolution sampling from the image to be processed;

[0024] Obtain a direction vector formed by the current convolution point and the reference point, and obtain an angle θ between the direction vector and a horizontal direction; wherein the horizontal direction is the x-axis direction of a pixel coordinate system;

[0025] The offset step size of the convolution sampling is obtained based on the convolution form and the angle θ.

[0026] In a technical solution of the above-mentioned image processing method, the step of obtaining the offset step of the convolution sampling based on the convolution form and the angle θ includes:

[0027] If the convolution form is the radial convolution, the offset step is calculated based on the following formula:

[0028] dx=t*r*cosθ, dy=t*r*sinθ;

[0029] If the convolution form is the rotation convolution, the offset step is calculated based on the following formula:

[0030] dx=t*cos(θ+t*θ0), dy=t*sin(θ+t*θ0);

[0031] Among them, dx is the offset step in the horizontal direction, dy is the offset step in the vertical direction, t is the number of current convolution sampling, r is the displacement increment of a single convolution sampling, θ0 is the angle increment of a single convolution sampling; wherein the vertical direction is the y-axis direction of the pixel coordinate system.

[0032] In a technical solution of the above-mentioned image processing method, the method further includes obtaining a width W and a height H of the image to be processed; and judging whether the current offset sampling point exceeds the boundary of the image to be processed includes:

[0033] If the current offset sampling point meets at least one of the following conditions, it is determined that it exceeds the boundary of the image to be processed;

[0034] x<0; or, x>W; or, y<0; or, y>H;

[0035] Wherein, x is the horizontal coordinate of the current offset sampling point in the pixel coordinate system, and y is the vertical coordinate of the current offset sampling point in the pixel coordinate system.

[0036] In a technical solution of the above-mentioned image processing method, adjusting the position of the current offset sampling point according to a preset rule includes:

[0037] If x<0, then adjust the horizontal coordinate of the current offset sampling point to x=0;

[0038] If x>W, then adjust the horizontal coordinate of the current offset sampling point to x=W;

[0039] If y<0, the ordinate of the current offset sampling point is adjusted to y=0;

[0040] If y>H, the ordinate of the current offset sampling point is adjusted to y=H.

[0041] In a technical solution of the above-mentioned image processing method, after performing the convolution sampling based on the current offset sampling point position, the method further includes:

[0042] Obtaining a gradient coefficient based on the direction vector;

[0043] The image to be processed after the convolution sampling is subjected to gradient processing based on the gradient coefficient to obtain a target image.

[0044] In a second aspect, an electronic device is provided, which includes a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and the program codes are suitable for being loaded and run by the processor to execute the image processing method described in any one of the technical solutions of the above-mentioned image processing method.

[0045] In a third aspect, an image acquisition device is provided, which includes the image acquisition device body and the electronic device described in the technical solution of the above electronic device.

[0046] In a fourth aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the image processing method described in any one of the technical solutions of the above-mentioned image processing method.

[0047] Solution 1. An image processing method, characterized in that the method comprises:

[0048] Acquire an image to be processed based on an image acquisition device;

[0049] Acquiring position and posture information of the image acquisition device;

[0050] Determine a convolution form based on the posture information;

[0051] The image to be processed is processed based on the convolution form.

[0052] Solution 2. The image processing method according to Solution 1 is characterized in that the posture information includes angular velocity data and acceleration data; and the acquiring of the posture information of the image acquisition device includes:

[0053] Acquiring the angular velocity data based on a gyro sensor of the image acquisition device;

[0054] The acceleration data is acquired based on an acceleration sensor of the image acquisition device.

[0055] Solution 3. The image processing method according to Solution 2 is characterized in that determining the convolution form based on the posture information comprises:

[0056] Processing the angular velocity data and the acceleration data;

[0057] comparing the processed angular velocity data and the acceleration data;

[0058] If the angular velocity data is greater than or equal to the acceleration data, it is determined that the convolution form is a radial convolution; otherwise, it is determined that the convolution form is a rotational convolution.

[0059] Solution 4. The image processing method according to Solution 3 is characterized in that the processing of the image to be processed based on the convolution form comprises:

[0060] Obtaining a current convolution point in the image to be processed;

[0061] Determining an offset step size of convolution sampling based on the convolution form;

[0062] Acquire a current offset sampling point based on the current convolution point and the offset step size, and determine whether the current offset sampling point exceeds a boundary of the image to be processed;

[0063] If so, the position of the current offset sampling point is adjusted according to a preset rule; otherwise, the convolution sampling is performed based on the current offset sampling point.

[0064] Solution 5. The image processing method according to Solution 4 is characterized in that determining the offset step size of the convolution sampling based on the convolution form comprises:

[0065] Acquire a reference point of the convolution sampling from the image to be processed;

[0066] Obtain a direction vector formed by the current convolution point and the reference point, and obtain an angle θ between the direction vector and a horizontal direction; wherein the horizontal direction is the x-axis direction of a pixel coordinate system;

[0067] The offset step size of the convolution sampling is obtained based on the convolution form and the angle θ.

[0068] Solution 6. The image processing method according to Solution 5 is characterized in that the offset step size of the convolution sampling is obtained based on the convolution form and the angle θ, comprising:

[0069] If the convolution form is the radial convolution, the offset step is calculated based on the following formula:

[0070] dx=t*r*cosθ, dy=t*r*sinθ;

[0071] If the convolution form is the rotation convolution, the offset step is calculated based on the following formula:

[0072] dx=t*cos(θ+t*θ0), dy=t*sin(θ+t*θ0);

[0073] Among them, dx is the offset step in the horizontal direction, dy is the offset step in the vertical direction, t is the number of current convolution sampling, r is the displacement increment of a single convolution sampling, θ0 is the angle increment of a single convolution sampling; wherein the vertical direction is the y-axis direction of the pixel coordinate system.

[0074] Solution 7. The image processing method according to Solution 6 is characterized in that the method further comprises: obtaining a width W and a height H of the image to be processed; and the determining whether the current offset sampling point exceeds a boundary of the image to be processed comprises:

[0075] If the current offset sampling point meets at least one of the following conditions, it is determined that it exceeds the boundary of the image to be processed;

[0076] x<0; or, x>W; or, y<0; or, y>H;

[0077] Wherein, x is the horizontal coordinate of the current offset sampling point in the pixel coordinate system, and y is the vertical coordinate of the current offset sampling point in the pixel coordinate system.

[0078] Solution 8. The image processing method according to Solution 7 is characterized in that adjusting the position of the current offset sampling point according to a preset rule comprises:

[0079] If x<0, then adjust the horizontal coordinate of the current offset sampling point to x=0;

[0080] If x>W, then adjust the horizontal coordinate of the current offset sampling point to x=W;

[0081] If y<0, the ordinate of the current offset sampling point is adjusted to y=0;

[0082] If y>H, the ordinate of the current offset sampling point is adjusted to y=H.

[0083] Solution 9. The image processing method according to Solution 5 is characterized in that after performing the convolution sampling based on the current offset sampling point position, the method further comprises:

[0084] Obtaining a gradient coefficient based on the direction vector;

[0085] The image to be processed after the convolution sampling is subjected to gradient processing based on the gradient coefficient to obtain a target image.

[0086] Solution 10. An electronic device, comprising a processor and a storage device, wherein the storage device is suitable for storing multiple program codes, and is characterized in that the program codes are suitable for being loaded and run by the processor to execute the image processing method described in any one of Solutions 1 to 9.

[0087] Solution 11. An image acquisition device, comprising the image acquisition device body and the electronic device described in Solution 10.

[0088] Solution 12. A computer-readable storage medium storing a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the image processing method described in any one of Solutions 1 to 9.

[0089] The above one or more technical solutions of the present invention have at least one or more of the following

[0090] Beneficial effects:

[0091] In the technical solution of the present invention, the image to be processed is obtained based on the image acquisition device, and the posture information of the image acquisition device is obtained, the convolution form is determined based on the posture information, and finally the image to be processed is processed based on the convolution form. Through the above implementation, the corresponding convolution form can be selected according to the motion state of the image acquisition device, which increases the flexibility of image processing, can more accurately express the current shooting scene and the shooting semantics that the user wants to convey, expands the shooting effect, and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] The disclosure of the present invention will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In addition, similar numbers in the figures are used to represent similar components, among which:

[0093] Figure 1 is a schematic flow chart of main steps of an image processing method according to an embodiment of the present invention;

[0094] Figure 2 1 is a flow chart of main steps of determining a convolution form based on posture information according to an embodiment of the present invention;

[0095] Figure 3 It is a schematic flow chart of main steps of processing an image to be processed based on a convolution form according to one embodiment of the present invention;

[0096] Figure 4 1 is a schematic flow chart of main steps of determining the offset step size of convolution sampling based on the convolution form according to one embodiment of the present invention;

[0097] Figure 5 is a schematic diagram of current offset sampling points of radial convolution according to an embodiment of the present invention;

[0098] Figure 6 is a schematic diagram of a current offset sampling point of a rotational convolution according to an embodiment of the present invention;

[0099] Figure 7 is a schematic flow chart of main steps of an image processing method according to another embodiment of the present invention;

[0100] Figure 8a is an image to be processed according to an embodiment of the present invention;

[0101] Figure 8b In one embodiment of the present invention, a target image is obtained by subjecting an image to be processed to radial convolution processing;

[0102] Figure 9a is an image to be processed according to an embodiment of the present invention;

[0103] Figure 9b In one embodiment of the present invention, the image to be processed is subjected to rotational convolution processing to obtain a target image;

[0104] Fig.10 It is a main structural diagram of an electronic device embodiment according to the present invention.

[0105] List of reference numerals:

[0106] 1001: processor; 1002: storage device. DETAILED DESCRIPTION

[0107] Some embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0108] In the description of the present invention, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, and memories, and may also include software parts, such as program codes, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor, or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware, or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program codes, such as a magnetic disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, and the like. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B", and may include only A, only B, or A and B. The singular terms "one" and "the" may also include plural forms.

[0109] Here we first explain some terms involved in the present invention.

[0110] Pose: A description of the position and motion of a rigid body in three-dimensional space.

[0111] Dimension: The basic property of a physical quantity. The study of physics can quantitatively describe various physical phenomena. The various physical quantities used in the description have a close relationship with each other, that is, they have a certain functional relationship. In order to accurately describe these relationships, physical quantities can be divided into basic quantities and derived quantities. Basic quantities are physical quantities with independent dimensions, and derived quantities are physical quantities whose dimensions can be expressed as a combination of basic dimensions; all derived quantities can be derived from basic quantities, thereby establishing a functional relationship between the entire physical quantities, which is usually called a quantity system.

[0112] Dimensionlessness: refers to removing part or all of the units of an equation involving physical quantities by replacing them with an appropriate variable in order to simplify experiments or calculations. It is an important processing idea in scientific research.

[0113] As described in the background art, currently the technology for performing convolution on captured images is usually relatively fixed, and the convolution effect achieved is relatively simple.

[0114] In order to solve the above problems, the present invention provides an image processing method, an electronic device, an image acquisition device and a storage medium.

[0115] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart of the main steps of an image processing method according to an embodiment of the present invention. Figure 1 As shown, the image processing method in the embodiment of the present invention mainly includes the following steps S101 to S104.

[0116] Step S101: Acquire an image to be processed based on an image acquisition device.

[0117] Step S102: Acquire the position and posture information of the image acquisition device.

[0118] Step S103: Determine the convolution form based on the posture information.

[0119] Step S104: Process the image to be processed based on the convolution form.

[0120] Based on the method described in steps S101 to S104 above, through the above implementation, the corresponding convolution form can be selected according to the motion state of the image acquisition device, which increases the flexibility of image processing, can more accurately express the current shooting scene and the shooting semantics that the user wants to convey, expands the shooting effect, and improves the user experience.

[0121] The above steps S101 to S104 are further explained below.

[0122] In some implementations of the above step S101, the image acquisition device starts a shooting program to obtain an original image of the current scene, that is, the image to be processed, and simultaneously obtains a width W and a height H of the image to be processed.

[0123] In some embodiments, the image acquisition device includes a camera, a webcam, a camcorder, a scanner, a vehicle-mounted camera, and other devices with a camera function, such as a mobile phone, a tablet computer, etc., which are not limited here.

[0124] The above is a further description of step S101.

[0125] In some implementations of the above step S102, the position information of the image acquisition device includes angular velocity data and acceleration data.

[0126] Further, in some implementations, obtaining the position and posture information of the image acquisition device mainly includes:

[0127] Acquire angular velocity data based on a gyroscope sensor of an image acquisition device;

[0128] The acceleration data is obtained based on the acceleration sensor of the image acquisition device.

[0129] Among them, the gyroscope sensor senses the movement and tilt of the device by measuring the angular velocity data of the image acquisition device; the acceleration sensor detects the vibration / shake amplitude of the image acquisition device by measuring the acceleration data of the image acquisition device.

[0130] It should be noted that the above examples of obtaining the posture information of the image acquisition device are only schematic illustrations. In practical applications, the posture information that can be obtained is not limited to angular velocity data and acceleration data, but can also include attitude data, angle data, etc. Different sensors can obtain data at the same time or in other orders. As long as it does not violate the technical concept of the solution of the present invention, those skilled in the art can set it according to the use scenario, which is not limited here.

[0131] The above is a further description of step S102.

[0132] In some implementations of the above step S103, the acquired posture information is taken as angular velocity data and acceleration data as an example. Figure 2 , Figure 2 FIG. 1 is a flow chart of the main steps of determining the convolution form based on the pose information according to an embodiment of the present invention. Figure 2 As shown, it mainly includes the following steps S201 to S202.

[0133] Step S201: Processing the angular velocity data and the acceleration data.

[0134] Since the angular velocity data (rad / s) and acceleration data (m / s 2 ) are not the same physical quantities, so they are first processed and converted into the same physical quantities to unify the data before comparison.

[0135] In some embodiments, both the angular velocity data and the acceleration data may be converted into the tilt angle of the camera (i.e., the tilt angle in the Z-axis direction in the camera coordinate system). Specifically, the angular velocity data measured by the gyroscope sensor may be time-integrated to obtain the tilt angle; and the acceleration data measured by the gyroscope sensor may be used to determine the tilt angle using the gravity vector and its projection on the Z-axis.

[0136] In other embodiments, both angular velocity data and acceleration data can be expressed in terms of dimensions. Dimension is an important concept in physics. Different physical quantities have different units, and these units are interrelated. Therefore, by properly defining some basic units (called basic units), any other units (called derived units) can be expressed as the product of these units, which are unified to facilitate the study of the relationship between various physical quantities.

[0137] Specifically, in the International System of Units, the dimension symbols of the seven basic physical quantities length, mass, time, electric current, thermodynamic temperature, amount of substance, and luminous intensity are L, M, T, I, θ, N, and J, respectively. Other physical quantities can be derived from the basic quantities, such as the dimension formula of speed is LT -1 , the dimension formula of force is LMT -2 , the dimension of acceleration is LT -2 , the dimension of angular velocity is T -1 wait.

[0138] Furthermore, the acceleration data and angular velocity data can be converted into dimensionless form. Dimensionless means removing part or all of the units of an equation involving physical quantities by replacing them with a suitable variable. Dimensionless data processing mainly solves the comparability of data. Dimensionless methods include standardization, normalization, Z-score standardization, etc.

[0139] It should be pointed out that the data processing methods listed above are only illustrative. In practical applications, those skilled in the art can choose them on their own as long as they do not violate the technical concept of the present invention, and no limitation is made here.

[0140] Step S202: Compare the processed angular velocity data and acceleration data.

[0141] Once the data is unified, it can be compared.

[0142] Specifically, if the angular velocity data is greater than or equal to the acceleration data, the convolution form is determined to be radial convolution; otherwise, the convolution form is determined to be rotational convolution.

[0143] For example, after converting the angular velocity data and acceleration data into angles, if the angle value corresponding to the angular velocity data is π / 3, and the angle value corresponding to the acceleration data is π / 6, then the angular velocity data is greater than or equal to the acceleration data, and the convolution form is determined to be radial convolution; if the angle value corresponding to the angular velocity data is 5π / 12, and the angle value corresponding to the acceleration data is π / 2, then the angular velocity data is less than the acceleration data, and the convolution form is determined to be rotational convolution.

[0144] Furthermore, after determining the convolution form, the number of convolution sampling can be set according to the image effect desired by the user. Generally, according to the image effect and processing speed, it is recommended to set the lower limit of the number of convolution sampling to 3 and the upper limit to 11-17. In actual applications, users can set it according to their needs, and there is no limitation here.

[0145] The above is a further description of step S103.

[0146] In some implementations of the above step S104, see the attached Figure 3 , Figure 3 FIG. 1 is a flow chart of the main steps of processing an image to be processed based on a convolution method according to an embodiment of the present invention. Figure 3 As shown, it mainly includes the following steps S301 to S303

[0147] Step S301: Obtain the current convolution point in the image to be processed.

[0148] The current convolution point is any point P in the pixel coordinate system of the image to be processed. i (x i ,y i ), where the origin of the pixel coordinate system is O, which is the upper left corner of the pixel coordinate system, the horizontal direction is the x-axis, and the vertical direction is the y-axis.

[0149] Step S302: Determine the offset step size of the convolution sampling based on the convolution form.

[0150] Specifically, the offset step size of the convolution sampling is determined based on the convolution form (radial convolution or rotational convolution) determined in the above steps.

[0151] In some embodiments, see Appendix Figure 4 , Figure 4 FIG. 1 is a flow chart of the main steps of determining the offset step size of convolution sampling based on the convolution form according to an embodiment of the present invention. Figure 4 As shown, step S302 mainly includes steps S3021 to S3023.

[0152] Step S3021: Obtain a reference point for convolution sampling from the image to be processed.

[0153] The reference point of the convolution sampling can be set by yourself or it can be set by default to the center point P0 (x0, y0) of the image pixel coordinate system, which is not limited here.

[0154] Step S3022: Obtain a direction vector formed by the current convolution point and the reference point, and obtain an angle θ between the direction vector and the horizontal direction; wherein the horizontal direction is the x-axis direction of the pixel coordinate system.

[0155] Specifically, first calculate the reference point P0(x0, y0) and the current convolution point P i (x i ,y i ) formed by the direction vector (x i -x0,y i -y0), and then calculate the direction vector The angle θ with the x-axis direction of the pixel coordinate system is calculated as follows:

[0156] θ=arctan[(y i -y0) / (x i –x0+a)]

[0157] Among them, a is a very small number, such as 0.001, x i -x0+a to prevent x i =x0, the denominator is meaningless.

[0158] Step S3023: Obtain the offset step size of the convolution sampling based on the convolution form and the angle θ.

[0159] Specifically, in some implementations, if the convolution form is radial convolution, the offset step is calculated based on the following formula:

[0160] dx=t*r*cosθ,dy=t*r*sinθ

[0161] Among them, dx is the offset step size in the x-axis direction of the pixel coordinate system, dy is the offset step size in the y-axis direction of the pixel coordinate system, t is the number of current convolution sampling, and r is the displacement increment of a single convolution sampling. Specifically, r = min(1 / W, 1 / H), that is, the minimum value of the inverse of the width and the inverse of the height of the image to be processed is taken.

[0162] In other implementations, if the convolution form is a rotation convolution, the offset step is calculated based on the following formula:

[0163] dx=t*cos(θ+t*θ0), dy=t*sin(θ+t*θ0);

[0164] Among them, dx is the offset step size of the pixel coordinate system in the x-axis direction, dy is the offset step size of the pixel coordinate system in the y-axis direction, t is the number of current convolution sampling, θ0 is the angle increment of a single convolution sampling, specifically, θ0 = k*θz / N, wherein θz is the tilt angle of the Z-axis direction in the camera coordinate system (calculated based on the angular velocity data obtained by the gyroscope sensor), and K is a coefficient, K∈[0.1, 0.6].

[0165] The above is a further description of step S302.

[0166] Step S303: obtaining a current offset sampling point based on the current convolution point and the offset step size, and determining whether the current offset sampling point exceeds the boundary of the image to be processed.

[0167] Specifically, the current convolution point P i (x i ,y i ) plus the offset step of the x-axis and y-axis, i.e., dx and dy, respectively, we can get the current offset sampling point P(x, y), where x = x i +dx,y=y i +dy.

[0168] When the convolution form is radial convolution, refer to the attached Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the current offset sampling point of the radial convolution according to an embodiment of the present invention. When the convolution form is a rotational convolution, refer to FIG. Figure 6 , Figure 6 is a schematic diagram of the current offset sampling point of the rotational convolution according to an embodiment of the present invention.

[0169] Further, based on the position of the current offset sampling point, it is determined whether the current offset sampling point exceeds the boundary of the image to be processed. If so, the position of the current offset sampling point is adjusted according to a preset rule; otherwise, convolution sampling is performed based on the current offset sampling point.

[0170] In some implementations, the judgment and adjustment logic of step S303 is as follows:

[0171] x=x i +dx>0? x i +dx:0;x=x i +dx <W?x:W

[0172] y=y i +dy>0? y i +dy:0;y=y i+dy < H? y : H

[0173] Specifically, if x i +dx > 0 and x i +dx < W, it means that the abscissa x of the current offset sampling point P does not exceed the boundary of the image to be processed. At this time, x = x i +dx; if when y i +dy > 0 and y i +dy < H, it means that the ordinate y of the current offset sampling point P does not exceed the boundary of the image to be processed. At this time, y = y i +dy.

[0174] On the contrary, if the current offset sampling point meets at least one of the following conditions, it is determined that it exceeds the boundary of the image to be processed;

[0175] x < 0; or, x > W; or, y < 0; or, y > H;

[0176] Among them, if x < 0, it means that the current offset sampling point exceeds the left boundary of the image to be processed; if x > W, it means that the current offset sampling point exceeds the right boundary of the image to be processed; if y < 0, it means that the current offset sampling point exceeds the upper boundary of the image to be processed; if y > H, it means that the current offset sampling point exceeds the lower boundary of the image to be processed.

[0177] Furthermore, if the current offset sampling point exceeds the image to be processed, the position of the current offset sampling point is adjusted according to a preset rule.

[0178] Specifically, it includes:

[0179] If x < 0, then adjust the abscissa of the current offset sampling point to x = 0;

[0180] If x > W, then adjust the abscissa of the current offset sampling point to x = W;

[0181] If y < 0, then adjust the ordinate of the current offset sampling point to y = 0;

[0182] If y > 0, then adjust the ordinate of the current offset sampling point to y = H.

[0183] In some embodiments, if the current offset sampling point does not exceed the boundary of the image to be processed, convolution sampling is performed based on the current offset sampling point. The specific program code is as follows:

[0184] long sum = 0;

[0185] for(int i = 0; i < H; ++i)

[0186] for(int j = 0; j < W; ++j) {

[0187] for(int t=0;t <N;++t){

[0188] p0=Msrc[i+dx,j+dy];

[0189] sum+=p0;

[0190] }

[0191] Mdir[i,j]=sum / N;

[0192] }

[0193] The above is a further explanation of step S104.

[0194] Further, after executing the above step S104, refer to the attached Figure 7 , Figure 7 FIG. 1 is a flow chart of the main steps of an image processing method according to another embodiment of the present invention. Figure 7 As shown, step S701 to step S702 also include:

[0195] Step S701: obtaining a gradient coefficient based on a direction vector;

[0196] Specifically, according to the direction vector Calculate the gradient coefficient using the following formula:

[0197]

[0198] Among them, fr represents the gradient coefficient, Represents the direction vector The modulus of pow represents the function of finding the power. Represents a vector Model Perform 5th power operation.

[0199] Step S702: performing gradient processing on the image to be processed after convolution sampling based on the gradient coefficient to obtain a target image.

[0200] The specific program code is as follows:

[0201] for(int i=0i <H;++i)

[0202] for(int j = 0; j <W;++j)

[0203] {

[0204] Mout[i,j]=Msrc[i,j]*(1.-fr)+Mc[i,j]*fr;

[0205] }

[0206] After the above processing, the target image can be obtained. For details, please refer to FIG. 8. Figure 8a is an image to be processed according to an embodiment of the present invention, Figure 8b In one embodiment of the present invention, the image to be processed is subjected to radial convolution processing to obtain a target image.

[0207] Refer to Figure 9, Figure 9a is an image to be processed according to an embodiment of the present invention, Figure 9b In one embodiment of the present invention, the image to be processed is subjected to rotational convolution processing to obtain a target image.

[0208] Furthermore, after obtaining the target image, the shooting procedure can be ended, the shooting data can be saved and the database information of the image acquisition device can be updated.

[0209] The above is a further description of the image processing method provided by the present invention.

[0210] Based on the above method, the corresponding convolution form can be selected according to the motion state of the image acquisition device, which increases the flexibility of image processing, can more accurately express the current shooting scene and the shooting semantics that the user wants to convey, expands the shooting effect, and improves the user experience.

[0211] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effects of the present invention, different steps do not have to be performed in such an order. They can be performed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present invention.

[0212] It should be noted that the image frames to be processed and the posture information involved in the embodiments of the present disclosure (including but not limited to the angular velocity data, acceleration data, stored data, displayed data, etc. for analysis) are all information and data authorized by the user or fully authorized by all parties. The operations such as acquisition, processing, and display of images involved in the embodiments of the present disclosure are all performed after authorization by the user or the object or after full authorization by all parties.

[0213] It is understood by those skilled in the art that the present invention implements all or part of the processes in the method of the above embodiment, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electric carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0214] Furthermore, the present invention also provides an electronic device. Fig.10 , Fig.10 1 is a schematic diagram of the main structure of an electronic device according to an embodiment of the present invention. Fig.10 As shown, the electronic device in the embodiment of the present invention mainly includes a processor 1001 and a storage device 1002. The storage device 1002 can be configured to store a program for executing the image processing method of the above method embodiment, and the processor 1001 can be configured to execute the program in the storage device 1002, which includes but is not limited to the program for executing the image processing method of the above method embodiment. For ease of explanation, only the part related to the embodiment of the present invention is shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention.

[0215] In some possible implementations of the present invention, the electronic device may include multiple processors 1001 and multiple storage devices 1002. The program for executing the image processing method of the above method embodiment may be divided into multiple subprograms, each of which may be loaded and run by the processor 1001 to execute different steps of the image processing method of the above method embodiment. Specifically, each subprogram may be stored in different storage devices 1002, and each processor 1001 may be configured to execute the programs in one or more storage devices 1002 to jointly implement the image processing method of the above method embodiment, that is, each processor 1001 executes different steps of the image processing method of the above method embodiment to jointly implement the image processing method of the above method embodiment.

[0216] The above-mentioned multiple processors 1001 may be processors deployed on the same device. For example, the above-mentioned electronic device may be a high-performance device composed of multiple processors, and the above-mentioned multiple processors 1001 may be processors configured on the high-performance device. In addition, the above-mentioned multiple processors 1001 may also be processors deployed on different devices. For example, the above-mentioned electronic device may be a server cluster, and the above-mentioned multiple processors 1001 may be processors on different servers in the server cluster; the above-mentioned electronic device may be a driving device cluster, and the above-mentioned multiple processors 1001 may be processors on different driving devices in the driving device cluster.

[0217] Furthermore, the present invention also provides an image acquisition device. In an embodiment of an image acquisition device according to the present invention, the image acquisition device may include an image acquisition device body and the electronic device described in the above electronic device embodiment.

[0218] Furthermore, the present invention also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present invention, the computer-readable storage medium can be configured to store a program for executing the image processing method of the above method embodiment, and the program can be loaded and run by the processor to implement the above image processing method. For ease of explanation, only the parts related to the embodiment of the present invention are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present invention. The computer-readable storage medium can be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present invention is a non-temporary computer-readable storage medium.

[0219] So far, the technical solution of the present invention has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an image to be processed based on an image acquisition device; Acquire the position and posture information of the image acquisition device; the position and posture information includes angular velocity data and acceleration data; Determine a convolution form based on the posture information; Processing the image to be processed based on the convolution form; Among them, determining the convolution form based on the posture information includes: processing the angular velocity data and the acceleration data; comparing the processed angular velocity data and the acceleration data; if the angular velocity data is greater than or equal to the acceleration data, determining that the convolution form is radial convolution; otherwise, determining that the convolution form is rotational convolution.

2. The image processing method according to claim 1, characterized in that: The obtaining of the position information of the image acquisition device comprises: Acquiring the angular velocity data based on a gyro sensor of the image acquisition device; The acceleration data is acquired based on an acceleration sensor of the image acquisition device.

3. The image processing method according to claim 1, characterized in that: The processing of the image to be processed based on the convolution form includes: Obtaining a current convolution point in the image to be processed; Determining an offset step size of convolution sampling based on the convolution form; Acquire a current offset sampling point based on the current convolution point and the offset step size, and determine whether the current offset sampling point exceeds a boundary of the image to be processed; If so, the position of the current offset sampling point is adjusted according to a preset rule; otherwise, the convolution sampling is performed based on the current offset sampling point.

4. The image processing method according to claim 3, characterized in that: The step of determining the offset step of convolution sampling based on the convolution form includes: Acquire a reference point of the convolution sampling from the image to be processed; Obtain a direction vector formed by the current convolution point and the reference point, and obtain an angle θ between the direction vector and a horizontal direction; wherein the horizontal direction is the x-axis direction of a pixel coordinate system; The offset step size of the convolution sampling is obtained based on the convolution form and the angle θ.

5. The image processing method according to claim 4, characterized in that: The step of obtaining the offset step of the convolution sampling based on the convolution form and the angle θ includes: If the convolution form is the radial convolution, the offset step is calculated based on the following formula: dx=t*r*cosθ, dy=t*r*sinθ; If the convolution form is the rotation convolution, the offset step is calculated based on the following formula: dx=t*cos(θ+t*θ0), dy=t*sin(θ+t*θ0); Among them, dx is the offset step in the horizontal direction, dy is the offset step in the vertical direction, t is the number of current convolution sampling, r is the displacement increment of a single convolution sampling, θ0 is the angle increment of a single convolution sampling; wherein the vertical direction is the y-axis direction of the pixel coordinate system.

6. The image processing method according to claim 5, characterized in that: The method further comprises, Acquiring the width W and height H of the image to be processed; and determining whether the current offset sampling point exceeds the boundary of the image to be processed includes: If the current offset sampling point meets at least one of the following conditions, it is determined that it exceeds the boundary of the image to be processed; x<0; or, x>W; or, y<0; or, y>H; Wherein, x is the horizontal coordinate of the current offset sampling point in the pixel coordinate system, and y is the vertical coordinate of the current offset sampling point in the pixel coordinate system.

7. The image processing method according to claim 6, characterized in that: The adjusting the position of the current offset sampling point according to a preset rule comprises: If x<0, then adjust the horizontal coordinate of the current offset sampling point to x=0; If x>W, then adjust the horizontal coordinate of the current offset sampling point to x=W; If y<0, the ordinate of the current offset sampling point is adjusted to y=0; If y>H, the ordinate of the current offset sampling point is adjusted to y=H.

8. The image processing method according to claim 4, characterized in that: After performing the convolution sampling based on the current offset sampling point position, the method further includes: Obtaining a gradient coefficient based on the direction vector; The image to be processed after the convolution sampling is subjected to gradient processing based on the gradient coefficient to obtain a target image.

9. An electronic device comprising a processor and a storage device, wherein the storage device is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by the processor to execute the image processing method according to any one of claims 1 to 8.

Images