Method, apparatus, terminal device and medium for digitizing vibration images

Abstract

The present invention discloses a method, apparatus, terminal device and medium for numerical conversion of vibration images. The method includes: acquiring a vibration image captured by a camera module, where the vibration image corresponds to a vibration angle; performing convolution and numerical conversion processing on the vibration image using a preset convolution kernel to obtain a corresponding convolution image; performing numerical statistics on the convolution values of m pixel points in the convolution image to obtain corresponding numerical conversion data, and the numerical conversion data is used to reflect the vibration angle corresponding to the vibration image. By adopting the present invention, the mapping relationship between the vibration image and the vibration angle can be accurately and conveniently obtained, thus solving the problem in the prior art that the OIS vibration angle in the camera module cannot be detected.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method, device, terminal device and medium for numerical conversion of vibration images. Background Art

[0002] Users often like to record every bit of life in the form of photos or videos. Especially with the popularization of terminal devices with photo-taking and video-recording functions, people's needs for photography are satisfied. However, it is found in practice that due to reasons such as the rotation of the camera lens by the user or the type of the lens, the situation of blurry captured images caused by camera jitter / vibration may occur.

[0003] To avoid the above situation, currently, an optical image stabilizer (OIS) for the rotation (Roll) axis is added to the camera module to offset the vibration of the camera. However, currently, there is no tool or solution to detect the vibration angle of the OIS. Summary of the Invention

[0004] An embodiment of the present application provides a method for numerical conversion of vibration images, which can characterize / detect the corresponding vibration angle by numerical conversion of vibration images.

[0005] On the one hand, an embodiment of the present application provides a method for numerical conversion of vibration images, the method comprising:

[0006] Obtaining a vibration image captured by a camera module, the vibration image corresponding to a vibration angle;

[0007] Performing convolution and numerical conversion processing on the vibration image using a preset convolution kernel to obtain a corresponding convolution image, the convolution image including convolution values of m pixel points respectively, where m is a positive integer;

[0008] Performing numerical statistics on the convolution values of m pixel points in the convolution image to obtain corresponding numerical conversion data, the numerical conversion data being used to reflect the vibration angle corresponding to the vibration image.

[0009] Optionally, the performing convolution and numerical conversion processing on the vibration image using a preset convolution kernel to obtain a corresponding convolution image includes:

[0010] Determining a region of interest in the vibration image according to a preset region position;

[0011] Splitting and merging the region of interest according to a preset image channel to obtain a corresponding processed image region, the size of the processed image region not exceeding the size of the region of interest;

[0012] Perform convolution and numerical processing on the processed image region using a preset convolution kernel to obtain the convolution image corresponding to the processed image region.

[0013] Optionally, the size of the convolution kernel is a×b, the convolution kernel includes convolution vectors in a×b directions, the processed image region includes m pixel points, and the performing convolution and numerical processing on the processed image region using a preset convolution kernel to obtain the convolution image includes:

[0014] Taking each pixel point in the processed image region as the central pixel point, determine the adjacent pixel points except the central pixel point within the window size range corresponding to the convolution kernel;

[0015] Calculate the vertex normal vector of the central pixel point according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction;

[0016] Calculate the convolution value of each of the m pixel points according to the vertex normal vector of the central pixel point and the unit vector in the preset direction;

[0017] Obtain the convolution image according to the convolution values of each of the m pixel points.

[0018] Optionally, the calculating the vertex normal vector of the central pixel point according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction includes:

[0019] Construct target vectors in each direction according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction;

[0020] Perform normal vector calculation and normalization processing on the target vectors in each direction to obtain unit normal vectors in each direction;

[0021] Perform accumulation processing on the unit normal vectors in each direction to obtain the vertex normal vector of the central pixel point.

[0022] Optionally, the performing numerical statistics on the convolution values of each of the m pixel points in the convolution image to obtain the corresponding numerical data includes:

[0023] According to the convolution values of each of the m pixel points, screen out the number of pixel points whose convolution values exceed the preset value from the m pixel points as the numerical data.

[0024] Optionally, the number of vibration images is n, where n is a positive integer greater than 1, and the method further includes:

[0025] Perform curve fitting on the numerical data and the vibration angle corresponding to each of the n vibration images to obtain corresponding fitting curves;

[0026] Obtain the numerical data of the image to be predicted;

[0027] According to the fitting curve and the numerical data of the image to be predicted, perform vibration angle prediction on the image to be predicted to obtain the target prediction angle corresponding to the image to be predicted.

[0028] Optionally, the number of regions of interest in each vibration image is k, where k is a positive integer greater than 1. The performing curve fitting on the numerical data and the vibration angle corresponding to each of the n vibration images to obtain corresponding fitting curves includes:

[0029] According to the preset region positions, determine k groups of n regions of interest with the same region positions and the numerical data corresponding to each of the n regions of interest in each group from the n vibration images;

[0030] Perform curve fitting on the numerical data corresponding to each of the n regions of interest in each group and the vibration angles corresponding to the n vibration images respectively to obtain k fitting curves;

[0031] The performing vibration angle prediction on the image to be predicted according to the fitting curve and the numerical data of the image to be predicted to obtain the target prediction angle corresponding to the image to be predicted includes:

[0032] Use the k fitting curves and the numerical data of the image to be predicted to perform vibration angle prediction on the image to be predicted, and correspondingly obtain k prediction angles corresponding to the image to be predicted;

[0033] Process the k prediction angles to obtain the target prediction angle.

[0034] On the other hand, the present application provides a device for numerical conversion of vibration images through an embodiment of the present application. The device includes: an acquisition module, a processing module, and a statistics module, where:

[0035] The acquisition module is used to acquire a vibration image, and the vibration image corresponds to a vibration angle;

[0036] The processing module is used to perform convolution and numerical conversion processing on the vibration image using a preset convolution kernel to obtain a corresponding convolution image. The convolution image includes the convolution values of m pixel points respectively, and m is a positive integer;

[0037] The statistical module is used to numerically count the convolution values of m pixel points in the convolution image respectively to obtain corresponding numerical data, and the numerical data is used to reflect the vibration angle corresponding to the vibration image.

[0038] For the content not introduced or described in the embodiments of the present application, reference can be made to the relevant introduction in the foregoing method embodiments, which will not be elaborated here.

[0039] On the other hand, the present application provides a terminal device through an embodiment of the present application. The terminal device includes: a processor, a memory, a communication interface, and a bus; the processor, the memory, and the communication interface are connected through the bus and complete communication with each other; the memory stores executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory to execute the method for numerical conversion of the vibration image as described above.

[0040] On the other hand, the present application provides a computer-readable storage medium through an embodiment of the present application. The computer-readable storage medium stores a program, and when the program runs on a terminal device, it executes the method for numerical conversion of the vibration image as described above.

[0041] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: The present application obtains a vibration image captured by a camera module, and the vibration image corresponds to a vibration angle. Then, a preset convolution kernel is used to perform convolution and numerical conversion processing on the vibration image to obtain a corresponding convolution image. Finally, the convolution values of m pixel points in the convolution image are numerically counted respectively to obtain corresponding numerical data, and the numerical data is used to reflect the vibration angle corresponding to the vibration image. In the above solution, the present application can use a preset convolution kernel to perform convolution and numerical conversion processing on the vibration image, and then numerically count the processed convolution image to obtain the numerical data corresponding to the vibration image. The numerical data is used to represent / reflect the vibration angle corresponding to the vibration image, so that the mapping relationship between the vibration image and the vibration angle can be accurately and conveniently obtained, and the problem that the OIS vibration angle in the camera module cannot be detected in the prior art is solved. Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1It is a schematic flowchart of a vibration image numericalization method provided by an embodiment of the present application.

[0044] Figure 2 It is a schematic diagram of the construction of a convolution kernel provided by an embodiment of the present application.

[0045] Figure 3 It is a schematic diagram of the construction of a vector plane provided by an embodiment of the present application.

[0046] Figure 4 It is a schematic diagram of processing a vibration image into a convolution image provided by an embodiment of the present application.

[0047] Figure 5 It is a schematic diagram of a fitting curve provided by an embodiment of the present application.

[0048] Figure 6 It is a schematic structural diagram of a vibration image numericalization device provided by an embodiment of the present application.

[0049] Figure 7 It is a schematic structural diagram of a terminal device provided by an embodiment of the present application. Specific embodiments

[0050] By providing a method for numericalizing vibration images in the embodiments of the present application, the technical problem in the prior art of being unable to effectively detect the OIS vibration angle is solved.

[0051] The technical solution of the embodiment of the present application for solving the above technical problem is generally as follows: Obtain a vibration image captured by a camera module, where the vibration image corresponds to a vibration angle; use a preset convolution kernel to perform convolution and numericalization processing on the vibration image to obtain a corresponding convolution image, where the convolution image includes the convolution values of m pixel points respectively, and m is a positive integer; perform numerical statistics on the convolution values of the m pixel points in the convolution image to obtain corresponding numericalized data, and the numericalized data is used to reflect the vibration angle corresponding to the vibration image.

[0052] To better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0053] First, it should be noted that the term "and / or" appearing in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the preceding and following associated objects.

[0054] Please refer to Figure 1, which is a schematic flowchart of a method for digitizing vibration images provided by an embodiment of the present application. As Figure 1 The method shown includes the following implementation steps:

[0055] S101. Obtain a vibration image captured by a camera module, where the vibration image corresponds to a vibration angle.

[0056] The vibration image in the present application is captured by a camera module equipped with an optical image stabilizer (OIS) and is used to analyze the vibration angle of the OIS. The number of the vibration images is not limited and is usually multiple. Each vibration image corresponds to a vibration angle of the OIS. Specifically, the present application can collect corresponding vibration images at different vibration angles. For example, when the OIS vibration angle is between 0° and 2°, a vibration image is collected every 0.2°, and so on.

[0057] S102. Use a preset convolution kernel to perform convolution and digitization processing on the vibration image to obtain a corresponding convolution image, where the convolution image includes the convolution values of m pixel points respectively, and m is a positive integer.

[0058] S103. Perform numerical statistics on the convolution values of the m pixel points in the convolution image to obtain corresponding digitized data, where the digitized data is used to reflect the vibration angle corresponding to the vibration image.

[0059] Here, the present application takes one vibration image as an example to introduce the specific implementation manners of steps S102 and S103.

[0060] In step S102, in a specific embodiment, the present application can first determine a corresponding region of interest (ROI) from the vibration image according to the preset region position. The region position (or the region of interest) can be set by the system itself. For example, it can be located in the upper left, lower left, upper right, or lower right position of the vibration image, and the present application does not make a limitation.

[0061] Next, the present application can perform splitting and merging processing on the region of interest according to the preset image channels to obtain a corresponding processed image region, where the size of the processed image region does not exceed the size of the region of interest. For example, taking the region of interest as an RGB image region with a size of 200×200 in the vibration image, the present application can split the 200×200 region of interest according to the R channel, GR channel, GB channel, and B channel to obtain 4 channel image regions, and the size of each channel image region is 100×100; then the present application can merge the 4 channel image regions with a size of 100×100 to obtain the processed image region with a size of 100×100.

[0062] Finally, the present application performs convolution and numerical processing on the processed image region using a preset convolution kernel to obtain the convolution image corresponding to the processed image region. In a possible implementation manner, when the size of the convolution kernel is a×b, the convolution kernel includes a×b convolution vectors in different directions, and the processed image region includes m pixel points, where m is a positive integer. At this time, the present application takes each pixel point in the processed image region as the central pixel point, and within the window size range corresponding to the convolution kernel, determines the adjacent pixel points except the central pixel point, and the number of the adjacent pixel points is usually multiple.

[0063] Furthermore, the present application can calculate the vertex normal vector of the central pixel point according to the pixel value of each adjacent pixel point, the pixel value of the central pixel point, and the convolution vectors in each direction. Specifically, the present application can calculate the pixel difference between the pixel value of each adjacent pixel point and the pixel value of the central pixel point, and then use the calculated pixel differences and the convolution vectors in each direction to correspondingly construct target vectors in each direction. Then, the present application can perform normal vector calculation and unitization processing on the target vectors in each direction to obtain the unit normal vectors in each direction. Furthermore, perform accumulation processing on the unit normal vectors in each direction to obtain the vertex normal vector of the central pixel point.

[0064] Furthermore, the present application can calculate the convolution value of each of the m pixel points according to the vertex normal vector of the central pixel point and the unit vector in a preset direction (such as the unit vector in the Z-axis direction). Finally, the convolution image is formed / obtained according to the convolution values of each of the m pixel points.

[0065] To help better understand the embodiments of the present application, the following is a detailed example. Taking the convolution kernel of 3×3 as an example, please refer to Figure 2 which shows a schematic diagram of the plane creation of a 3×3 convolution kernel. As Figure 2 shown, the 3×3 convolution kernel includes convolution vectors in 8 directions. Among them, these 8 directions are V1 to V8 as shown in the figure respectively. The convolution vector in each direction can be: (-1, -1, 0), (0, -1, 0), (1, -1, 0), (-1, 0, 0), (0, 0, 0), (1, 0, 0), (-1, 1, 0), (0, 1, 0), and (1, 1, 0). Taking any central pixel point in the processed image region as an example, its vertex normal vector can be calculated using the following formula (1).

[0066]

[0067] Among them, denotes the vertex normal vector of the central pixel point, i denotes the i-th adjacent pixel point of the central pixel point, and in this example, i can be at most 8 in a 3×3 convolution kernel. h i denotes the difference between the pixel value of the i-th adjacent pixel point and the pixel value of the central pixel point. denotes the convolution vector in the corresponding direction in the convolution kernel.

[0068] Next, the present application uses the following formula (2) to calculate the angle between the vertex normal vector of the central pixel point and the unit vector (0, 0, 1) in the preset direction, which is the convolution value of the central pixel point.

[0069]

[0070] For example, taking the convolution and numerical processing of data A with a 3×3 convolution kernel as an example. Among them, data A is First, data A can be understood as including the pixel values of 9 pixel points. The numerical values in data A all represent pixel values, such as 111 and 112, etc. When calculating the convolution value of the central pixel point (the pixel point corresponding to the pixel value of 106) in data A, the present application can subtract the pixel values (which can also be called brightness values / height values) of the remaining 8 adjacent pixel points adjacent to the central pixel point from the pixel value of the central pixel point respectively, and then construct 8 target vectors in 8 directions with the convolution vectors in 8 directions, which are respectively represented as follows:

[0071]

[0072] Next, the present application can respectively establish planes parallel to the Z-axis for the 8 target vectors; then calculate the normal vectors corresponding to each target vector on this plane and normalize them, so as to obtain the unit normal vectors in each direction. Among them, please refer to Figure 3 shows a schematic diagram of a possible vector creation plane. As Figure 3 respectively shows 8 target vectors, and the illustration is represented as T1~T8. Exemplarily, the present application calculates the corresponding unit normal vectors based on the 8 target vectors, which can be respectively shown as follows:

[0073]

[0074] Furthermore, the present application can perform an accumulative calculation on the 8 unit normal vectors in the calculated 8 directions, so as to obtain the vertex normal vector of the central pixel point, which can be specifically: (-0.03, 1.95, 1.09).

[0075] Optionally, to better display the vector branches of interest to the user, the present application can preset the sensitivity coefficient of the Z-axis according to the actual needs of the system, such as 0.125, etc. At this time, the present application can multiply the Z-axis value of the vertex normal vector of the central pixel by 0.125 to obtain the updated vertex normal vector of the central pixel, which can specifically be: (-0.03, 1.95, 0.13).

[0076] Finally, the present application can calculate the convolution value of the central pixel according to the vertex normal vector of the central pixel and the unit vector (0, 0, 1) in the preset direction, which is specifically: Arccos(0.13 / sqrt(-0.03 2 +1.95 2 +0.13 2 )) = 86.21, where sqrt refers to the square root calculation in a mathematical formula.

[0077] It should be noted that according to the above-described principle, the present application can calculate the convolution values of each of the m pixels in the processed image area, and thus obtain the convolution image corresponding to the processed image area. For example, please refer to Figure 4 a schematic diagram showing a possible convolution image processed from a vibration image. As Figure 4 shown in, the left diagram is the vibration image, and the right diagram is the convolution image obtained after convolution and numerical processing of the region of interest in the vibration image.

[0078] In step S103, after the present application calculates the convolution values of each of the m pixels in the convolution image, it can screen out the number of pixels whose convolution values exceed a preset value from the m pixels, so as to be used as the numerical data corresponding to the vibration image. The preset value is set by the system itself, such as an empirical value set according to user experience, or set according to the actual needs of the system, etc., and the present application does not make any limitations.

[0079] The following introduces some optional embodiments involved in the present application.

[0080] In an optional embodiment, the number of the vibration images is n, the number of regions of interest in each vibration image is 1, and the regional positions of the regions of interest corresponding in each vibration image can be the same. Correspondingly, the present invention can calculate the numerical data of each of the n vibration images by adopting the above-mentioned process of steps S101 to S103. Then, curve fitting is performed on the numerical data of each of the n vibration images and the vibration angles corresponding to each of the n vibration images, so as to obtain a corresponding fitting curve, and this fitting curve is used to characterize the change relationship between the numerical data corresponding to the vibration image and the vibration angle. The fitting curve includes but is not limited to a straight line, a broken line, an arc, a line segment, a parabola, or other curves with continuous lines, etc.

[0081] For example, please refer to Figure 5 a schematic diagram showing a possible fitting curve. As Figure 5 shown, the fitting curve is a broken line, where the abscissa represents the vibration angle and the ordinate represents the numerical data of the vibration image.

[0082] Furthermore, the present application can obtain the numerical data of the image to be predicted for the vibration angle (also simply referred to as the image to be predicted), and then use the fitting curve and the numerical data of the image to be predicted to predict the vibration angle of the image to be predicted. Specifically, the numerical data of the image to be predicted is substituted into the fitting curve to calculate the target prediction angle corresponding to the image to be predicted.

[0083] In another alternative embodiment, the number of the vibration images is n, and the number of regions of interest in each vibration image is k, where k is a positive integer greater than 1. Correspondingly, the present invention can calculate k numerical data corresponding to each vibration image by using the flow of the above steps S101 to S103, and the numerical data corresponds to the regions of interest one by one.

[0084] Furthermore, the present application can determine k groups of n regions of interest with the same region position from the n vibration images according to the region position of each region of interest. In other words, each group of regions of interest includes n regions of interest, and the region positions of these n regions of interest in the n vibration images are the same, such as the upper left image region in the n vibration images.

[0085] Then, the present invention can respectively perform curve fitting on the numerical data corresponding to each of the n regions of interest in each of the k groups and the vibration angles corresponding to the n vibration images, so as to obtain k fitting curves. The introduction of the fitting curve can be referred to the relevant introduction in the foregoing embodiments, and will not be elaborated here.

[0086] Then the present application can obtain the numerical data of the image to be predicted, and then use the k fitting curves and the numerical data of the image to be predicted to predict the vibration angle of the image to be predicted, and calculate the corresponding k predicted angles corresponding to the image to be predicted. Specifically, the numerical data of the image to be predicted can be respectively substituted into the k fitting curves to calculate k predicted angles. Finally, the present application can process the k predicted angles, such as performing arithmetic operations such as averaging, taking the median, and taking the mode, to calculate the target predicted angle corresponding to the image to be predicted.

[0087] By implementing the embodiments of the present application, the present application obtains a vibration image captured by a camera module. The vibration image corresponds to a vibration angle. Then, a preset convolution kernel is used to perform convolution and numerical processing on the vibration image to obtain a corresponding convolution image. Finally, numerical statistics are performed on the convolution values of m pixel points in the convolution image respectively to obtain corresponding numerical data, and the numerical data is used to reflect the vibration angle corresponding to the vibration image. In the above solution, the present application can use a preset convolution kernel to perform convolution and numerical processing on the vibration image, and then perform numerical statistics on the processed convolution image to obtain the numerical data corresponding to the vibration image. The numerical data is used to characterize / reflect the vibration angle corresponding to the vibration image, so that the mapping relationship between the vibration image and the vibration angle can be accurately and conveniently obtained, and the problem that the OIS vibration angle in the camera module cannot be detected in the prior art is solved.

[0088] Based on the same inventive concept, another embodiment of the present application provides a device and a terminal device corresponding to the method for numericalizing the vibration image in the embodiments of the present application.

[0089] Please refer to Figure 6 , which is a schematic structural diagram of a device for numericalizing a vibration image provided by an embodiment of the present application. As Figure 6 shown, the device 60 includes: an acquisition module 601, a processing module 602, and a statistical module 603, where:

[0090] The acquisition module 601 is configured to acquire a vibration image, and the vibration image corresponds to a vibration angle;

[0091] The processing module 602 is configured to use a preset convolution kernel to perform convolution and numerical processing on the vibration image to obtain a corresponding convolution image. The convolution image includes the convolution values of m pixel points respectively, and m is a positive integer;

[0092] The statistical module 603 is configured to perform numerical statistics on the convolution values of m pixel points in the convolution image respectively to obtain corresponding numerical data, and the numerical data is used to reflect the vibration angle corresponding to the vibration image.

[0093] Optionally, the processing module 602 is specifically configured to:

[0094] Determine a region of interest in the vibration image according to a preset region position;

[0095] Split and merge the region of interest according to a preset image channel to obtain a corresponding processed image region, and the size of the processed image region does not exceed the size of the region of interest;

[0096] Perform convolution and numerical processing on the processed image region using a preset convolution kernel to obtain the convolution image corresponding to the processed image region.

[0097] Optionally, the size of the convolution kernel is a×b, the convolution kernel includes convolution vectors in a×b directions, the processed image region includes m pixel points, and the processing module 602 is further specifically configured to:

[0098] Taking each pixel point in the processed image region as the central pixel point, determine adjacent pixel points except the central pixel point within the window size range corresponding to the convolution kernel;

[0099] Calculate the vertex normal vector of the central pixel point according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction;

[0100] Calculate the convolution value of each of the m pixel points according to the vertex normal vector of the central pixel point and the unit vector in the preset direction;

[0101] Obtain the convolution image according to the convolution values of each of the m pixel points.

[0102] Optionally, the processing module 602 is further specifically configured to:

[0103] Construct target vectors in each direction according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction;

[0104] Perform normal vector calculation and unitization processing on the target vectors in each direction to obtain unit normal vectors in each direction;

[0105] Perform accumulation processing on the unit normal vectors in each direction to obtain the vertex normal vector of the central pixel point.

[0106] Optionally, the statistical module 603 is specifically configured to:

[0107] According to the convolution values of each of the m pixel points, screen out the number of pixel points whose convolution values exceed a preset value from the m pixel points as the numerical data.

[0108] Optionally, the number of vibration images is n, n is a positive integer greater than 1, and the device further includes a fitting module 604 and a prediction module 605, where:

[0109] The fitting module 604 is further configured to perform curve fitting on the numerical data and the vibration angle respectively corresponding to the n vibration images to obtain corresponding fitting curves;

[0110] The obtaining module 601 is further configured to obtain the numerical data of the image to be predicted;

[0111] The prediction module 605 is further configured to predict the vibration angle of the image to be predicted according to the fitting curve and the numerical data of the image to be predicted, so as to obtain the target prediction angle corresponding to the image to be predicted.

[0112] Optionally, the number of regions of interest in each of the vibration images is k, where k is a positive integer greater than 1. The fitting module 604 is specifically configured to:

[0113] Determine k groups of n regions of interest with the same region positions and the numerical data corresponding to each of the n regions of interest in each group from the n vibration images according to the preset region positions;

[0114] Perform curve fitting on the numerical data corresponding to each of the n regions of interest in each group and the vibration angles corresponding to the n vibration images respectively to obtain k fitting curves;

[0115] The prediction module 605 is specifically configured to:

[0116] Predict the vibration angle of the image to be predicted by using the k fitting curves and the numerical data of the image to be predicted, and correspondingly obtain k prediction angles corresponding to the image to be predicted;

[0117] Process the k prediction angles to obtain the target prediction angle.

[0118] Please refer to FIG. 7 together, which is a schematic structural diagram of a terminal device provided by an embodiment of the present application. As Figure 7 shown, the terminal device 70 includes: at least one processor 701, a communication interface 702, a user interface 703, and a memory 704. The processor 701, the communication interface 702, the user interface 703, and the memory 704 can be connected through a bus or other means. In this embodiment of the present invention, taking the connection through the bus 705 as an example. Among them,

[0119] The processor 701 may be a general-purpose processor, such as a central processing unit (CPU).

[0120] The communication interface 702 may be a wired interface (such as an Ethernet interface) or a wireless interface (such as a cellular network interface or a wireless local area network interface), and is used for communicating with other terminals or websites. In this embodiment of the present invention, the communication interface 702 is specifically configured to obtain vibration images.

[0121] The user interface 703 can specifically be a touch panel, including a touch screen and a touch display screen, for detecting operation instructions on the touch panel. The user interface 703 can also be a physical button or a mouse. The user interface 703 can also be a display screen for outputting and displaying images or data.

[0122] The memory 704 can include volatile memory, such as random access memory (RAM); the memory can also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); the memory 704 can also include a combination of the above types of memory. The memory 704 is used to store a set of program codes, and the processor 701 is used to call the program codes stored in the memory 704 to perform the following operations:

[0123] Obtain a vibration image captured by the camera module, where the vibration image corresponds to a vibration angle;

[0124] Use a preset convolution kernel to perform convolution and quantization processing on the vibration image to obtain a corresponding convolution image, where the convolution image includes the convolution values of m pixel points respectively, and m is a positive integer;

[0125] Perform numerical statistics on the convolution values of the m pixel points in the convolution image to obtain corresponding quantization data, where the quantization data is used to reflect the vibration angle corresponding to the vibration image.

[0126] Optionally, the step of using a preset convolution kernel to perform convolution and quantization processing on the vibration image to obtain a corresponding convolution image includes:

[0127] Determine a region of interest in the vibration image according to a preset region position;

[0128] Split and merge the region of interest according to a preset image channel to obtain a corresponding processed image region, where the size of the processed image region does not exceed the size of the region of interest;

[0129] Use a preset convolution kernel to perform convolution and quantization processing on the processed image region to obtain the convolution image corresponding to the processed image region.

[0130] Optionally, the size of the convolution kernel is a×b. The convolution kernel includes convolution vectors in a×b directions. The processed image region includes m pixel points. Using the preset convolution kernel to perform convolution and numerical processing on the processed image region to obtain the convolution image includes:

[0131] Taking each pixel point in the processed image region as the central pixel point, determining adjacent pixel points except the central pixel point within the window size corresponding to the convolution kernel;

[0132] Calculating the vertex normal vector of the central pixel point according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction;

[0133] Calculating the convolution value of each of the m pixel points according to the vertex normal vector of the central pixel point and the unit vector in the preset direction;

[0134] Obtaining the convolution image according to the convolution values of each of the m pixel points.

[0135] Optionally, the calculating the vertex normal vector of the central pixel point according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction includes:

[0136] Constructing target vectors in each direction according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction;

[0137] Performing normal vector calculation and unitization processing on the target vectors in each direction to obtain unit normal vectors in each direction;

[0138] Performing accumulation processing on the unit normal vectors in each direction to obtain the vertex normal vector of the central pixel point.

[0139] Optionally, the performing numerical statistics on the convolution values of each of the m pixel points in the convolution image to obtain corresponding numerical data includes:

[0140] According to the convolution values of each of the m pixel points, screening out the number of pixel points whose convolution values exceed the preset value from the m pixel points as the numerical data.

[0141] Optionally, the number of vibration images is n, where n is a positive integer greater than 1. The processor 701 is further configured to:

[0142] Performing curve fitting on the numerical data and the vibration angle corresponding to each of the n vibration images to obtain corresponding fitting curves;

[0143] Obtaining the numerical data of the image to be predicted;

[0144] Based on the fitting curve and the numerical data of the image to be predicted, perform a vibration angle prediction on the image to be predicted to obtain the target prediction angle corresponding to the image to be predicted.

[0145] Optionally, the number of regions of interest in each of the vibration images is k, where k is a positive integer greater than 1. The curve fitting of the numerical data and the vibration angles corresponding to the n vibration images respectively to obtain the corresponding fitting curves includes:

[0146] According to the preset region positions, determine k groups of n regions of interest with the same region positions and the numerical data corresponding to each of the n regions of interest in each group from the n vibration images;

[0147] Perform curve fitting on the numerical data corresponding to each of the n regions of interest in each group and the vibration angles corresponding to the n vibration images respectively to obtain k fitting curves;

[0148] The performing a vibration angle prediction on the image to be predicted based on the fitting curve and the numerical data of the image to be predicted to obtain the target prediction angle corresponding to the image to be predicted includes:

[0149] Use the k fitting curves and the numerical data of the image to be predicted to perform a vibration angle prediction on the image to be predicted, and correspondingly obtain k prediction angles corresponding to the image to be predicted;

[0150] Process the k prediction angles to obtain the target prediction angle.

[0151] Since the terminal device introduced in this embodiment is the terminal device used for implementing the method for numerical conversion of vibration images in the embodiments of the present application, based on the method for numerical conversion of vibration images introduced in the embodiments of the present application, those skilled in the art can understand the specific implementation manners and various variations of the terminal device in this embodiment. Therefore, the specific implementation of how this terminal device implements the method in the embodiments of the present application will not be described in detail here. As long as it is the terminal device used by those skilled in the art to implement the method for numerical conversion of vibration images in the embodiments of the present application, it falls within the scope of protection of the present application.

[0152] The technical solutions in the embodiments of the present application at least have the following technical effects or advantages: The present application obtains a vibration image captured by a camera module, where the vibration image corresponds to a vibration angle, and then uses a preset convolutional kernel to perform convolution and numerical processing on the vibration image to obtain a corresponding convolutional image. Finally, numerical statistics are performed on the convolution values of each of the m pixel points in the convolutional image to obtain corresponding numerical data, and the numerical data is used to reflect the vibration angle corresponding to the vibration image. In the above solution, the present application can use a preset convolutional kernel to perform convolution and numerical processing on the vibration image, and then perform numerical statistics on the processed convolutional image to obtain the numerical data corresponding to the vibration image. This numerical data is used to characterize / reflect the vibration angle corresponding to the vibration image, so that the mapping relationship between the vibration image and the vibration angle can be accurately and conveniently obtained, solving the problem in the prior art that the OIS vibration angle in the camera module cannot be detected.

[0153] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0155] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process or a plurality of processes and / or blocks Figure 1 one process or a plurality of processes and / or blocks Figure 1 and / or steps for implementing the functions specified in one block or a plurality of blocks.

[0157] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made by those skilled in the art once they learn of the basic creative concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0158] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for numericalization of vibration images, characterized in that, The method includes: Obtaining a vibration image captured by a camera module, where the vibration image corresponds to a vibration angle; Performing convolution and numerical processing on the vibration image using a preset convolution kernel to obtain a corresponding convolution image, where the convolution image includes the convolution values of each of m pixel points, and m is a positive integer; Performing numerical statistics on the convolution values of each of the m pixel points in the convolution image to obtain corresponding numerical data, where the numerical data is used to reflect the vibration angle corresponding to the vibration image; The performing convolution and numerical processing on the vibration image using a preset convolution kernel to obtain a corresponding convolution image includes: Determining a region of interest in the vibration image according to a preset region position; Splitting and merging the region of interest according to a preset image channel to obtain a corresponding processed image region, where the size of the processed image region does not exceed the size of the region of interest; Performing convolution and numerical processing on the processed image region using a preset convolution kernel to obtain the convolution image corresponding to the processed image region; The size of the convolution kernel is a×b, the convolution kernel includes convolution vectors in a×b directions, the processed image region includes m pixel points, and the performing convolution and numerical processing on the processed image region using a preset convolution kernel to obtain the convolution image includes: Taking each pixel point in the processed image region as a central pixel point, and determining adjacent pixel points except the central pixel point within the window size corresponding to the convolution kernel; Calculating the vertex normal vector of the central pixel point according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction; Calculating the convolution values of each of the m pixel points according to the vertex normal vector of the central pixel point and the unit vector in a preset direction; Obtaining the convolution image according to the convolution values of each of the m pixel points.

2. The method according to claim 1, wherein The calculating the vertex normal vector of the central pixel point according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction includes: Constructing target vectors in each direction according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction; Performing normal vector calculation and unitization processing on the target vectors in each direction to obtain unit normal vectors in each direction; Performing accumulation processing on the unit normal vectors in each direction to obtain the vertex normal vector of the central pixel point.

3. The method according to claim 1, wherein The performing numerical statistics on the convolution values of each of the m pixel points in the convolution image to obtain corresponding numerical data includes: According to the convolution values of each of the m pixel points, screening out the number of pixel points whose convolution values exceed a preset value from the m pixel points as the numerical data, where the preset value is a custom value.

4. The method according to claim 1, wherein The number of the vibration images is n, and n is a positive integer greater than 1. The method further includes: Performing curve fitting on the numerical data and the vibration angle corresponding to each of the n vibration images to obtain a corresponding fitting curve; Obtaining the numerical data of the image to be predicted; Based on the fitting curve and the numerical data of the image to be predicted, perform a vibration angle prediction on the image to be predicted to obtain the target prediction angle corresponding to the image to be predicted.

5. The method according to claim 4, wherein The number of regions of interest in each of the vibration images is k, where k is a positive integer greater than 1. The curve fitting of the numerical data and the vibration angles corresponding to the n vibration images respectively includes: According to the preset region positions, determine k groups of n regions of interest with the same region positions from the n vibration images, and the numerical data corresponding to each of the n regions of interest in each group; Perform curve fitting on the numerical data corresponding to each of the n regions of interest in each group and the vibration angles corresponding to the n vibration images respectively to obtain k fitting curves; The performing a vibration angle prediction on the image to be predicted according to the fitting curve and the numerical data of the image to be predicted to obtain the target prediction angle corresponding to the image to be predicted includes: Use the k fitting curves and the numerical data of the image to be predicted to perform a vibration angle prediction on the image to be predicted, and correspondingly obtain the prediction angles corresponding to the k images to be predicted; Process the k prediction angles to obtain the target prediction angle.

6. A device for digitizing vibration images, characterized in that, The device includes: an acquisition module, a processing module, and a statistics module, where: The acquisition module is used to acquire a vibration image, and the vibration image corresponds to a vibration angle; The processing module is used to perform convolution and numerical processing on the vibration image using a preset convolution kernel to obtain a corresponding convolution image, and the convolution image includes the convolution values of m pixel points respectively, where m is a positive integer; The processing module is specifically used for: Determine the region of interest in the vibration image according to the preset region position; Split and merge the region of interest according to a preset image channel to obtain a corresponding processed image region, and the size of the processed image region does not exceed the size of the region of interest; Perform convolution and numerical processing on the processed image region using a preset convolution kernel to obtain the convolution image corresponding to the processed image region; The size of the convolution kernel is a×b, the convolution kernel includes convolution vectors in a×b directions, and the processed image region includes m pixel points. The processing module is also specifically used for: Taking each pixel point in the processed image region as the central pixel point, determine the adjacent pixel points except the central pixel point within the window size corresponding to the convolution kernel; Calculate the vertex normal vector of the central pixel point according to the pixel values of the adjacent pixel points, the pixel value of the central pixel point, and the convolution vectors in each direction; Calculate the convolution values of the m pixel points respectively according to the vertex normal vector of the central pixel point and the unit vector in the preset direction; Obtain the convolution image according to the convolution values of the m pixel points respectively. The statistical module is configured to perform numerical statistics on the convolution values of m pixel points in the convolution image respectively, to obtain corresponding numerical data, and the numerical data is used to reflect the vibration angle corresponding to the vibration image.

7. A terminal device, characterized in that, The terminal device includes: a processor, a memory, a communication interface, and a bus; the processor, the memory, and the communication interface are connected through the bus and communicate with each other; the memory stores executable program codes; the processor runs a program corresponding to the executable program codes by reading the executable program codes stored in the memory, so as to execute the method for numerical conversion of the vibration image according to any one of claims 1-5 above.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program, and when the program runs on the terminal device, it executes the method for numerical conversion of the vibration image according to any one of claims 1-5 above.

Images