Intelligent image zooming method

Abstract

The invention relates to the technical field of image correction, in particular to an intelligent image zooming method, which comprises the following steps of: acquiring a to-be-zoomed original image; determining a selection coefficient of each row or column in an original image by analyzing a dominant hue, a contour direction and connectivity in the original image to be zoomed; and screening out a part of rows or columns from the original image by using the selection coefficients of all rows or columns, trimming the original image, and carrying out scaling processing on the trimmed image according to a preset scaling ratio. Through the visual detection technology, the visual effect and identifiability of the zoomed image content can be greatly improved, the possibility that the image content is deformed when the image is zoomed in an unequal proportion by directly using an interpolation-based image zooming method is reduced, and the accuracy and efficiency of image correction are improved.

Description

Technical Field

[0001] This application relates to the field of image correction technology, and specifically relates to an image intelligent scaling method. Background Art

[0002] In image transmission and storage, image scaling is usually part of data compression. By reducing the image size, the size of the image file is reduced to save the storage space of the image file and improve the transmission efficiency and loading speed of the image during transmission.

[0003] In image scaling technology, the interpolation-based image scaling method can quickly achieve image scaling. However, when performing non-uniform scaling on an image, especially when the image contains important information or details, it usually causes visual distortion of the image content, resulting in a large visual difference between the two images before and after scaling, thereby affecting the observer's recognition and understanding of the image content. Summary of the Invention

[0004] To solve the above technical problems, this application provides an image intelligent scaling method to solve the existing problems.

[0005] An image intelligent scaling method of this application adopts the following technical solutions:

[0006] An embodiment of this application provides an image intelligent scaling method, which includes the following steps:

[0007] Step 1: Obtain the original image to be scaled;

[0008] Step 2: Determine the selection coefficient of each row or column in the original image by analyzing the main color, contour direction, and connectivity in the original image to be scaled; specifically:

[0009] S1: Cluster the pixel points in the original image, and use the clustering labels of all pixel points to construct a main color image; and analyze the average level of all element values in each row or column of the main color image as the main color influence degree of the corresponding row or column;

[0010] S2: For any row or column of pixel points in the original image, respectively use the distribution similarity of the autocorrelation values of the adjacent pixel points in the corresponding row or column in the edge image and the main color image of the original image in different directions to determine the contour color feature value of the row or column of pixel points;

[0011] S3: Obtain the distance between all contour pixel points in the adjacent rows or columns at the corresponding positions between the image before and after cropping the row or column of pixel points on the edge image; combine the distance and the contour color feature value of each row or column in the original image to determine the connectivity change degree of each row or column in the original image;

[0012] S4: Construct the selection coefficient for each row or column in the original image by combining the main color influence degree and the connectivity change degree;

[0013] Step 3: Use the selection coefficients of all rows or columns to screen out some rows or columns from the original image, trim the original image, and scale the trimmed image according to a preset scaling ratio.

[0014] Preferably, before step S1, first obtain the number of rows or columns that need to be trimmed when the original image is scaled proportionally according to the number of rows and columns after preset scaling.

[0015] Preferably, in the clustering process of step S1, the clustering distance is further determined as the color value difference between pixel points.

[0016] Preferably, before analyzing the main color influence degree using the main color image in step S1, further use the proportion degree of the clustering labels of pixel points to replace the clustering labels of the corresponding pixel points to obtain a replaced main color image for analyzing the main color influence degree.

[0017] Preferably, the proportion degree is the ratio of the number of pixel points included in the clustering cluster where the pixel point is located to the total number of pixel points in the original image.

[0018] Preferably, in addition to the method of determining the contour color tone feature value of the pixel points in this row or column in step S2, further consider the average level of the autocorrelation values of the neighboring pixel points at the corresponding position of the pixel points in this row or column in the edge image in different directions.

[0019] Preferably, in step S3, the connectivity change degree of each row or column is further determined by the product of the contour color tone feature value of the corresponding row or column and the distance of the corresponding row or column.

[0020] Preferably, in addition to the method of determining the connectivity change degree of each row or column in step S3, further consider the distribution of all contour pixel points in this row or column, and jointly determine the connectivity change degree of the corresponding row or column with the contour color tone feature value of the corresponding row or column and the distance.

[0021] Preferably, in step S4, the selection coefficient of each row or column is determined by the negative correlation mapping result after the positive fusion of the main color influence degree and the connectivity change degree of the corresponding row or column.

[0022] Preferably, the method of screening the partial rows or columns is further determined as: screening out the number of rows or columns with the largest cropping coefficients from the original image.

[0023] This application has at least the following beneficial effects:

[0024] 1. This application analyzes the dominant colors in the original image that contribute to maintaining the recognizability of the image content, and constructs the influence degree of the dominant color of the corresponding row or column based on the distribution of the dominant colors in the original image. The beneficial effect is to avoid the situation where details in the important color regions that are beneficial to image content recognition in the original image are cropped off during subsequent cropping of the original image, and the situation where details in the important color regions that are beneficial to image content recognition are repeatedly expanded during interpolation of the original image, thereby avoiding deviations in the content of the image caused by image correction.

[0025] 2. This application constructs a contour color feature value based on the consistency change between the obvious directional contours in the original image and the change of the dominant color in the region where the contour is located. The beneficial effect is that it can more accurately evaluate the degree of consistency between the change direction of the contour and the change direction of the dominant color in each column or row of the original image in its corresponding region, improving the accuracy of visual detection.

[0026] 3. This application analyzes the connectivity of the pixel points in the region of the original image where the contour with consistent hue direction appears, and constructs a connectivity change degree by combining the distribution differences of the contour pixel points in the adjacent columns or rows after trimming the corresponding row or column and the contour color feature value. The beneficial effect is to reduce the possibility that the key features carried by the contour with consistent hue direction in the original image are divided into multiple parts or cut off during subsequent cropping of the original image, and the possibility that the key features carried by the contour with consistent hue direction in the original image are overly magnified during interpolation of the original image, ensuring that the content of the image is distorted after image correction.

[0027] 4. This application constructs a selection coefficient by combining the influence degree of the dominant color and the connectivity change degree, completes the trimming of the original image according to the obtained preset scaling ratio, trimming quantity, and the constructed selection coefficient, and realizes the scaling process of the original image based on the obtained trimmed image. The beneficial effect is that by selecting appropriate columns or rows in the original image for trimming, it reduces the possibility that details in the important color regions that are beneficial to image content recognition in the original image are cropped off, the key features carried are divided into multiple parts or cut off, and the situation where details in the important color regions that are beneficial to image content recognition are repeatedly expanded or overly magnified during interpolation of the original image. Through the integrated visual intelligence technology of this application, complex visual detection can be achieved, not only greatly improving the visual effect and recognizability of the image content after scaling processing, but also reducing the possibility of image content distortion caused by non-uniform scaling of the image using the interpolation-based image scaling method directly, improving the accuracy and efficiency of image correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0029] Figure 1 It is a flowchart of an image intelligent scaling method provided by the present application;

[0030] Figure 2 It is a flowchart of the determination process of the selection coefficient for each row or column in the original image provided by the present application;

[0031] Figure 3 It is a schematic diagram of the process of cropping and shrinking the first type of original image provided by an embodiment of the present application;

[0032] Figure 4 It is a schematic diagram of the process of cropping and shrinking the second type of original image provided by an embodiment of the present application;

[0033] Figure 5 It is a schematic diagram of the process of cropping and shrinking the third type of original image provided by an embodiment of the present application. Detailed implementation manners

[0034] To further elaborate on the technical means and effects adopted by the present application to achieve the intended invention purpose, the following, in combination with the drawings and preferred embodiments, details the specific implementation manners, structures, features, and effects of an image intelligent scaling method proposed according to the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0035] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0036] The following specifically describes the specific solution of an image intelligent scaling method provided by the present application in combination with the drawings.

[0037] Specifically, the following image intelligent scaling method is provided. Please refer to Figure 1 , and the method includes the following steps:

[0038] Embodiment 1

[0039] Step 1: Obtain the original image to be scaled.

[0040] In this application, several rows or columns are selected from the original image for trimming to ensure that the number of rows and columns of the image can meet the conditions for equal-proportion scaling of the image, and an image scaling method based on interpolation is used to scale the trimmed original image, so as to reduce the situation of image content distortion when the image is scaled and improve the accuracy and efficiency of image correction.

[0041] For the image correction method of intelligent image scaling, in this embodiment, taking the reduction of the image as an example, first, for the original image A to be scaled, the RGB image A1 and the grayscale image A2 of the original image A are respectively obtained. The acquisition of the RGB image and the grayscale image is a well-known technology, and the specific process will not be elaborated here.

[0042] In other embodiments of this application, several rows or columns can also be selected from the original image to be scaled for cropping, and the number of rows and columns of the image is cropped to the total number of rows and columns of the preset scaled image to complete the scaling process of the original image.

[0043] Step 2: Determine the selection coefficient of each row or column in the original image by analyzing the main color, contour direction, and connectivity in the original image to be scaled.

[0044] To avoid the situation that the key visual information in the original image A to be scaled is cropped after row cropping or column cropping, which affects the recognition of key objects and features in the subsequent image, this application analyzes the main color, contour direction, and connectivity in the original image, and selects appropriate rows or columns to be cropped in the original image A to avoid the loss of key content in the image.

[0045] Before analyzing the selection coefficient of each row or column in the original image, first obtain the number of rows or columns that need to be cropped when the original image is scaled in equal proportion according to the number of rows and columns of the preset scaled image.

[0046] Preferably, in this embodiment, the number of rows or columns that need to be cropped is the minimum number that needs to be cropped when the original image is scaled in equal proportion.

[0047] In other embodiments of this application, the number of rows or columns that need to be cropped and the number of rows and columns can also be selected by itself when the original image is scaled in equal proportion according to the number of color types in the actual original image. Specifically, as an implementation manner in this embodiment, the method for obtaining the number of rows or columns that need to be cropped is specifically as follows:

[0048] (1) Before cropping the original image A, first judge the cropping direction of the original image A.

[0049] Specifically, obtain the total number of rows M1 and the total number of columns N1 of the original image A, as well as the preset total number of rows M2 and the total number of columns N2 of the scaled original image A.

[0050] When the row scaling ratio of the original image A is less than the column scaling ratio, that is, M1 / M2 < N1 / N2, perform column cropping on the original image A;

[0051] Otherwise, when the row scaling ratio of the original image A is greater than the column scaling ratio, that is, M1 / M2 > N1 / N2, perform row cropping on the original image A.

[0052] When M1 / M2 = N1 / N2, that is, at this time, equal-scale scaling can be performed. Before equal-scale scaling, column cropping and row cropping can also be performed on the original image A first.

[0053] Among them, the preset total number of rows M2 and the total number of columns N2 of the scaled original image A are specifically set by the implementer himself / herself when actually scaling the image.

[0054] (2) Secondly, obtain the scaling ratio of the original image A and the number of rows or columns to be cropped. Specifically, taking the column cropping process of the original image A as an example, that is, when M1 / M2 < N1 / N2, and record M1 / M2 as the scaling ratio of the original image A, and obtain the number L that the original image A needs to be cropped, which is used to represent the number of columns of the pixel points that need to be cropped in the original image A.

[0055] Specifically, the method for obtaining the number L that the original image A needs to be cropped is: L = N1 - round(N2×(M1 / M2)); in the formula, M1 and N1 respectively represent the total number of rows and the total number of columns of the original image A; M2 and N2 respectively represent the preset total number of rows and the total number of columns of the scaled original image A; round() is a rounding function.

[0056] When M1 / M2 = N1 / N2, that is, at this time, it is equal-scale scaling. Row cropping and column cropping can also be performed on the original image in sequence according to the cropping requirements of the implementer. Among them, the maximum number of row cropping is M1 - M2, and the maximum number of column cropping is N1 - N2.

[0057] Next, in this application, the process flow chart for determining the selection coefficient of each row or column in the original image is as shown in the appendix Figure 2 as follows:

[0058] S1: Cluster the pixel points in the original image, and use the clustering labels of all pixel points to construct a main color image; and analyze the average level of all element values in each row or column of the main color image as the main color influence degree of the corresponding row or column.

[0059] The dominant color in an image is usually one of the first features that an observer notices when viewing the image, and the dominant color can help convey the theme and information of the image and identify the key content in the image. For example, in a landscape photo, the blue of the sky; in food photography, the color of the food, etc. Therefore, when cropping the original image A, preserving these dominant colors helps ensure the recognizability of the image content while ensuring that the image colors are natural and realistic.

[0060] Preferably, in this embodiment, the clustering distance in the clustering process is the color value difference between pixel points. In other embodiments of the present application, the clustering distance in the clustering process can also be set as the position difference between pixel points.

[0061] It should be noted that for the clustering label of a pixel point, the clustering label is determined according to the clustering algorithm for all pixel points after clustering, where the process of obtaining the clustering label of a pixel point is a well-known technology and the specific process will not be elaborated here.

[0062] In addition, in other embodiments of the present application, before analyzing the influence degree of the dominant color using the dominant color image, the clustering label of the corresponding pixel point is replaced with the proportion degree of the clustering label of the pixel point to obtain a replaced dominant color image for analyzing the influence degree of the dominant color; where the proportion degree is the proportion of the number of pixel points included in the clustering cluster where the pixel point is located to the number of all pixel points in the original image.

[0063] Specifically, as an implementation manner in this embodiment, step S1 is specifically as follows:

[0064] S11: For all pixel points in the original image A, cluster all pixel points according to the RGB value difference corresponding to them in the RGB image A1, and construct a dominant color image B1 with the clustering labels of all pixel points obtained by clustering.

[0065] Specifically, use the K-means clustering algorithm to divide all pixel points in the original image A into K categories, where the Euclidean distance between the RGB values of the pixel points in the RGB image A1 is used as the metric distance in the K-means clustering algorithm, and K is set to 16, which can be set by the implementer himself. The image formed by the clustering labels of all pixel points obtained after clustering according to their positions in the original image A is denoted as the dominant color image B1 of the original image A, which is used to characterize the distribution of the dominant color of the original image A.

[0066] Among them, the K-means clustering algorithm is a well-known technology and the specific process will not be elaborated here. In other implementation manners of this embodiment, other clustering algorithms can also be used, such as the Mean shift clustering algorithm, the DBSCAN algorithm, and the hierarchical clustering algorithm, etc.

[0067] S12: Use the average level of the element values of each column of pixel points in the main color image B1 as the influence degree of the main color for the corresponding column.

[0068] Specifically, taking the j-th column A(j) of the original image A as an example, the mean value of the element values of all pixel points in the j-th column of the main color image B1 is denoted as the influence degree S(j) of the main color for the j-th column A(j), which is used to characterize the influence degree of the main color contained in the pixel points of the j-th column in the original image A on the visual effect of the entire image.

[0069] It should be understood that the larger the proportion of the pixel points contained in the main color corresponding to the j-th column of pixel points in the original image A, that is, the larger the mean value, the greater the influence degree, that is, the larger the influence degree S(j) of the main color. Therefore, in order to retain these main colors that are helpful for maintaining the content recognition of the original image A, the j-th column A(j) of the original image A is less likely to be selected as the column to be cropped.

[0070] As an implementation manner in other embodiments of the present application, before step S12, use the proportion of the number of all pixel points in the cluster where each pixel point in the main color image B1 is located in the original image A to replace the cluster label of the corresponding pixel point to obtain the main color image B2.

[0071] Specifically, respectively count the proportion of the number of all pixel points in the cluster where each pixel point in the main color image B1 is located in the number of all pixel points in the original image A, which is denoted as the proportion degree of the cluster label of the corresponding pixel point and is used to characterize the proportion degree of the pixel points in each main color in the original image A. Then, replace the cluster labels of each pixel point in the main color image B1 with the proportion degree of the cluster label of the corresponding pixel point to obtain the re-replaced main color image B2.

[0072] S2: For any row or column of pixel points in the original image, respectively use the distribution similarity of the autocorrelation values in different directions of the adjacent pixel points in the corresponding row or column in the edge image and the main color image of the original image to determine the contour color feature value of the row or column of pixel points.

[0073] Since the contour information in the image usually helps in the recognition of key objects and features in the image, when cropping the image, if the loss degree of the image contour information closely related to the main color in the image is too large, there will be a difference in the color distribution between the cropped image and the original image, which will further lead to the loss of information about the overall structure of the image.

[0074] Generally, in an image, contours with obvious directions, such as straight lines and curves, usually can provide important information about the shape and structure of the objects in the image. Moreover, the more consistent the change direction of these contours is with the main change direction of the main color tone in the area where they are located, the clearer these contours can define the boundaries of the objects, and thus better reflect the characteristics of the key objects in the image.

[0075] This is because edges usually correspond to abrupt changes in brightness or color in the image. When the direction of the contour is consistent with the main change direction of the main color tone in the area, the consistency of the color tone change direction will strengthen this abrupt change, making the edge more obvious. And when the human visual system processes an image, it tends to interpret consistent contours and color tone changes as continuous boundaries. This consistency helps to identify the objects in the image. Therefore, it is more necessary to reduce the possibility that the pixel points on these contours are cropped off. And for the convenience of subsequent processing, these contours are called contours with consistent color tone directions.

[0076] Accordingly, in this embodiment, the Canny edge detection algorithm is used to extract the edge image C1 of the grayscale image A2. The edge image C1 is a binary image. For the convenience of subsequent processing, the pixel points in the foreground part (gray value is 255) of the edge image C1 are denoted as contour pixel points.

[0077] Among them, the Canny edge detection algorithm is a well-known technology, and the specific process will not be elaborated here. In other embodiments of the present application, the Sobel operator or the Prewitt operator can also be used to obtain the edge image of the grayscale image A2.

[0078] Specifically, as an implementation manner of this embodiment, step S2 is specifically as follows:

[0079] S21: For any column of pixel points in the original image A, respectively analyze the autocorrelation values of all pixel points in the window in different directions by using the adjacent pixel points corresponding to this column of pixel points in the edge image C1 and the main color tone image B1, and respectively form a contour direction vector p1(j) and a main color tone direction vector p2(j).

[0080] Specifically, taking the j-th column A(j) in the original image A as an example, a window with a size of M1×5 centered on the j-th column in the edge image C1 is obtained, where M1 is the total number of rows of the original image A, and the window size can be set by the implementer himself.

[0081] It should be noted that in the edge image C1, when a complete window does not exist for a certain column, its incomplete part is analyzed according to the case where the element value of the pixel point is 0.

[0082] Next, the autocorrelation values of the window in all preset angular directions are calculated using the gray-level co-occurrence matrix, and all the autocorrelation values obtained within the window are arranged into a vector in ascending order of the preset angles, denoted as the contour direction vector p1(j) of the j-th column A(j), which is used to characterize the consistency of the change directions of all contour pixel points of the j-th column A(j) in the original image A within its neighboring region in the preset angular directions.

[0083] It should be understood that the larger the correlation value corresponding to a certain preset angle in the contour direction vector p1(j), the more the contours in the window region have the characteristic of continuous change in the direction corresponding to the preset angle.

[0084] The preset angles in this embodiment include 0°, 45°, 90°, and 135°, which can be set by the implementer himself. The calculation of the autocorrelation value in the gray-level co-occurrence matrix is a well-known technology, and the specific process will not be elaborated here.

[0085] Then, a window of size M1×5 centered on the j-th column in the dominant color image B1 is obtained, and using the same method as the contour direction vector p1(j), the dominant color direction vector p2(j) of the j-th column A(j) is obtained, which is used to characterize the consistency of the change directions of the dominant colors corresponding to all pixel points of the j-th column A(j) in the original image A within its window region in the preset angular directions.

[0086] It should be noted that in the dominant color image B1, when a complete window does not exist for a certain column, the incomplete part is assigned according to the average element value of all pixel points in the dominant color image B1.

[0087] It should be understood that the larger the correlation value corresponding to a certain preset angle in the dominant color direction vector p2(j), the more the dominant color in its window region has the characteristic of continuous change in the direction corresponding to the preset angle.

[0088] S22: Calculate the similarity between the contour direction vector p1(j) and the dominant color direction vector p2(j) as the contour color feature value h(j,i) of the j-th column A(j) in the original image A.

[0089] Specifically, the cosine similarity between the dominant color direction vector p1(j) and the dominant color direction vector p2(j) is denoted as the contour color feature value h(j) of the j-th column A(j), which is used to characterize the degree of consistency between the change direction of the contour of the j-th column A(j) in the original image A within its window region and the change direction of the dominant color in its window region.

[0090] It should be understood that the greater the cosine similarity, the more consistent the change direction of the contour is with the change direction of the main color tone, that is, the greater the contour color tone eigenvalue h(j).

[0091] Among them, the calculation method of the cosine similarity is a well-known technology, and the specific calculation process will not be elaborated here. In other implementation manners of this embodiment, other similarity calculation methods between vectors can also be used, such as the Pearson correlation coefficient, the reciprocal of the Euclidean distance, etc.

[0092] As an implementation manner in other embodiments of this application, before using similarity to determine the contour color tone eigenvalue h(j,i) of the j-th column A(j) in step S22, the mean value of all components in the contour direction vector p1(j) of the j-th column A(j) can also be considered, and it is denoted as the contour direction distinctness h1(j) of the j-th column A(j), which is used to characterize the degree of obvious directionality of the contour of the j-th column A(j) in the original image A in its corresponding region. The greater the mean value, the more obvious the directionality of the contour of the j-th column A(j) in its corresponding window region, that is, the greater the contour direction distinctness h1(j).

[0093] Thus, in an implementation manner in other embodiments of this application, the contour color tone eigenvalue h(j) of the j-th column A(j) can be further determined by the product result of the similarity between the contour direction vector p1(j) and the main color tone direction vector p2(j), and the contour direction distinctness h1(j) of the j-th column A(j).

[0094] This method is not only used to characterize the degree of consistency between the change direction of the contour and the change direction of the main color tone of the j-th column A(j) in the original image A in its corresponding window region, but also takes into account the degree of obvious directionality of the contour of the j-th column A(j) in its corresponding region. The greater the degree of consistency and obviousness, the greater the contour color tone eigenvalue h(j).

[0095] In addition, in other embodiments of this application, in addition to the method of determining the contour color tone eigenvalue of the pixel points in this row or column in step S2, the average level of the autocorrelation values of the adjacent pixel points corresponding to this row or column in the edge image at different angles is further considered.

[0096] S3: Obtain the distances between all contour pixel points in the adjacent rows or columns at the corresponding positions between the images before and after cropping the pixel points in this row or column on the edge image; combine the distances of each row or column in the original image and the contour color tone eigenvalue to determine the connectivity change degree of each row or column in the original image.

[0097] Under normal circumstances, the connectivity of pixel points on the contour in an image is very important for the recognition of key objects and features in the image. The connectivity of contour pixel points refers to the connection relationship between contour pixel points in the image. If the image is improperly cropped, it may damage the connectivity of contour pixel points in the image, resulting in key objects and features in the image being segmented into multiple parts or being cut off.

[0098] Therefore, to reduce the possibility that the key features carried in the contour of the hue direction consistency of the original image A are segmented into multiple parts or cut off, the connectivity of the contour pixel points belonging to the contour of the hue direction consistency in the original image A should remain unchanged or change slightly before and after cropping.

[0099] Preferably, in this embodiment, the degree of change in connectivity of each row or column is determined by the result of multiplying the contour hue feature value of the corresponding row or column by the distance of the corresponding row or column.

[0100] It should be noted that when a certain column or row does not have its complete adjacent row or column, its incomplete part is analyzed according to the situation where there are no contour pixel points.

[0101] Specifically, as an implementation manner of this embodiment, step S3 is specifically as follows:

[0102] S31: Combine all the contour pixel points within the three adjacent columns corresponding to the j-th column A(j) in the edge image C1 to form the connectivity matrix u1(j) of the j-th column A(j). Combine all the contour pixel points within the three adjacent columns corresponding to the position after removing the j-th column in the edge image C1 to form the cropped connectivity matrix u2(j) of the j-th column A(j).

[0103] Specifically, taking the j-th column A(j) in the original image A as an example, extract all the pixel points within the three adjacent columns (the (j - 1), j, and (j + 1) columns) of the j-th column in the edge image C1. Arrange all the extracted pixel points in ascending order of the horizontal and vertical coordinates of the pixel points to form a matrix, and assign the element value corresponding to all the contour pixel points among all the pixel points in this matrix as 1, and assign the element values corresponding to the remaining pixel points as 0. Denote the matrix after assignment as the connectivity matrix u1(j) of the j-th column A(j).

[0104] Using the same method as the connectivity matrix u1(j), obtain the connectivity matrix u1(j + 1) of the (j + 1)-th column A(j + 1) in the original image A. Replace the first column in the connectivity matrix u1(j + 1) with the first column in the connectivity matrix u1(j). Denote the matrix obtained after replacement as the cropped connectivity matrix u2(j) of the j-th column A(j), which is used to characterize the distribution of the contour pixel points within the three adjacent columns corresponding to the position of the j-th column A(j) in the cropped original image A after the j-th column A(j) in the original image A is cropped.

[0105] In other embodiments of this embodiment, 5-neighbor columns, 7-neighbor columns, etc. can also be used, and the specific settings are determined by the implementer himself.

[0106] S32: Denote the distance between the connectivity matrix u1(j) of the j-th column A(j) and the cropped connectivity matrix u2(j) as the contour connectivity change degree U(j) of the j-th column A(j).

[0107] Denote the Manhattan distance between the connectivity matrix u1(j) and the cropped connectivity matrix u2(j) as the contour connectivity change degree U(j) of the j-th column A(j), which is used to characterize the degree of change in the connectivity of the pixel points in the j-th column of the original image A after cropping. The greater the difference degree between the corresponding elements in the connectivity matrix u1(j) and the cropped connectivity matrix u2(j), that is, the greater the Manhattan distance, the greater the degree of change in connectivity, that is, the greater the contour connectivity change degree U(j).

[0108] Among them, the calculation process of the Manhattan distance is a well-known technology and will not be elaborated here. In other embodiments of this embodiment, other methods for calculating the difference degree between matrices can also be used, such as the Euclidean distance, the Minkowski distance, etc.

[0109] S33: Combine the contour connectivity change degree and the contour tone feature value of each column in the original image A to determine the connectivity change degree of each column in the original image A.

[0110] Specifically, taking the j-th column A(j) in the original image A as an example, use the contour connectivity change degree and the contour tone feature value of the j-th column A(j) to obtain the connectivity change degree H(j) of the j-th column A(j), which is used to characterize the degree of change in the connectivity of the contour pixel points belonging to the contour of the tone direction consistency in the j-th column of the original image A before and after cropping in the j-th column.

[0111] Among them, the expression of the connectivity change degree H(j) of the j-th column A(j) is: H(j) = h(j) × U(j); in the formula, U(j) represents the contour connectivity change degree of the j-th column A(j) in the original image A; h(j) represents the contour tone feature value of the j-th column A(j) in the original image A.

[0112] It should be understood that the more obvious the directionality of the contour of the j-th column A(j) in the original image A within its window area, and at the same time, the more consistent the change direction of the contour and the main tone corresponding to the pixel points within its window area, that is, the greater h(j), the greater the possibility that the contour pixel points in the j-th column A(j) belong to the contour pixel points of the contour of the tone direction consistency in the original image A;

[0113] Moreover, the greater the degree of change in the connectivity of the pixel points in the j-th column A(j) before and after cropping the j-th column in the original image A, that is, the greater U(j) is, the greater the degree of change in the connectivity of the contour pixel points belonging to the contour of the hue direction consistency contour in the original image A in the j-th column before and after cropping the j-th column, that is, the greater the connectivity change degree H(j) is.

[0114] In other embodiments of the present application, in addition to using the contour connectivity change degree and the contour hue feature value of each column to determine the connectivity change degree of each column in step S33, the distribution of the contour pixel points in this column can also be considered, and jointly with the contour hue feature value and the contour connectivity change degree of this column, to determine the connectivity change degree of this column.

[0115] As an implementation manner in other embodiments of the present application, specifically, the expression of the connectivity change degree H(j) of the j-th column A(j) of the original image A can also be set as: H(j) = b(j) × h(j) × U(j); where b(j) represents the mean value of the pixel points in the j-th column of the edge image C1.

[0116] It should be understood that the more the contour pixel points are distributed in the j-th column A(j), that is, the greater b(j) is, and the more obvious the directionality of the contour of the j-th column A(j) in its window area is, and at the same time, the more consistent the change direction of the contour and the main hue corresponding to the pixel points in its window area is, that is, the greater h(j) is, then the more the number of contour pixel points in the j-th column A(j) that contain the contour of the hue direction consistency contour in the original image A, that is, the greater h(j) × b(j) is, and the greater the degree of change in the connectivity of the pixel points in the j-th column A(j) before and after cropping the j-th column in the original image A, that is, the greater U(j) is, then after cropping the j-th column in the original image A, the greater the degree of change in the connectivity of the contour pixel points belonging to the contour of the hue direction consistency contour in the original image A in the j-th column before and after cropping, that is, the greater the connectivity change degree H(j) is.

[0117] S4: Construct the selection coefficient of each row or column in the original image by combining the main hue influence degree and the connectivity change degree.

[0118] Preferably, in this embodiment, the selection coefficient of each row or column is determined by the negative correlation mapping result after the positive fusion of the main hue influence degree and the connectivity change degree of the corresponding row or column.

[0119] It can be understood that the positive fusion is a fusion method such as addition and multiplication between data. The specific positive fusion method is determined by the implementer according to the actual situation to select a suitable fusion method, and the present application does not make special restrictions. Optionally, the negative correlation mapping can be implemented by methods such as negative linear mapping, negative exponential mapping or setting adjustment parameters.

[0120] In other embodiments of the present application, it is also possible to first perform negative correlation mapping on the main color influence degree and connectivity change degree of each row or column respectively, and then perform positive fusion on the two negative correlation mapping results to obtain the selection coefficient corresponding to the row or column.

[0121] As an implementation manner of this embodiment, taking the j-th column A(j) of the original image A as an example, the selection coefficient of the j-th column A(j) of the original image A is denoted as W(j), which is used to characterize the possibility that the j-th column in the original image A is selected as the column to be cropped.

[0122] Specifically, the expression of the selection coefficient W(j) of the j-th column A(j) of the original image A is: In the formula, S(j) and H(j) respectively represent the main color influence degree and connectivity change degree of the j-th column A(j) of the original image A; β is a preset tuning parameter coefficient. To prevent the denominator from being 0, where β takes a value of 0.01.

[0123] It should be understood that the greater the influence degree of the main color contained in the pixel points of the j-th column in the original image A on the visual effect of the entire image, that is, the greater S(j), in order to retain the main color that helps to maintain the content recognition of the original image A, the j-th column A(j) of the original image A is less likely to be selected as the column to be cropped, that is, W(j) should be smaller; at the same time, the greater the degree of change in the connectivity of the contour pixel points belonging to the contour of the hue direction consistency in the original image A in the j-th column before and after cropping, in order to reduce the possibility that the key features carried in the contour of the hue direction consistency in the original image A are divided into multiple parts or cut off, the j-th column in the original image A is less likely to be selected as the column to be cropped, that is, W(j) should be smaller.

[0124] Step three: Use the selection coefficients of all rows or columns to screen out some rows or columns from the original image, trim the original image, and perform scaling processing on the trimmed image according to a preset scaling ratio.

[0125] According to step two, the cropping coefficients can be calculated for each row or column in the original image A, and the L rows or columns with the largest cropping coefficients are screened out from the original image A, where L is the number of rows or columns that the original image A needs to be cropped. The selected L rows or columns are deleted from the original image A, and the original image A after deletion is denoted as the cropped image of the original image A.

[0126] In other embodiments of the present application, it is also possible to respectively screen out the row or column region with the largest sum value of the selection coefficients of all rows or columns in the row or column region composed of all consecutive L rows or columns in the original image A. Delete the screened row or column region in the original image A, and denote the original image A after deletion as the cropped image of the original image A.

[0127] Furthermore, use an interpolation-based image scaling method to downscale the cropped image of the original image A, and the downscaling ratio is scaled using the scaling ratio of the original image A calculated in step two.

[0128] Among them, the interpolation-based image scaling method is a well-known technology, and the specific process will not be elaborated here. The downscaled cropped interpolation image obtained will be used as the output to complete the downscaling process of the original image.

[0129] In this embodiment, Figure 3 、 Figure 4 and Figure 5 are respectively the schematic diagrams of the cropping and downscaling processes of three different original images in this embodiment.

[0130] Among them, the original RGB image, the main color image B2, and the edge image C1 are all obtained on the basis of their corresponding original images, and are all used to analyze the rows or columns to be cropped in the original image.

[0131] The cropped image is the image obtained by screening out some rows or columns from the original image and trimming the original image in step three. The cropped interpolation image is the image obtained by downscaling the cropped image using an interpolation-based image scaling method. The ordinary interpolation image is the image obtained by directly downscaling the original image using an interpolation-based image scaling method.

[0132] Embodiment 2

[0133] For the trimming method of image upscaling, obtain the total number of rows M1 and the total number of columns N1 of the original image A, as well as the preset total number of rows M3 and the total number of columns N3 of the upscaled original image A.

[0134] 1. When the original image A is upscaled in an unequal ratio, that is, when M3 / M1≠N3 / N1, first obtain the minimum upscaling ratio and the interpolation direction of the original image A, and perform an equal-ratio upscaling process on the original image A according to the minimum upscaling ratio. Then, perform interpolation processing on the rows or columns that have not reached the preset size.

[0135] It should be noted that the minimum upscaling ratio is to upscale one direction of the rows or columns of the original image A to the preset size, and the other direction needs to be expanded.

[0136] 1.1 Specifically, when M3 / M1 < N3 / N1, perform column expansion processing on the original image A, and use M3 / M1 as the minimum magnification ratio of the original image A. Otherwise, perform row expansion processing on the original image A, and use N3 / N1 as the minimum magnification ratio of the original image A. Then, perform proportional magnification processing on the original image A using an interpolation-based image scaling method, where the minimum magnification ratio of the original image A is used as the magnification ratio in the method to obtain the proportional magnification image DA of the original image A.

[0137] 1.2 Secondly, obtain the interpolation quantity according to the interpolation direction of the original image A, and perform row or column expansion processing on the proportional magnification image DA of the original image A according to the interpolation quantity. Then, output the image obtained after the expansion processing as the final magnification image of the original image A to complete the non-uniform magnification processing of the original image A.

[0138] Specifically, taking the column expansion processing of the original image A as an example, the method for obtaining the column expansion processing quantity L1 is: L1 = N3 - round(N1×(M3 / M1)), where round() is the rounding function.

[0139] Divide the proportional magnification image DA into L1 column regions according to the number of columns to obtain L1 column regions of the proportional magnification image DA, where L1 is the quantity that the original image A needs to be expanded. The l1-th (l1 < L1) column region DA(l1) of the proportional magnification image DA is the region formed by the (l1 - 1)*round(N4 / L1)-th to the l1*round(N4 / L1)-th columns in the proportional magnification image DA, and the L1-th column region DA(L1) of the proportional magnification image DA is the region formed by the (L1 - 1)*round(N4 / L1)-th to the N4-th columns in the proportional magnification image DA. N4 is the total number of columns in the proportional magnification image DA, and round() is the rounding function.

[0140] Use the same calculation method for the selection coefficient in the shrinking embodiment of the original image A to calculate the selection coefficient of each column in the proportional magnification image DA respectively, and screen out the column region with the largest sum of selection coefficients from all column regions in the proportional magnification image DA. Use the nearest neighbor interpolation algorithm to perform column interpolation processing on the screened column regions in the proportional magnification image DA respectively, and output the image obtained after the interpolation processing as the final magnification image of the original image A to implement the column expansion processing of the original image. Among them, the nearest neighbor interpolation algorithm is a well-known technology and will not be elaborated. In other embodiments, the bilinear interpolation method can also be used.

[0141] 2. When M3 / M1 = N3 / N1, that is, it is an equal-proportion magnification at this time. Before the equal-proportion magnification, the original image A can also be sequentially subjected to row expansion processing and column expansion processing, and the image obtained after the expansion processing is used as the final magnified image of the original image A for output, where the maximum number of row expansions is M3 - M1, and the maximum number of column expansions is N3 - N1.

[0142] Another simple implementation method for image magnification can be to magnify the image by the maximum ratio, and then crop the rows and columns that exceed the preset size, and finally obtain the magnified image.

[0143] Each embodiment in this application is described in a progressive manner. For the same or similar parts between each embodiment, reference can be made to each other, and the key point of each embodiment is to illustrate the differences from other embodiments.

[0144] It should be noted that unless otherwise specified and limited, terms such as "including", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such article or device. Without more limitations, an element defined by the statement "including one..." does not exclude the existence of another identical element in the article or device including the said element. In addition, the term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0145] Those skilled in the art will easily think of other implementation schemes of this application after considering the specification and practicing the invention herein. This application aims to cover any variations, uses or adaptive changes of this application, and these variations, uses or adaptive changes follow the general principles of this application and include common general knowledge or conventional technical means in the technical field not invented by this application.

[0146] It should be understood that this application is not limited to the exact structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. An image intelligent scaling method, characterized in that: The method comprises the following steps: Step 1: Get the original image to be scaled; Step 2: Determine the selection coefficient of each row or column in the original image by analyzing the main color tone, contour direction and connectivity in the original image to be scaled; specifically: S1: Cluster the pixels in the original image and construct a main tone image using the clustering labels of all pixels; and analyze the average level of all element values ​​in each row or column in the main tone image as the main tone influence of the corresponding row or column; S2: For any row or column of pixels in the original image, the contour tone feature value of the pixel in the row or column is determined by using the distribution similarity of the autocorrelation values ​​of the neighboring pixels in the corresponding row or column in the edge image and the main tone image of the original image in different directions; S3: obtaining the distances of all contour pixels in the corresponding adjacent rows or columns between the image before and after cropping of the row or column of pixels on the edge image; combining the distances of each row or column in the original image and the contour tone feature values ​​to determine the connectivity change degree of each row or column in the original image; S4: Construct the selection coefficient of each row or column in the original image by combining the influence of the main color tone and the connectivity change degree; Step 3: Filter out some rows or columns from the original image using the selection coefficients of all rows or columns, trim the original image, and scale the trimmed image according to a preset scaling ratio.

2. The method for intelligent image scaling according to claim 1, characterized in that: Before step S1, the number of rows or columns that need to be trimmed when the original image is proportionally scaled according to the preset number of rows and columns after scaling is first obtained.

3. The method for intelligent image scaling according to claim 1, wherein: In the clustering process of step S1, the cluster distance is further determined as the color value difference between the pixels.

4. The method for intelligent image scaling according to claim 3, characterized in that: Before using the main tone image to analyze the main tone influence in step S1, the proportion of the cluster labels of the pixels is further used to replace the cluster labels of the corresponding pixels to obtain the replaced main tone image for analyzing the main tone influence.

5. The method for intelligent image scaling according to claim 4, characterized in that: The proportion is the ratio of the number of pixels in the cluster where the pixel is located to the number of all pixels in the original image.

6. The method for intelligent image scaling according to claim 1, characterized in that: In addition to the method of determining the contour tone characteristic value of the row or column of pixels in step S2, the average level of the autocorrelation values ​​of the neighboring pixels at the corresponding position of the row or column of pixels in the edge image in different directions is further considered.

7. The method for intelligent image scaling according to claim 1, characterized in that: In step S3, the connectivity variation degree of each row or column is further determined by the result of multiplying the contour tone characteristic value of the corresponding row or column by the distance of the corresponding row or column.

8. The method for intelligent image scaling according to claim 7, characterized in that: In addition to the method of determining the connectivity variation of each row or column in step S3, the distribution of all contour pixels of the row or column is further considered, and the connectivity variation of the corresponding row or column is determined together with the contour tone characteristic value of the corresponding row or column and the distance.

9. The method for intelligent image scaling according to claim 1, characterized in that: In step S4, the selection coefficient of each row or column is determined by the negative correlation mapping result after positive fusion of the main color influence and the connectivity change degree of the corresponding row or column.

10. The method for intelligent image scaling according to claim 2, wherein: The method for screening some rows or columns is further determined as: screening out rows or columns with the largest number of rows or columns of cropping coefficient from the original image.

Images