An image processing method, related device, storage medium and program product

Abstract

Embodiments of the present application disclose an image processing method, related equipment, a storage medium and a program product, wherein the method comprises: obtaining a target gradient image of a target image to be processed, the value of each pixel point in the target gradient image being used to indicate the pixel change amplitude of the pixel point at the corresponding position in the target image, the pixel change amplitude comprising the difference between the pixel values of the pixel point and the adjacent pixel point at the corresponding position in the target image; determining an amplitude change parameter according to the pixel change amplitude corresponding to each pixel point in the target gradient image, the amplitude change parameter being used to indicate the change amplitude of the pixel change amplitude in the target gradient image; and performing scanning processing on the target image based on the amplitude change parameter, determining a target jump position with a change amplitude greater than a target change amplitude from the target image according to the corresponding pixel change amplitude, and determining the image boundary of the target image according to the boundary line where the target jump position is located, which can improve the accuracy of the determined image boundary.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method, related equipment, storage medium, and program product. Background Technology

[0002] With the development of computer technology, image processing methods for digital images have become increasingly diverse. For example, parts of an image can be deleted, or frosted glass effects or black-and-white borders can be added to make the subject of the image more prominent. In traditional image processing methods, because the degree of blurring in frosted glass effects is unrestricted and the effect depends on the specific image content, there is currently no fixed pattern for recognizing frosted glass effects in images. Therefore, how to accurately detect added frosted glass effects in images has become a current research hotspot. Summary of the Invention

[0003] This application provides an image processing method, related equipment, storage medium, and program product that can improve the accuracy of the determined image boundaries.

[0004] On one hand, embodiments of this application provide an image processing method, including:

[0005] Obtain the target gradient map of the target image to be processed. The value of each pixel in the target gradient map is used to indicate the pixel change amplitude of the pixel at the corresponding position in the target image. The pixel change amplitude includes the difference between the pixel value of the pixel at the corresponding position in the target image and the pixel value of the adjacent pixel.

[0006] Based on the pixel change magnitude corresponding to each pixel in the target gradient map, an amplitude change parameter is determined, which is used to indicate the magnitude of the pixel change magnitude in the target gradient map.

[0007] The target image is scanned based on the amplitude change parameter to determine the target jump position in the target image where the change amplitude of the corresponding pixel is greater than the target change amplitude, and the image boundary of the target image is determined based on the boundary line where the target jump position is located.

[0008] In another aspect, embodiments of this application provide an image processing apparatus, including:

[0009] The acquisition unit is used to acquire the target gradient map of the target image to be processed. The value of each pixel in the target gradient map is used to indicate the pixel change amplitude of the pixel at the corresponding position in the target image. The pixel change amplitude includes the difference between the target pixel value of the pixel at the corresponding position in the target image and the target pixel value of the adjacent pixel.

[0010] The determining unit is used to determine the amplitude change parameter based on the pixel change amplitude corresponding to each pixel point in the target gradient map. The amplitude change parameter is used to indicate the change magnitude of the pixel change amplitude in the target gradient map.

[0011] The scanning unit is used to scan the target image based on the amplitude change parameter, determine the target jump position in the target image where the change amplitude of the corresponding pixel change is greater than the target change amplitude, and determine the image boundary of the target image based on the boundary line where the target jump position is located.

[0012] Furthermore, embodiments of this application also provide a computer device, including:

[0013] A processor, the processor being adapted to implement one or more computer instructions;

[0014] A storage medium storing one or more computer instructions adapted to be loaded by the processor and executed by the image processing method described above.

[0015] In another aspect, embodiments of this application provide a storage medium storing one or more computer instructions, which are adapted to be loaded by a processor and executed by the above-described image processing method.

[0016] In another aspect, embodiments of this application provide a computer program product or a computer program, the computer program product including a computer program stored in a storage medium; a processor reads the computer program from the storage medium, and the processor executes the computer program, causing the computer device to perform the above-described image processing method.

[0017] In this embodiment, the principle that pixel change amplitudes are typically larger in image regions without special effects (e.g., frosted glass effect) and smaller in image regions with special effects is fully utilized. By acquiring a target gradient map of the target image using a computer device, and further constructing amplitude change parameters based on the pixel values ​​in this target gradient map, a parameter reflecting the amplitude change of pixel values ​​within the corresponding image region is obtained. This allows the computer device to accurately determine the image region with special effects based on the amplitude change parameter, thereby enabling the computer device to determine a more accurate image boundary. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of a target image provided in an embodiment of this application;

[0020] Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0021] Figure 3a This is a schematic diagram of a target image and its target gradient map provided in an embodiment of this application;

[0022] Figure 3b This is a schematic diagram of a scanning direction and its corresponding boundary line direction provided in an embodiment of this application;

[0023] Figure 4 This is a schematic flowchart of another image processing method provided in the embodiments of this application;

[0024] Figure 5 This is a schematic diagram of a video file corresponding to a video frame provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0027] This application proposes an image processing scheme that can be executed by a computer device. In this scheme, the computer device can calculate the magnitude of pixel change in the target image by determining the magnitude of pixel change at each pixel point. Since there is a significant difference between the magnitude of pixel change in the image region with a frosted glass effect and that in the image region without a frosted glass effect, the computer device can determine the image boundary in the target image based on the relationship between the magnitude of the change and the target magnitude of the change. This further improves the accuracy of the image boundary determined by the computer device. The target image can be a single image or a video frame in a video file; the image boundary can refer to the display boundary of the image region without a frosted glass effect in the target image. For example, see [link to relevant documentation]. Figure 1 As shown, Figure 1 Both the boundary lines marked with 10 and the boundary lines marked with 11 can be referred to as the image boundaries of the target image. Therefore, it is easy to understand that when a computer device processes the target image using the image processing scheme proposed in this application, it can accurately determine the image regions in the target image that have a frosted glass effect. To clearly describe the data processing scheme proposed in this application, the following examples all use regions in the target image that have a frosted glass effect as examples to provide a detailed description of the embodiments of this application.

[0028] Based on the above description, the general principle of the image processing scheme proposed in this application can be summarized as follows: The computer device first acquires a target gradient map of the target image. The values ​​of the pixels in this target gradient map can be used to describe the magnitude of pixel change at the corresponding pixel in the target image. Based on this, the computer device can determine an amplitude change parameter according to the pixel values ​​in the target gradient map. This amplitude change parameter indicates the magnitude of pixel change in the target gradient map. Therefore, the computer device can use the amplitude change parameter to determine locations in the target image where the magnitude of pixel change is large (referred to as "jump positions"). Consequently, the computer device can determine the boundary line based on the jump positions, and this boundary line can serve as the image boundary. In other words, the computer device can determine the image boundary of the target image based on the amplitude change parameter.

[0029] In specific implementations, the computer devices mentioned above can be terminal devices or servers. Specifically, terminal devices can include, but are not limited to: smartphones, tablets, laptops, desktop computers, in-vehicle terminals, smart home appliances, smartwatches, smart voice interaction devices, etc.; servers can include, but are not limited to: independent physical servers, server clusters or distributed systems composed of multiple physical servers, and cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. This application does not impose specific limitations on these.

[0030] Please see Figure 2 , Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this application. This image processing method is based on the aforementioned image processing scheme; therefore, it can be understood that this image processing method can be executed by the aforementioned computer device. Specifically, as... Figure 2 As shown, the method includes steps S201-S203:

[0031] S201. Obtain the target gradient map of the target image to be processed.

[0032] As described above, the value of each pixel in the target gradient map is used to indicate the pixel change amplitude of the corresponding pixel in the target image. The pixel change amplitude includes the difference between the target pixel value of the corresponding pixel in the target image and its neighboring pixels.

[0033] In a specific embodiment, the target gradient map can be a second-order gradient map, such as a Laplacian gradient map or a difference-of-Gaussian gradient map. In this case, the value of each pixel in the target gradient map is the second-order gradient magnitude. The second-order gradient magnitude can be used to describe the magnitude of change in the grayscale value of a pixel in the target image corresponding to that second-order gradient magnitude. This magnitude specifically indicates the difference in grayscale value between a pixel in the target image and its neighboring pixels. Therefore, when the target gradient map is a second-order gradient map, the aforementioned target pixel value refers to the grayscale value of the pixel in the target image. To better understand the embodiments of this application, the following detailed explanation of how a computer device acquires a target gradient map is given using a Laplacian gradient map as an example.

[0034] Specifically, the computer device can use the Laplacian operator to determine the second-order gradient corresponding to each pixel in the target image, thereby determining the magnitude of the second-order gradient for each pixel. This allows the computer device to construct a target gradient map based on the determined second-order gradient magnitudes. For example, the target image and its corresponding second-order gradient map can be found in [reference needed]. Figure 3a As shown, Figure 3a In this diagram, the image marked with 31 is the target image, and the image marked with 32 is the corresponding Laplacian gradient map of the target image. The Laplacian operator mentioned above is a second-order differential operator and a commonly used edge detection operator. Therefore, computer devices can use the Laplacian gradient map to measure the second-order gradient of local regions in the target image. In this case, if there is a frosted glass effect region in the target image, the second-order gradient magnitude within the frosted glass effect region is usually relatively small, and the variation between different second-order gradient magnitudes is not significant. However, the variation between different second-order gradient magnitudes is larger in the normal image region (i.e., the image region without the frosted glass effect).

[0035] S202. Determine the amplitude change parameter based on the pixel change amplitude corresponding to each pixel point in the target gradient map.

[0036] Since the amplitude variation parameter can be used to indicate the magnitude of pixel changes in the target gradient map, it can be understood that the amplitude variation parameter is determined based on multiple pixel changes, and essentially, it can be understood as the variance of these multiple pixel changes. In practical applications, since these multiple pixel changes are usually composed of the pixel changes corresponding to each pixel in any image region of the target image, the amplitude variation parameter can refer to the variance of the pixel changes corresponding to that arbitrary image region. For example, assuming the target image includes image region A, which consists of 200 pixels, the computer device can determine the amplitude variation parameter of image region A based on the variance of the pixel changes corresponding to each of these 200 pixels.

[0037] Based on the above description, it is easy to understand that because the changes in the amplitudes of the second-order gradients within the frosted glass effect region are small, the amplitude variation parameter corresponding to the frosted glass effect region is usually small. Similarly, because the changes in the amplitudes of the second-order gradients within the normal image region are large, the amplitude variation parameter corresponding to the normal image region is usually large. Therefore, a computer device can determine whether an arbitrary image region is a frosted glass effect region based on the magnitude of the amplitude variation parameter corresponding to that region; or, in other words, a computer device can determine whether an arbitrary image region is a frosted glass effect region based on the magnitude of the change in pixel amplitude within that region.

[0038] S203. Based on the amplitude change parameter, the target image is scanned and processed to determine the target jump position in the target image where the change amplitude of the corresponding pixel is greater than the target change amplitude, and the image boundary of the target image is determined according to the boundary line where the target jump position is located.

[0039] The image boundary of the target image can be the display boundary where the non-frosted glass effect area begins to appear, specifically the boundary line between the frosted glass effect area and the non-frosted glass effect area. The target transition position can be represented by the position of a pixel in the target image. Therefore, the corresponding pixel change amplitude can be understood as the change amplitude of each pixel in the image area for which the amplitude change parameter needs to be determined. The boundary line of the target transition position can be in any direction (e.g., horizontal, vertical, etc.), determined specifically by the direction in which the computer device scans the target image. Please refer to [link to details]. Figure 3b Assuming Figure 3b The area marked 331 represents a pixel in the target image. Therefore, if a computer scans the target image horizontally (from left to right or from right to left), the direction of the boundary line where the target jumps is located is vertical (i.e., longitudinal). Figure 3bThe boundary line marked with 332 in the image. If the computer device scans the target image vertically (from top to bottom or from bottom to top), the direction of the boundary line where the target jumps is located is horizontal (i.e., laterally, such as...). Figure 3b (The boundary line marked by 341). For ease of explanation, unless otherwise specified, the following will use the example of a computer device scanning the target image horizontally, with the boundary line where the target jump position is located as the horizontal direction, to describe the relevant steps of the embodiments proposed in this application in detail.

[0040] In this embodiment, the image boundary is determined based on the boundary line where the target jump position is located. The target jump position is determined by a computer device scanning the target image based on the magnitude of pixel changes. Since the magnitude of pixel changes within the effective image area is typically small, determining the target jump position based on the magnitude of changes improves the accuracy of the determined target jump position, thereby improving the accuracy of the image boundary determined based on the boundary line where the target jump position is located. Furthermore, by scanning the target image, each pixel in the target image can be used to determine the target jump position, further improving the accuracy of the target jump position to a certain extent, and thus further improving the accuracy of the image boundary.

[0041] Please see Figure 4 , Figure 4 This is a schematic flowchart illustrating another image processing method provided in an embodiment of this application. This image processing method can be executed by the aforementioned computer device, such as... Figure 4 As shown, the method includes steps S401-S404:

[0042] S401. Obtain the target gradient map of the target image to be processed.

[0043] In one embodiment, the specific implementation of step S401 can be found in the relevant description in step S201, and will not be repeated here.

[0044] S402. Determine the amplitude change parameter based on the pixel change amplitude corresponding to each pixel point in the target gradient map.

[0045] As mentioned above, the amplitude variation parameter is essentially the variance corresponding to the amplitude changes of multiple pixels. Therefore, in specific implementations, the computer device calculates the amplitude variation parameter as shown in Equation 1:

[0046] Var(X)=E[(X-μ) 2 ] = E[X 2 -2Xμ+μ 2 ] = E(X 2 )-2μ 2+μ 2 =E(X) 2 )-μ 2 Formula 1

[0047] Where X represents the pixel change amplitude, μ represents the average of these pixel change amplitudes, Var(X) represents the amplitude change parameter; E(X) 2 This represents the average of the squares of the magnitude changes of each pixel among these multiple pixel change magnitudes. Based on this, in a specific implementation, when determining the magnitude change parameters, the computer device can first construct a first integral image based on the pixel values ​​of each pixel in the target gradient image, and then construct a second integral image based on the squares of the pixel values ​​of each pixel in the target gradient image. It should be noted that, for the sake of clarity in understanding the embodiments of this application, the pixels in the target gradient image are referred to as target pixels, the pixels in the first integral image are referred to as first pixels, and the pixels in the second integral image are referred to as second pixels, in order to provide a clear and detailed description of the embodiments of this application.

[0048] Based on the above description, the value of a pixel in the first integral image (i.e., the value of the first pixel) is used to represent the sum of pixel values ​​in the corresponding pixel region of the target gradient image. Here, the corresponding pixel region can be understood as: in the target gradient image, a rectangular region with the target pixel corresponding to the first pixel as its lower right corner and the target image's upper left corner as its upper left corner. Therefore, the value of the first pixel can represent: the sum of pixel values ​​of all target pixels within the rectangular region corresponding to the first pixel in the target gradient image. Based on this, it is easy to understand that the computer device can determine μ in Equation 1 based on the value of the first pixel in the first integral image and the number of first pixels within the rectangular region corresponding to the first pixel, and thus obtain μ. 2 .

[0049] Correspondingly, the value of a pixel in the second integral image (i.e., the value of the second pixel) represents the sum of squares of the corresponding pixel values ​​in the target gradient image. This corresponding pixel region can be understood as a rectangular area in the target gradient image with the target pixel corresponding to the second pixel as its lower right corner and the upper left corner of the target gradient image as its upper left corner. In other words, the value of the second pixel can represent the sum of the squares of the pixel values ​​of all target pixels within the rectangular area corresponding to the second pixel in the target gradient image. Based on this, it is easy to understand that a computer device can calculate E(X) in Equation 1 above based on the value of the second pixel in the second integral image and the rectangular area corresponding to the second pixel. 2 Furthermore, the computer device can be configured based on the determined E(X). 2 ) and μ 2The amplitude variation parameter is then calculated. Furthermore, it is easy to understand that the computer device can determine the amplitude variation parameter based on the pixel values ​​in the second integral image and the corresponding pixel values ​​in the first integral image.

[0050] The following detailed explanation, using a specific example, illustrates how a computer device determines amplitude variation parameters. Assume a rectangular image region A exists in the target image, comprising multiple pixels, with the top-left corner of region A being the top-left corner of the target image. When determining the amplitude variation parameters corresponding to rectangular image region A, the computer device can identify the multiple target pixels corresponding to region A in the target gradient map. Further, the computer device can identify a first pixel in the first integral map representing the sum of the pixel values ​​corresponding to these multiple target pixels, and a second pixel in the second integral map representing the sum of the squares of the pixel values ​​corresponding to these multiple target pixels. Based on this, the computer device can calculate the mean μ (i.e., the mean of the pixel variation amplitude) of the pixel values ​​corresponding to these multiple target pixels based on the determined pixel value of the first pixel. Furthermore, the computer device can calculate the mean E(X) of the squares of the pixel values ​​corresponding to these multiple target pixels based on the determined pixel value of the second pixel. 2 (i.e., the mean of the squares of the pixel change amplitude), thus enabling the computer device to calculate the amplitude change parameter of the rectangular image region A using Equation 1.

[0051] It should be noted that in practical applications, the top-left corner of the rectangular image region A mentioned in the above example can be at any position. Therefore, in this case, if the computer device wants to calculate the average value of the pixel change amplitude corresponding to the rectangular image region A, it can determine the first pixel value corresponding to the top-left corner pixel in the first integral image, and then calculate the average value of the pixel change amplitude corresponding to the rectangular image region A based on the determined first pixel value. Similarly, if the computer device wants to calculate the average value of the square of the pixel change amplitude corresponding to the rectangular image region A, it can determine the second pixel value corresponding to the top-left corner pixel in the second integral image, and then calculate the average value of the square of the pixel change amplitude corresponding to the rectangular image region A based on the determined second pixel value.

[0052] S403. Perform symmetrical scanning processing on the target image based on the amplitude change parameter, and determine the first jump position and the second jump position in the target image where the change amplitude of the corresponding pixel is greater than the change amplitude of the target.

[0053] In this context, symmetrical scanning of a target image by a computer device can refer to left-right symmetrical scanning, i.e., the computer device scans the target image from left to right and from right to left respectively; or it can refer to top-bottom symmetrical scanning, i.e., the computer device scans the target image from top to bottom and from bottom to top respectively. In specific applications, if the frosted glass effect area included in the target image is located on the left or right sides of the target image, then the computer device can determine the frosted glass effect area in the target image by performing left-right symmetrical scanning. Correspondingly, if the frosted glass effect area included in the target image is located on the top or bottom sides of the target image (e.g., ... Figure 1 As shown in the figure, the computer device can determine the frosted glass effect area in the target image by performing a symmetrical vertical scan of the target image.

[0054] The following example, using a left-right symmetrical scan of a target image, illustrates in detail the method by which the computer device performs symmetrical scanning of a target image in this embodiment. Specifically, the computer device can first divide the target image into one or more horizontal strip-shaped regions, and then scan each horizontal strip-shaped region from left to right to determine the first transition position. Further, after determining the first transition position, the computer device can then scan the horizontal strip-shaped region from right to left to determine the second transition position, thus obtaining a pair of transition positions. Therefore, it can be understood that if the computer device divides the target image into multiple horizontal strip-shaped regions, multiple pairs of transition positions can ultimately be obtained.

[0055] The computer device scans the horizontal strip-shaped region in the following manner: From left to right, the computer device sequentially determines a first rectangular region and a second rectangular region within the horizontal strip-shaped region. The bottom-right corner pixel of the first rectangular region (hereinafter referred to as the "first bottom-right pixel") and the bottom-right corner pixel of the second rectangular region (hereinafter referred to as the "second bottom-right pixel") are two adjacent pixels. Based on this, the computer device can determine the amplitude change parameters corresponding to the first rectangular region and the second rectangular region, respectively. When the difference between the amplitude change parameters of the first and second rectangular regions exceeds a certain threshold, the pixel change amplitude corresponding to the first bottom-right pixel is determined. If the amplitude change between the first and second bottom-right pixels exceeds the target amplitude, the computer device can use the first bottom-right pixel as the first transition position. Furthermore, after the computer device determines the first lower right corner pixel, it can re-determine the first rectangular region and the second rectangular region sequentially from right to left within the horizontal strip region. At this point, the lower left corner pixel of the first rectangular region (hereinafter referred to as the first lower left corner) and the lower left corner pixel of the second rectangular region (hereinafter referred to as the second lower left corner) are two adjacent pixels. Based on the above description, similarly, the computer device can determine the pixel change amplitude corresponding to the first lower left corner pixel after the difference between the amplitude change parameter corresponding to the first rectangular region and the amplitude change parameter corresponding to the second rectangular region exceeds a certain threshold. If the change amplitude between the first and second lower left corner pixels exceeds the target change amplitude, the first lower left corner pixel is further designated as the second jump position.

[0056] It should be noted that, in specific implementations, if the computer device fails to determine the transition position within the currently determined first and second rectangular regions, the first and second rectangular regions can be shifted one pixel in the scanning direction to redetermine them. Furthermore, if the computer device wants to perform a vertically symmetrical scan of the target image, it can divide the target image into multiple vertical strip-shaped regions, scan each strip-shaped region from top to bottom to determine the first transition position, and then scan that strip-shaped region from bottom to top to determine the second transition position. The specific implementation principle can be found in the above-described method for horizontally symmetrical scanning of the target image, which will not be elaborated upon in this embodiment.

[0057] S404. Determine the symmetrical deviation between the first jump position and the second jump position, and determine whether the first jump position and the second jump position are the target jump positions based on the symmetrical deviation.

[0058] The symmetry deviation describes the distance difference between the first jump position and the corresponding display boundary (hereinafter referred to as the "first distance"), and the second jump position and the corresponding display boundary (hereinafter referred to as the "second distance"). Specifically, if the first and second jump positions are determined from a horizontal strip region, then the display boundary corresponding to the first jump position is the left display boundary of the target image, and the display boundary corresponding to the second jump position is the right display boundary of the target image. Correspondingly, if the first and second jump positions are determined from a vertical strip region, then the display boundary corresponding to the first jump position is the upper display boundary of the target image, and the display boundary corresponding to the second jump position is the lower display boundary of the target image. The display boundary can be understood as the position where the target image begins to be displayed, and it can be composed of the outermost pixels of the target image.

[0059] To more clearly illustrate the embodiments of this application, the following description, taking the first and second jump positions determined by the computer device from the horizontal strip region as an example, will detail the method by which the computer device determines the symmetry deviation. The computer device first determines a first distance between the first jump position and the display boundary on the left side of the target image, and a second distance between the second jump position and the display boundary on the right side of the target image. Further, the computer device can calculate the difference between the first and second distances and use this difference as the symmetry deviation.

[0060] Based on the above description, the specific method by which the computer device determines whether the first jump position and the second jump position are target jump positions based on the symmetry deviation can be as follows: If the symmetry deviation is less than the deviation threshold, then the first jump position and the second jump position are determined to be symmetrical, and the symmetrical first jump position and the symmetrical second jump position are taken as the target jump position. Correspondingly, if the symmetry deviation is greater than the deviation threshold, then the first jump position and the second jump position are determined not to be target jump positions.

[0061] S405. Determine the image boundary of the target image based on the boundary line where the target jump position is located.

[0062] In practice, the computer device first obtains the associated jump positions of the target jump position and performs a consistency check between the associated jump positions and the target jump position. The consistency check can be understood as verifying whether the positional change between the target jump position and the associated jump position is less than a change threshold. Under normal circumstances, if the target image has a frosted glass effect area, then all the first jump positions determined in the target image are consistent, and all the second jump positions are also consistent. Therefore, the purpose of the consistency check is to ensure that the determined boundary lines are more accurate, thereby improving the accuracy of the determined image boundaries. It should be noted that "consistent jump positions" means that all jump positions are located on a straight line. For example, assuming the first jump position is determined from a horizontal striped region of the target image, then all the first jump positions should normally be located on the same vertical straight line; correspondingly, it is easy to understand that if the first jump position is determined from a vertical striped region of the target image, then all the first jump positions should normally be located on the same horizontal straight line.

[0063] Based on the above description, a computer device can divide a target image into multiple (e.g., N, where N is a positive integer) strip-shaped regions and perform symmetrical scanning on each strip-shaped region to determine N pairs of transition positions (i.e., N first transition positions and N second transition positions). In this case, assuming any pair of transition positions among the N pairs is determined to be the target transition position, the aforementioned associated transition positions can be understood as: the other transition positions among the N pairs that are not determined to be the target transition position. Specifically, the first transition position determined to be the target transition position is associated with other first transition positions, and the second transition position determined to be the target transition position is associated with other second transition positions.

[0064] Based on the above description, if the positional change between the associated jump position and the target jump position is less than the change threshold, the computer device can determine the corresponding boundary line based on the associated jump position and the target jump position, and use the determined boundary line as the boundary line where the target jump position is located. Then, the boundary line where the target jump position is located can be used as the image boundary of the target image.

[0065] In one embodiment, since the amplitude variations between pixels corresponding to the frosted glass effect area are relatively small, to further improve the accuracy of the determined image boundary, the computer device can obtain the amplitude variation parameters corresponding to the frosted glass effect area to verify the correctness of the determined image boundary, and use the true image boundary that passes the correctness verification as the final image boundary. This can make the determined image boundary more accurate. Specifically, the computer device can obtain the target area composed of the display boundary and the image boundary of the target image, and obtain the amplitude variation parameters corresponding to the target area. Further, the computer device can use the amplitude variation parameters corresponding to the target area to verify the correctness of the image boundary. The specific implementation can be as follows: if the amplitude variation parameter is less than or equal to a parameter threshold, the computer device determines the image boundary as the true image boundary of the target image; if the amplitude variation parameter is greater than the parameter threshold, the computer device redetermines the image boundary of the target image.

[0066] In yet another embodiment, the target image can be any video frame in a video file. Therefore, based on the above... Figure 2 and Figure 4 The description of the image processing method includes a computer device that can also acquire associated images of the target image and verify the image boundaries in the target image based on the image boundaries determined from the associated images, thereby determining more accurate image boundaries. The associated images include images whose playback interval with the target image is less than or equal to a duration threshold. For example, the associated images can be the previous frame or the next frame of the target image; the target image is taken from a currently popular short video file. Traditional black and white borders are often added to short video files to solve the problem of inconsistent screen width matching when switching between portrait and landscape modes. However, these black and white borders significantly affect the viewing experience. Therefore, adding a frosted glass effect to short video files has become increasingly popular. The so-called frosted glass effect refers to the visual effect achieved by adding a certain degree of blurring to a specific area of ​​an image, giving the impression of being covered by frosted glass, achieving a hazy beauty. The frosted glass effect is not only common in short videos but also in some operating systems, such as the frosted glass effect in iOS (a mobile operating system). Furthermore, accurate detection of the frosted glass effect helps to filter out videos with added frosted glass effects from massive amounts of video data, preventing them from being played on platforms such as television stations and broadcast media that do not allow direct playback of videos with such effects. Therefore, the image processing method proposed in this application can be used for frosted glass effect detection in video files, video quality filtering, copyright protection, etc. Specifically, the following uses the frosted glass effect detection scenario in video files (i.e., detecting whether a video file has added frosted glass effects) as an example to further illustrate the embodiments of this application in detail.

[0067] Since the position of the frosted glass effect is usually consistent across all video frames when a video file contains a frosted glass effect, in order to improve the accuracy of computer equipment in detecting the frosted glass effect in video files, the computer equipment can first obtain the image boundary of the associated image, and then verify the image boundary of the associated image with the image boundary of the target image to ensure that the position of the image boundary of the target image in the target image is consistent with the position of the image boundary of the associated image in the associated image.

[0068] Specifically, the computer device can first determine the distance between the image boundary of the associated image and its display boundary (hereinafter referred to as the "association distance"), then determine the distance between the image boundary of the target image and its display boundary (hereinafter referred to as the "target distance"). Further, the computer device can determine whether the image boundary of the associated image and the image boundary of the target image are consistent based on the association distance and the target distance. If the association distance and the target distance are equal (or less than a certain threshold), then the image boundary of the associated image is considered consistent with the image boundary of the target image, thus determining that a frosted glass effect has been added to the video file. Furthermore, it should be noted that there can be multiple associated images. Therefore, to avoid the influence of darker video frames (e.g., video frames shot at night) on the frosted glass effect detection, this application embodiment also proposes that if the number of associated images whose image boundaries are inconsistent with the target image is less than a certain threshold, then the video file can also be considered to have a frosted glass effect. Figure 5 In each of the consecutive video frames shown, if Figure 5 No frosted glass effect was detected in one or more video frames marked with 51, while Figure 5 Multiple video frames marked with 52 and Figure 5 If multiple video frames marked with 53 all show a frosted glass effect, then it can be assumed that a frosted glass effect has been added to the video file corresponding to each consecutive video frame.

[0069] In this embodiment, because the computer device constructs a first integral image based on the target gradient map, it can quickly calculate the average value of the change amplitude of multiple pixels within any rectangular region. Similarly, because the computer device also constructs a second integral image based on the target gradient map, it can quickly calculate the average value of the squares of the change amplitudes of multiple pixels within any rectangular region. Based on this, the computer device can quickly determine the change amplitude of pixels within the arbitrary rectangular region, thereby improving the speed at which the computer device determines the target jump position, and further improving the speed at which the computer device determines the image boundary. Furthermore, because the computer device performs consistency verification on the determined jump position, the determined target jump position is more accurate, thereby improving the accuracy of the image boundary determined based on the target jump position.

[0070] Based on the description of the above-described embodiments of the image processing method, this application also discloses an image processing apparatus, which can be a computer program (including program code) running on the aforementioned computer device. This image processing apparatus can execute... Figure 2 or Figure 4 The method shown. Please refer to [link / reference]. Figure 6 The image processing device may include: an acquisition unit 601, a determination unit 602, and a scanning unit 603.

[0071] The acquisition unit 601 is used to acquire a target gradient map of the target image to be processed. The value of each pixel in the target gradient map is used to indicate the pixel change amplitude of the pixel at the corresponding position in the target image. The pixel change amplitude includes the difference between the target pixel value of the pixel at the corresponding position in the target image and the target pixel value of the adjacent pixel.

[0072] The determining unit 602 is used to determine an amplitude change parameter based on the pixel change amplitude corresponding to each pixel point in the target gradient map. The amplitude change parameter is used to indicate the change magnitude of the pixel change amplitude in the target gradient map.

[0073] The scanning unit 603 is used to scan the target image based on the amplitude change parameter, determine the target jump position in the target image where the change amplitude of the corresponding pixel change is greater than the target change amplitude, and determine the image boundary of the target image based on the boundary line where the target jump position is located.

[0074] In one embodiment, the determining unit 602 can also be used to perform:

[0075] A first integral image is constructed based on the pixel values ​​of each pixel in the target gradient image, and a second integral image is constructed based on the squares of the pixel values ​​of each pixel in the target gradient image. The pixel values ​​in the first integral image are used to represent the sum of pixels in the corresponding pixel region in the target gradient image, and the pixel values ​​in the second integral image are used to represent the sum of squares of pixels in the corresponding pixel region in the target gradient image.

[0076] The amplitude variation parameter is determined based on the pixel values ​​in the second integral image and the pixel values ​​at corresponding positions in the second integral image.

[0077] In yet another embodiment, the scanning unit 603 may specifically be used to perform:

[0078] Based on the amplitude change parameter, the target image is symmetrically scanned to determine the first jump position and the second jump position in the target image where the change amplitude of the corresponding pixel is greater than the target change amplitude.

[0079] Determine the symmetrical deviation between the first jump position and the second jump position, and determine whether the first jump position and the second jump position are target jump positions based on the symmetrical deviation.

[0080] In yet another embodiment, the scanning unit 603 may also be specifically used to perform:

[0081] If the symmetry deviation is less than the deviation threshold, then the first jump position and the second jump position are determined to be symmetrical, wherein the first jump position and the second jump position that are symmetrical are the target jump positions;

[0082] If the symmetry deviation is greater than the deviation threshold, then the first jump position and the second jump position are determined not to be the target jump position.

[0083] In yet another embodiment, the scanning unit 603 may also be specifically used to perform:

[0084] Obtain the associated transition position of the target transition position, and perform a consistency check between the associated transition position and the target transition position;

[0085] When the positional change between the associated jump position and the target jump position is less than a change threshold, the boundary line formed by the associated jump position and the target jump position is determined as the boundary line where the target jump position is located.

[0086] In another embodiment, the image processing device further includes a verification unit 604, which can be specifically used to perform:

[0087] Obtain the target region formed by the display boundary of the target image and the image boundary of the target image, and obtain the amplitude change parameter corresponding to the target region;

[0088] The correctness of the image boundary is verified by using the amplitude change parameter corresponding to the target region.

[0089] If the amplitude change parameter is less than or equal to the parameter threshold, the image boundary is determined to be the true image boundary of the target image;

[0090] If the amplitude change parameter is greater than the parameter threshold, the image boundary of the target image is redefined.

[0091] In another embodiment, the verification unit 604 can also be specifically used to perform:

[0092] Obtain associated images of the target image, wherein the associated images include images whose playback interval between them and the target image is less than or equal to a duration threshold;

[0093] The image boundary of the associated image is determined, and the image boundary of the target image is verified based on the determined image boundary to ensure that the position of the image boundary of the target image in the target image is consistent with the position of the image boundary of the associated image in the associated image.

[0094] According to another embodiment of this application, Figure 2 and Figure 4 The steps involved in the method shown can be derived from... Figure 6 This is performed by the individual units in the image processing apparatus shown. For example: Figure 2 The step S201 shown can be performed by Figure 6 The acquisition unit 601 in the image processing apparatus shown is responsible for performing step S202; step S202 can be performed by... Figure 6 The determination unit 602 in the image processing apparatus shown is responsible for executing step S203; step S203 can be performed by... Figure 6 The scanning unit 603 in the image processing apparatus shown performs this operation. For example, Figure 4 The step S401 shown can be performed by Figure 6 The acquisition unit 601 in the image processing apparatus shown is responsible for performing step S402; step S402 can be performed by... Figure 6 The determination unit 602 in the image processing apparatus shown is responsible for executing steps S403 to S405; each step can be performed by... Figure 6 The scanning unit 603 in the image processing apparatus shown performs the operation.

[0095] According to another embodiment of this application, Figure 6The units in the illustrated image processing apparatus are divided based on logical functions. These units can be individually or entirely merged into one or more other units, or some of these units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. In other embodiments of this application, the image processing apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0096] According to another embodiment of the present application, the following can be achieved by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), a device capable of performing operations such as... Figure 2 or Figure 4 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 6 The image processing apparatus shown herein, and the image processing method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a storage medium, loaded onto the aforementioned computing device via the storage medium, and run therein.

[0097] In this embodiment, the image processing device fully utilizes the principle that in image regions without special effects (such as frosted glass), the amplitude of pixel change is typically large, while in image regions with special effects, the amplitude of pixel change is typically small. By acquiring a target gradient map of the target image using the image processing device, and further constructing amplitude change parameters based on the pixel values ​​in the target gradient map, the device can accurately determine the image regions with special effects based on the amplitude change parameters, thereby enabling the image processing device to determine more accurate image boundaries.

[0098] Based on the descriptions of the above method and device embodiments, this application also proposes a computer device. Please refer to... Figure 7 The computer device includes at least a processor 701 and a storage medium 702, and the processor 701 and the storage medium 702 within the computer device can be connected via a bus or other means.

[0099] The storage medium 702 is a memory device in the computer device, used to store programs and data. It is understood that the storage medium 702 can include the built-in storage medium of the computer device, or it can include extended storage media supported by the computer device. The storage medium 702 provides storage space, which stores the operating system of the computer device. Furthermore, this storage space also stores one or more computer instructions suitable for loading and execution by the processor 701. These computer instructions can be one or more computer programs (including program code). It should be noted that the storage medium can be high-speed RAM, or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one storage medium located remotely from the aforementioned processor. The processor 701 (or CPU (Central Processing Unit)) is the computing and control core of the computer device, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve corresponding method flows or corresponding functions.

[0100] In one embodiment, processor 701 may load and execute one or more computer instructions stored in storage medium 702 to implement the aforementioned related... Figure 2 and Figure 4 The corresponding method steps in the image processing method embodiment shown; in specific implementation, one or more computer instructions in the computer storage medium 702 are loaded by the processor 701 and executed as follows:

[0101] Obtain the target gradient map of the target image to be processed. The value of each pixel in the target gradient map is used to indicate the pixel change amplitude of the pixel at the corresponding position in the target image. The pixel change amplitude includes the difference between the pixel value of the pixel at the corresponding position in the target image and the pixel value of the adjacent pixel.

[0102] Based on the pixel change magnitude corresponding to each pixel in the target gradient map, an amplitude change parameter is determined, which is used to indicate the magnitude of the pixel change magnitude in the target gradient map.

[0103] The target image is scanned based on the amplitude change parameter to determine the target jump position in the target image where the change amplitude of the corresponding pixel is greater than the target change amplitude, and the image boundary of the target image is determined based on the boundary line where the target jump position is located.

[0104] In one embodiment, the processor 701 can also be used to load and execute:

[0105] A first integral image is constructed based on the pixel values ​​of each pixel in the target gradient image, and a second integral image is constructed based on the squares of the pixel values ​​of each pixel in the target gradient image. The pixel values ​​in the first integral image are used to represent the sum of pixels in the corresponding pixel region in the target gradient image, and the pixel values ​​in the second integral image are used to represent the sum of squares of pixels in the corresponding pixel region in the target gradient image.

[0106] The amplitude variation parameter is determined based on the pixel values ​​in the second integral image and the pixel values ​​at corresponding positions in the second integral image.

[0107] In yet another embodiment, the processor 701 can also be used to load and execute:

[0108] Based on the amplitude change parameter, the target image is symmetrically scanned to determine the first jump position and the second jump position in the target image where the change amplitude of the corresponding pixel is greater than the target change amplitude.

[0109] Determine the symmetrical deviation between the first jump position and the second jump position, and determine whether the first jump position and the second jump position are target jump positions based on the symmetrical deviation.

[0110] In yet another embodiment, the processor 701 can also be used to load and execute:

[0111] If the symmetry deviation is less than the deviation threshold, then the first jump position and the second jump position are determined to be symmetrical, wherein the first jump position and the second jump position that are symmetrical are the target jump positions;

[0112] If the symmetry deviation is greater than the deviation threshold, then the first jump position and the second jump position are determined not to be the target jump position.

[0113] In yet another embodiment, the processor 701 can also be used to load and execute:

[0114] Obtain the associated transition position of the target transition position, and perform a consistency check between the associated transition position and the target transition position;

[0115] When the positional change between the associated jump position and the target jump position is less than a change threshold, the boundary line formed by the associated jump position and the target jump position is determined as the boundary line where the target jump position is located.

[0116] In yet another embodiment, the processor 701 can also be used to load and execute:

[0117] Obtain the target region formed by the display boundary of the target image and the image boundary of the target image, and obtain the amplitude change parameter corresponding to the target region;

[0118] The correctness of the image boundary is verified by using the amplitude change parameter corresponding to the target region.

[0119] If the amplitude change parameter is less than or equal to the parameter threshold, the image boundary is determined to be the true image boundary of the target image;

[0120] If the amplitude change parameter is greater than the parameter threshold, the image boundary of the target image is redefined.

[0121] In yet another embodiment, the processor 701 can also be used to load and execute:

[0122] Obtain associated images of the target image, wherein the associated images include images whose playback interval between them and the target image is less than or equal to a duration threshold;

[0123] The image boundary of the associated image is determined, and the image boundary of the target image is verified based on the determined image boundary to ensure that the position of the image boundary of the target image in the target image is consistent with the position of the image boundary of the associated image in the associated image.

[0124] In this embodiment, the computer device fully utilizes the principle that in image regions without special effects (such as frosted glass), the amplitude of pixel change is typically large, while in image regions with special effects, the amplitude of pixel change is typically small. By acquiring a target gradient map of the target image using the computer device, and further constructing amplitude change parameters based on the pixel values ​​in the target gradient map, an amplitude change parameter is generated to reflect the amplitude of pixel change within the corresponding image region. This allows the computer device to accurately determine the image region with special effects based on the amplitude change parameter, thereby enabling the computer device to determine a more accurate image boundary.

[0125] This application also provides a storage medium storing one or more computer instructions for the image processing method described above. When one or more processors load and execute these computer instructions, the image processing method described in the embodiments can be implemented, and will not be repeated here. The beneficial effects of using the same method will also not be repeated here. It is understood that the computer instructions can be deployed on one or more devices capable of communicating with each other.

[0126] It should be noted that, according to one aspect of the embodiments of this application, a computer program product or computer program is also provided, which includes computer instructions stored in a storage medium. The processor of a computer device reads the computer instructions from the storage medium and executes the computer instructions, causing the computer device to perform the aforementioned actions. Figure 2 and Figure 4 The image processing method shown is provided in various alternative embodiments.

[0127] Furthermore, it should be understood that the various embodiments disclosed above are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application are still within the scope of this application.

Claims

1. An image processing method, characterized in that, include: Obtain the target gradient map of the target image to be processed. The value of each pixel in the target gradient map is used to indicate the pixel change amplitude of the pixel at the corresponding position in the target image. The pixel change amplitude includes the difference between the target pixel value of the pixel at the corresponding position in the target image and the target pixel value of the adjacent pixel. Based on the pixel change magnitude corresponding to each pixel in the target gradient map, an amplitude change parameter is determined, which is used to indicate the magnitude of the pixel change magnitude in the target gradient map. Based on the amplitude change parameters, the target image is subjected to symmetrical scanning processing. The first jump position and the second jump position in the target image are determined respectively, where the change amplitude of the corresponding pixel change is greater than the target change amplitude. The symmetry deviation between the first jump position and the second jump position is determined. Based on the symmetry deviation, it is determined whether the first jump position and the second jump position are the target jump positions. Based on the boundary line where the target jump position is located, the image boundary of the target image is determined.

2. The method as described in claim 1, characterized in that, The step of determining the amplitude change parameter based on the pixel change amplitude corresponding to each pixel point in the target gradient map includes: A first integral image is constructed based on the pixel values ​​of each pixel in the target gradient image, and a second integral image is constructed based on the squares of the pixel values ​​of each pixel in the target gradient image. The pixel values ​​in the first integral image are used to represent the sum of pixels in the corresponding pixel region in the target gradient image, and the pixel values ​​in the second integral image are used to represent the sum of squares of pixels in the corresponding pixel region in the target gradient image. The amplitude variation parameter is determined based on the pixel values ​​in the second integral image and the pixel values ​​at corresponding positions in the second integral image.

3. The method as described in claim 1, characterized in that, The step of determining whether the first jump position and the second jump position are target jump positions based on the symmetry deviation includes: If the symmetry deviation is less than the deviation threshold, then the first jump position and the second jump position are determined to be symmetrical, wherein the first jump position and the second jump position that are symmetrical are the target jump positions; If the symmetry deviation is greater than the deviation threshold, then the first jump position and the second jump position are determined not to be the target jump position.

4. The method as described in claim 1, characterized in that, The methods for determining the boundary line where the target transition position is located include: Obtain the associated transition position of the target transition position, and perform a consistency check between the associated transition position and the target transition position; When the positional change between the associated jump position and the target jump position is less than a change threshold, the boundary line formed by the associated jump position and the target jump position is determined as the boundary line where the target jump position is located.

5. The method as described in claim 1, characterized in that, The method further includes: Obtain the target region formed by the display boundary of the target image and the image boundary of the target image, and obtain the amplitude change parameter corresponding to the target region; The correctness of the image boundary is verified by using the amplitude change parameter corresponding to the target region. If the amplitude change parameter is less than or equal to the parameter threshold, the image boundary is determined to be the true image boundary of the target image; If the amplitude change parameter is greater than the parameter threshold, the image boundary of the target image is redefined.

6. The method as described in claim 1, characterized in that, The method further includes: Obtain associated images of the target image, wherein the associated images include images whose playback interval between them and the target image is less than or equal to a duration threshold; The image boundary of the associated image is determined, and the image boundary of the target image is verified based on the determined image boundary to ensure that the position of the image boundary of the target image in the target image is consistent with the position of the image boundary of the associated image in the associated image.

7. An image processing apparatus, characterized in that, include: The acquisition unit is used to acquire the target gradient map of the target image to be processed. The value of each pixel in the target gradient map is used to indicate the pixel change amplitude of the pixel at the corresponding position in the target image. The pixel change amplitude includes the difference between the target pixel value of the pixel at the corresponding position in the target image and the target pixel value of the adjacent pixel. The determining unit is used to determine the amplitude change parameter based on the pixel change amplitude corresponding to each pixel point in the target gradient map. The amplitude change parameter is used to indicate the change magnitude of the pixel change amplitude in the target gradient map. The scanning unit is configured to perform symmetrical scanning processing on the target image based on the amplitude change parameter, and to determine, respectively, a first jump position and a second jump position in the target image where the change amplitude of the corresponding pixel change is greater than the target change amplitude; determine the symmetrical deviation between the first jump position and the second jump position, and determine whether the first jump position and the second jump position are target jump positions based on the symmetrical deviation, and determine the image boundary of the target image based on the boundary line where the target jump position is located.

8. A computer device, characterized in that, include: A processor, the processor being adapted to implement one or more computer instructions; A storage medium storing one or more computer instructions adapted to be loaded by the processor and executed as described in any one of claims 1-6.

9. A storage medium, characterized in that, The storage medium stores one or more computer instructions, which are adapted to be loaded by a processor and executed by the image processing method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-6.

Images