Image processing method and device, electronic equipment and storage medium
By performing parallax map segmentation and clustering on the image, the problem of difficulty in extracting stationary foreground in the prior art is solved, and the foreground and background separation under complex backgrounds are achieved, and more accurate foreground images are generated.
Patent Information
- Application Number
- CN202510004471.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-02
- Publication Date
- 2025-05-02
AI Technical Summary
Existing foreground extraction methods are difficult to extract static foregrounds and cannot effectively separate foreground objects from the background, especially when complex dynamic backgrounds and large changes in lighting.
By acquiring the image and its corresponding disparity map, segmenting the disparity map as an image block, determining the boundary block and non-boundary block, clustering the pixel values in the boundary block to obtain the foreground area, and combining the non-boundary blocks based on the average value, forming the combined foreground area, and processing the image to generate the foreground image.
The demarcation edges of the foreground and background are achieved more accurately handled in complex backgrounds, allowing for the foreground image to be determined more completely and accurately, thereby effectively separating the foreground objects from the background.
Smart Images
Figure CN119919441A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of data processing technology, and specifically relates to an image processing method, a processing device, an electronic device and a storage medium. Background Art
[0002] In the related technical solutions, foreground extraction methods mainly include methods based on background modeling and frame difference.
[0003] Among them, methods based on background modeling, such as Gaussian mixture models, distinguish foreground from background by modeling the background. However, this method often makes misjudgments in complex dynamic backgrounds, misjudging the dynamic part of the background as the foreground. At the same time, when the lighting changes greatly, the stability and accuracy of the model will also be seriously affected.
[0004] The frame difference based method uses the difference between adjacent frames to detect the foreground. Although this method is computationally simple, it is easily affected by the speed of object movement and noise. When the object moves slowly, the frame difference may not be able to effectively detect the object, and when there is noise in the image, a large number of false foreground areas will be generated.
[0005] Obviously, the existing foreground extraction methods only focus on the simplistic setting of clearly separating the foreground and the background. It is difficult to extract a static foreground and cannot effectively separate the foreground object from the background. Summary of the invention
[0006] The purpose of the embodiments of the present application is to provide an image processing method, a processing device, an electronic device and a storage medium, which can solve the problem that the existing foreground extraction method only focuses on the simplified setting of clearly separating the foreground and the background, making it difficult to extract a static foreground and unable to effectively separate the foreground object from the background.
[0007] In a first aspect, an embodiment of the present application provides an image processing method, including: obtaining a first image and a disparity map corresponding to the first image; segmenting the disparity map to obtain multiple image blocks; determining boundary blocks and non-boundary blocks in the multiple image blocks; clustering pixel values within the boundary blocks to obtain a foreground area of the boundary blocks, the foreground area corresponding to a first parameter value, and the first parameter value is the average value of the pixel values within the foreground area; merging the non-boundary blocks into the foreground area based on the first parameter value with the boundary block as the center to obtain a merged foreground area; using the merged foreground area to process the first image to obtain a foreground image.
[0008] In a second aspect, an embodiment of the present application provides an image processing device, including: an acquisition module, used to acquire a first image and a disparity map corresponding to the first image; a segmentation module, used to segment the disparity map to obtain multiple image blocks; a determination module, used to determine boundary blocks and non-boundary blocks in the multiple image blocks; a clustering module, used to cluster pixel values within the boundary blocks to obtain a foreground area of the boundary blocks, the foreground area corresponds to a first parameter value, and the first parameter value is the average value of the pixel values within the foreground area; a merging module, used to merge the non-boundary blocks into the foreground area based on the first parameter value with the boundary block as the center to obtain a merged foreground area; a processing module, used to use the merged foreground area to process the first image to obtain a foreground image.
[0009] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be executed on the processor, and when the programs or instructions are executed by the processor, the steps of the method of the first aspect are implemented.
[0010] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.
[0011] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the method of the first aspect.
[0012] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method of the first aspect.
[0013] In an embodiment of the present application, an image processing method is proposed. In the image processing method, a disparity map corresponding to a first image is divided into image blocks so as to distinguish local foreground and background information, thereby ensuring that there is no confusion between the background and foreground when the relationship between the background and foreground is complex and the relatively blurred foreground and background are not confused. The pixel values in the boundary blocks are clustered so as to adaptively divide the foreground and background, thereby more accurately processing the boundary edge of the foreground and background. The first image is processed using the merged foreground area, so that the foreground image can be determined more completely and accurately, thereby effectively separating the foreground object from the background. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 One of the flowcharts of the image processing method provided in the embodiment of the present application is shown;
[0015] Figure 2 One of the schematic diagrams of image block merging proposed in the embodiment of the present application is shown;
[0016] Figure 3 A second schematic diagram of image block merging proposed in an embodiment of the present application is shown;
[0017] Figure 4 The third schematic diagram of the image block merging proposed in the embodiment of the present application is shown;
[0018] Figure 5 The second flowchart of the image processing method proposed in the embodiment of the present application is shown;
[0019] Figure 6 A schematic block diagram of an image processing device provided by an embodiment of the present application is shown;
[0020] Figure 7 is a schematic block diagram of an electronic device in an embodiment of the present application;
[0021] Figure 8 It is a schematic diagram of the hardware structure of the electronic device in the embodiment of the present application. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present application to clearly describe the embodiments in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.
[0023] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0024] The image processing method, processing device, electronic device and storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.
[0025] In one embodiment, if Figure 1 As shown, an image processing method is proposed, comprising:
[0026] Step 102: Obtain a first image and a disparity map corresponding to the first image.
[0027] The disparity map is an image that records the disparity information of each pixel value in the scene. Disparity refers to the position difference of the same object in two different perspective images due to different observation positions (for example, the position difference of two cameras in a binocular vision system).
[0028] In simple terms, each pixel value in the disparity map represents the offset of the corresponding position of the pixel in the binocular image. This offset is directly related to the depth of the object. Through the principle of triangulation, the depth information of the object can be calculated based on the disparity.
[0029] Step 104 , segment the disparity map to obtain a plurality of image blocks.
[0030] Step 106: Determine boundary blocks and non-boundary blocks among the multiple image blocks.
[0031] Step 108, clustering is performed on the pixel values in the boundary block to obtain a foreground area of the boundary block, the foreground area corresponds to a first parameter value, and the first parameter value is an average value of the pixel values in the foreground area.
[0032] Step 110 , taking the boundary block as the center, based on the first parameter value, merge the non-boundary blocks into the foreground area to obtain a merged foreground area.
[0033] Step 112: Process the first image using the merged foreground area to obtain a foreground image.
[0034] In an embodiment of the present application, an image processing method is proposed. In the image processing method, a disparity map corresponding to a first image is divided into image blocks so as to distinguish local foreground and background information, thereby ensuring that there is no confusion between the background and foreground when the relationship between the background and foreground is complex and the relatively blurred foreground and background are not confused. The pixel values in the boundary blocks are clustered so as to adaptively divide the foreground and background, thereby more accurately processing the boundary edge of the foreground and background. The first image is processed using the merged foreground area, so that the foreground image can be determined more completely and accurately, thereby effectively separating the foreground object from the background.
[0035] In some embodiments of the present application, determining boundary blocks and non-boundary blocks in multiple image blocks specifically includes: obtaining multiple gradient maps corresponding to the multiple image blocks, each gradient map including multiple gradient values; when a target gradient value in a first gradient map is greater than or equal to a first gradient value, determining that the image block corresponding to the first gradient value is a boundary block; when the gradient value of the first gradient map is less than the first gradient value, determining that the image block corresponding to the first gradient value is a non-boundary block; wherein the first gradient map is any one of the multiple gradient maps.
[0036] In this embodiment, the size of the gradient value can reflect whether the pixel changes smoothly. Specifically, if the gradient value is small, it is considered that the pixel changes smoothly and is unlikely to be in the boundary area between the foreground and the background. Conversely, if the gradient value is large, it is considered that the pixel changes drastically and may be in the boundary area between the foreground and the background. Based on this, a first gradient value can be set in advance, and the image block can be judged according to the first gradient value to determine whether the image block is a boundary block.
[0037] In some embodiments of the present application, a Sobel operator is used to perform convolution processing on pixel values in an image block to obtain a gradient map corresponding to the image block.
[0038] Among them, the Sobel operator highlights the edge information of the image by calculating the approximate values of the first-order derivatives in the horizontal and vertical directions, and then obtains the gradient value corresponding to each pixel value in the image block.
[0039] In some embodiments of the present application, the target gradient value is the gradient value at the Nth position when multiple gradient values are sorted from large to small, where N is a positive integer greater than 1.
[0040] In this embodiment, N is not equal to 1, so it is possible to avoid taking the maximum gradient value as the target gradient value, thereby reducing the misjudgment of the boundary block and improving the accuracy of image block detection.
[0041] In some embodiments, the target gradient value is a maximum 99% gradient value among the plurality of gradient values.
[0042] In some embodiments of the present application, the disparity map is segmented to obtain multiple image blocks, which specifically includes: obtaining a segmentation size; segmenting the disparity map using the segmentation size to obtain multiple image blocks; wherein the multiple image blocks include a first image block, a second image block and a third image block, the first image block partially overlaps with the second image block, the first image block partially overlaps with the third image block, the second image block is an image block adjacent to the first image block in a first direction, the third image block is an image block adjacent to the first image block in a second direction, and the first direction is perpendicular to the second direction.
[0043] In this embodiment, the segmentation size may be a side length of n pixels, and the value of n may be determined according to the resolution of the disparity map and the actual application scenario, which will not be described in detail here.
[0044] In the above embodiment, the image blocks partially overlap in the first direction and the second direction, which helps to process the disparity map more finely, especially when processing the edges of objects and complex texture areas, it can ensure the integrity and continuity of the information and avoid losing or ignoring local disparity information changes due to overly coarse division methods, thus laying a foundation for subsequent accurate judgment of boundary blocks and extraction of foregrounds.
[0045] In some embodiments of the present application, a 50% overlap strategy is adopted between adjacent image blocks.
[0046] In this embodiment, for the overlapping area between two adjacent image blocks, as long as any one of the two adjacent image blocks is determined to be a boundary block, the overlapping area will also be determined as the boundary area of the boundary block, which effectively solves the problem of misjudgment caused by few boundary points in certain areas.
[0047] For example, Figure 2 As shown, the gray portion of the divided small block 202 is the overlapping portion with other small blocks.
[0048] like Figure 3 As shown, the small block 302 is not a boundary block, but the overlapping area is a boundary block, and the small block 302 is divided into a boundary block.
[0049] In some embodiments of the present application, pixel values in a boundary block are clustered to obtain a foreground area of the boundary block, specifically including: clustering the pixel values in the boundary block using a clustering algorithm to obtain a first pixel value set and a second pixel value set, wherein the pixel values in one of the first pixel value set and the second pixel value set are less than or equal to a preset value, and the pixel values in the other of the first pixel value set and the second pixel value set are greater than a preset value, and the preset value is a median value of the pixel values in the boundary block; obtaining a first median pixel value and a second median pixel value, wherein the first median pixel value is a median value of the pixel values in the first pixel value set, and the second median pixel value is a median value of the pixel values in the second pixel value set; and taking a pixel value set corresponding to a larger median pixel value between the first median pixel value and the second median pixel value as the foreground area of the boundary block.
[0050] In this embodiment, the clustering algorithm is a kmeans clustering algorithm, wherein the kmeans clustering algorithm is a classic distance-based clustering algorithm, whose goal is to divide data points into a specified number of clusters so that the distance between data points within a cluster is as small as possible, while the distance between clusters is as large as possible.
[0051] In this embodiment, each pixel point in the boundary block is input into the kmeans clustering algorithm as a data point according to its disparity information, and the number of clustering categories is set to two.
[0052] Among them, during the operation of the algorithm, the pixels are grouped according to the similarity of their disparity. After the clustering operation, two different sets of pixels are obtained. Since the foreground object is usually closer to the observation point than the background object, according to the inverse relationship between disparity and depth (that is, the closer the distance, the greater the disparity, and the farther the distance, the smaller the disparity), the set of pixels with larger disparity can be determined as the foreground, and the other set of pixels with smaller disparity can be determined as the background. That is, the set of pixel values corresponding to the larger median pixel value between the first median pixel value and the second median pixel value is used as the foreground area of the boundary block.
[0053] In this way, the disparity information in the disparity map is utilized and the kmeans clustering algorithm is used to achieve a preliminary distinction between the foreground and background within each boundary block.
[0054] In some embodiments of the present application, a clustering algorithm is used to cluster pixel values in a boundary block to obtain a first pixel value set and a second pixel value set, specifically including: randomly selecting K points from the boundary block as centroids; assigning each pixel value in the boundary block to the cluster where the centroid closest to it is located; calculating a new centroid based on the data points in each cluster, and moving the centroid to a new position. Repeat this process until the centroid no longer changes or a preset number of iterations is reached.
[0055] The pixel value in one of the first pixel value set and the second pixel value set is less than or equal to a preset value, and the pixel value in the other of the first pixel value set and the second pixel value set is greater than the preset value.
[0056] The preset value is the median value of the pixel values in the boundary block.
[0057] In some embodiments of the present application, a non-boundary block is merged into a foreground area based on a first parameter value with a boundary block as the center to obtain a merged foreground area, specifically including: obtaining a target non-boundary block, where the target non-boundary block is a non-boundary block centered on the boundary block and adjacent to the boundary block; obtaining a second parameter value, where the second parameter value is an average value of pixel values in the target non-boundary block; and merging the target non-boundary block into the foreground area when the difference between the second parameter value and the first parameter value is less than or equal to a difference threshold.
[0058] In this embodiment, the boundary block is used as the seed point, that is, the center, to perform region growth. In this process, the sub-unit of growth is not a single pixel, but a segmented image block.
[0059] Obviously, the growth method based on image blocks can process disparity map information more efficiently and stably than that based on pixels, and reduce errors caused by factors such as single pixel noise.
[0060] In the above embodiment, for each target non-border block to be judged (four small blocks adjacent to the border block above, below, left and right), the difference in mean between it and the border block is calculated, that is, the difference between the second parameter value and the first parameter value. If the difference is within the difference threshold, the target non-border block is included in the foreground area.
[0061] In this embodiment, starting from the boundary block and gradually expanding outward, qualified target non-boundary blocks are continuously merged into the foreground area until no new target non-boundary blocks meet the growth conditions. Ultimately, all small blocks judged to be foregrounds constitute a complete foreground area, which is used to accurately represent the foreground part of the image and provide accurate foreground information for subsequent related visual processing tasks.
[0062] For example, Figure 4 As shown, it is determined whether the adjacent blocks of the seed point (due to 50% overlap, the adjacent blocks overlap with the seed point) meet the growth conditions, and if so, they are included in the seed point.
[0063] In some embodiments, Figure 5 As shown, the image processing method includes:
[0064] Step 502: Obtain an image to be processed and a disparity map corresponding thereto.
[0065] Step 504: divide the disparity map into small blocks with a preset size.
[0066] The preset size is also the segmentation size in this application.
[0067] Step 506, determine whether each small block is a boundary block, if the determination result is yes, execute step 508, if the determination result is no, execute step 506.
[0068] Step 508: cluster the original block into two categories, foreground and background, by clustering the disparity within the block.
[0069] Step 510, using the boundary block as a seed point, obtain all foreground regions through a region growing algorithm.
[0070] In this embodiment, by utilizing the disparity map and using kmeans clustering to distinguish the foreground and background based on the disparity size, the close connection between disparity and object depth is fully utilized. Compared with the traditional method that relies on single feature distinction, it can effectively reduce misjudgment in complex scenes and accurately identify the foreground and background pixel sets in the boundary block.
[0071] In addition, the region growing algorithm with boundary blocks as seed points and small blocks as units, combined with the judgment of small block mean and standard deviation, avoids misjudgment caused by single pixel outliers, expands the foreground area more robustly, and ensures that the generated foreground mask accurately covers the real foreground.
[0072] What is pointed out is that the method of dividing the disparity map with preset size and 50% overlap is highly flexible when dealing with complex shapes and changing foreground objects. The overlapping small blocks can capture the disparity gradient information of edges and transition areas, and is effective in dealing with irregular object edges and mutual occlusion of foreground objects.
[0073] By using the Sobel operator to determine the boundary blocks and combining the gradient value with the preset value screening, the foreground-background boundary area can be accurately located, which is more advantageous in processing scenes with complex textures and rich layers.
[0074] The image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.
[0075] In some embodiments of the present application, Figure 6 As shown, an image processing device 600 is provided, including: an acquisition module 602, used to acquire a first image and a disparity map corresponding to the first image; a segmentation module 604, used to segment the disparity map to obtain multiple image blocks; a determination module 606, used to determine boundary blocks and non-boundary blocks in the multiple image blocks; a clustering module 608, used to cluster the pixel values in the boundary blocks to obtain a foreground area of the boundary blocks, the foreground area corresponds to a first parameter value, and the first parameter value is the average value of the pixel values in the foreground area; a merging module 610, used to merge the non-boundary blocks into the foreground area based on the first parameter value with the boundary block as the center to obtain a merged foreground area; a processing module 612, used to use the merged foreground area to process the first image to obtain a foreground image.
[0076] In some embodiments of the present application, the determination module 606 is specifically used to: obtain multiple gradient maps corresponding to multiple image blocks, each gradient map includes multiple gradient values; when the target gradient value in the first gradient map is greater than or equal to the first gradient value, determine that the image block corresponding to the first gradient value is a boundary block; when the gradient value of the first gradient map is less than the first gradient value, determine that the image block corresponding to the first gradient value is a non-boundary block; wherein the first gradient map is any one of the multiple gradient maps.
[0077] In some embodiments of the present application, the target gradient value is the gradient value at the Nth position when multiple gradient values are sorted from large to small, where N is a positive integer greater than 1.
[0078] In some embodiments of the present application, the segmentation module 604 is specifically used to: obtain a segmentation size; use the segmentation size to segment the disparity map to obtain multiple image blocks; wherein the multiple image blocks include a first image block, a second image block and a third image block, the first image block partially overlaps with the second image block, the first image block partially overlaps with the third image block, the second image block is an image block adjacent to the first image block in a first direction, the third image block is an image block adjacent to the first image block in a second direction, and the first direction is perpendicular to the second direction.
[0079] In some embodiments of the present application, the clustering module 608 is specifically used to: cluster the pixel values in the boundary block using a clustering algorithm to obtain a first pixel value set and a second pixel value set, wherein the pixel values in one of the first pixel value set and the second pixel value set are less than or equal to a preset value, and the pixel values in the other of the first pixel value set and the second pixel value set are greater than a preset value, and the preset value is the median value of the pixel values in the boundary block; obtain a first median pixel value and a second median pixel value, wherein the first median pixel value is the median value of the pixel values in the first pixel value set, and the second median pixel value is the median value of the pixel values in the second pixel value set; and use the pixel value set corresponding to the larger median pixel value of the first median pixel value and the second median pixel value as the foreground area of the boundary block.
[0080] In some embodiments of the present application, the merging module 610 is specifically used to: obtain a target non-border block, which is a non-border block centered on the border block and adjacent to the border block; obtain a second parameter value, which is an average value of pixel values in the target non-border block; and merge the target non-border block into the foreground area when the difference between the second parameter value and the first parameter value is less than or equal to a difference threshold.
[0081] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0082] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.
[0083] The image processing device provided in the embodiment of the present application can achieve Figure 1 The various processes implemented by the image processing method embodiment can achieve the same technical effect. To avoid repetition, they will not be described here.
[0084] like Figure 7 As shown, an embodiment of the present application also provides an electronic device 700, including a processor 702 and a memory 704, wherein the memory 704 stores programs or instructions that can be executed on the processor 702, and when the program or instructions are executed by the processor 702, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.
[0085] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.
[0086] Figure 8 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.
[0087] like Figure 8As shown, the electronic device 800 includes but is not limited to: a radio frequency unit 801, a network module 802, an audio output unit 803, an input unit 804, a sensor 805, a display unit 806, a user input unit 807, an interface unit 808, a memory 809, and a processor 810 and other components.
[0088] Those skilled in the art will appreciate that the electronic device 800 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 810 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.
[0089] In some embodiments of the present application, the processor 810 is used to obtain a first image and a disparity map corresponding to the first image; segment the disparity map to obtain multiple image blocks; determine boundary blocks and non-boundary blocks in the multiple image blocks; cluster the pixel values in the boundary blocks to obtain a foreground area of the boundary blocks, the foreground area corresponds to a first parameter value, and the first parameter value is the average value of the pixel values in the foreground area; with the boundary block as the center, the non-boundary blocks are merged into the foreground area based on the first parameter value to obtain a merged foreground area; the first image is processed using the merged foreground area to obtain a foreground image.
[0090] In some embodiments of the present application, the processor 810 is specifically used to obtain multiple gradient maps corresponding to multiple image blocks, each gradient map includes multiple gradient values; when the target gradient value in the first gradient map is greater than or equal to the first gradient value, determine that the image block corresponding to the first gradient value is a boundary block; when the gradient value of the first gradient map is less than the first gradient value, determine that the image block corresponding to the first gradient value is a non-boundary block; wherein the first gradient map is any one of the multiple gradient maps.
[0091] In some embodiments of the present application, the target gradient value is the gradient value at the Nth position when multiple gradient values are sorted from large to small, where N is a positive integer greater than 1.
[0092] In some embodiments of the present application, the processor 810 is specifically used to obtain a segmentation size; the disparity map is segmented using the segmentation size to obtain multiple image blocks; wherein the multiple image blocks include a first image block, a second image block, and a third image block, the first image block partially overlaps with the second image block, the first image block partially overlaps with the third image block, the second image block is an image block adjacent to the first image block in a first direction, the third image block is an image block adjacent to the first image block in a second direction, and the first direction is perpendicular to the second direction.
[0093] In some embodiments of the present application, the processor 810 is specifically used to cluster the pixel values in the boundary block using a clustering algorithm to obtain a first pixel value set and a second pixel value set, wherein the pixel values in one of the first pixel value set and the second pixel value set are less than or equal to a preset value, and the pixel values in the other of the first pixel value set and the second pixel value set are greater than the preset value, and the preset value is the median value of the pixel values in the boundary block; obtain a first median pixel value and a second median pixel value, wherein the first median pixel value is the median value of the pixel values in the first pixel value set, and the second median pixel value is the median value of the pixel values in the second pixel value set; and use the pixel value set corresponding to the larger median pixel value of the first median pixel value and the second median pixel value as the foreground area of the boundary block.
[0094] In some embodiments of the present application, the processor 810 is specifically used to obtain a target non-border block, which is a non-border block centered on the boundary block and adjacent to the boundary block; obtain a second parameter value, which is an average value of pixel values in the target non-border block; and when the difference between the second parameter value and the first parameter value is less than or equal to a difference threshold, merge the target non-border block into the foreground area.
[0095] It should be understood that in the embodiment of the present application, the input unit 804 may include a graphics processor (Graphics Processing Unit, GPU) 8041 and a microphone 8042, and the graphics processor 8041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 806 may include a display panel 8061, and the display panel 8061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 807 includes a touch panel 8071 and at least one of other input devices 8072. The touch panel 8071 is also called a touch screen. The touch panel 8071 may include two parts: a touch detection device and a touch controller. Other input devices 8072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.
[0096] The memory 809 can be used to store software programs and various data. The memory 809 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 809 may include a volatile memory or a non-volatile memory, or the memory 809 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 809 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.
[0097] The processor 810 may include one or more processing units; optionally, the processor 810 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 810.
[0098] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0099] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0100] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0101] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0102] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0103] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "including one" do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the embodiments of the present application are essentially or partly contributed to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.
[0105] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image processing method, characterized in that: include: Acquire a first image and a disparity map corresponding to the first image; Segmenting the disparity map to obtain a plurality of image blocks; Determine boundary blocks and non-boundary blocks among the plurality of image blocks; Performing clustering processing on the pixel values in the boundary block to obtain a foreground area of the boundary block, the foreground area corresponds to a first parameter value, and the first parameter value is an average value of the pixel values in the foreground area; Taking the boundary block as the center, merging the non-boundary block into the foreground area based on the first parameter value to obtain a merged foreground area; The first image is processed using the merged foreground area to obtain a foreground image.
2. The image processing method according to claim 1, characterized in that: The determining of the boundary blocks and the non-boundary blocks in the plurality of image blocks specifically comprises: Acquire a plurality of gradient maps corresponding to the plurality of image blocks, each of the gradient maps comprising a plurality of gradient values; When the target gradient value in the first gradient map is greater than or equal to the first gradient value, determining that the image block corresponding to the first gradient value is a boundary block; When the gradient value of the first gradient map is less than the first gradient value, determining that the image block corresponding to the first gradient value is a non-boundary block; The first gradient map is any one of the multiple gradient maps.
3. The image processing method according to claim 2, characterized in that: The target gradient value is the gradient value at the Nth position when the plurality of gradient values are sorted from large to small, where N is a positive integer greater than 1.
4. The image processing method according to claim 2, characterized in that: The segmenting of the disparity map to obtain a plurality of image blocks specifically includes: Get the segmentation size; Segmenting the disparity map using the segmentation size to obtain a plurality of image blocks; The multiple image blocks include a first image block, a second image block and a third image block, the first image block partially overlaps with the second image block, the first image block partially overlaps with the third image block, the second image block is an image block adjacent to the first image block in a first direction, the third image block is an image block adjacent to the first image block in a second direction, and the first direction is perpendicular to the second direction.
5. The image processing method according to any one of claims 1 to 4, characterized in that: The clustering process of the pixel values in the boundary block to obtain the foreground area of the boundary block specifically includes: Clustering the pixel values in the boundary block using a clustering algorithm to obtain a first pixel value set and a second pixel value set, wherein the pixel values in one of the first pixel value set and the second pixel value set are less than or equal to a preset value, and the pixel values in the other of the first pixel value set and the second pixel value set are greater than a preset value, and the preset value is a median value of the pixel values in the boundary block; Obtaining a first median pixel value and a second median pixel value, wherein the first median pixel value is a median value of pixel values in the first pixel value set, and the second median pixel value is a median value of pixel values in the second pixel value set; A pixel value set corresponding to a larger median pixel value between the first median pixel value and the second median pixel value is used as a foreground area of the boundary block.
6. The image processing method according to any one of claims 1 to 4, characterized in that: The step of taking the boundary block as the center and merging the non-boundary block into the foreground area based on the first parameter value to obtain the merged foreground area specifically includes: Acquire a target non-boundary block, where the target non-boundary block is the non-boundary block centered on the boundary block and adjacent to the boundary block; Obtaining a second parameter value, where the second parameter value is an average value of pixel values in the target non-boundary block; When the difference between the second parameter value and the first parameter value is less than or equal to a difference threshold, the target non-boundary block is merged into the foreground area.
7. An image processing device, characterized in that: include: An acquisition module, used to acquire a first image and a disparity map corresponding to the first image; A segmentation module, used to segment the disparity map to obtain multiple image blocks; A determination module, used to determine boundary blocks and non-boundary blocks in the plurality of image blocks; A clustering module, used for performing clustering processing on the pixel values in the boundary block to obtain a foreground area of the boundary block, wherein the foreground area corresponds to a first parameter value, and the first parameter value is an average value of the pixel values in the foreground area; a merging module, configured to merge the non-boundary block into the foreground area based on the first parameter value with the boundary block as the center, to obtain a merged foreground area; A processing module is used to process the first image using the merged foreground area to obtain a foreground image.
8. The image processing device according to claim 7, characterized in that: The determination module is specifically used for: Acquire a plurality of gradient maps corresponding to the plurality of image blocks, each of the gradient maps comprising a plurality of gradient values; When the target gradient value in the first gradient map is greater than or equal to the first gradient value, determining that the image block corresponding to the first gradient value is a boundary block; When the gradient value of the first gradient map is less than the first gradient value, determining that the image block corresponding to the first gradient value is a non-boundary block; The first gradient map is any one of the multiple gradient maps.
9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 6 are implemented.
10. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 6 are implemented.