An image key region segmentation method, device, equipment and storage medium

Abstract

The application relates to the technical field of image segmentation, in particular to an image key region segmentation method and device, equipment and a storage medium. The method is used for solving the problem of low segmentation effect of a traditional image segmentation scheme. The method is as follows: based on edge information of an image to be segmented, region fusion is performed on a key region and a non-key region in a key region set in the image to be segmented, and corresponding fused boundary information is obtained; based on the corresponding fused boundary information, region fusion is performed on a candidate region except the key region in the image to be segmented, and initial boundary information of each region included in the image to be segmented is obtained; and based on the edge information, each initial boundary information is corrected to obtain target boundary information of a region where a target object is located; thus, the overall calculation amount of processing the image to be segmented is reduced, the final region classification accuracy is improved, and the segmentation effect is improved without affecting the image quality effect of the image key region.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image segmentation, in particular to an image key region segmentation method, device and equipment and storage medium. BACKGROUND

[0002] With the rapid development of image processing technology, image segmentation has become a hot spot in the field of image understanding. Image segmentation is the first step of image analysis, the basis of computer vision, and an important part of image understanding.

[0003] Under the traditional image segmentation technology, the image is usually evenly divided into partitions, and then the features of the partitions are extracted. Then, based on the features extracted by the partitions, the regions of the image are fused to obtain the image segmentation result. However, since the traditional image segmentation technology often needs image information of continuous multiple frames of images or multiple different processing of the same image to determine the regions of the image, the image segmentation of the image is realized, and the corresponding image segmentation result is obtained. Obviously, using the traditional image segmentation technology has the following problems: the time-consuming of processing the image is long; the evenly divided partitions of the image have low flexibility and large calculation amount, and the accuracy of the obtained image segmentation result is low.

[0004] Therefore, with the continuous development of artificial intelligence, the appearance of neural network optimizes the partition method of traditional image segmentation technology to a certain extent and improves the accuracy of image segmentation result. However, using neural network usually needs to train the model first, and then the trained model can be used for image segmentation. Before training the model, different training sample sets need to be collected for different models, that is, the training samples need to be directionally segmented, and then the training sample set is used to directionally train the model until the model in the actual application meets the preset condition in terms of accuracy. It can be seen that using neural network also has the problems of long time-consuming, large calculation amount and poor flexibility. SUMMARY

[0005] The embodiments of the present application provide an image key region segmentation method, device, equipment and storage medium, which can accurately segment the image key region under the premise of reducing the calculation amount, so as to facilitate subsequent image enhancement processing of the image key region.

[0006] The specific technical solutions provided by the embodiments of the present application are as follows:

[0007] In a first aspect, the embodiments of the present application provide an image key region segmentation method, comprising:

[0008] obtaining edge information of a to-be-segmented image, wherein the to-be-segmented image comprises at least one target object;

[0009] setting a key region in the image to be segmented according to a preset region configuration rule, wherein the key region comprises at least one focus region and at least one non-focus region, and the focus region comprises part or all of the region where the target object is located;

[0010] performing region fusion on the image to be segmented in the key region based on the edge information, to obtain the fused boundary information corresponding to the focus region and the non-focus region, respectively;

[0011] performing region fusion on the candidate region in the image to be segmented except the key region based on the fused boundary information corresponding to the focus region and the non-focus region, to obtain the initial boundary information of each region included in the image to be segmented;

[0012] correcting the initial boundary information of each region based on the edge information, to obtain the target boundary information of the region where the target object is located.

[0013] The image focus region segmentation method provided by the application greatly reduces the calculation amount without affecting the image quality of the image focus region by analyzing the features of the key region instead of the features of the complete image to be segmented. In the feature extraction stage of the image to be segmented, not only the pixel feature information of the image to be segmented is extracted, but also the edge information of the image to be segmented, which is high-dimensional space information, is obtained. The edge information of the image to be segmented is introduced to guide the subsequent feature extraction of the candidate region in the image to be segmented except the key region, image region classification, and correction of the initial boundary information of the final region, thereby improving the classification accuracy of the image. Further, the classification is based on the similarity between the non-key region and the key region of the image to be segmented, which has certain robustness to the illumination change between frames.

[0014] In a possible implementation, before the edge information of the image to be segmented is obtained, the method further comprises:

[0015] selecting at least one candidate image from an image set, and taking the candidate image as the original image corresponding to the image to be segmented, wherein the image set comprises each candidate image that needs to be segmented in the image focus region; or obtaining video data, extracting any frame image from the video data, and taking the frame image as the original image corresponding to the image to be segmented;

[0016] performing smoothing filtering on the original image by using a preset filtering mode, to obtain the image to be segmented;

[0017] The edge information of the image to be segmented is obtained by:

[0018] The Canny edge detection algorithm is used to perform edge detection on the image to be segmented, thereby obtaining the edge information of the image to be segmented.

[0019] In one possible implementation, setting key regions in the image to be segmented according to preset region configuration rules includes:

[0020] According to statistical data, the key region is set in the central region of the image to be segmented, and the non-key region is set in the edge region of the image to be segmented, wherein each key region has the same size, and each non-key region has the same size.

[0021] In one possible implementation, the step of performing region fusion on the image to be segmented within the key region based on the edge information to obtain fused boundary information corresponding to the key region and the non-key region includes:

[0022] Based on the edge information, feature extraction is performed on the image to be segmented within the reference region to obtain each reference feature vector corresponding to the reference region, wherein the reference region is the key region or the non-key region;

[0023] For any reference feature vector among the various reference feature vectors, determine the corresponding similarity based on the reference feature vector and each other reference feature vector;

[0024] The reference feature vectors with similarity greater than a preset value are fused to obtain the corresponding fused feature vectors. The regions corresponding to the reference feature vectors with similarity greater than the preset value are then fused to obtain the fused boundary information of the reference regions.

[0025] In one possible implementation, the step of extracting features from the image to be segmented within the reference region based on the edge information to obtain various reference feature vectors corresponding to the reference region includes:

[0026] Based on any sub-edge information in the edge information, for each pixel in the reference area, the weight value of the pixel on the same side of the sub-edge information is set as a first weight value, and the weight value of the pixel on the opposite side of the sub-edge information is set as a second weight value, wherein the first weight value is greater than the second weight value.

[0027] Based on the weight values ​​of each pixel, a preset filtering method is used to extract the pixel features of the image to be segmented within the reference region, thereby obtaining each reference feature vector corresponding to the reference region.

[0028] In a possible implementation, the method further includes: performing region merging on the candidate region other than the key region in the to-be-segmented image based on the fused boundary information corresponding to the key region and the non-key region, to obtain initial boundary information of each region included in the to-be-segmented image, including:

[0029] extending the candidate region other than the key region in the to-be-segmented image, performing region classification on the extended region based on the fused boundary information corresponding to the key region and the non-key region, to obtain a classification information set corresponding to the candidate region, wherein the classification information set includes at least one piece of classification information;

[0030] performing feature extraction on the candidate region based on the fused boundary information corresponding to the key region and the non-key region and the classification information set, to obtain a plurality of other feature vectors;

[0031] traversing the plurality of other feature vectors, determining, for each traversed other feature vector, a first difference value between the other feature vector and each first feature vector, and determining a second difference value between the other feature vector and each second feature vector, wherein the first feature vector is any of the fused feature vector corresponding to the key region or any of the remaining reference feature vectors, and the second feature vector is any of the fused feature vector corresponding to the non-key region or any of the remaining reference feature vectors;

[0032] if the first difference value is smaller than the second difference value, performing region merging on the regions corresponding to the first feature vector and the other feature vector respectively, or if the first difference value is greater than the second difference value, performing region merging on the regions corresponding to the second feature vector and the other feature vector respectively;

[0033] determining, based on the key region and the non-key region after region merging respectively, initial boundary information corresponding to the key region and the non-key region respectively.

[0034] In a possible implementation, the method further includes: based on the edge information, respectively correcting the initial boundary information of each region to obtain target boundary information of the region where the target object is located, including:

[0035] traversing each pixel point included in the to-be-segmented image, and based on the edge information, determining whether the pixel point meets a preset condition for each traversed pixel point, wherein the preset condition includes that pixel points on the same side of the same boundary information have the same classification label, and pixel points in a closed boundary information have the same classification label, the boundary information is any sub-edge information included in the edge information, and the classification label of each pixel point is labeled after determining the classification information of the region where the pixel point is located.

[0036] If yes, the classification label of the pixel point is saved; otherwise, the classification label corresponding to the pixel point with the largest number in the region where the pixel point is located is used to replace the classification label of the pixel point;

[0037] Based on the classification labels of the pixel points after the traversal, target boundary information of each region included in the image to be segmented is determined.

[0038] The target boundary information corresponding to the key region in the each region is used as the target boundary information of the region where the target object is located.

[0039] In a second aspect, an embodiment of the present application provides an image key region segmentation device, which comprises:

[0040] An acquisition module is configured to acquire edge information of an image to be segmented, wherein the image to be segmented comprises at least one target object.

[0041] A setting module is configured to set a key region in the image to be segmented according to a preset region configuration rule, wherein the key region comprises at least one key region and at least one non-key region, and the key region comprises part or all of the region where the target object is located.

[0042] A first fusion module is configured to perform region fusion on the image to be segmented in the key region based on the edge information, and obtain fusion boundary information corresponding to the key region and the non-key region, respectively.

[0043] A second fusion module is configured to perform region fusion on a candidate region in the image to be segmented except the key region based on the fusion boundary information corresponding to the key region and the non-key region, and obtain initial boundary information of each region included in the image to be segmented.

[0044] A correction module is configured to correct the initial boundary information of each region based on the edge information, and obtain target boundary information of the region where the target object is located.

[0045] In a possible implementation, before the edge information of the image to be segmented is acquired, the acquisition module is further configured to:

[0046] At least one candidate image is selected from an image set, and the candidate image is used as an original image corresponding to the image to be segmented, wherein the image set comprises each candidate image that needs to be subjected to image key region segmentation; or video data is acquired, and any frame image is extracted from the video data, and the frame image is used as an original image corresponding to the image to be segmented.

[0047] Smoothly filter the original image by using a preset filtering mode to obtain the image to be segmented;

[0048] The edge information of the image to be segmented is obtained, and the obtaining module is configured to:

[0049] An edge detection algorithm is used to detect edges of the image to be segmented to obtain the edge information of the image to be segmented.

[0050] In a possible implementation, the key region is set in the image to be segmented according to a preset region configuration rule, and the setting module is configured to:

[0051] According to statistical data, the key region is set in a central region of the image to be segmented, and the non-key region is set in an edge region of the image to be segmented, wherein the size of each key region is the same, and the size of each non-key region is the same.

[0052] In a possible implementation, the image to be segmented in the key region is fused based on the edge information to obtain fused boundary information corresponding to the key region and the non-key region respectively, and the first fusion module is configured to:

[0053] Based on the edge information, feature vectors of the reference region corresponding to the image to be segmented in the reference region are obtained, wherein the reference region is the key region or the non-key region;

[0054] For any one of the reference feature vectors, a similarity corresponding to the any one reference feature vector and each other reference feature vector is determined;

[0055] The reference feature vectors with a similarity greater than a preset value are fused to obtain a fused feature vector corresponding to the reference feature vectors, and the reference regions corresponding to the reference feature vectors with a similarity greater than a preset value are fused to obtain fused boundary information corresponding to the reference region.

[0056] In a possible implementation, the image to be segmented in the reference region is fused based on the edge information to obtain feature vectors of the reference region corresponding to the image to be segmented in the reference region, and the first fusion module is configured to:

[0057] Based on any sub-edge information in the edge information, for each pixel point in the reference region, a weight value of a pixel point on a same side of the sub-edge information is set as a first weight value, and a weight value of a pixel point on an opposite side of the sub-edge information is set as a second weight value, wherein the first weight value is greater than the second weight value;

[0058] respectively based on the weight values of the respective pixels, pixel features of the to-be-segmented image in the reference region are extracted using a preset filtering manner, to obtain respective reference feature vectors corresponding to the reference region.

[0059] In a possible implementation, based on the fused boundary information corresponding to the key region and the non-key region, region fusion is performed on the candidate region in the to-be-segmented image except the key region, to obtain initial boundary information of each region included in the to-be-segmented image, and the second fusion module is configured to:

[0060] extend the candidate region in the to-be-segmented image except the key region, based on the fused boundary information corresponding to the key region and the non-key region, region classification is performed on the extended region, to obtain a classification information set corresponding to the candidate region, wherein the classification information set includes at least one piece of classification information;

[0061] based on the fused boundary information corresponding to the key region and the non-key region and the classification information set, feature extraction is performed on the candidate region, to obtain a plurality of other feature vectors;

[0062] each time one of the other feature vectors is traversed, a first difference value between the other feature vector and each first feature vector is determined, and a second difference value between the other feature vector and each second feature vector is determined, wherein the first feature vector is any of the fused feature vectors corresponding to the key region or any of the remaining reference feature vectors, and the second feature vector is any of the fused feature vectors corresponding to the non-key region or any of the remaining reference feature vectors;

[0063] if the first difference value is smaller than the second difference value, region fusion is performed on the regions corresponding to the first feature vector and the other feature vector respectively, or if the first difference value is greater than the second difference value, region fusion is performed on the regions corresponding to the second feature vector and the other feature vector respectively;

[0064] based on the key region and the non-key region after region fusion respectively, initial boundary information corresponding to each is determined.

[0065] In a possible implementation, based on the edge information, the initial boundary information of each region is respectively corrected, to obtain target boundary information of the region where the target object is located, and the correction module is configured to:

[0066] The method comprises the following steps: traversing each pixel point included in the image to be segmented; determining, for each pixel point, whether the pixel point meets a preset condition based on the edge information, wherein the preset condition comprises that the pixel points on the same side of the same boundary information have the same classification label, and the pixel points within the closed boundary information have the same classification label, the boundary information is any sub-edge information included in the edge information, and the classification label of each pixel point is labeled after determining the classification information of the region where the pixel point is located; if yes, saving the classification label of the pixel point; otherwise, replacing the classification label of the pixel point with the classification label corresponding to the pixel point with the largest number in the region where the pixel point is located.

[0067] If yes, the classification label of the pixel point is saved; otherwise, the classification label of the pixel point is replaced with the classification label corresponding to the pixel point with the largest number in the region where the pixel point is located.

[0068] Based on the classification labels of the pixel points after the traversal is completed, target boundary information of each region included in the image to be segmented is determined.

[0069] The target boundary information corresponding to the key region in the regions is taken as the target boundary information of the region where the target object is located.

[0070] In a third aspect, an electronic device is provided, comprising:

[0071] a memory configured to store a computer program executable by a controller;

[0072] The controller is connected with the memory and is configured to execute the method of any one of the above first aspect.

[0073] In a fourth aspect, a computer readable storage medium is provided, when instructions in the storage medium are executed by a processor, the processor can execute the method of any one of the above first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0074] Figure 1A is a schematic diagram of an application scenario in an embodiment of the present application;

[0075] Figure 1A is a schematic diagram of another application scenario in an embodiment of the present application;

[0076] Figure 2 is a flowchart of a method for segmenting a key region of an image in an embodiment of the present application;

[0077] Figure 3 is a flowchart of a method for obtaining an image to be segmented in an embodiment of the present application;

[0078] Figure 4 is a schematic diagram of obtaining an image to be segmented by performing a smoothing filter on an original image in an embodiment of the present application;

[0079] Figure 5 A process schematic diagram of obtaining edge information of an image to be segmented by using a Canny edge detection algorithm in an embodiment of the present application;

[0080] Figure 6 A setting effect schematic diagram of a key area and a non-key area in an embodiment of the present application;

[0081] Figure 7 A process schematic diagram of determining fusion boundary information corresponding to a reference area in an embodiment of the present application;

[0082] Figure 8 A process schematic diagram of extracting each reference feature vector corresponding to a reference area in an embodiment of the present application;

[0083] Figure 9 A principle schematic diagram of extracting pixel features by using a filter in an embodiment of the present application;

[0084] Figure 10 A boundary line schematic diagram of sub-edge information in a reference area in an embodiment of the present application;

[0085] Figure 11 A boundary line effect schematic diagram of fusion boundary information in an embodiment of the present application;

[0086] Figure 12 A process schematic diagram of a method for determining initial boundary information of each area in an image to be segmented in an embodiment of the present application;

[0087] Figure 13 A classification information marking effect schematic diagram of a pixel in an embodiment of the present application;

[0088] Figure 14 A process schematic diagram of a method for obtaining target boundary information of an area where a target object is located in an embodiment of the present application;

[0089] Figure 15 A boundary line effect schematic diagram of initial boundary information in an embodiment of the present application;

[0090] Figure 16 A boundary line effect schematic diagram of target boundary information in an embodiment of the present application;

[0091] Figure 17 A classification result effect schematic diagram obtained in an actual application in an embodiment of the present application;

[0092] Figure 18 A logical architecture schematic diagram of a segmentation device of an image key area in an embodiment of the present application;

[0093] Figure 19 Fig. 1 is a schematic diagram of an entity architecture of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0094] The technical solutions in the embodiments of the present application will be clearly and completely described in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of the present application.

[0095] It should be noted that the terms "first", "second", "third", etc. in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0096] The design idea of the embodiments of the present application will be briefly introduced below.

[0097] The present application relates to the technical field of image segmentation, and mainly relates to an image key region segmentation method, device, equipment and storage medium.

[0098] With the rapid development of image processing technology, under the prior art, there are generally two kinds of image key region segmentation schemes as follows:

[0099] Scheme one: traditional image segmentation technology.

[0100] In scheme one, usually, image information of continuous multiple frames of images is needed, or the same image is processed multiple times differently to realize image segmentation. Specifically, first, the image is evenly divided into partitions, then the features of the partitions are extracted respectively, and based on the features extracted from the partitions, the image is fused by region to obtain the image segmentation result.

[0101] Obviously, using the traditional image segmentation technology has the problems of long time consumption for processing the image, low flexibility for partitioning the image, and low accuracy of the image segmentation result based on the evenly divided partitioning method.

[0102] Scheme two, using a neural network model.

[0103] The emergence of neural networks has to some extent optimized the partitioning method of traditional image segmentation technology, and improved the accuracy of the image segmentation result. For example, a traditional neural network model that distinguishes between foreground information and background information to realize segmentation of each region contained in the image, or an instance segmentation network model for a target object.

[0104] However, the neural network usually needs to train a model first, and then the trained model can be used for image segmentation. Before training the model, a training sample set needs to be collected, and different training sample sets are collected for models with different needs, that is, the training sample needs to be directionally segmented first, and then the training sample set is used to train the model directionally to obtain the model in the above actual application which meets the preset condition in accuracy. It can be seen that the neural network also has the problems of long time consumption and poor flexibility.

[0105] For example, for the application scenario of display panels, the image algorithm accuracy in most image enhancement algorithms is often proportional to the calculation amount. If finer and better image quality effects are desired to be presented in the display panel, a central processing unit with higher calculation capacity needs to be configured for the display panel. However, for images, due to the image composition characteristics, the color difference between adjacent pixels is not large, and the image usually contains a variety of redundant information. Directly using the existing trained neural network model cannot achieve the expected image segmentation effect.

[0106] Further, due to the perception characteristics of the human eye, the emphasis degree of image content is not completely consistent. Usually, the human eye often only focuses on a few key parts to obtain effective information, and therefore, it is not necessary to perform equal image enhancement on each region in the image. Only the image key region needs to use the image enhancement algorithm with large calculation amount. In this way, not only the calculation amount in the image segmentation process is reduced, but also the key content contained in the image has finer and better image quality effects.

[0107] Therefore, in order to solve the problems of long time consumption, large calculation amount, and low image segmentation effect of the traditional image segmentation technology and the neural network model, in the embodiments of the present application, the edge information of the to-be-segmented image is obtained, and a key region is set in the to-be-segmented image according to a preset region configuration rule. The to-be-segmented image includes at least one target object, the key region includes at least one key region and at least one non-key region, and the key region includes part or all of the region where the target object is located. Then, based on the edge information, the to-be-segmented image in the key region is subjected to region fusion to obtain the fused boundary information corresponding to the key region and the non-key region, respectively. Then, based on the fused boundary information corresponding to the key region and the non-key region, the candidate region in the to-be-segmented image except the key region is subjected to region fusion to obtain the initial boundary information of each region included in the to-be-segmented image. Finally, based on the edge information, the initial boundary information of each region is corrected to obtain the target boundary information of the region where the target object is located.

[0108] The image key region segmentation method provided in the application greatly reduces the overall calculation amount of processing the image to be segmented under the premise of not affecting the image quality effect of the image key region by analyzing the features of the key region instead of analyzing the features of the complete image to be segmented; in the feature extraction stage of the image to be segmented, not only the pixel feature information of the image to be segmented is extracted, but also the edge information and high-dimensional space information of the image to be segmented are obtained; the edge information of the image to be segmented is introduced to guide the feature extraction of the candidate region other than the key region in the image to be segmented, the image region classification, and the correction of the initial boundary information of the final region, thereby improving the classification accuracy of the image.

[0109] Further, the classification is performed based on the pixel feature similarity between the non-key region (the candidate region other than the key region in the image to be segmented) and the key region of the image to be segmented, and has a certain robustness to illumination changes between frames. The image key region segmentation method provided in the embodiments of the application is suitable for an electronic device. Illustratively, the electronic device can be an electronic device with an image display device (for example, a display panel); for example, it can be a smart terminal, a smart mobile terminal, a tablet computer, a notebook computer, a smart palm device, a personal computer (PC), a computer, a smart screen, a display device, various wearable devices, a personal digital assistant (PDA), etc.; the electronic device can also be a server connected with a device with a display function (for example, a display device), such as a standalone physical server, a server cluster composed of multiple physical servers or a distributed system, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms, etc. basic cloud computing services of cloud server; wherein the server can be connected with the device with the display function through a wired or wireless manner, and the application does not limit the connection manner.

[0110] It can be understood that the specific form of the electronic device is not limited in the application.

[0111] The following takes the electronic device as the display device to illustrate an application scenario in the embodiments of the application, as shown in Figure 1A . Referring to Figure 1A , the application scenario includes a display device 100. The display device 100 performs the image key region segmentation process provided in the embodiments of the application based on the obtained image data (the specific process will be introduced later), and displays the image data after subsequent processing or directly displays the image data.

[0112] Another application scenario of the embodiment of the present application is illustrated below taking an electronic device as a server as an example. As shown in Figure 1B , the application scenario includes a server 110 and a display device 100, and the server 110 communicates with the display device 100 through the Internet. The server 110 performs the segmentation process of the image key area provided by the embodiment of the present application based on the obtained image data (the specific process will be described later), and then transmits the segmented image data to the display device 100 through the Internet, so that the display device 100 displays after subsequent processing or directly displays the segmented image data. Figure 1B

[0113] After introducing the application scenario of the embodiment of the present application, the preferred embodiment of the present application is further described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the features of the embodiments of the present application and the embodiments can be combined with each other without conflict.

[0114] As shown in Figure 2 , in the embodiment of the present application, a segmentation method of image key area is provided, and the specific process of the method is as follows:

[0115] Step 200: Obtain the edge information of the image to be segmented, wherein the image to be segmented includes at least one target object.

[0116] In some feasible embodiments, before performing step 200, the following steps need to be performed, as shown in Figure 3

[0117] Step 300: Select at least one candidate image from the image set, and take the candidate image as the original image corresponding to the image to be segmented, wherein the image set contains various candidate images that need to be segmented by the image key area; or, obtain video data, extract any frame image from the video data, and take the frame image as the original image corresponding to the image to be segmented.

[0118] In the embodiment of the present application, when performing step 300, at least one candidate image can be selected from the pre-collected image set, and the candidate image is taken as the original image corresponding to the image to be segmented, wherein the image set contains various candidate images that need to be segmented by the image key area; video data can also be obtained, and then any frame image is extracted from the obtained video data, and the frame image is taken as the original image corresponding to the image to be segmented, wherein the frame image contains at least one target object that needs to be segmented by the image key area.

[0119] Step 310: Smoothly filter the original image by using a preset filtering method to obtain the image to be segmented.​​

[0120] In the embodiment of the present application, step 310 is performed to reduce the complexity of subsequent processing of the image to be segmented, reduce the pixel value range, and remove part of the low-frequency noise.

[0121] For example, referring to FIG. 2, an original image A is taken as an example. Figure 4

[0122] It is assumed that the preset filtering mode is Gaussian filtering.

[0123] Then, the Gaussian filtering is used to perform smoothing filtering on the original image A to obtain the image to be segmented A.

[0124] It should be noted that in the embodiment of the present application, the Gaussian filtering is only an example, and other forms of smoothing filtering mode can also be used to perform smoothing filtering on the original image to obtain the image to be segmented, so as to achieve the purpose of reducing the subsequent image processing calculation amount, and the smoothing filtering mode is not limited here.

[0125] Then, after obtaining the image to be segmented by performing steps 300-310, step 200 is performed to obtain the edge information of the image to be segmented, wherein the image to be segmented includes at least one target object.

[0126] In the implementation, when step 200 is performed, the canny edge detection algorithm can be used to perform edge detection on the image to be segmented to obtain the edge information of the image to be segmented, as shown in FIG. 3. Figure 5

[0127] It should be noted that similarly, in the embodiment of the present application, other forms of edge detection mode can also be used to perform edge filtering on the image to be segmented to obtain the corresponding edge information, and the specific form of edge detection is not limited here.

[0128] Step 210: setting a key region in the image to be segmented according to a preset region configuration rule, wherein the key region includes at least one key region and at least one non-key region, and the key region includes part or all of the region where the target object is located.

[0129] Before the implementation of the present scheme, a large amount of image sample data is collected in advance, and the statistical data of the boundary information corresponding to the key region and the non-key region of each image sample data is obtained through big data statistics. According to the statistical data, due to the subjective perception characteristics of the human eye, the image content in the center of the display panel is more likely to be paid attention to than the boundary content.

[0130] ​​Therefore, in the embodiment of the present application, when step 210 is performed, according to the statistical data, the key regions are set in the center region of the image to be segmented, and the non-key regions are set in the edge region of the image to be segmented, wherein the size of each key region is the same, and the size of each non-key region is the same.

[0131] In a specific implementation, a plurality of key regions can be set in the center region of the image to be segmented, and a plurality of non-key regions can be set in the edge region of the image to be segmented, so as to improve the perception ability of the electronic device to the image to be segmented.

[0132] For example, referring to FIG. 2A, take the image A to be segmented as an example. Figure 6

[0133] Suppose that the preset region configuration rule is to set four key regions in the center region of the image A to be segmented, and to set four non-key regions in the edge region (i.e., the four corner regions) of the image A to be segmented, and the size of each key region is the same, and the size of each non-key region is the same.

[0134] Then, in the embodiment of the present application, according to the above-mentioned preset region configuration rule, four key regions are set in the center region of the image A to be segmented, and four non-key regions are set in the four corner regions of the image A to be segmented, as shown in FIG. 2B. Figure 6

[0135] In the embodiment of the present application, after the key regions and the non-key regions are set in the image to be segmented, a classification label corresponding to the key region is marked for each pixel point in the key region, and a classification label corresponding to the non-key region is marked for each pixel point in the non-key region.

[0136] Step 220: based on the edge information, performing region fusion on the image to be segmented in the key region, to obtain the fusion boundary information corresponding to the key region and the non-key region, respectively.

[0137] In the embodiment of the present application, for the convenience of subsequent description, the key region and the non-key region are collectively referred to as a reference region, but it should be clear that based on the edge information, the feature extraction is performed on the image to be segmented in the key region and the non-key region, to obtain the respective fusion boundary information.

[0138] In a specific implementation, referring to FIG. 2C, when step 220 is performed, the following steps are specifically performed: Figure 7

[0139] Step 2201: based on the edge information, performing feature extraction on the image to be segmented in the reference region, to obtain each reference feature vector corresponding to the reference region, wherein the reference region is the key region or the non-key region.​​​

[0140] In the embodiment of the present application, when the step 2201 is performed, a plurality of filtering methods can be used to extract the features of the image to be segmented in the key region and the non-key region respectively based on the edge information, so as to obtain the respective reference feature vectors.

[0141] In the specific implementation, referring to FIG. 8, when the step 2201 is performed, the following steps are specifically performed: Figure 8

[0142] Step 800: Based on any sub-edge information in the edge information, the weight value of the pixel point on the same side of the sub-edge information is set as a first weight value, and the weight value of the pixel point on the opposite side of the sub-edge information is set as a second weight value, wherein the first weight value is greater than the second weight value.

[0143] In the embodiment of the present application, when the step 800 is performed, the weight value of each pixel point in the reference region can be set based on any sub-edge information in the edge information, that is, the weight value of the pixel point on the same side of the sub-edge information is set as a first weight value, and the weight value of the pixel point on the opposite side of the sub-edge information is set as a second weight value, wherein the first weight value is greater than the second weight value, so that the filter used in the embodiment of the present application can extract the pixel features of the pixel points on the same side and the opposite side of the sub-edge information respectively.

[0144] Step 810: Based on the weight value of each pixel point, the pixel features of the image to be segmented in the reference region are extracted using a preset filtering method, and each reference feature vector corresponding to the reference region is obtained.

[0145] In the specific implementation, referring to FIG. 8, when the steps 800-810 are performed, it is assumed that any sub-edge information in the edge information represents a curved boundary line in the image to be segmented in the reference region. Figure 9

[0146] Therefore, when the image to be segmented in the reference region is extracted, the image to be segmented in the reference region is divided into two parts, i.e., region 1 (upper part) and region 2 (lower part) based on the boundary line, and the pixel features of the pixel points contained in region 1 and region 2 are extracted respectively.

[0147] It is assumed that the size of the filter corresponding to the preset filtering method is 3*3.

[0148] In actual application, when the above filter is used to extract the features of the image to be segmented in the reference region, the filter is divided into two parts: sub-filter 1 corresponding to region 1 and sub-filter 2 corresponding to region 2.

[0149] ​​Specifically, for the sub-filter 1, the weight value of each pixel point of the to-be-segmented image in the region 1 is set as the first weight value, the weight value of each pixel point of the to-be-segmented image in the region 2 where the filter is located is set as the second weight value, and then the pixel features of each pixel point of the to-be-segmented image in the region 1 are extracted based on the weight values of the pixel points, so as to obtain each reference feature vector corresponding to the region 1.

[0150] Similarly, for the sub-filter 2, the weight value of each pixel point of the to-be-segmented image in the region 1 is set as the second weight value, the weight value of each pixel point of the to-be-segmented image in the region 2 where the filter is located is set as the first weight value, and then the pixel features of each pixel point of the to-be-segmented image in the region 2 are extracted based on the weight values of the pixel points, so as to obtain each reference feature vector corresponding to the region 2.

[0151] In this way, when the to-be-segmented image is filtered, the filter can be guided to better extract pixel features of the same type in the to-be-segmented image based on the edge information of the to-be-segmented image, so as to improve the accuracy of the final region classification.

[0152] Step 2202: For any reference feature vector in each reference feature vector, the similarity corresponding to each other reference feature vector is determined based on any reference feature vector.

[0153] In the embodiment of the application, after the step 2201 is performed to obtain each reference feature vector corresponding to the reference region, the step 2202 is performed to select any one reference feature vector from each reference feature vector, and the similarity of the reference feature vector to each other reference feature vector is calculated.

[0154] For example, referring to FIG. 22B, it is assumed that there are two boundary lines in the reference region and there is a closed region in the reference region. Figure 10

[0155] Then, any one reference feature vector is selected from the four reference feature vectors, for example, the selected reference feature vector is the reference feature vector 1.

[0156] Then, any one reference feature vector is selected from the four reference feature vectors, for example, the selected reference feature vector is the reference feature vector 1.

[0157] Then, the similarity of the reference feature vector 1 to the reference feature vector 2, the reference feature vector 3 and the reference feature vector 4 is calculated, and is sequentially recorded as similarity 1, similarity 2 and similarity 3.

[0158] ​In a specific implementation, any method for calculating the similarity of two vectors can be used to determine the similarity values of any two reference feature vectors, which are not limited in the embodiments of the present application.

[0159] In step 2203, the reference feature vectors with similarity values greater than the preset value are fused to obtain a corresponding fused feature vector, and the regions corresponding to the reference feature vectors with similarity values greater than the preset value are fused to obtain the fused boundary information corresponding to the reference region.

[0160] For example, referring to FIG. 8, the image to be segmented is still taken as an example. Figure 11

[0161] Suppose that two boundary lines pass through the reference region in the image to be segmented, and the two boundary lines form a closed region in the reference region.

[0162] Then, step 2201 is performed to obtain four reference feature vectors corresponding to the reference region, and step 2202 is performed to obtain the similarity of reference feature vector 1 to reference feature vector 2, reference feature vector 3, and reference feature vector 4, which are sequentially denoted as similarity 1, similarity 2, and similarity 3, wherein the selected reference feature vector is reference feature vector 1.

[0163] Then, the reference feature vectors with similarity values greater than the preset value are fused, and the regions corresponding to the reference feature vectors with similarity values greater than the preset value are fused to obtain the fused boundary information corresponding to the reference region.

[0164] Suppose that similarity 1 and similarity 2 are greater than the preset value in the above similarity 1, similarity 2, and similarity 3, i.e., the similarity of reference feature vector 1, reference feature vector 2, and reference feature vector 3 is high.

[0165] Then, reference feature vector 1, reference feature vector 2, and reference feature vector 3 are fused to obtain a fused feature vector, and the regions corresponding to reference feature vector 1, reference feature vector 2, and reference feature vector 3 are fused into one region, and then the fused region is used to obtain the fused boundary information corresponding to the reference region, as shown in FIG. 9. Figure 11

[0166] In a specific implementation, the mean of reference feature vector 1, reference feature vector 2, and reference feature vector 3 can be taken as the fused feature vector fused into one region, and then the fused feature vector corresponding to the reference region and reference feature vector 4 are obtained. In this way, redundant calculation can be reduced, and the calculation amount of image processing can be reduced.

[0167] ​​By performing the steps 2201-2203, the fused boundary information corresponding to the reference region can be obtained. Since the reference region is the key region or the non-key region, after performing the steps 2201-2203 (i.e., the step 220), the fused boundary information corresponding to the key region and the non-key region, respectively, can be obtained. Then, the step 230 is performed. Based on the fused boundary information corresponding to the key region and the non-key region, respectively, the candidate regions in the to-be-segmented image except the key region are extended, so as to fuse the part or all of the candidate regions having similar or identical features with the key region and the key region, and fuse the part or all of the candidate regions having similar or identical features with the non-key region and the non-key region, thereby realizing the image segmentation of the to-be-segmented image.

[0168] The step 230: based on the fused boundary information corresponding to the key region and the non-key region, the candidate regions in the to-be-segmented image except the key region are fused, so as to obtain the initial boundary information of each region included in the to-be-segmented image.

[0169] In the embodiment of the present application, when the step 230 is performed, the following steps are performed with reference to the to-be-segmented image: Figure 12 As shown in the figure, the following steps are specifically performed:

[0170] The step 2301: the candidate regions in the to-be-segmented image except the key region are extended, based on the fused boundary information corresponding to the key region and the non-key region, the extended regions are classified, so as to obtain the classification information set corresponding to the candidate regions, wherein the classification information set at least includes one piece of classification information.

[0171] In the embodiment of the present application, the step 2301 is performed, taking the key region (i.e., the key region and the non-key region) as the reference, the candidate regions in the to-be-segmented image except the key region are extended, the part or all of the candidate regions having similar or identical features with the key region are set as the classification information corresponding to the key region, the part or all of the candidate regions having similar or identical features with the non-key region are set as the classification information corresponding to the non-key region, and the part or all of the candidate regions having different features with the key region and the non-key region are set as the classification information corresponding to other regions, thereby obtaining the classification information set corresponding to the candidate regions.

[0172] In a specific implementation, based on the fused boundary information in the key region, the candidate region in the to-be-segmented image except the key region is extended, if there is a boundary connected with the key region in the extended region, it is determined that the classification information of the extended region belongs to the key region, and the pixel points in the extended region are marked as the classification label corresponding to the key region; if there is a boundary connected with the non-key region in the extended region, it is determined that the classification information of the extended region belongs to the non-key region, and the pixel points in the extended region are marked as the classification label corresponding to the non-key region.

[0173] In actual application, due to the improper setting of the preset region configuration rule, the key region is not set well, or the to-be-segmented image is too smooth, the obtained edge information is too little, or the connection between each region in the to-be-segmented image and other regions is less, and the like, there may be pixel points in part of the candidate region that are not classified, and then the pixel points in the part of the unclassified region are classified as one region, which is recorded as other regions, and each pixel point in the other region is marked as the classification label corresponding to the other region.

[0174] For example, referring to FIG. 8, still taking the to-be-segmented image A as an example. Figure 13

[0175] Suppose that the candidate region in the to-be-segmented image except the key region is extended, and there is a boundary line in the extended region that is the same as the fused boundary information in the key region A.

[0176] Then, the pixel points in the region corresponding to the boundary line are marked as the classification label corresponding to the key region A, for example, the classification label of the pixel point A in FIG. 8, and the corresponding classification information belongs to the key region A. Figure 13

[0177] In this way, through the above region extension and classification operation, each pixel point contained in the to-be-segmented image is marked with a corresponding classification label, then step 2302 is executed, pixel features are extracted through filtering, and fine classification of the to-be-segmented image is realized.

[0178] Step 2302: based on the fused boundary information and the classification information set corresponding to the key region and the non-key region, feature extraction is performed on the candidate region, and a plurality of other feature vectors are obtained.

[0179] In the embodiment of the application, when step 2302 is executed, the filtering mode for feature extraction of the to-be-segmented image in the aforementioned reference region is adopted, that is, based on the fused boundary information and the classification information set, the filter is divided into two parts, and the pixel features of the partition are extracted respectively, so that a plurality of other feature vectors are obtained; at this time, a filter with a size larger than the filter size used in the aforementioned reference region can be used to complete the feature extraction of the candidate region, so as to reduce the calculation amount of image processing. ​​

[0180] Step 2303: traversing the plurality of other feature vectors, and determining, for each other feature vector, a first difference value of the other feature vector and each first feature vector, and determining, for each other feature vector, a second difference value of the other feature vector and each second feature vector, wherein the first feature vector is any of the fusion feature vector corresponding to the key region or any of the remaining reference feature vectors, and the second feature vector is any of the fusion feature vector corresponding to the non-key region or any of the remaining reference feature vectors.

[0181] In the embodiment of the present application, after the step 2302, since the obtained other feature vectors are usually multiple, the first difference value corresponding to each other feature vector and each fusion feature vector corresponding to the key region and each remaining reference feature vector (the reference feature vector not participating in the vector fusion) is determined, and the second difference value corresponding to each other feature vector and each fusion feature vector corresponding to the non-key region and each remaining reference feature vector (the reference feature vector not participating in the vector fusion) is determined.

[0182] Then, for any other feature vector, the first difference value and the second difference value are compared, if the first difference value is less than the second difference value, it indicates that the other feature vector is more similar to the fusion feature vector corresponding to the key region, otherwise, if the first difference value is greater than the second difference value, it indicates that the other feature vector is more similar to the fusion feature vector corresponding to the non-key region.

[0183] It should be noted that if multiple key regions and multiple non-key regions are set, when the step 2303 is performed, the first difference value and the second difference value of each other feature vector of the candidate region and the fusion feature vector and the remaining reference feature vector corresponding to the multiple key regions and the multiple non-key regions are determined in turn.

[0184] Step 2304: if the first difference value is less than the second difference value, the region corresponding to the first feature vector and the other feature vector is fused, or if the first difference value is greater than the second difference value, the region corresponding to the second feature vector and the other feature vector is fused.

[0185] In the embodiments of the present application, if the first difference value is less than the second difference value, it indicates that the other feature vector is more similar to the first feature vector corresponding to the key region, and based on the feature vectors with high similarity belonging to the same region, the region corresponding to the first feature vector in the key region and the region corresponding to the other feature vector are fused; if the first difference value is greater than the second difference value, it indicates that the other feature vector is more similar to the second feature vector, and based on the feature vectors with high similarity belonging to the same region, the region corresponding to the second feature vector in the non-key region and the region corresponding to the other feature vector are fused.

[0186] Step 2305: Based on the fused key region and non-key region respectively, the initial boundary information corresponding to each is determined.

[0187] For example, still taking the image to be segmented A as an example. It is assumed that each pixel point contained in the image to be segmented is marked with a classification label, and a classification information set is obtained.

[0188] Then, based on the fused boundary information corresponding to the key region and the non-key region and the classification information set, feature extraction is performed on the candidate regions in the image to be segmented except the key region, and a plurality of other feature vectors are obtained, denoted as Sj, wherein j is a serial number, representing the number of other feature vectors.

[0189] It is assumed that the fused feature vector corresponding to the key region is denoted as Ln, and the fused feature vector corresponding to the non-key region is denoted as Mi, wherein n and i are serial numbers, respectively representing the number of the fused feature vectors corresponding to each.

[0190] Then, the first difference value of each other feature vector and each first feature vector corresponding to the key region is calculated, and the second difference value of each other feature vector and each second feature vector corresponding to the non-key region is calculated.

[0191] In specific implementation, any method for calculating the difference value of two vectors can be used to determine the first difference value and the second difference value, which is not specifically limited in the embodiments of the present application.

[0192] Then, the region corresponding to the other feature vector with the first difference value less than the second difference value and the region corresponding to the first feature vector in the key region are fused, and the region corresponding to the other feature vector with the first difference value greater than the second difference value and the region corresponding to the second feature vector in the non-key region are fused.

[0193] Suppose that there are three other feature vectors, denoted as S1, S2, and S3, and that there are four key regions, and five corresponding feature vectors (including the fused feature vector and the remaining reference feature vectors), denoted as L1, L2, L3, L4, and L5. There are four non-key regions, and four corresponding feature vectors (including the fused feature vector and the remaining reference feature vectors), denoted as M1, M2, M3, and M4.

[0194] Suppose that the first difference value of each first feature vector corresponding to S1 and the key region is less than each second feature vector corresponding to S1 and the non-key region, the first difference value of each first feature vector corresponding to S2 and the key region is less than each second feature vector corresponding to S2 and the non-key region, and the first difference value of each first feature vector corresponding to S3 and the key region is greater than each second feature vector corresponding to S3 and the non-key region.

[0195] Then, the regions corresponding to S1 and S2 are fused with the key region, and the region corresponding to S3 is fused with the non-key region.

[0196] In this way, the candidate region can be fused with the key region or the non-key region, thereby realizing the region segmentation of the image to be segmented.

[0197] Finally, the initial boundary information corresponding to each of the key region and the non-key region after region fusion is obtained.

[0198] Step 240: Based on the edge information, the initial boundary information of each region is corrected to obtain the target boundary information of the region where the target object is located.

[0199] In order to avoid the existence of local singular values and misclassification caused by noise, in the embodiment of the present application, step 240 is performed to correct the initial boundary information of each region obtained in step 230 based on the edge information of the image to be segmented, as shown in FIG. 8, and the specific execution is as follows: Figure 14

[0200] Step 2401: Each pixel point included in the image to be segmented is traversed, and for each pixel point, it is determined whether the pixel point meets a preset condition based on the edge information, wherein the preset condition includes that the pixel points on the same side of the boundary information have the same classification label, and the pixel points within the closed boundary information have the same classification label, the boundary information is any sub-edge information included in the edge information, and the classification label of each pixel point is marked after determining the classification information of the region where the pixel point is located.

[0201] Step 2402: If yes, the classification label of the pixel point is saved; otherwise, the classification label corresponding to the pixel point with the largest number in the region where the pixel point is located is replaced by the classification label of the pixel point.​

[0202] For example, still taking the image A to be segmented as an example.

[0203] Suppose that the sub-edge information of any region in the image A to be segmented represents that the boundary is a smooth boundary (such as Figure 10 the boundary line 1 corresponding to the sub-edge information shown), and the initial boundary information of the region in the image A to be segmented obtained is as shown in Figure 15 The initial boundary information represents that there is an obvious bulge boundary line A on the boundary, as shown in Figure 15

[0204] Then, based on the sub-edge information of the image A to be segmented, the initial boundary information of the region is corrected, and the classification label of the pixel point in the bulge is replaced by the classification label corresponding to the pixel point with the largest number in the region where the pixel point is located. The final target boundary information is as shown in Figure 16

[0205] If the classification label of the pixel point in the bulge is a key region, and the classification label corresponding to the pixel point with the largest number in the region where the pixel point is located is replaced from the key region to a non-key region, the classification label of the pixel point in the bulge is replaced by the non-key region; otherwise, if the classification label of the pixel point in the bulge is a non-key region, and the classification label corresponding to the pixel point with the largest number in the region where the pixel point is located is a key region, the classification label of the pixel point in the bulge is replaced from the non-key region to the key region.

[0206] Step 2403: based on the classification labels of the pixels after the traversal is completed, determining the target boundary information of each region included in the image to be segmented.

[0207] Step 2404: taking the target boundary information corresponding to the key region in each region as the target boundary information of the region where the target object is located.

[0208] In the embodiments of the present application, since the key regions set in the image to be segmented according to the preset region configuration rule include part or all of the regions in the region where the target object is located, after the target boundary information corresponding to each region is determined in step 2403, step 2404 is performed to take the target boundary information corresponding to the key region in each region as the target boundary information of the region where the target object is located. In this way, when image enhancement or other image detail optimization is performed on the image to be segmented subsequently, a large amount of calculation of the image enhancement algorithm or other image detail optimization algorithm can be used for the key region (i.e. the internal region of the target boundary information), and a small amount of calculation of the image enhancement algorithm or other image detail optimization algorithm can be used for the regions other than the key region in the image to be segmented, so that the overall calculation amount of processing the image to be segmented is reduced, and the final region classification accuracy is improved, without affecting the image quality of the key region.​​

[0209] In the embodiments of the present application, all feature vectors corresponding to the plurality of key regions can be combined to form a key feature vector group, and all feature vectors corresponding to the plurality of non-key regions can be combined to form a non-key feature vector group. Then, at least one difference threshold is set, each other feature vector corresponding to a candidate region is compared with the key feature vector group and the non-key feature group, so as to obtain target boundary information of each region of the image to be segmented. Different colors are filled for different regions, and the obtained boundary is clear. Since the subsequent image processing is performed in a region-by-region manner, it is not necessary to detect and retain the detailed features in each region. Referring to FIG. 8, a flowchart of a method for segmenting an image according to an embodiment of the present application is shown. Figure 17 Figure 17 A classification result corresponding to the obtained image A to be segmented is shown.

[0210] Based on the same inventive concept, referring to FIG. 8, a method for segmenting an image according to an embodiment of the present application is shown. The method comprises the following steps. Figure 18 According to the embodiments of the present application, a device for segmenting an image key region is provided, which comprises:

[0211] The acquisition module 1810 is configured to acquire edge information of an image to be segmented, wherein the image to be segmented comprises at least one target object.

[0212] The setting module 1820 is configured to set a key region in the image to be segmented according to a preset region configuration rule, wherein the key region comprises at least one key region and at least one non-key region, and the key region comprises part or all of the region where the target object is located.

[0213] The first fusion module 1830 is configured to perform region fusion on the image to be segmented in the key region based on the edge information, and obtain fusion boundary information corresponding to the key region and the non-key region, respectively.

[0214] The second fusion module 1840 is configured to perform region fusion on a candidate region in the image to be segmented except the key region based on the fusion boundary information corresponding to the key region and the non-key region, and obtain initial boundary information of each region included in the image to be segmented.

[0215] The correction module 1850 is configured to correct the initial boundary information of each region based on the edge information, and obtain target boundary information of the region where the target object is located.

[0216] In a possible implementation, before the edge information of the image to be segmented is acquired, the acquisition module 1810 is further configured to:

[0217] ​select at least one candidate image from an image set, and take the candidate image as an original image corresponding to the image to be segmented, wherein the image set contains various candidate images that need to be segmented in a key area of an image; or, obtain video data, extract any frame image from the video data, and take the frame image as an original image corresponding to the image to be segmented;

[0218] Smoothly filter the original image by using a preset filtering mode to obtain the image to be segmented;

[0219] The edge information of the image to be segmented is obtained, and the acquisition module 1810 is configured to:

[0220] An edge detection algorithm is used to perform edge detection on the image to be segmented to obtain the edge information of the image to be segmented.

[0221] In a possible implementation, the key area is set in the image to be segmented according to a preset area configuration rule, and the setting module 1820 is configured to:

[0222] According to statistical data, the key area is set in a center area of the image to be segmented, and the non-key area is set in an edge area of the image to be segmented, wherein the size of each key area is the same, and the size of each non-key area is the same.

[0223] In a possible implementation, the image to be segmented in the key area is fused based on the edge information to obtain fused boundary information corresponding to the key area and the non-key area, respectively, and the first fusion module 1830 is configured to:

[0224] Based on the edge information, feature extraction is performed on the image to be segmented in a reference area to obtain various reference feature vectors corresponding to the reference area, wherein the reference area is the key area or the non-key area;

[0225] For any reference feature vector in the various reference feature vectors, a corresponding similarity is determined based on the any reference feature vector and each other reference feature vector;

[0226] The various reference feature vectors with a similarity greater than a preset value are fused to obtain a corresponding fused feature vector, and the reference area corresponding to the various reference feature vectors with a similarity greater than a preset value is fused to obtain fused boundary information corresponding to the reference area.

[0227] In a possible implementation, the feature extraction is performed on the to-be-segmented image in the reference region based on the edge information, to obtain each reference feature vector corresponding to the reference region, and the first fusion module 1830 is configured to:

[0228] based on any sub-edge information in the edge information, set a weight value of a pixel point on a same side of the sub-edge information as a first weight value, and set a weight value of a pixel point on an opposite side of the sub-edge information as a second weight value, where the first weight value is greater than the second weight value;

[0229] based on the weight value of each pixel point, respectively, pixel features of the to-be-segmented image in the reference region are extracted by using a preset filtering manner, to obtain each reference feature vector corresponding to the reference region.

[0230] In a possible implementation, the region fusion is performed on the candidate region in the to-be-segmented image except the key region based on the fused boundary information corresponding to the key region and the non-key region, to obtain initial boundary information of each region included in the to-be-segmented image, and the second fusion module 1840 is configured to:

[0231] extend the candidate region in the to-be-segmented image except the key region, perform region classification on the extended region based on the fused boundary information corresponding to the key region and the non-key region, to obtain a classification information set corresponding to the candidate region, where the classification information set includes at least one piece of classification information;

[0232] based on the fused boundary information corresponding to the key region and the non-key region and the classification information set, perform feature extraction on the candidate region, to obtain a plurality of other feature vectors;

[0233] traverse the plurality of other feature vectors, and for each other feature vector, determine a first difference value between the other feature vector and each first feature vector, and determine a second difference value between the other feature vector and a second feature vector, where the first feature vector is any fused feature vector corresponding to the key region or any remaining reference feature vector, and the second feature vector is any fused feature vector corresponding to the non-key region or any remaining reference feature vector;

[0234] if the first difference value is less than the second difference value, perform region fusion on regions corresponding to the first feature vector and the other feature vector, respectively; or if the first difference value is greater than the second difference value, perform region fusion on regions corresponding to the second feature vector and the other feature vector, respectively;

[0235] Based on the key areas and non-key areas after regional fusion, the corresponding initial boundary information is determined respectively.

[0236] In one possible implementation, the initial boundary information of each region is corrected based on the edge information to obtain the target boundary information of the region where the target object is located. The correction module 1850 is used for:

[0237] The process iterates through each pixel in the image to be segmented. For each pixel traversed, it determines whether the pixel meets a preset condition based on the edge information. The preset condition includes that pixels on the same side of the same boundary information have the same classification label, and pixels within closed boundary information have the same classification label. The boundary information is any sub-edge information included in the edge information. The classification label of each pixel is marked after the classification information of the region where the pixel is located is determined.

[0238] If yes, then save the category label of the pixel; otherwise, replace the category label of the pixel with the category label corresponding to the pixel with the largest number of pixels in the area where the pixel is located.

[0239] Based on the classification labels of each pixel after traversal, the target boundary information of each region included in the image to be segmented is determined.

[0240] The target boundary information corresponding to the key areas in each of the regions is used as the target boundary information of the region where the target object is located.

[0241] Based on the same inventive concept, this application provides an electronic device, including:

[0242] Memory is used to store computer programs that can be executed by the controller;

[0243] The controller is connected to the memory and is configured to execute any of the methods described in the above embodiments.

[0244] For example, taking the electronic device as a display device, see [reference]. Figure 19 As shown, the display device includes: a memory 191, a processor 192, and a display 193, wherein,

[0245] Memory 191 is used to store computer programs that can be executed by processor 192;

[0246] The processor 192 may include one or more central processing units (CPUs) or digital processing units, etc.

[0247] A display 193 for displaying a picture.

[0248] The specific connection medium between the memory 191, the processor 192 and the display 193 is not limited in the embodiments of the present application. The embodiments of the present application are not limited to the connection mode between the other components, which is only illustrative. Figure 19 In the embodiments of the present application, the memory 191, the processor 192 and the display 193 are connected through the bus 194, and the connection mode between the other components is only illustrative, and is not limited. The bus 194 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 19 In the embodiments of the present application, only one thick line is used to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0249] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium, when the instructions in the storage medium are executed by a processor, the processor can execute any one of the methods in the above embodiments. Since the principle of solving the problem of the above computer readable storage medium is similar to the image key area segmentation method, the implementation of the above computer readable storage medium can be referred to the implementation of the method, and the repeated parts will not be described here.

[0250] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0251] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (system) and computer program product of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram. These computer program instructions can be provided to the processor of a general purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce means for implementing the functions specified in one or more flows in the flowchart and / or one or more blocks in the block diagram.

[0252] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks and / or flowchart flow or flows and / or block or blocks of the block diagrams.

[0253] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks and / or flowchart flow or flows and / or block or blocks of the block diagrams.

[0254] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for segmenting key regions of an image, characterized in that, include: Obtain edge information of an image to be segmented, wherein the image to be segmented includes at least one target object; According to the preset region configuration rules, key regions are set in the image to be segmented, wherein the key regions include at least one important region and at least one non-important region, and the important region includes part or all of the region where the target object is located. Based on the edge information, feature extraction is performed on the image to be segmented within the reference region to obtain each reference feature vector corresponding to the reference region, wherein the reference region is the key region or the non-key region; For any reference feature vector among the various reference feature vectors, a similarity is determined based on the reference feature vector and each other reference feature vector; the regions corresponding to each reference feature vector with a similarity greater than a preset value are fused to obtain the fused boundary information corresponding to the reference region. Based on the fused boundary information corresponding to the key region and the non-key region, region fusion is performed on the candidate regions in the image to be segmented, excluding the key region, to obtain the initial boundary information of each region included in the image to be segmented. Based on the edge information, the initial boundary information of each region is corrected to obtain the target boundary information of the region where the target object is located.

2. The method as described in claim 1, characterized in that, Before obtaining the edge information of the image to be segmented, the following steps are also included: Select at least one candidate image from the image set and use the candidate image as the original image corresponding to the image to be segmented, wherein the image set contains each candidate image that needs to be segmented into key regions of the image; or, acquire video data, extract any frame image from the video data, and use the frame image as the original image corresponding to the image to be segmented. The original image is smoothed using a preset filtering method to obtain the image to be segmented; The step of obtaining the edge information of the image to be segmented includes: The Canny edge detection algorithm is used to perform edge detection on the image to be segmented, thereby obtaining the edge information of the image to be segmented.

3. The method as described in claim 1, characterized in that, The step of setting key regions in the image to be segmented according to preset region configuration rules includes: According to statistical data, the key region is set in the central region of the image to be segmented, and the non-key region is set in the edge region of the image to be segmented, wherein each key region has the same size, and each non-key region has the same size.

4. The method as described in claim 1, characterized in that, Based on the edge information, feature extraction is performed on the image to be segmented within the reference region to obtain various reference feature vectors corresponding to the reference region, including: Based on any sub-edge information in the edge information, for each pixel in the reference area, the weight value of the pixel on the same side of the sub-edge information is set as a first weight value, and the weight value of the pixel on the opposite side of the sub-edge information is set as a second weight value, wherein the first weight value is greater than the second weight value. Based on the weight values ​​of each pixel, a preset filtering method is used to extract the pixel features of the image to be segmented within the reference region, thereby obtaining each reference feature vector corresponding to the reference region.

5. The method as described in claim 1, characterized in that, After determining the similarity between any one of the reference feature vectors and each of the other reference feature vectors, the process further includes: The reference feature vectors with similarity greater than the preset value are fused to obtain the corresponding fused feature vector.

6. The method as described in claim 5, characterized in that, Based on the fused boundary information corresponding to the key region and the non-key region, region fusion is performed on the candidate regions in the image to be segmented, excluding the key region, to obtain the initial boundary information of each region included in the image to be segmented, including: Extend the candidate regions within the image to be segmented, excluding the key regions. Based on the fused boundary information corresponding to the key regions and the non-key regions, classify the extended regions to obtain a set of classification information corresponding to the candidate regions, wherein the set of classification information includes at least one set of classification information. Based on the fused boundary information of the key region and the non-key region and the classification information set, feature extraction is performed on the candidate region to obtain multiple other feature vectors. The process involves iterating through the plurality of other feature vectors. For each of the other feature vectors, a first difference value between the other feature vector and each first feature vector is determined, and a second difference value between the other feature vector and each second feature vector is determined. The first feature vector is any of the fused feature vectors corresponding to the key region or any of the remaining reference feature vectors, and the second feature vector is any of the fused feature vectors corresponding to the non-key region or any of the remaining reference feature vectors. If the first difference value is less than the second difference value, then the regions corresponding to the first feature vector and the other feature vectors are merged; or, if the first difference value is greater than the second difference value, then the regions corresponding to the second feature vector and the other feature vectors are merged. Based on the key areas and non-key areas after regional fusion, the corresponding initial boundary information is determined respectively.

7. The method according to any one of claims 1-6, characterized in that, The step of correcting the initial boundary information of each region based on the edge information to obtain the target boundary information of the region where the target object is located includes: The process iterates through each pixel in the image to be segmented. For each pixel traversed, it determines whether the pixel meets a preset condition based on the edge information. The preset condition includes that pixels on the same side of the same boundary information have the same classification label, and pixels within closed boundary information have the same classification label. The boundary information is any sub-edge information included in the edge information. The classification label of each pixel is marked after the classification information of the region where the pixel is located is determined. If yes, then save the category label of the pixel; otherwise, replace the category label of the pixel with the category label corresponding to the pixel with the largest number of pixels in the area where the pixel is located. Based on the classification labels of each pixel after traversal, the target boundary information of each region included in the image to be segmented is determined. The target boundary information corresponding to the key areas in each of the regions is used as the target boundary information of the region where the target object is located.

8. A device for segmenting key regions of an image, characterized in that, include: An acquisition module is used to acquire edge information of an image to be segmented, wherein the image to be segmented includes at least one target object; The setting module is used to set key regions in the image to be segmented according to preset region configuration rules. The key regions include at least one important region and at least one non-important region. The important region includes part or all of the region where the target object is located. The first fusion module is used to extract features from the image to be segmented within the reference region based on the edge information, to obtain various reference feature vectors corresponding to the reference region, wherein the reference region is the key region or the non-key region; for any reference feature vector among the various reference feature vectors, to determine the corresponding similarity between the any reference feature vector and each other reference feature vector; and to fuse the regions corresponding to the reference feature vectors whose similarity is greater than a preset value to obtain the fused boundary information corresponding to the reference region. The second fusion module is used to perform region fusion on candidate regions in the image to be segmented, excluding the key regions, based on the fused boundary information corresponding to the key regions and the non-key regions, to obtain the initial boundary information of each region included in the image to be segmented. The correction module is used to correct the initial boundary information of each region based on the edge information to obtain the target boundary information of the region where the target object is located.

9. An electronic device, characterized in that, include: Memory is used to store computer programs that can be executed by the controller; The controller is connected to the memory and is configured to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor, the processor is able to perform the method as described in any one of claims 1-7.

Images