Image processing method, device and equipment

Abstract

The embodiment of the present application provides an image processing method, apparatus and device, the method comprising: obtaining foreground information, the foreground information being generated by extracting foreground pixels from an image to be processed; performing parameter calculation based on the foreground information to obtain processing parameters for the foreground information; and performing morphological processing on the foreground information according to the processing parameters. The present application can reduce the probability of losing an object of interest or leaving noise in morphological processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device and equipment. Background Art

[0002] The application of mathematical morphology in image processing has become more and more extensive. For example, by processing images based on mathematical morphology, the region of interest in the image can be obtained. Among them, morphological processing includes erosion processing and dilation processing.

[0003] In the image processing process, the foreground of an image can be subjected to morphological processing. Specifically, the erosion processing can set the foreground pixels at the edge as background pixels, so that the boundary of the foreground shrinks to eliminate noise. The dilation processing can set the background pixels adjacent to the foreground pixels at the edge as foreground pixels, so that the boundary of the foreground expands to fill the holes. Usually, the processing parameters used when morphologically processing the foreground are fixed, that is, all foregrounds are subjected to morphological processing using fixed processing parameters.

[0004] However, the above-mentioned method of fixing processing parameters has the problem that due to inappropriate processing parameters, the morphological processing has a high probability of losing the object of interest or leaving noise. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, apparatus and device to solve the problem of a high probability of missing objects of interest or residual noise in morphological processing in the prior art.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0007] Obtaining foreground information, wherein the foreground information is generated by extracting foreground pixels from the image to be processed;

[0008] Perform parameter calculation based on the foreground information to obtain processing parameters for the foreground information;

[0009] The foreground information is subjected to morphological processing according to the processing parameters.

[0010] In a second aspect, an embodiment of the present application provides an image processing method, including:

[0011] Acquire image data of an image to be processed, wherein the image data is used to indicate a region of interest of the image to be processed; the region of interest is determined based on a final result of morphological processing obtained by the image processing method according to any one of the first aspects;

[0012] Based on the image data, the region of interest of the image to be processed is displayed.

[0013] In a third aspect, an embodiment of the present application provides an image processing device, including:

[0014] An acquisition module, used for acquiring foreground information, wherein the foreground information is generated by extracting foreground pixels from the image to be processed;

[0015] An obtaining module, used for performing parameter calculation based on the foreground information to obtain processing parameters for the foreground information;

[0016] A processing module is used to perform morphological processing on the foreground information according to the processing parameters.

[0017] In a fourth aspect, an embodiment of the present application provides an image processing device, including:

[0018] An acquisition module, used for acquiring image data of an image to be processed, wherein the image data is used for indicating a region of interest of the image to be processed; the region of interest is determined based on a final result of morphological processing obtained by the image processing method according to any one of the first aspects;

[0019] A display module is used to display the region of interest of the image to be processed based on the image data.

[0020] In a fifth aspect, an embodiment of the present application provides a computer device, comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method described in any one of the first aspects.

[0021] In a sixth aspect, an embodiment of the present application provides a computer device, comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method described in the second aspect.

[0022] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes at least one section of code, and the at least one section of code can be executed by a computer to control the computer to execute the method as described in any one of the first aspects.

[0023] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes at least one section of code, and the at least one section of code can be executed by a computer to control the computer to execute a method as described in any one of the second aspects.

[0024] An embodiment of the present application further provides a computer program, which, when executed by a computer, is used to implement the method as described in any one of the first aspects.

[0025] An embodiment of the present application further provides a computer program, which, when executed by a computer, is used to implement the method as described in any one of the second aspects.

[0026] The image processing method, apparatus and device provided in the embodiments of the present application obtain foreground information, perform parameter calculation based on the foreground information to obtain processing parameters for the foreground information, and perform morphological processing on the foreground information according to the processing parameters, thereby achieving the calculation of morphological processing parameters for the foreground information. Since the morphological processing parameters can determine the processing degree of morphological processing, the processing degree of morphological processing of foreground information can be adaptive to different foreground information, thereby reducing the probability of excessive or insufficient processing degree, thereby reducing the probability of losing objects of interest or leaving noise in morphological processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 A schematic diagram of an application scenario of an embodiment of the present application;

[0029] Figure 2A A schematic diagram of foreground information provided by an embodiment of the present application;

[0030] Figure 2B For Figure 2A A schematic diagram of the foreground information after being eroded;

[0031] Figure 2C For Figure 2A The schematic diagram of the result after the foreground information shown is expanded;

[0032] Figure 3A A schematic diagram of an image to be processed provided in an embodiment of the present application;

[0033] Figure 3B For Figure 3A Schematic diagram of the result after the foreground information is subjected to excessive corrosion treatment;

[0034] Figure 3C For Figure 3ASchematic diagram of the result after the foreground information is corroded with too little intensity;

[0035] Figure 4 A flowchart of an image processing method provided in one embodiment of the present application;

[0036] Figure 5A A schematic diagram of foreground information provided by an embodiment of the present application;

[0037] Figure 5B Based on Figure 5A Schematic diagram of the connected domain generated by the foreground information shown;

[0038] Figure 6 A principle block diagram of the image processing method provided in the embodiment of the present application;

[0039] Figure 7 is the processing parameter pair obtained based on parameter calculation Figure 3A Schematic diagram of the result after morphological processing of foreground information;

[0040] Figure 8 A flowchart of an image processing method provided by another embodiment of the present application;

[0041] Fig. 9 A schematic diagram of the structure of an image processing device provided in one embodiment of the present application;

[0042] Fig.10 A schematic diagram of the structure of a computer device provided in one embodiment of the present application;

[0043] Fig.11 A schematic diagram of the structure of an image processing device provided by another embodiment of the present application;

[0044] Fig.12 A schematic diagram of the structure of a computer device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0047] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0048] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0049] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a product or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a product or system. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the product or system including the elements.

[0050] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0051] In order to facilitate those skilled in the art to understand the technical solution provided by the embodiments of the present application, the technical environment in which the technical solution is implemented is described below.

[0052] In the related art, the processing parameters used when morphologically processing the foreground are fixed, that is, fixed processing parameters are used for morphological processing of all foregrounds. There is a problem that due to inappropriate processing parameters, the morphological processing has a high probability of losing objects of interest or leaving noise. Therefore, there is an urgent need in the related art for an image processing method that can reduce the probability of losing objects of interest or leaving noise in morphological processing.

[0053] Based on actual technical requirements similar to those described above, the image processing method provided in this application can use technical means to reduce the probability of losing objects of interest or leaving noise in morphological processing.

[0054] The image processing methods provided in various embodiments of the present application are described in detail below through an exemplary application scenario.

[0055] Figure 1 Schematic diagram of application scenarios of the image processing method provided in the embodiment of the present application. Figure 1 As shown, the application scenario may include an image acquisition device 11, an image processing device 12 and an image display device 13. The image acquisition device 11 may be used to acquire an image to be processed, and the image to be processed may be, for example, a video frame in a video frame sequence.

[0056] After the image acquisition device 11 acquires the image to be processed, the image processing device 12 can extract foreground pixels in the image to be processed by background modeling to generate foreground information. Of course, in other embodiments, foreground pixels can also be extracted by other methods to generate foreground information, and the present application does not limit this. Among them, the foreground information can be used to distinguish foreground pixels and non-foreground pixels (i.e., background pixels) in at least a part of the area or the whole area of ​​the image to be processed. In one embodiment, the foreground information may include a foreground mask, and the pixel value of a pixel in the foreground mask may be 0 or 1, 1 represents a foreground pixel, and 0 represents a background pixel. Including the foreground mask in the foreground information is conducive to simplifying the calculation. It should be noted that the image acquisition device 11 and the image processing device 12 can be integrated into the same device, or can be located in different devices respectively.

[0057] After generating the foreground information of the image to be processed, the image processing device 12 may further use the method provided in the embodiment of the present application to perform morphological processing on the foreground information. It should be noted that: Figure 1 In the example, the image processing device 12 completes both the generation of foreground information and the morphological processing of the foreground information. It can be understood that in other embodiments, different devices can also be used to complete the generation of foreground pixels and the morphological processing of foreground information, and the different devices can be integrated into the same device or can be located in different devices.

[0058] Among them, morphological processing can include corrosion processing, expansion processing, opening processing and closing processing, among which corrosion processing and expansion processing are the most basic processing. The process of corrosion first and then expansion is called opening processing, and the process of expansion first and then corrosion is called closing processing.

[0059] Foreground information such as Figure 2A As shown in the figure, the updated foreground information obtained after the foreground information is eroded can be as follows Figure 2B As shown in FIG. 1 , the updated foreground information obtained after the foreground information is expanded can be as follows: Figure 2C shown. Figure 2A-2C A small box in the image can represent a pixel, a small box filled with gray can represent a foreground pixel, and a small box filled with white can represent a background pixel. Figure 2A and Figure 2B It can be seen that by corroding the foreground information, the foreground boundary can be shrunk. Figure 2A and Figure 2C It can be seen that by dilating the foreground information, the boundary of the foreground can be expanded.

[0060] Specifically, for a certain foreground pixel in the foreground information, if one or more of its adjacent pixels are background pixels, the erosion process can set the foreground pixel to a background pixel. Similarly, for a certain background pixel in the foreground information, if one or more of its adjacent pixels are foreground pixels, the dilation process can set the background pixel to a foreground pixel. From the above definition, it can be seen that the degree of erosion and dilation processing is of great significance to the effect of morphological processing, wherein the degree of processing can be determined by intensity and number of times.

[0061] Among them, the corrosion process can use a template window to process one or more pixels of the foreground information. For example, the corrosion process can align the center position of a 3×3 square window with the pixel (i.e., the current pixel) in the foreground information that is currently to be processed. If n1 (n1 is greater than 0 and less than (window size-1)) or more adjacent pixels of the current pixel in the square window are background pixels, then n2 (n2 is greater than 0 and less than the window size) pixels in the square window can be set as background pixels. When n2 is equal to 1, the current pixel can be set as a background pixel. When n2 is greater than 1, other foreground pixels other than the current pixel can also be set as background pixels. Similarly, the corrosion process can also use a template window of 5×5, 7×7, 9×9, or other sizes or shapes. Among them, adjacent can be, for example, 4-adjacent and 8-adjacent. If the coordinates of the current pixel are (x, y), then: the 4-adjacent adjacent pixels are pixels with coordinates of (x+d, y) or (x, y+d), d=1 or -1; the 8-adjacent adjacent pixels are pixels with coordinates of (x+d, y), (x, y+d) or (x+d, y+d), d=1 or -1.

[0062] Assuming that the specific method of corrosion processing 1 is for a 3×3 square window, if one or more adjacent pixels of the current pixel in the 3×3 square window are background pixels, the current pixel is set as a background pixel, and the specific method of corrosion processing 2 is for a 3×3 square window, if two or more adjacent pixels of the current pixel in the 3×3 square window are background pixels, the current pixel is set as a background pixel, then the intensity of corrosion processing 1 is greater than the intensity of corrosion processing 2, that is, corrosion processing 1 can set more foreground pixels as background pixels than corrosion processing 2. It can be seen that the smaller n1 is, the greater the intensity of the corrosion processing can be. It should be noted that for template windows of 5×5, 7×7, 9×9, or other sizes or shapes, there is a similar corrosion processing intensity relationship.

[0063] Assuming that the specific method of corrosion processing 3 is for a 3×3 square window, if one or more adjacent pixels of the current pixel in the 3×3 square window are background pixels, only the current pixel is set as a background pixel, and the specific method of corrosion processing 4 is for a 3×3 square window, if one or more adjacent pixels of the current pixel in the 3×3 square window are background pixels, all foreground pixels in the 3×3 square window are set as background pixels, then the intensity of corrosion processing 4 is greater than that of corrosion processing 3, that is, corrosion processing 4 can set more foreground pixels as background pixels than corrosion processing 3. It can be seen that the larger n2 is, the greater the intensity of the corrosion processing can be. It should be noted that for template windows of 5×5, 7×7, 9×9, or other sizes or shapes, there is also a similar corrosion processing intensity relationship.

[0064] It should be noted that, since the corrosion process is specifically processed based on the pixel conditions within the template window used, in addition to the aforementioned n1 and n2, the size of the template window used in the corrosion process can also affect the intensity of the corrosion process.

[0065] The dilation process can use a template window to process one or more pixels of the foreground information. For example, the dilation process can align the center position of a 3×3 square window with the pixel (i.e., the current pixel) in the foreground information that is currently to be processed. If n3 (n3 is greater than 0 and less than (window size-1)) or more adjacent pixels of the current pixel in the square window are foreground pixels, then n4 (n4 is greater than 0 and less than the window size) pixels in the square window can be set as foreground pixels. When n4 is equal to 1, the current pixel can be set as a foreground pixel. When n4 is greater than 1, other background pixels other than the current pixel can also be set as foreground pixels. Similarly, the dilation process can also use a template window of 5×5, 7×7, 9×9, or other sizes or shapes.

[0066] Assuming that the specific method of dilation processing 1 is that for a 3×3 square window, if one or more neighboring pixels of the current pixel in the 3×3 square window are foreground pixels, the current pixel is set as a foreground pixel, and the specific method of dilation processing 2 is that for a 3×3 square window, if two or more neighboring pixels of the current pixel in the 3×3 square window are foreground pixels, the current pixel is set as a foreground pixel, then the intensity of dilation processing 1 is greater than the intensity of dilation processing 2, that is, dilation processing 1 can set more background pixels as foreground pixels than dilation processing 2. It can be seen that the smaller n3 is, the greater the intensity of the dilation processing can be. It should be noted that for template windows of 5×5, 7×7, 9×9, or other sizes or shapes, there is also a similar dilation processing intensity relationship.

[0067] Assuming that the specific method of dilation processing 3 is that for a 3×3 square window, if one or more neighboring pixels of the current pixel in the 3×3 square window are foreground pixels, only the current pixel is set as a foreground pixel, and the specific method of dilation processing 4 is that for a 3×3 square window, if one or more neighboring pixels of the current pixel in the 3×3 square window are foreground pixels, all background pixels in the 3×3 square window are set as foreground pixels, then the intensity of dilation processing 4 is greater than that of dilation processing 3, that is, dilation processing 4 can set more background pixels as foreground pixels than dilation processing 3. It can be seen that the larger n4 is, the greater the intensity of the dilation processing can be. It should be noted that for template windows of 5×5, 7×7, 9×9, or other sizes or shapes, there is also a similar dilation processing intensity relationship.

[0068] It should be noted that, since the expansion process is specifically processed based on the pixel conditions in the template window used, in addition to the aforementioned n3 and n4, the size of the template window used in the expansion process can also affect the intensity of the expansion process.

[0069] refer to Figure 3A and Figure 3B , the corrosion processing with too strong intensity (for example, when one or more adjacent pixels are background pixels, the current pixel is set as the background pixel) can effectively filter the noise, but it will also lose some small-sized objects of interest, resulting in missed detection of objects of interest. Figure 3A is the image to be processed, Figure 3A The three objects framed by the middle rectangle are objects of interest. Figure 3B The two white areas in the figure are the two objects of interest after the corrosion process with too strong intensity. Figure 3A and Figure 3B It can be seen that the object of interest may be lost after the corrosion process is too strong. It can be understood that the number of corrosion processes can also affect the degree of corrosion, and the greater the number of corrosion processes, the greater the degree of corrosion can be.

[0070] On the contrary, refer to Figure 3A and Figure 3C , corrosion processing with too low intensity (for example, when two or more adjacent pixels are background pixels, the current pixel is set as a background pixel) can retain small-sized objects of interest, but it will also leave noise and cause false detection of objects of interest. Figure 3A is the image to be processed, Figure 3A The three objects framed by the middle rectangle are objects of interest. Figure 3C The five white areas in the figure are the five objects of interest after being processed with less intensive corrosion. Figure 3A and Figure 3C It can be seen that the corrosion process with too low intensity will leave noise, that is, the noise will be mistaken for the object of interest (i.e. Figure 3C The two areas are framed by the middle rectangle).

[0071] Similarly, a dilation process with too high a strength can fill in the holes in the object contour, but it will also expand the edge of the noise, resulting in residual noise and misdetection of the object of interest. A dilation process with too low a strength can reduce the expansion of the noise edge, but it will also reduce the contour filling effect of the object of interest, resulting in partial missing or even missing of the object of interest. It can be understood that the number of dilation processes can also affect the degree of dilation. The more dilation times, the greater the degree of dilation.

[0072] From this, it can be seen that the method of using fixed processing parameters to perform morphological processing on foreground information may result in the fixed processing parameters being too large or too small for the foreground information, resulting in a higher probability of losing objects of interest or leaving noise in the morphological processing.

[0073] The image processing method provided by the embodiment of the present application obtains foreground information, performs parameter calculation based on the foreground information to obtain processing parameters for the foreground information, and performs morphological processing on the foreground information according to the processing parameters, thereby achieving the calculation of morphological processing parameters for the foreground information. Since the morphological processing parameters can determine the processing degree of morphological processing, the processing degree of morphological processing on the foreground information can be adaptive to different foreground information, thereby reducing the probability of excessive or insufficient processing degree, thereby reducing the probability of losing objects of interest or leaving noise in morphological processing.

[0074] In one embodiment, the final result obtained by the image processing device 12 performing morphological processing on the foreground information can be used to determine the region of interest of the image to be processed. Optionally, the region of interest of the image to be processed can be determined by the image processing device 12, or the region of interest can be determined by other devices other than the image processing device 12. It can be understood that the information of the region of interest can be included in the image data of the image to be processed. Of course, in other embodiments, the final result obtained by performing morphological processing on the foreground information can also be used to determine other information, which is not limited in this application.

[0075] After determining the region of interest of the image to be processed, the image display device 13 can display the region of interest of the image to be processed based on the image data of the image to be processed. It should be noted that the image processing device 12 and the image display device 13 can be integrated into the same device, or can be located in different devices.

[0076] It should be noted that the method provided in the embodiment of the present application can be applied to any device that can obtain the image to be processed or the foreground information of the image to be processed, and the device can use the method to process the image. Taking the image acquisition device 11, the image processing device 12 and the image display device 13 integrated into one device as an example, the device can be, for example, a camera, a video camera, a personal computer with a camera device, a laptop computer, a mobile phone, a television, etc. Of course, in other embodiments, the device for executing the image processing method can also be other types of devices, and the present application does not limit this.

[0077] It should be noted that the method provided in the embodiments of the present application can be applied to any scenario that requires morphological processing of foreground information, such as intelligent detection, target detection, target tracking, face recognition, media processing, video on demand, video painting, film production, animation production and other application scenarios. The fields involved include but are not limited to security, monitoring, inspection or online streaming video.

[0078] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0079] Figure 4 This is a flow chart of an image processing method provided by an embodiment of the present application. The execution subject of this embodiment may be Figure 1 The image processing device 12 in FIG. Figure 4 As shown, the method of this embodiment may include:

[0080] Step 41, obtaining foreground information, wherein the foreground information is generated by extracting foreground pixels from the image to be processed;

[0081] Step 42, performing parameter calculation based on the foreground information to obtain processing parameters for the foreground information;

[0082] Step 43: Perform morphological processing on the foreground information according to the processing parameters.

[0083] In one embodiment, obtaining foreground information may specifically include: extracting foreground pixels from the image to be processed to generate foreground information of the image to be processed. In another embodiment, obtaining foreground information may specifically include: receiving foreground information sent by other devices, and the foreground information may be generated by the other devices by extracting foreground pixels from the image to be processed. In yet another embodiment, obtaining foreground information may specifically include: reading foreground information from a storage device, and the foreground information may be generated by a local device or other device.

[0084] It is understandable that the foreground of an image is relative to the background of the image, and the meaning of foreground pixels may be different in different scenarios. Foreground pixels may refer to pixels of suspected objects of interest in a certain scenario. Whether the suspected objects of interest are real objects of interest or noise requires further processing. For example, in a target detection scenario, foreground pixels may refer to pixels of suspected moving targets. For another example, in a face recognition scenario, foreground pixels may refer to pixels of suspected faces. Of course, in other embodiments, foreground pixels may also have other meanings, which are not limited in this application.

[0085] In the embodiment of the present application, after obtaining the foreground information, parameter calculation can be performed based on the foreground information to obtain processing parameters for the foreground information, and the processing parameters are used to perform morphological processing on the foreground information. Since the processing parameters determine the processing degree of morphological processing on the foreground information, and the processing parameters are calculated based on the foreground information parameters, the processing parameters used for morphological processing of the foreground information can be adaptive to the specific situation of the suspected object of interest in the foreground information, reducing the probability that the processing degree is too large or too small for the foreground information, thereby reducing the probability of losing the object of interest or leaving noise in the morphological processing.

[0086] Exemplarily, considering that the proportion of foreground pixels in the foreground information can reflect the specific situation of the suspected object of interest in the foreground information, the parameter calculation can be performed based on the proportion of foreground pixels in the foreground information. Based on this, step 42 can specifically include: performing a foreground pixel proportion analysis on the foreground information to obtain a proportion result; and, based on the relationship between the proportion result and the proportion threshold, obtaining a first processing parameter for the foreground information. The proportion result can refer to the ratio of the number of foreground pixels in the foreground information to the number of all pixels in the foreground pixels.

[0087] Among them, if the proportion result is less than the proportion threshold, it can be indicated that the proportion of foreground pixels in the foreground information is small, and the possible situation is that the size of the suspected object of interest (i.e., the number of foreground pixels) is relatively small, so the first processing parameter that can better retain the object of interest can be calculated. Conversely, if the proportion result is greater than the proportion threshold, it can be indicated that the proportion of foreground pixels in the foreground information is large, and the possible situation is that the size of the suspected object of interest is relatively large, so the first processing parameter that can better eliminate noise can be calculated.

[0088] Optionally, the number of percentage thresholds may be one or two.

[0089] When the number of proportion thresholds is one, a proportion result less than or equal to the proportion threshold may indicate that the proportion of foreground pixels in the foreground information is small, and a proportion result greater than the proportion threshold may indicate that the proportion of foreground pixels in the foreground information is large.

[0090] When the number of proportion thresholds is two, the two proportion thresholds may be recorded as a first proportion threshold and a second proportion threshold, respectively. Assuming that the first proportion threshold is less than the second proportion threshold, if the proportion result is less than the first proportion threshold, it may indicate that the proportion of foreground pixels in the foreground information is large; if the proportion result is greater than the second proportion threshold, it may indicate that the proportion of foreground pixels in the foreground information is small; if the proportion result is greater than the first proportion threshold and less than the second proportion threshold, it may indicate that the proportion of foreground pixels in the foreground information is average.

[0091] In practical applications, the first processing parameter can be flexibly obtained according to demand to achieve self-adaptation of the first processing parameter.

[0092] In one embodiment, two groups of processing parameters can be pre-set, one group of processing parameters can better retain the object of interest, and the other group of processing parameters can better eliminate noise. Based on this, the first processing parameter for the foreground information is obtained based on the size relationship between the proportion result and the proportion threshold, which can specifically include: when the proportion result is less than or equal to the proportion threshold, adaptively selecting a group of processing parameters that can better retain the object of interest as the first processing parameter; when the proportion result is greater than the proportion threshold, adaptively selecting a group of processing parameters that can better eliminate noise as the first processing parameter. It can be understood that in other embodiments, the number of proportion thresholds can be multiple, and multiple proportion thresholds can divide more than 2 proportion intervals, so that more than two groups of processing parameters can be further pre-set to further improve adaptability.

[0093] In another embodiment, obtaining the first processing parameter for the foreground information based on the size relationship between the proportion result and the proportion threshold may specifically include: determining a first parameter strategy that matches the size relationship based on the size relationship between the proportion result and the proportion threshold; and, based on preset processing parameters, adopting the first parameter strategy to obtain the first processing parameter for the foreground information.

[0094] In the case where the morphological processing includes corrosion processing, since the corrosion processing is used to shrink the boundary, the smaller the degree of corrosion processing, the smaller the degree of boundary shrinkage, and the more conducive to retaining the object of interest. Therefore, when the proportion result is less than the proportion threshold (for example, the proportion result is less than the first proportion threshold), in order to better retain the object of interest, the first parameter strategy includes: a strategy to reduce the degree of corrosion processing.

[0095] Reducing the degree of corrosion processing may include reducing the intensity and / or number of corrosion. For example, the intensity of corrosion may be reduced by increasing n1, reducing n2, or reducing the size of the template window. It should be noted that the specific contents of n1, n2, and the template window can be referred to the above-mentioned related description, which will not be repeated here.

[0096] In practical applications, the method of reducing the degree of corrosion treatment can be flexibly implemented according to needs. Optionally, the degree of reduction of the degree of corrosion treatment can be fixed. For example, assuming that n1 and n2 in the preset processing parameters are both equal to 2, n1 can be fixedly increased by 1, 2+1 to obtain 3, and n2 can be fixedly reduced by 1, 2-1 to obtain 1.

[0097] Alternatively, optionally, the degree of reduction of the corrosion processing degree may be positively correlated with the degree of difference between the proportion threshold and the proportion result. The degree of difference may be, for example, the absolute value of the difference between the proportion threshold and the proportion result. By making the degree of reduction of the corrosion processing degree positively correlated with the degree of difference, the smaller the proportion result is than the proportion threshold, the smaller the boundary shrinkage degree can be, which is more conducive to retaining the object of interest. It can be understood that the degree of reduction of the corrosion processing degree can be understood as a kind of degree of change of the corrosion processing degree.

[0098] When the morphological processing includes corrosion processing, since the dilation processing is used to expand the boundary, the greater the degree of the dilation processing, the greater the degree of boundary expansion, and the more conducive to retaining the object of interest. Therefore, when the proportion result is less than the proportion threshold (for example, the proportion result is less than the first proportion threshold), in order to better retain the object of interest, the first parameter strategy includes: a strategy of increasing the degree of dilation processing.

[0099] Increasing the expansion processing degree may include increasing the intensity and / or number of expansions. For example, the expansion intensity may be increased by reducing n3, increasing n4, or increasing the size of the template window. It should be noted that for the specific contents of n3, n4, and the template window, please refer to the above-mentioned related description, which will not be repeated here.

[0100] In practical applications, the method of increasing the expansion degree can be flexibly implemented according to needs. Optionally, the increase degree of the expansion degree can be fixed. For example, assuming that n3 and n4 in the preset processing parameters are both equal to 2, n3 can be fixedly reduced by 1, 2-1 to obtain 1, and n4 can be fixedly increased by 1, 2+1 to obtain 3.

[0101] Alternatively, optionally, the increase in the expansion degree may be positively correlated with the difference between the proportion threshold and the proportion result. The difference may be, for example, the absolute value of the difference between the proportion threshold and the proportion result. By making the increase in the expansion degree positively correlated with the difference, the smaller the proportion result is than the proportion threshold, the greater the boundary expansion degree can be, which is more conducive to retaining the object of interest. It can be understood that the increase in the expansion degree can be understood as a change in the expansion degree.

[0102] When the morphological processing includes corrosion processing, since the corrosion processing is used to shrink the boundary, the greater the degree of corrosion processing, the greater the degree of boundary shrinkage, and the more conducive to eliminating noise. Therefore, when the proportion result is greater than the proportion threshold (for example, the proportion result is greater than the second proportion threshold), in order to better eliminate noise, the first parameter strategy includes: a strategy of increasing the degree of corrosion processing.

[0103] Increasing the degree of corrosion processing may include increasing the intensity and / or number of corrosion. For example, the intensity of corrosion may be increased by reducing n1, increasing n2, or increasing the size of the template window. It should be noted that the specific contents of n1, n2, and the template window can be referred to the above-mentioned related description, which will not be repeated here.

[0104] In practical applications, the method of increasing the degree of corrosion treatment can be flexibly implemented according to needs. Optionally, the degree of increase in the degree of corrosion treatment can be fixed. For example, assuming that n1 and n2 in the preset processing parameters are both equal to 2, n1 can be fixedly reduced by 1, 2-1 to obtain 1, and n2 can be fixedly increased by 1, 2+1 to obtain 3.

[0105] Alternatively, optionally, the increase in the degree of the corrosion treatment may be positively correlated with the difference between the proportion threshold and the proportion result. The difference may be, for example, the absolute value of the difference between the proportion threshold and the proportion result. By making the decrease in the degree of corrosion treatment positively correlated with the difference, the greater the proportion result is than the proportion threshold, the greater the boundary shrinkage can be, which is more conducive to eliminating noise. It can be understood that the increase in the degree of corrosion treatment can be understood as another degree of change in the degree of corrosion treatment.

[0106] When the morphological processing includes corrosion processing, since the dilation processing is used to expand the boundary, the smaller the degree of dilation processing, the smaller the degree of boundary expansion, and the more conducive to eliminating noise. Therefore, when the proportion result is greater than the proportion threshold (for example, the proportion result is greater than the second proportion threshold), in order to better eliminate noise, the first parameter strategy includes: a strategy to reduce the degree of dilation processing.

[0107] Reducing the expansion processing degree may include reducing the intensity and / or number of expansions. For example, the intensity of the expansion may be reduced by increasing n3, decreasing n4, or increasing the size of the template window. It should be noted that for the specific contents of n3, n4, and the template window, please refer to the above-mentioned related description, which will not be repeated here.

[0108] In practical applications, the method of reducing the expansion processing degree can be flexibly implemented according to needs. Optionally, the reduction degree of the expansion processing degree can be fixed. For example, assuming that n3 and n4 in the preset processing parameters are both equal to 2, n3 can be fixedly increased by 1, 2+1 to obtain 3, and n4 can be fixedly reduced by 1, 2-1 to obtain 1.

[0109] Alternatively, optionally, the degree of reduction of the expansion processing degree may be positively correlated with the degree of difference between the proportion threshold and the proportion result. The degree of difference may be, for example, the absolute value of the difference between the proportion threshold and the proportion result. By making the degree of reduction of the expansion processing degree positively correlated with the degree of difference, the greater the proportion result is than the proportion threshold, the smaller the boundary expansion degree can be, which is more conducive to eliminating noise. It can be understood that the degree of reduction of the expansion processing degree can be understood as another degree of change of the expansion processing degree.

[0110] It should be noted that a combination of corrosion processing with different intensities, expansion processing with different intensities, corrosion processing with different times, and expansion processing with different times can be used flexibly to obtain the first processing parameters.

[0111] It can be understood that, for general situations other than the large and small proportions of foreground pixels in the foreground information, for example, the proportion result is greater than the first proportion threshold and less than the second proportion threshold, the preset processing parameters can be directly used as the first processing parameters, that is, there is no need to adjust the preset processing parameters, and the preset processing parameters can be directly used to perform morphological processing on the foreground information.

[0112] In the embodiment of the present application, the proportion threshold may be a fixed threshold, or the proportion threshold may be set according to the sum of the sizes of the objects of interest that may appear in a single picture in the shooting scene of the image to be processed. By setting the proportion threshold according to the sum of the sizes of the objects of interest that may appear in a single picture, the setting of the proportion threshold can be adapted to the size characteristics of the objects of interest that may appear in the picture, which is conducive to improving adaptability.

[0113] Exemplarily, considering that the connected domains in the updated foreground information after morphological processing is performed on the foreground information, the specific conditions of the suspected objects of interest in the foreground information can be reflected to a certain extent, therefore, the parameter calculation can be performed based on the updated foreground information after a round of morphological processing. Based on this, step 42 can specifically include: determining the number of connected domains in the updated foreground information obtained after a round of morphological processing is performed on the foreground information; determining whether the number is greater than a first number threshold; and if the number of connected domains in the updated foreground information is greater than a first number threshold, determining a second processing parameter for the foreground information based on the difference between the number and the first number threshold, the second processing parameter being used to perform another round of morphological processing on the foreground information.

[0114] In one embodiment, if the number of connected domains in the updated foreground information is less than a first number threshold, the updated foreground information obtained by performing the most recent round of morphological processing on the foreground information may be used as the final result of performing morphological processing on the foreground information.

[0115] The connected domain of the updated foreground information can be formed by connecting adjacent foreground pixels in the updated foreground information. The connected domain can be realized, for example, by a FloodFill algorithm. Of course, in other embodiments, the connected domain can also be obtained by other algorithms, which is not limited in this application.

[0116] Taking the Floodfill algorithm as an example, we first obtain the color or intensity of the current pixel in the foreground information (which can be 0 or 1 for the foreground mask), then obtain all pixels with the same color or intensity in all 4-adjacent or 8-adjacent areas of the current pixel, and connect these pixels to the current pixel. Figure 5A As shown in the figure, the connected domain generated by the Floodfill algorithm can be Figure 5B shown. Figure 5A middle Figure 5B A small box in the image can represent a pixel. Figure 5A The 0 and 1 in are pixel values. A pixel value of 0 represents a background pixel, and a pixel value of 1 represents a foreground pixel. Figure 5B The area formed by the small squares filled with medium gray can represent the connected domain.

[0117] Optionally, the number of connected domains in the updated foreground information may include the total number of connected domains in the updated foreground information. Based on this, the number of connected domains in the updated foreground information can, to a certain extent, represent the total number of suspected objects of interest in the updated foreground information. If the number of connected domains in the updated foreground information is greater than the first number threshold, it may indicate that the total number of suspected objects of interest in the updated foreground information is too large, and a new round of morphological processing is required for the updated foreground information. If the number of connected domains in the updated foreground information is less than the first number threshold, it may indicate that the total number of suspected objects of interest in the updated foreground information is not too large, and a new round of morphological processing is not required for the updated foreground information, that is, the updated foreground information can be used as the final result obtained by morphological processing of the foreground information.

[0118] Alternatively, optionally, the number of connected domains in the updated foreground information may include the number of connected domains in the updated foreground information whose number of pixels is less than or equal to the second number threshold. Considering that the suspected objects of interest corresponding to the connected domains with larger sizes (number of foreground pixels) are usually real objects of interest, and the sizes of the connected domains composed of noise pixels are usually smaller, in order to prevent the retention of a large number of smaller noises in a targeted manner, the connected domains may be screened based on the second number threshold.

[0119] Based on this, the number of connected domains in the updated foreground information can, to a certain extent, represent the number of suspected objects of interest with smaller sizes in the updated foreground information. If the number of connected domains in the updated foreground information is greater than the first number threshold, it can be indicated that the total number of suspected objects of interest with smaller sizes in the updated foreground information is too large, and a new round of morphological processing needs to be performed on the updated foreground information. If the number of connected domains in the updated foreground information is less than the first number threshold, it can be indicated that the total number of suspected objects of interest with smaller sizes in the updated foreground information is not too large, and a new round of morphological processing does not need to be performed on the updated foreground information, that is, the updated foreground information can be used as the final result obtained by performing morphological processing on the foreground information.

[0120] Optionally, the first quantity threshold may be a fixed threshold; or the first quantity threshold may be set according to the number of objects of interest that may appear in a single picture in the shooting scene of the image to be processed. The first quantity threshold is obtained according to the number of objects of interest that may appear in a single picture, so that the setting of the first quantity threshold can adapt to the number characteristics of the objects of interest that may appear in the picture, which is conducive to improving adaptability.

[0121] Optionally, multiple first quantity thresholds can also be set for the same shooting scene. For example, for the same shooting scene, if the number of objects of interest that may appear in a single picture in different time periods is different, multiple first quantity thresholds can correspond to multiple time periods so that the first quantity thresholds can better adapt to changes in image content.

[0122] Optionally, the second quantity threshold may be a fixed threshold; or the second quantity threshold is set based on the size of the object of interest that may appear in the shooting scene of the image to be processed. Exemplarily, it can be set based on the size of the smaller object of interest that may appear. The second quantity threshold is obtained based on the size of the object of interest that may appear in a single picture, so that the setting of the second quantity threshold can adapt to the size characteristics of the object of interest that may appear in the picture, which is conducive to improving adaptability.

[0123] If the number of connected domains in the updated foreground information is greater than the first number threshold, it may be that too much noise is contained, resulting in too many connected domains, or it may be that the contour of the object of interest is divided into multiple parts and is not connected together. Therefore, the noise is eliminated by corroding the shrinking boundary, and the different parts of the same object of interest are connected by dilating the expanding boundary, both of which can reduce the number of connected domains.

[0124] In practical applications, the second processing parameter can be flexibly obtained according to demand to achieve self-adaptation of the second processing parameter.

[0125] In one embodiment, two groups of processing parameters may be pre-set, and the two groups of processing parameters have different degrees of shrinking and / or expanding boundaries. Based on this, the second processing parameter for the foreground information is determined based on the difference between the number of connected domains in the updated foreground information and the first quantity threshold, which may specifically include: when the difference is less than or equal to the first quantity threshold, a group of processing parameters with a smaller degree of adaptive selection is used as the second processing parameter; when the difference is greater than the first quantity threshold, a group of processing parameters with a larger degree of adaptive selection is used as the second processing parameter. It can be understood that in other embodiments, the number of first quantity thresholds may be multiple, and the multiple first quantity thresholds may be divided into a number interval greater than 2, so that more than two groups of processing parameters may be further pre-set to further improve adaptability.

[0126] In another embodiment, the determining of the second processing parameter for the foreground information based on the difference between the quantity and the first quantity threshold may specifically include: determining a second parameter strategy that matches the difference based on the difference between the quantity and the first quantity threshold; and obtaining the second processing parameter for the foreground information by adopting the second parameter strategy based on the processing parameters used for the most recent round of morphological processing of the foreground information.

[0127] Optionally, a first difference threshold and a second difference threshold can be used to divide three difference intervals, and the first difference threshold is less than the second difference threshold. In the case where the difference between the number of connected domains in the updated foreground information and the first number threshold is less than the first difference threshold, it can be indicated that the number of connected domains in the updated foreground information obtained by the most recent round of morphological processing that needs to be reduced is small, and continuing to use the processing parameters used for the most recent round of morphological processing of the foreground information may cause the problem of excessive processing degree. Based on this, the second parameter strategy includes: a strategy for reducing the degree of corrosion processing, and / or a strategy for reducing the degree of expansion processing. It should be noted that the specific methods for reducing the degree of corrosion processing and reducing the degree of expansion processing can be referred to the aforementioned related description, which will not be repeated here.

[0128] In the case where the difference between the number of connected domains in the updated foreground information and the first number threshold is greater than the second difference threshold, it can be indicated that the number of connected domains in the updated foreground information obtained by the most recent round of morphological processing needs to be reduced more, and the use of the processing parameters used for the most recent round of morphological processing of the foreground information may cause the problem of too low a processing degree. Based on this, the second parameter strategy includes: a strategy for increasing the degree of corrosion processing, and / or a strategy for increasing the degree of expansion processing. It should be noted that the specific methods for increasing the degree of corrosion processing and increasing the degree of expansion processing can be referred to the aforementioned related description, which will not be repeated here.

[0129] When the difference between the number of connected domains in the updated foreground information and the first number threshold is greater than the first difference threshold and less than the second difference threshold, it can be indicated that the number of connected domains in the updated foreground information obtained by the most recent round of morphological processing needs to be reduced, and the degree of processing of the processing parameters used in the most recent round of morphological processing of the foreground information will not be too high or too low. Based on this, the second parameter strategy includes: a strategy of maintaining the degree of corrosion processing, and / or a strategy of maintaining the degree of expansion processing.

[0130] It should be noted that a combination of corrosion processing with different intensities, expansion processing with different intensities, corrosion processing with different times, and expansion processing with different times can be used flexibly to obtain the second processing parameters.

[0131] In one embodiment, the maximum number of rounds of morphological processing for the foreground information can be limited. Exemplarily, the foreground information can be limited to a maximum of two rounds of morphological processing, that is, the aforementioned determination of the number of connected domains can be performed only on the updated foreground information obtained by the first round of morphological processing. Based on this, the determination of the number of connected domains in the updated foreground information obtained after one round of morphological processing on the foreground information can specifically include: determining the number of connected domains in the updated foreground information obtained after the first round of morphological processing on the foreground information. In this case, the updated foreground information obtained after the first or second round of morphological processing on the foreground information is the final result obtained by morphological processing on the foreground information.

[0132] In another embodiment, the maximum number of rounds of morphological processing for the foreground information may not be limited, that is, the aforementioned number of connected domains may be determined for the updated foreground information obtained after each round of morphological processing. Based on this, the number of connected domains in the updated foreground information obtained after a round of morphological processing for the foreground information is determined, which may specifically include: for the updated foreground information obtained after each round of morphological processing for the foreground information, the number of connected domains in the updated foreground information is determined. This ensures that the number of connected domains in the final result obtained by morphological processing for the foreground information is not greater than the first number threshold.

[0133] It is understandable that one or more erosion and / or dilation processes may be performed in each round of morphological processing. Performing a second round of morphological processing on the foreground information refers to further morphological processing on the updated foreground information obtained by the first round of morphological processing, performing a third round of morphological processing on the foreground information refers to further morphological processing on the updated foreground information obtained by the second round of morphological processing, and so on.

[0134] It should be noted that the aforementioned method of calculating processing parameters based on the proportion of foreground pixels and the aforementioned method of calculating processing parameters based on the number of connected domains can be used alone or in combination.

[0135] Take the combination of the two methods as an example, Figure 6As shown, the image processing module 12 may include a first parameter determination module 61, an erosion module 62, an expansion module 63, a decision module 64, and a second parameter determination module 65. First, the first parameter calculation module 61 may perform a ratio analysis of foreground pixels on the foreground information a to obtain a ratio result, and based on the size relationship between the ratio result and the ratio threshold, obtain a first processing parameter for the foreground information a, and the first processing parameter may include a processing parameter 1 for erosion processing and a processing parameter 2 for expansion processing. Then, the erosion module 62 may first perform an erosion processing on the foreground information a according to the processing parameter 1, and then the expansion module 63 may further expand the erosion processing result of the erosion processing performed by the erosion module 62 according to the processing parameter 2, thereby implementing a first round of morphological processing on the foreground information a and obtaining the updated foreground information a1. Afterwards, the decision module 64 may perform a decision processing based on the number of connected domains in the updated foreground information a1. If the number of connected domains in the updated foreground information a1 is not greater than the first number threshold, it may be decided to output the updated foreground information a1 as the final result of the morphological processing performed on the foreground information a. If the number of connected domains in the updated foreground information a1 is greater than the first number threshold, the updated foreground information a1 may be passed to the second parameter determination module 65, and the second parameter determination module 65 determines the second processing parameter for the foreground information a based on the difference between the number of connected domains in the updated foreground information a1 and the first number threshold, and the second processing parameter may include a processing parameter 3 for corrosion processing and a processing parameter 4 for expansion processing. Further, the corrosion module 62 may first perform corrosion processing on the updated foreground information a1 according to the processing parameter 3, and then the expansion module 63 may further expand the corrosion processing result of the corrosion processing by the corrosion module 62 according to the processing parameter 4, so as to implement a second round of morphological processing on the foreground information a and obtain the updated foreground information a2. Similarly, the decision module 64 may decide whether to use the updated foreground information a2 as the final result or to pass the updated foreground information a2 to the second parameter determination module to determine the second processing parameter for the third round of morphological processing on the foreground information a based on the updated foreground information a2.

[0136] It should be noted that Figure 6 The various modules shown in the figure may specifically be software function modules.

[0137] It should be noted that Figure 6 In the example, in each round of morphological processing, the corrosion processing is performed first and then the dilation processing is performed. It can be understood that in one or more rounds of morphological processing, only the corrosion processing or the dilation processing can be performed, and the order of the corrosion processing and the dilation processing in the same round of morphological processing can also be interchanged.

[0138] Using the image processing method provided in the embodiment of the present application, Figure 3A The final result after processing the foreground information of the image to be processed can be as shown in Figure 7 shown. Figure 7 The three white areas are three objects of interest identified after morphological processing by the method provided in the embodiment of the present application. Figure 3A , Figure 3B and Figure 7 It can be seen that the final result of morphological processing by the method provided in the embodiment of the present application can retain the object of interest that is mistakenly regarded as noise by the corrosion processing with too strong intensity. Figure 3A , Figure 3C and Figure 7 It can be seen that the final result of morphological processing by the method provided in the embodiment of the present application can eliminate the noise that is mistakenly regarded as an object of interest by the corrosion processing with too low intensity.

[0139] In the embodiment of the present application, optionally, after the foreground information is morphologically processed to obtain a final result, the region of interest of the image to be processed can also be determined based on the final result obtained by morphologically processing the foreground information. It should be noted that the specific method of determining the region of interest based on the final result obtained by morphologically processing the foreground information is not limited in the present application.

[0140] The experimental comparison results of the object of interest recognition of video frames in multiple videos based on the morphological processing with fixed processing parameters in the traditional technology and the morphological processing with parameter calculation processing parameters provided by the embodiment of the present application are as follows: the average accuracy of the object of interest recognition based on the morphological processing with fixed processing parameters in the traditional technology is 64.07%, and the average recall rate is 82.95%; the average accuracy of the object of interest recognition based on the morphological processing with parameter calculation processing parameters provided by the embodiment of the present application is 86.81%, and the average recall rate is 96.90%. It can be seen that the morphological processing with parameter calculation processing parameters provided by the embodiment of the present application can reduce the probability of missing objects of interest and residual noise in morphological processing compared with the morphological processing based on fixed processing parameters in the traditional technology.

[0141] The image processing method provided by the embodiment of the present application obtains foreground information, performs parameter calculation based on the foreground information to obtain processing parameters for the foreground information, and performs morphological processing on the foreground information according to the processing parameters, thereby achieving the calculation of morphological processing parameters for the foreground information. Since the morphological processing parameters can determine the processing degree of morphological processing, the processing degree of morphological processing on the foreground information can be adaptive to different foreground information, thereby reducing the probability of excessive or insufficient processing degree, thereby reducing the probability of losing objects of interest or leaving noise in morphological processing.

[0142] Figure 8 This is a flow chart of an image processing method provided by another embodiment of the present application. The execution subject of this embodiment may be Figure 1 The image display device 13 in FIG. Figure 8 As shown, the method of this embodiment may include:

[0143] Step 81, obtaining image data of the image to be processed, wherein the image data is used to indicate the region of interest of the image to be processed; the region of interest is based on Figure 4 The final result of the morphological processing obtained by the image processing method shown is determined;

[0144] Step 82: display the region of interest of the image to be processed based on the image data.

[0145] It should be noted that for details on the morphological processing of the foreground information of the processed image, please refer to Figure 4 The detailed description of the illustrated embodiment will not be repeated here.

[0146] The image processing method provided in the embodiment of the present application obtains image data of the image to be processed, and displays the region of interest of the image to be processed based on the image data. Figure 4 The final result of the morphological processing obtained by the image processing method shown is determined, so the probability of missing an object of interest or leaving noise in the displayed content can be reduced.

[0147] Fig. 9 FIG. 1 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application; Fig. 9 As shown, this embodiment provides an image processing device, which can perform the above Figure 4 The image processing method shown, specifically, the image processing device may include:

[0148] An acquisition module 91 is used to obtain foreground information, wherein the foreground information is generated by extracting foreground pixels from the image to be processed;

[0149] An obtaining module 92, configured to perform parameter calculation based on the foreground information to obtain processing parameters for the foreground information;

[0150] The processing module 93 is used to perform morphological processing on the foreground information according to the processing parameters.

[0151] Optionally, the obtaining module 92 is specifically used to perform a foreground pixel ratio analysis on the foreground information to obtain a ratio result; and, based on a size relationship between the ratio result and a ratio threshold, obtain a first processing parameter for the foreground information.

[0152] Optionally, the obtaining module 92 is used to obtain a first processing parameter for the foreground information based on the size relationship between the proportion result and the proportion threshold, specifically including: based on the size relationship between the proportion result and the proportion threshold, determining a first parameter strategy that matches the size relationship; and, based on preset processing parameters, adopting the first parameter strategy to obtain the first processing parameter for the foreground information.

[0153] Optionally, the proportion result is less than a proportion threshold; the first parameter strategy includes: a strategy for reducing the degree of corrosion treatment, and / or a strategy for increasing the degree of expansion treatment.

[0154] Optionally, the proportion result is greater than a proportion threshold; the first parameter strategy includes: a strategy for increasing the degree of corrosion treatment, and / or a strategy for reducing the degree of expansion treatment.

[0155] Optionally, the degree of change in the corrosion treatment degree is positively correlated with the degree of difference between the proportion threshold and the proportion result.

[0156] Optionally, the degree of change in the expansion processing degree is positively correlated with the degree of difference between the proportion threshold and the proportion result.

[0157] Optionally, the proportion threshold is set according to the sum of sizes of objects of interest that may appear in a single picture in the shooting scene of the image to be processed.

[0158] Optionally, the obtaining module 92 is specifically used to determine the number of connected domains in the updated foreground information obtained after a round of morphological processing is performed on the foreground information; determine whether the number is greater than a first number threshold; and, if so, determine a second processing parameter for the foreground information based on the difference between the number and the first number threshold, the second processing parameter being used to perform another round of morphological processing on the foreground information.

[0159] Optionally, the number includes the number of connected domains in which the number of pixels in the updated foreground information is less than or equal to a second number threshold.

[0160] Optionally, the second quantity threshold is set according to the size of an object of interest that may appear in a shooting scene of the image to be processed.

[0161] Optionally, the first quantity threshold is set according to the number of objects of interest that may appear in a single picture in the shooting scene of the image to be processed.

[0162] Optionally, the obtaining module 92 is used to determine a second processing parameter for the foreground information based on the difference between the quantity and the first quantity threshold, specifically including: determining a second parameter strategy that matches the difference based on the difference between the quantity and the first quantity threshold; and, based on the processing parameters used for the most recent round of morphological processing of the foreground information, adopting the second parameter strategy to obtain the second processing parameter for the foreground information.

[0163] Optionally, the difference is greater than a first difference threshold and less than a second difference threshold; the second parameter strategy includes: a strategy for maintaining the degree of corrosion treatment, and / or a strategy for maintaining the degree of expansion treatment.

[0164] Optionally, the difference is less than a first difference threshold; the second parameter strategy includes: a strategy for reducing the degree of corrosion treatment, and / or a strategy for reducing the degree of expansion treatment.

[0165] Optionally, the difference is greater than a second difference threshold; the second parameter strategy includes: a strategy for increasing the degree of corrosion treatment, and / or a strategy for increasing the degree of expansion treatment.

[0166] Optionally, reducing the extent of corrosion treatment includes reducing the intensity and / or frequency of corrosion.

[0167] Optionally, increasing the degree of expansion treatment includes increasing the intensity and / or frequency of expansion.

[0168] Optionally, increasing the degree of corrosion treatment includes increasing the intensity and / or frequency of corrosion.

[0169] Optionally, reducing the degree of expansion treatment includes reducing the intensity and / or frequency of expansion.

[0170] Optionally, the obtaining module 92 is further configured to, if the number of connected domains in the updated foreground information is less than a first quantity threshold, use the updated foreground information obtained by performing the most recent round of morphological processing on the foreground information as the final result of performing morphological processing on the foreground information.

[0171] Optionally, the processing module 93 is further configured to determine a region of interest of the image to be processed based on a final result obtained by performing morphological processing on the foreground information.

[0172] Fig. 9 The device shown can perform Figure 4 For the method of the embodiment shown in the figure, the part not described in detail in this embodiment can be referred to Figure 4 The implementation process and technical effects of this technical solution refer to Figure 4 The description in the illustrated embodiment will not be repeated here.

[0173] In one possible implementation, Fig. 9 The structure of the image processing device shown can be implemented as a computer device. Fig.10 As shown, the computer device may include: a processor 101 and a memory 102. The memory 102 is used to store information that supports the computer device to execute the above Figure 4 The program of the image processing method provided in the illustrated embodiment, the processor 101 is configured to execute the program stored in the memory 102 .

[0174] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 101, the following steps can be implemented:

[0175] Obtaining foreground information, wherein the foreground information is generated by extracting foreground pixels from the image to be processed;

[0176] Perform parameter calculation based on the foreground information to obtain processing parameters for the foreground information;

[0177] The foreground information is subjected to morphological processing according to the processing parameters.

[0178] Optionally, the processor 101 is further configured to execute the aforementioned Figure 4 All or part of the steps in the illustrated embodiments.

[0179] The structure of the computer device may further include a communication interface 103 for the computer device to communicate with other devices or a communication network.

[0180] Fig.11 FIG. 1 is a structural diagram of an image processing device provided in another embodiment of the present application; Fig.11 As shown, this embodiment provides an image processing device, which can perform the above Figure 8 The image processing method shown, specifically, the image processing device may include:

[0181] The acquisition module 111 is used to acquire image data of the image to be processed, wherein the image data is used to indicate the region of interest of the image to be processed; the region of interest is based on Figure 4 The final result of the morphological processing obtained by the image processing method shown is determined;

[0182] The display module 112 is configured to display the region of interest of the image to be processed based on the image data.

[0183] Fig.11 The device shown can perform Figure 8 For the method of the embodiment shown in the figure, the part not described in detail in this embodiment can be referred to Figure 8The implementation process and technical effects of this technical solution refer to Figure 8 The description in the illustrated embodiment will not be repeated here.

[0184] In one possible implementation, Fig.11 The structure of the image processing device shown can be implemented as a computer device. Fig.12 As shown, the computer device may include: a processor 121 and a memory 122. The memory 122 is used to store information that supports the computer device to execute the above Figure 8 The program of the image processing method provided in the illustrated embodiment, the processor 121 is configured to execute the program stored in the memory 122 .

[0185] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 121, the following steps can be implemented:

[0186] Acquire image data of an image to be processed, wherein the image data is used to indicate a region of interest of the image to be processed; the region of interest is based on Figure 4 The final result of the morphological processing obtained by the image processing method shown is determined;

[0187] Based on the image data, the region of interest of the image to be processed is displayed.

[0188] Optionally, the processor 121 is further configured to execute the aforementioned Figure 8 All or part of the steps in the illustrated embodiments.

[0189] The structure of the computer device may further include a communication interface 123 for the computer device to communicate with other devices or a communication network.

[0190] In addition, the present application embodiment provides a computer storage medium for storing computer software instructions used by the terminal, which includes instructions for executing the above Figure 4 The procedures involved in the illustrated method embodiment.

[0191] The present application embodiment provides a computer storage medium for storing computer software instructions used by a terminal, which includes instructions for executing the above Figure 8 The procedures involved in the illustrated method embodiment.

[0192] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0193] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0194] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0195] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0196] These computer program instructions may also be loaded onto a computer or other programmable device so that a series of processing steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide the instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0197] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0198] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0199] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0200] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, It is characterized in that include: Obtaining foreground information, wherein the foreground information is generated by extracting foreground pixels from the image to be processed; Parameter calculation is performed based on the foreground information to obtain processing parameters for the foreground information, the parameter calculation includes determining the proportion of foreground pixels and / or the number of connected domains in the foreground pixels, and based on the relationship between the proportion and / or the number and the corresponding threshold, the processing parameters are obtained by adopting a strategy corresponding to the relationship, the strategy including: reducing the degree of corrosion processing and / or increasing the degree of expansion processing when the proportion is less than the proportion threshold, increasing the degree of corrosion processing and / or reducing the degree of expansion processing when the proportion is greater than the proportion threshold, increasing the degree of corrosion processing and / or the degree of expansion processing when the difference between the number and the number threshold is greater than the difference threshold, and / or reducing the degree of corrosion processing and / or the degree of expansion processing when the difference between the number and the number threshold is less than the difference threshold; The foreground information is subjected to morphological processing according to the processing parameters.

2. The method according to claim 1, It is characterized in that The performing parameter calculation based on the foreground information to obtain processing parameters for the foreground information includes: Performing a foreground pixel ratio analysis on the foreground information to obtain a ratio result; Based on the magnitude relationship between the proportion result and the proportion threshold, a first processing parameter for the foreground information is obtained.

3. The method according to claim 2, It is characterized in that The obtaining of a first processing parameter for the foreground information based on the size relationship between the proportion result and the proportion threshold includes: Based on the size relationship between the proportion result and the proportion threshold, determining a first parameter strategy that matches the size relationship; Based on the preset processing parameters, the first parameter strategy is adopted to obtain the first processing parameters for the foreground information.

4. The method according to claim 3, It is characterized in that The proportion result is less than the proportion threshold; the first parameter strategy includes: a strategy to reduce the degree of corrosion treatment, and / or a strategy to increase the degree of expansion treatment.

5. The method according to claim 3, It is characterized in that The proportion result is greater than the proportion threshold; the first parameter strategy includes: a strategy for increasing the degree of corrosion treatment, and / or a strategy for reducing the degree of expansion treatment.

6. The method according to claim 4 or 5, It is characterized in that The degree of change in the corrosion treatment degree is positively correlated with the degree of difference between the proportion threshold and the proportion result.

7. The method according to claim 4 or 5, It is characterized in that The degree of change in the expansion processing degree is positively correlated with the degree of difference between the proportion threshold and the proportion result.

8. The method according to claim 2, It is characterized in that The percentage threshold is set according to the sum of sizes of objects of interest that may appear in a single picture in the shooting scene of the image to be processed.

9. The method according to claim 1, It is characterized in that The performing parameter calculation based on the foreground information to obtain processing parameters for the foreground information includes: for updated foreground information obtained after performing a round of morphological processing on the foreground information, determining the number of connected domains in the updated foreground information; determining whether the quantity is greater than a first quantity threshold; If yes, then based on the difference between the number and the first number threshold, a second processing parameter for the foreground information is determined, and the second processing parameter is used to perform another round of morphological processing on the foreground information.

10. The method according to claim 9, It is characterized in that The number includes the number of connected domains in which the number of pixels in the updated foreground information is less than or equal to a second number threshold.

11. The method according to claim 10, It is characterized in that The second quantity threshold is set according to the size of the object of interest that may appear in the shooting scene of the image to be processed.

12. The method according to claim 10, It is characterized in that The first quantity threshold is set according to the number of objects of interest that may appear in a single picture in the shooting scene of the image to be processed.

13. The method according to claim 10, It is characterized in that The determining, based on the difference between the quantity and the first quantity threshold, a second processing parameter for the foreground information comprises: Based on a difference between the quantity and the first quantity threshold, determining a second parameter strategy matching the difference; Based on the processing parameters used in the most recent round of morphological processing on the foreground information, the second parameter strategy is adopted to obtain the second processing parameters for the foreground information.

14. The method according to claim 13, It is characterized in that The difference is greater than a first difference threshold and less than a second difference threshold; the second parameter strategy includes: a strategy for maintaining a corrosion treatment degree, and / or a strategy for maintaining a dilation treatment degree.

15. The method according to claim 13, It is characterized in that The difference is less than a first difference threshold; the second parameter strategy includes: a strategy for reducing the degree of corrosion treatment, and / or a strategy for reducing the degree of expansion treatment.

16. The method according to claim 13, It is characterized in that The difference is greater than a second difference threshold; the second parameter strategy includes: a strategy for increasing the degree of corrosion treatment, and / or a strategy for increasing the degree of expansion treatment.

17. The method according to claim 4 or 15, It is characterized in that Reducing the extent of the corrosion treatment includes reducing the intensity and / or frequency of corrosion.

18. The method according to claim 4 or 16, It is characterized in that Increasing the extent of the expansion treatment includes increasing the intensity and / or frequency of the expansion.

19. The method according to claim 5 or 16, It is characterized in that Increasing the degree of etching treatment includes increasing the intensity and / or the number of etchings.

20. The method according to claim 5 or 15, It is characterized in that Reducing the extent of the expansion treatment includes reducing the intensity and / or the number of expansions.

21. The method according to claim 9, It is characterized in that The method further comprises: If the number of connected domains in the updated foreground information is less than the first number threshold, the updated foreground information obtained by performing the most recent round of morphological processing on the foreground information is the final result of performing the morphological processing on the foreground information.

22. The method according to claim 1, It is characterized in that The method further comprises: Based on a final result obtained by morphologically processing the foreground information, a region of interest of the image to be processed is determined.

23. The method according to claim 1, It is characterized in that The foreground information includes a foreground mask.

24. An image processing method, It is characterized in that include: Acquire image data of an image to be processed, wherein the image data is used to indicate a region of interest of the image to be processed; The region of interest is determined based on the final result of morphological processing obtained by the image processing method according to any one of claims 1 to 23; Based on the image data, the region of interest of the image to be processed is displayed.

25. An image processing device, It is characterized in that include: An acquisition module, used for acquiring foreground information, wherein the foreground information is generated by extracting foreground pixels from the image to be processed; A obtaining module, used for performing parameter calculation based on the foreground information to obtain processing parameters for the foreground information, wherein the parameter calculation includes determining the proportion of foreground pixels and / or the number of connected domains in the foreground pixels, and based on the relationship between the proportion and / or the number and the corresponding threshold, obtaining the processing parameters by adopting a strategy corresponding to the relationship, wherein the strategy includes: reducing the degree of corrosion processing and / or increasing the degree of expansion processing when the proportion is less than the proportion threshold, increasing the degree of corrosion processing and / or reducing the degree of expansion processing when the proportion is greater than the proportion threshold, increasing the degree of corrosion processing and / or the degree of expansion processing when the difference between the number and the number threshold is greater than the difference threshold, and / or reducing the degree of corrosion processing and / or the degree of expansion processing when the difference between the number and the number threshold is less than the difference threshold; A processing module is used to perform morphological processing on the foreground information according to the processing parameters.

26. An image processing device, It is characterized in that include: An acquisition module, used for acquiring image data of an image to be processed, wherein the image data is used for indicating a region of interest of the image to be processed; The region of interest is determined based on the final result of morphological processing obtained by the image processing method according to any one of claims 1 to 23; A display module is used to display the region of interest of the image to be processed based on the image data.

27. A computer device, It is characterized in that include: A memory, a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as described in any one of claims 1 to 23.

28. A computer device, It is characterized in that include: A memory, a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the method as claimed in claim 24.

Images