Image boundary determination method, device and storage medium

By dividing the image of the earth and rock material pile into image blocks and using the compensation convolution kernel and filtered convolution kernel to process the block classification matrix, the problem of inaccurate image boundary detection under complex operating conditions in the existing technology is solved, and the earth and rock material pile is realized Accurate detection of image boundaries.

CN113139980BActive Publication Date: 2024-12-03GUANGXI LIUGONG MASCH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202110521199.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-05-13
Publication Date
2024-12-03
Estimated Expiration
2041-05-13

AI Technical Summary

Technical Problem

The existing image boundary detection methods cannot accurately determine the boundaries of the earth and stone material pile images under complex operating conditions, especially due to the large number of material categories and close coloring, the detection is inaccurate.

Method used

By dividing the target image into multiple image blocks, extracting the image texture features of the image blocks, building a block classification matrix, and using compensation convolution kernel and filter convolution kernel to convolution process the block classification matrix, obtaining the compensation filter matrix to determine The boundary of the image.

Benefits of technology

The accuracy of fitting the image boundary of earth and rock material piles is improved, and the boundaries of complex images can be accurately determined.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113139980B_ABST
    Figure CN113139980B_ABST
Patent Text Reader

Abstract

The present application discloses a method, device and storage medium for determining image boundaries, which relates to the field of image processing technology and can detect the boundaries of earth and stone material pile images. The method includes: dividing the target image into image blocks, extracting the image texture features of the image blocks and determining the block classification results of the image blocks according to the image texture features, and determining the block classification matrix according to the block classification results; classifying the target image to obtain the classification results corresponding to the target image; performing convolution processing on the block classification matrix based on the compensation convolution kernel and the filter convolution kernel according to the classification results corresponding to the target image to obtain the compensation filter matrix; determining at least one boundary of the target image based on the elements in the compensation filter matrix. The compensation filter matrix is ​​used to correct the elements in the block classification matrix; the elements in the block classification matrix represent the block classification results of the image blocks at the corresponding positions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular, to a method, device and storage medium for determining an image boundary. Background Art

[0002] Currently, when determining the boundary of an image, a boundary detection method based on image features, a boundary detection method based on a model, or a boundary detection method based on deep learning is often used. The above methods can all be used to determine the boundary of the image to be detected.

[0003] However, for the image of earth and stone material pile under complex working conditions, due to the large number of material categories and the close coloring of the materials, the above method cannot accurately fit the boundary of the image of earth and stone material pile, that is, the existing boundary detection method cannot accurately determine the boundary of the image of earth and stone material pile. Therefore, it is urgent to propose a method for determining the image boundary to detect the boundary of the image of earth and stone material pile. Summary of the invention

[0004] The present application provides a method, device and storage medium for determining an image boundary, which can detect the boundary of an earth and stone material pile image.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In the first aspect, the present application provides a method for determining an image boundary, including: dividing a target image into image blocks, extracting image texture features of the image blocks and determining block classification results of the image blocks according to the image texture features, and determining a block classification matrix according to the block classification results; classifying the target image to obtain a classification result corresponding to the target image; performing convolution processing on the block classification matrix based on a compensation convolution kernel and a filter convolution kernel according to the classification result corresponding to the target image to obtain a compensation filter matrix; determining at least one boundary of the target image based on elements in the compensation filter matrix. The compensation filter matrix is ​​used to correct elements in the block classification matrix; and the elements in the block classification matrix represent the block classification results of the image blocks at corresponding positions.

[0007] In the method for determining the image boundary provided by the present application, the target image is divided into multiple image blocks, the block classification results of each image block are determined, and then the block classification matrix corresponding to the block classification results is determined. Afterwards, in order to improve the accuracy of the corresponding target image boundary fitting, based on the classification result of the target image, the block classification matrix is ​​convolved with a compensating convolution kernel and a filtering convolution kernel to obtain a compensation filtering matrix after the block classification matrix is ​​corrected, and then the boundary of the target image is fitted based on the elements in the compensating filtering matrix. It can be seen that the present application can accurately determine the boundary of the target image by dividing the target image into image blocks and optimizing the block classification matrix.

[0008] Optionally, in a possible design, the above “convolving the block classification matrix based on the compensation convolution kernel and the filter convolution kernel according to the classification result corresponding to the target image to obtain the compensation filter matrix” may include:

[0009] Determine the non-screened elements in the block classification matrix; the block classification results corresponding to the non-screened elements are different from the classification results corresponding to the target image;

[0010] Set the element values ​​of non-screened elements to zero to obtain a screening matrix;

[0011] Performing two-dimensional convolution processing on the screening matrix using a compensating convolution kernel to obtain a first intermediate matrix;

[0012] Performing compensation processing on the first intermediate matrix based on the compensation threshold to obtain a compensation matrix;

[0013] Performing two-dimensional convolution processing on the compensation matrix using a filter convolution kernel to obtain a second intermediate matrix;

[0014] The second intermediate matrix is ​​filtered based on the filtering threshold to obtain a compensation filtering matrix.

[0015] Optionally, in another possible design, the above “performing compensation processing on the first intermediate matrix based on the compensation threshold to obtain a compensation matrix” may include:

[0016] Determine, in the first intermediate matrix, elements whose element values ​​are greater than or equal to the compensation threshold as elements to be compensated;

[0017] The element value of the element to be compensated is set to the element value of the target element in the first intermediate matrix to obtain a compensation matrix; the block classification result corresponding to the target element is the same as the classification result corresponding to the target image.

[0018] Optionally, in another possible design, the above “filtering the second intermediate matrix based on the filtering threshold to obtain a compensation filtering matrix” may include:

[0019] Determine the elements in the second intermediate matrix whose element values ​​are less than the filtering threshold as elements to be filtered;

[0020] The element values ​​of the elements to be filtered are set to zero to obtain a compensation filter matrix.

[0021] Optionally, in another possible design, the above “determining at least one boundary of the target image based on the elements in the compensation filter matrix” may include:

[0022] Determine edge image blocks in the target image according to block classification results of image blocks corresponding to elements in the compensation filter matrix;

[0023] At least one boundary is determined based on the position of the edge image block.

[0024] Optionally, in another possible design, the above “determining at least one boundary based on the position of the edge image block” may include:

[0025] Determine the center point of the edge image block according to the position of the edge image block;

[0026] The center point of the edge image block is determined as a boundary fitting point, and at least one boundary is determined.

[0027] Optionally, in another possible design, the method for determining an image boundary provided by the present application may further include:

[0028] According to the sample image and the image block division results of the sample image, a block sub-model is obtained by training;

[0029] According to the segmentation result and the image texture features of the image blocks of the sample image, a texture feature extraction sub-model is trained;

[0030] According to the image texture features of the image blocks of the sample images, the block classification results of the image blocks of the sample images, and the block classification matrix of the image blocks of the sample images, a training sub-model is obtained by training;

[0031] Based on the block sub-model, the texture feature extraction sub-model and the training sub-model, a block classification model is constructed; the block classification model is used to: when a target image input is detected, output a block classification matrix of the target image.

[0032] In a second aspect, the present application provides a device for determining an image boundary, comprising: a determination module, a classification module, and a processing module;

[0033] Specifically, the determination module is used to divide the target image into image blocks, extract image texture features of the image blocks and determine the block classification results of the image blocks according to the image texture features, and determine the block classification matrix according to the block classification results; the elements in the block classification matrix represent the block classification results of the image blocks at corresponding positions;

[0034] A classification module is used to classify the target image and obtain the classification result corresponding to the target image;

[0035] A processing module, used for performing convolution processing on the block classification matrix based on the compensation convolution kernel and the filter convolution kernel according to the classification result corresponding to the target image, so as to obtain a compensation filter matrix;

[0036] The determination module is further used to determine at least one boundary of the target image based on the elements in the compensation filter matrix.

[0037] Optionally, in a possible design, the processing module is specifically used for:

[0038] Determine the non-screened elements in the block classification matrix; the block classification results corresponding to the non-screened elements are different from the classification results corresponding to the target image;

[0039] Set the element values ​​of non-screened elements to zero to obtain a screening matrix;

[0040] Performing two-dimensional convolution processing on the screening matrix using a compensating convolution kernel to obtain a first intermediate matrix;

[0041] Performing compensation processing on the first intermediate matrix based on the compensation threshold to obtain a compensation matrix;

[0042] Performing two-dimensional convolution processing on the compensation matrix using a filter convolution kernel to obtain a second intermediate matrix;

[0043] The second intermediate matrix is ​​filtered based on the filtering threshold to obtain a compensation filtering matrix.

[0044] Optionally, in another possible design, the processing module is further used for:

[0045] Determine, in the first intermediate matrix, elements whose element values ​​are greater than or equal to the compensation threshold as elements to be compensated;

[0046] The element value of the element to be compensated is set to the element value of the target element in the first intermediate matrix to obtain a compensation matrix; the block classification result corresponding to the target element is the same as the classification result corresponding to the target image.

[0047] Optionally, in another possible design, the processing module is further used for:

[0048] Determine the elements in the second intermediate matrix whose element values ​​are less than the filtering threshold as elements to be filtered;

[0049] The element values ​​of the elements to be filtered are set to zero to obtain a compensation filter matrix.

[0050] Optionally, in another possible design, the determination module is specifically used for:

[0051] Determine edge image blocks in the target image according to block classification results of image blocks corresponding to elements in the compensation filter matrix;

[0052] At least one boundary is determined based on the position of the edge image block.

[0053] Optionally, in another possible design, the determination module is further used to:

[0054] Determine the center point of the edge image block according to the position of the edge image block;

[0055] The center point of the edge image block is determined as a boundary fitting point, and at least one boundary is determined.

[0056] Optionally, in another possible design, the image boundary determination device provided by the present application may further include:

[0057] A training module, used for training and obtaining a block sub-model according to the sample image and the image block division result of the sample image;

[0058] The training module is further used to train a texture feature extraction sub-model according to the segmentation result and the image texture features of the image blocks of the sample image;

[0059] The training module is further used to train and obtain a training sub-model according to the image texture features of the image blocks of the sample images, the block classification results of the image blocks of the sample images, and the block classification matrix of the image blocks of the sample images;

[0060] The building module is used to build a block classification model based on the block sub-model, the texture feature extraction sub-model and the training sub-model; the block classification model is used to: when the target image input is detected, output the block classification matrix of the target image.

[0061] In a third aspect, the present application provides an image boundary determination device, comprising a memory, a processor, a bus and a communication interface; the memory is used to store computer execution instructions, and the processor is connected to the memory through a bus; when the image boundary determination device is running, the processor executes the computer execution instructions stored in the memory, so that the image boundary determination device performs the image boundary determination method provided in the first aspect above.

[0062] Optionally, the apparatus for determining the image boundary may further include a transceiver, which is used to execute the steps of sending and receiving data, signaling or information under the control of the processor of the apparatus for determining the image boundary, for example, to acquire a target image.

[0063] Further optionally, the image boundary determination device may be a physical machine for implementing the image boundary determination, or may be a part of the device in the physical machine, for example, may be a chip system in the physical machine. The chip system is used to support the image boundary determination device to implement the functions involved in the first aspect, for example, receiving, sending or processing the data and / or information involved in the above-mentioned image boundary determination method. The chip system includes a chip and may also include other discrete devices or circuit structures.

[0064] In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When a computer executes the instructions, the computer executes the method for determining an image boundary as provided in the first aspect.

[0065] In a fifth aspect, the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed on a computer, the computer executes the method for determining an image boundary as provided in the first aspect.

[0066] It should be noted that the above computer instructions may be stored in whole or in part on a computer-readable storage medium. The computer-readable storage medium may be packaged together with the processor of the image boundary determination device, or may be packaged separately from the processor of the image boundary determination device, and this application does not limit this.

[0067] The description of the second, third, fourth and fifth aspects in this application can refer to the detailed description of the first aspect; and the beneficial effects of the description of the second, third, fourth and fifth aspects can refer to the beneficial effect analysis of the first aspect, which will not be repeated here.

[0068] In this application, the name of the above-mentioned image boundary determination device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear with other names. As long as the functions of each device or functional module are similar to those of this application, they belong to the scope of the claims of this application and their equivalent technologies.

[0069] These and other aspects of the present application will become more apparent from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 An earth and stone material pile image provided in an embodiment of the present application;

[0071] Figure 2 A schematic diagram of a flow chart of a method for determining an image boundary provided in an embodiment of the present application;

[0072] Figure 3 A block classification matrix provided in an embodiment of the present application;

[0073] Figure 4 A screening matrix provided for an embodiment of the present application;

[0074] Figure 5 A first intermediate matrix provided in an embodiment of the present application;

[0075] Figure 6 A compensation matrix provided in an embodiment of the present application;

[0076] Figure 7 A second intermediate matrix provided in an embodiment of the present application;

[0077] Figure 8 A compensation filter matrix provided in an embodiment of the present application;

[0078] Fig. 9 A schematic diagram of a boundary line of a target image provided in an embodiment of the present application;

[0079] Fig.10 A schematic flow chart of another method for determining an image boundary provided in an embodiment of the present application;

[0080] Fig.11 A schematic flow chart of another method for determining an image boundary provided in an embodiment of the present application;

[0081] Fig.12 A schematic diagram of the structure of a device for determining an image boundary provided in an embodiment of the present application;

[0082] Fig.13 A schematic diagram of the structure of another device for determining image boundaries provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] The following is a detailed description of the image boundary determination method, device and storage medium provided in the embodiments of the present application in conjunction with the accompanying drawings.

[0084] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0085] The terms "first" and "second" and the like in the specification and drawings of this application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0086] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0087] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0088] In the description of the present application, unless otherwise specified, “plurality” means two or more.

[0089] Currently, when determining the boundary of an image, a boundary detection method based on image features, a boundary detection method based on a model, or a boundary detection method based on deep learning is often used. The above methods can all be used to determine the boundary of the image to be detected.

[0090] However, for the earthwork material pile image under complex working conditions, there are many material categories and the material colors are relatively close. Figure 1 , provides an image of earth and stone material pile, it can be seen that Figure 1 In the earth and stone pile image shown, the coloring of loess and stone is relatively close, so the above method cannot accurately fit the boundary of the earth and stone pile image, that is, the existing boundary detection method cannot accurately determine the boundary of the earth and stone pile image. Therefore, it is urgent to propose a method for determining the image boundary to detect the boundary of the earth and stone pile image.

[0091] In view of the problems existing in the above-mentioned prior art, the embodiments of the present application provide a method, device and storage medium for determining the boundary of an image, by dividing a target image into multiple image blocks, determining the block classification results of each image block, and then determining the block classification matrix corresponding to the block classification results. Afterwards, based on the classification result of the target image, a compensating convolution kernel and a filtering convolution kernel are used to perform convolution processing on the block classification matrix to obtain a compensation filtering matrix after correcting the block classification matrix, and then the boundary of the target image is fitted based on the elements in the compensation filtering matrix.

[0092] The method for determining the image boundary provided in the embodiment of the present application can be applied to the device for determining the image boundary, and the device for determining the image boundary can be a physical machine (such as a server) or a virtual machine (VM) deployed on a physical machine. When the device for determining the image boundary is a server, it can be a single server or a server cluster composed of multiple servers, which is not limited in the embodiment of the present application.

[0093] The following is a detailed description of the method for determining the image boundary provided by the present application.

[0094] Reference Figure 2 , the method for determining the image boundary provided in the embodiment of the present application includes S201-S204:

[0095] S201, dividing the target image into image blocks, extracting image texture features of the image blocks and determining block classification results of the image blocks according to the image texture features, and determining a block classification matrix according to the block classification results.

[0096] The target image may be an image of a pile of earth and stone materials under complex working conditions. Of course, in practical applications, the target image may also be other images that are not easy to fit the boundaries, and the present embodiment of the application does not limit this.

[0097] Optionally, in a possible implementation, the target image can be divided into image blocks by constructing a block classification model, image texture features of the image blocks can be extracted, and block classification results of the image blocks can be determined based on the image texture features, and a block classification matrix can be determined based on the block classification results. Specifically, a block division sub-model can be trained based on a sample image and the division results of the image blocks of the sample image; then, a texture feature extraction sub-model can be trained based on the division results and the image texture features of the image blocks of the sample image; then, a training sub-model can be trained based on the image texture features of the image blocks of the sample image, the block classification results of the image blocks of the sample image, and the block classification matrix of the image blocks of the sample image; finally, a block classification model can be constructed based on the block division sub-model, the texture feature extraction sub-model, and the training sub-model. The block classification model can be used to: when a target image input is detected, process the target image, obtain a block classification matrix of the target image, and output it.

[0098] The image block division result of the sample image or the target image can be determined according to the resolution of the image. Figure 1The image in is divided into 16*16 image blocks. Of course, in practical applications, in order to obtain the edge information of the image more accurately, the image can also be divided into 32*32 image blocks. However, the more image blocks are divided, the better. When the number of image blocks increases, the corresponding image blocks become smaller, and the image information carried will also decrease accordingly. It will be impossible to extract better image texture features, resulting in a decrease in recognition rate. Therefore, in practical applications, the number of divided image blocks can be determined in combination with the resolution of the image.

[0099] In a possible implementation, a gray level co-occurrence matrix may be used to extract image texture features of image blocks of an image.

[0100] The elements in the block classification matrix represent the block classification results of the image blocks at the corresponding positions. Figure 1 Taking the image shown in the figure as an example, the block classification results of the image block may include three types: background, stone and loess. The element "0" can be used to indicate that the block classification result of the image block is background, the element "1" can be used to indicate that the block classification result of the image block is stone, and the element "2" can be used to indicate that the block classification result of the image block is loess. By performing block classification on all image blocks in the target image or sample image, the following can be obtained: Figure 3 The 16*16 block classification matrix M is shown.

[0101] In a possible implementation, a support vector machine may be used to construct a block classification model. Of course, in practical applications, the block classification model may also be trained by other classifiers, which is not limited in the present embodiment of the application.

[0102] S202: classify the target image to obtain a classification result corresponding to the target image.

[0103] It is understandable that the embodiment of the present application does not limit the order of step S201 and step S202. In actual applications, step S202 may also be before step S201.

[0104] In a possible implementation, the target image can be classified by calling a deep learning model to obtain a classification result corresponding to the target image. The deep learning model can be trained based on sample images and the classification results corresponding to the sample images.

[0105] For example, Figure 1 Take the provided image as an example, the classification result of the image is stone.

[0106] S203. According to the classification result corresponding to the target image, convolution processing is performed on the block classification matrix based on the compensation convolution kernel and the filter convolution kernel to obtain a compensation filter matrix.

[0107] The compensation filter matrix is ​​used to correct the elements in the block classification matrix.

[0108] Optionally, in a possible implementation, the non-screening elements in the block classification matrix may be determined first, and the block classification results corresponding to the non-screening elements are different from the classification results corresponding to the target image. Figure 3 In the block classification matrix M shown in FIG. 1 , the elements corresponding to the image blocks whose block classification results are loess or background are non-screening elements, that is, the elements with element values ​​of 0 and 2 are non-screening elements. After determining the non-screening elements, the element values ​​of the non-screening elements are set to zero to obtain a screening matrix. For example, in Figure 3 In the block classification matrix M shown in the figure, the elements with values ​​of 0 and 2 can be set to zero, and the result is Figure 4 The screening matrix shown.

[0109] For example, if I c represents the target image, c represents the element value corresponding to the classification result of the target image (in the present embodiment, c=1 is used as an example for explanation), then the screening matrix M can be obtained according to expressions (1) and (2): c :

[0110] R c ={(m,n)|I c (i,j)=c,(i,j)∈N} (1)

[0111] M c ={M c (i,j)=c|(i,j)∈R c}∪{M c (i,j)=0|(i,j)∈NR c} (2)

[0112] Among them, R c express Figure 3 In the block classification matrix M shown in FIG. 1 , the coordinates of the image block corresponding to the element value c of the block classification result are represented by (m, n). c Represents the screening matrix of category c in M, N is the coordinate point in M, and the coordinate point in M ​​is represented by (i, j).

[0113] After obtaining the screening matrix, the compensation convolution kernel can be used to perform two-dimensional convolution processing on the screening matrix to obtain a first intermediate matrix, and the first intermediate matrix can be compensated based on a compensation threshold to obtain a compensation matrix; then, the compensation matrix can be two-dimensionally convolved using a filtering convolution kernel to obtain a second intermediate matrix, and the second intermediate matrix can be filtered based on the filtering threshold to obtain a compensation filtering matrix.

[0114] Among them, the compensating convolution kernel and the filtering convolution kernel can be convolution kernels determined in advance by humans. The compensating convolution kernel is used to correct the elements missed in the block classification matrix, and the filtering convolution kernel is used to correct the elements misdetected in the block classification matrix to improve the accuracy of the target image boundary fitting. Exemplarily, expression (3) is a compensating convolution kernel K provided in an embodiment of the present application, and expression (4) is a filtering convolution kernel F provided in an embodiment of the present application:

[0115]

[0116]

[0117] It is understandable that the compensation convolution kernel and the filter convolution kernel provided in the embodiment of the present application are only used as examples. In practical applications, other convolution kernels can also be determined as the compensation convolution kernel and the filter convolution kernel in the embodiment of the present application, and the embodiment of the present application does not limit this. It should be noted that the filter convolution kernel cannot make the element itself affect the convolution value, while the compensation convolution kernel needs to consider the influence of the element itself to avoid the edge information of the material pile not being correctly compensated and improved.

[0118] Optionally, in a possible implementation, elements in the first intermediate matrix whose element values ​​are greater than or equal to the compensation threshold can be determined as elements to be compensated; the element values ​​of the elements to be compensated are set to the element values ​​of the target elements in the first intermediate matrix to obtain a compensation matrix; and the block classification results corresponding to the target elements are the same as the classification results corresponding to the target image.

[0119] The compensation threshold may be an element value artificially determined in advance. For example, the compensation threshold may be 3c or 5c.

[0120] For example, Figure 5 As shown, the compensation convolution kernel K is used to Figure 4 The screening matrix M shown c Perform two-dimensional convolution processing to obtain the first intermediate matrix M K If T K Represents the compensation threshold, expressed as T K =5c as an example, the first intermediate matrix M can be expressed as K In the example, the elements whose element values ​​are greater than or equal to the compensation threshold are determined as the elements to be compensated, and the element values ​​of the elements to be compensated are set to the element values ​​c of the target elements. The classification results of the blocks corresponding to the target elements are the same as the classification results corresponding to the target images. K Except for the elements to be compensated, the element values ​​remain unchanged, and the following can be obtained: Figure 6 The compensation matrix M shown T :

[0121]

[0122] Optionally, in a possible implementation, elements in the second intermediate matrix whose element values ​​are less than a filtering threshold value may be determined as elements to be filtered; and the element values ​​of the elements to be filtered may be set to zero to obtain a compensation filtering matrix.

[0123] The filtering threshold may be an element value artificially determined in advance. For example, the filtering threshold may be 3c.

[0124] For example, Figure 7 As shown, the filter convolution kernel F is used to Figure 6 The compensation matrix M shown T Perform two-dimensional convolution processing to obtain the second intermediate matrix M F If T F Represents the filtering threshold, expressed as T F =3c as an example, the second intermediate matrix M can be expressed as F In the example, the elements whose element values ​​are less than the filtering threshold are determined as the elements to be filtered, and the element values ​​of the elements to be filtered are set to zero. F Except for the elements to be filtered, the element values ​​remain unchanged, and the following can be obtained: Figure 8 The compensation filter matrix M shown R :

[0125]

[0126] S204: Determine at least one boundary of the target image based on the elements in the compensation filter matrix.

[0127] Optionally, in a possible implementation, edge image blocks in the target image may be determined according to block classification results of image blocks corresponding to elements in the compensation filter matrix; and then at least one boundary may be determined based on the position of the edge image blocks.

[0128] Optionally, in a possible implementation manner, the center point of the edge image block may be determined according to the position of the edge image block, and then the center point of the edge image block may be determined as a boundary fitting point to determine at least one boundary.

[0129] from Figure 8It can be seen from the compensation filter matrix shown that the target image has two upper and lower boundaries. In practical applications, the upper boundary can be fitted or the lower boundary can be fitted. Of course, the upper and lower boundaries can also be fitted at the same time to determine the two boundaries of the target image. Exemplarily, taking the fitting of the upper boundary of the target image as an example, the edge image blocks of the upper boundary can be determined based on the block classification results of the image blocks corresponding to the elements in the compensation filter matrix, and then the center points of these edge image blocks are determined as fitting points. All the fitting points are connected to obtain the upper boundary of the target image. Exemplarily, as Fig. 9 As shown, the boundary line L is Figure 8 The compensation filter matrix M shown R The upper boundary obtained in .

[0130] In the method for determining the image boundary provided in the embodiment of the present application, the target image is divided into multiple image blocks, the block classification results of each image block are determined, and then the block classification matrix corresponding to the block classification results is determined. Afterwards, in order to improve the accuracy of the boundary fitting of the corresponding target image, based on the classification result of the target image, the block classification matrix is ​​convolved with a compensating convolution kernel and a filtering convolution kernel to obtain a compensation filtering matrix after the block classification matrix is ​​corrected, and then the boundary fitting of the target image is performed based on the elements in the compensating filtering matrix. It can be seen that the embodiment of the present application can accurately determine the boundary of the target image by dividing the target image into image blocks and optimizing the block classification matrix.

[0131] Based on the above description, Fig.10 As shown, Figure 2 Step S203 in can be replaced by S2031-S2036:

[0132] S2031. Determine the non-screening elements in the block classification matrix.

[0133] S2032. Set the element values ​​of non-screened elements to zero to obtain a screening matrix.

[0134] S2033. Perform two-dimensional convolution processing on the screening matrix using a compensating convolution kernel to obtain a first intermediate matrix.

[0135] S2034: Perform compensation processing on the first intermediate matrix based on the compensation threshold to obtain a compensation matrix.

[0136] S2035. Perform two-dimensional convolution processing on the compensation matrix using a filter convolution kernel to obtain a second intermediate matrix.

[0137] S2036: Filter the second intermediate matrix based on the filtering threshold to obtain a compensation filtering matrix.

[0138] Optional, such as Fig.11As shown, Figure 2 Step S204 in can be replaced by S2041-S2043:

[0139] S2041. Determine edge image blocks in the target image according to block classification results of image blocks corresponding to elements in the compensation filter matrix.

[0140] S2042: Determine the center point of the edge image block according to the position of the edge image block.

[0141] S2043: Determine the center point of the edge image block as a boundary fitting point, and determine at least one boundary.

[0142] like Fig.12 As shown, the embodiment of the present application further provides a device for determining an image boundary, and the device for determining an image boundary includes: a determination module 11, a classification module 12 and a processing module 13.

[0143] The determination module 11 executes S201 and S204 in the above method embodiment, the classification module 12 executes S202 in the above method embodiment, and the processing module 13 executes S203 in the above method embodiment.

[0144] Specifically, the determination module 11 is used to divide the target image into image blocks, extract the image texture features of the image blocks and determine the block classification results of the image blocks according to the image texture features, and determine the block classification matrix according to the block classification results; the elements in the block classification matrix represent the block classification results of the image blocks at the corresponding positions;

[0145] The classification module 12 is used to classify the target image and obtain the classification result corresponding to the target image;

[0146] The processing module 13 is used to perform convolution processing on the block classification matrix based on the compensation convolution kernel and the filter convolution kernel according to the classification result corresponding to the target image, so as to obtain a compensation filter matrix;

[0147] The determination module 11 is further configured to determine at least one boundary of the target image based on elements in the compensation filter matrix.

[0148] Optionally, in a possible implementation, the processing module 13 is specifically configured to:

[0149] Determine the non-screened elements in the block classification matrix; the block classification results corresponding to the non-screened elements are different from the classification results corresponding to the target image;

[0150] Set the element values ​​of non-screened elements to zero to obtain a screening matrix;

[0151] Performing two-dimensional convolution processing on the screening matrix using a compensating convolution kernel to obtain a first intermediate matrix;

[0152] Performing compensation processing on the first intermediate matrix based on the compensation threshold to obtain a compensation matrix;

[0153] Performing two-dimensional convolution processing on the compensation matrix using a filter convolution kernel to obtain a second intermediate matrix;

[0154] The second intermediate matrix is ​​filtered based on the filtering threshold to obtain a compensation filtering matrix.

[0155] Optionally, in another possible implementation, the processing module 13 is further configured to:

[0156] Determine, in the first intermediate matrix, elements whose element values ​​are greater than or equal to the compensation threshold as elements to be compensated;

[0157] The element value of the element to be compensated is set to the element value of the target element in the first intermediate matrix to obtain a compensation matrix; the block classification result corresponding to the target element is the same as the classification result corresponding to the target image.

[0158] Optionally, in another possible implementation, the processing module 13 is further configured to:

[0159] Determine the elements in the second intermediate matrix whose element values ​​are less than the filtering threshold as elements to be filtered;

[0160] The element values ​​of the elements to be filtered are set to zero to obtain a compensation filter matrix.

[0161] Optionally, in another possible implementation, the determining module 11 is specifically configured to:

[0162] Determine edge image blocks in the target image according to block classification results of image blocks corresponding to elements in the compensation filter matrix;

[0163] At least one boundary is determined based on the position of the edge image block.

[0164] Optionally, in another possible implementation, the determining module 11 is further configured to:

[0165] Determine the center point of the edge image block according to the position of the edge image block;

[0166] The center point of the edge image block is determined as a boundary fitting point, and at least one boundary is determined.

[0167] Optionally, in another possible implementation, the apparatus for determining an image boundary provided by the present application may further include:

[0168] A training module, used for training and obtaining a block sub-model according to the sample image and the image block division result of the sample image;

[0169] The training module is further used to train a texture feature extraction sub-model according to the segmentation result and the image texture features of the image blocks of the sample image;

[0170] The training module is further used to train and obtain a training sub-model according to the image texture features of the image blocks of the sample images, the block classification results of the image blocks of the sample images, and the block classification matrix of the image blocks of the sample images;

[0171] The building module is used to build a block classification model based on the block sub-model, the texture feature extraction sub-model and the training sub-model; the block classification model is used to: when the target image input is detected, output the block classification matrix of the target image.

[0172] Optionally, the device for determining the image boundary may further include a storage module, and the storage module is used to store program codes of the device for determining the image boundary, etc.

[0173] like Fig.13 As shown, an embodiment of the present application also provides an image boundary determination device, including a memory 41, a processor 42, a bus 43 and a communication interface 44; the memory 41 is used to store computer execution instructions, and the processor 42 is connected to the memory 41 through the bus 43; when the image boundary determination device is running, the processor 42 executes the computer execution instructions stored in the memory 41, so that the image boundary determination device performs the image boundary determination method provided in the above embodiment.

[0174] In a specific implementation, as an embodiment, the processor 42 (42-1 and 42-2) may include one or more central processing units (CPUs), such as Fig.13 As an embodiment, the image boundary determination device may include multiple processors 42, such as Fig.13 42-1 and processor 42-2 are shown in FIG. Each CPU in these processors 42 may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor 42 here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0175] The memory 41 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 41 may exist independently and be connected to the processor 42 via a bus 43. The memory 41 may also be integrated with the processor 42.

[0176] In a specific implementation, the memory 41 is used to store the data in the present application and the computer execution instructions corresponding to the software program for executing the present application. The processor 42 can perform various functions of the image boundary determination device by running or executing the software program stored in the memory 41 and calling the data stored in the memory 41.

[0177] The communication interface 44 uses any transceiver or other device for communicating with other devices or communication networks, such as a control system, a radio access network (RAN), a wireless local area network (WLAN), etc. The communication interface 44 may include a receiving unit to implement a receiving function, and a sending unit to implement a sending function.

[0178] The bus 43 may be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus. The bus 43 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.13 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0179] As an example, combining Fig.12 The functions implemented by the processing module in the image boundary determination device are similar to Fig.13 The function implemented by the processor in the image boundary determination device is the same as that implemented by the storage module in the image boundary determination device. Fig.13 The memory in implements the same function.

[0180] The explanation of the relevant contents in this embodiment can refer to the above method embodiment, which will not be repeated here.

[0181] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0182] The embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When a computer executes the instructions, the computer executes the method for determining the image boundary provided in the above embodiment.

[0183] Among them, the computer-readable storage medium, for example, can be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, RAM, ROM, an erasable programmable read only memory (EPROM), a register, a hard disk, an optical fiber, a CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above, or any other form of computer-readable storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application-specific integrated circuit (ASIC). In an embodiment of the present application, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or device.

[0184] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for determining an image boundary, characterized in that: include: Dividing the target image into image blocks, extracting image texture features of the image blocks and determining block classification results of the image blocks according to the image texture features, and determining a block classification matrix according to the block classification results; The elements in the block classification matrix represent the block classification results of the image blocks at the corresponding positions; Classifying the target image to obtain a classification result corresponding to the target image; According to the classification result corresponding to the target image, the block classification matrix is ​​convolved based on the compensation convolution kernel and the filter convolution kernel to obtain a compensation filter matrix; The compensation filter matrix is ​​used to modify the elements in the block classification matrix; Determining at least one boundary of the target image based on elements in the compensation filter matrix; The convolution processing of the block classification matrix based on the compensation convolution kernel and the filter convolution kernel to obtain the compensation filter matrix includes: Determining a non-screening element in the block classification matrix; the block classification result corresponding to the non-screening element is different from the classification result corresponding to the target image; Performing a zero operation on the element values ​​of the non-screening elements to obtain a screening matrix; Performing a two-dimensional convolution process on the screening matrix using the compensating convolution kernel to obtain a first intermediate matrix; Performing compensation processing on the first intermediate matrix based on a compensation threshold to obtain a compensation matrix; Performing two-dimensional convolution processing on the compensation matrix using the filter convolution kernel to obtain a second intermediate matrix; The second intermediate matrix is ​​filtered based on a filtering threshold to obtain the compensation filtering matrix.

2. The method according to claim 1, characterized in that The step of performing compensation processing on the first intermediate matrix based on the compensation threshold to obtain a compensation matrix includes: Determine, in the first intermediate matrix, elements whose element values ​​are greater than or equal to the compensation threshold as elements to be compensated; The element value of the element to be compensated is set to the element value of the target element in the first intermediate matrix to obtain the compensation matrix; the block classification result corresponding to the target element is the same as the classification result corresponding to the target image.

3. The method according to claim 1, characterized in that The filtering the second intermediate matrix based on the filtering threshold to obtain the compensation filtering matrix includes: Determine, in the second intermediate matrix, elements whose element values ​​are less than the filtering threshold as elements to be filtered; The element values ​​of the elements to be filtered are set to zero to obtain the compensation filter matrix.

4. The method according to claim 1, characterized in that The determining at least one boundary of the target image based on the elements in the compensation filter matrix comprises: Determining edge image blocks in the target image according to block classification results of image blocks corresponding to elements in the compensation filter matrix; The at least one boundary is determined based on the position of the edge image block.

5. The method according to claim 4, characterized in that The determining the at least one boundary based on the position of the edge image block comprises: Determine the center point of the edge image block according to the position of the edge image block; The center point of the edge image block is determined as a boundary fitting point to determine the at least one boundary.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: According to the sample image and the division result of the image block of the sample image, a block sub-model is obtained by training; According to the division result and the image texture features of the image blocks of the sample image, a texture feature extraction sub-model is trained; According to the image texture features of the image blocks of the sample images, the block classification results of the image blocks of the sample images, and the block classification matrix of the image blocks of the sample images, training a training sub-model; A block classification model is constructed based on the block division sub-model, the texture feature extraction sub-model and the training sub-model; the block classification model is used to: output a block classification matrix of the target image when the target image input is detected.

7. A device for determining an image boundary, characterized in that: include: A determination module, configured to divide the target image into image blocks, extract image texture features of the image blocks, determine block classification results of the image blocks according to the image texture features, and determine a block classification matrix according to the block classification results; The elements in the block classification matrix represent the block classification results of the image blocks at the corresponding positions; A classification module, used to classify the target image and obtain a classification result corresponding to the target image; A processing module, configured to perform convolution processing on the block classification matrix based on a compensation convolution kernel and a filter convolution kernel according to the classification result corresponding to the target image, so as to obtain a compensation filter matrix; The compensation filter matrix is ​​used to modify the elements in the block classification matrix; The determination module is further used to determine at least one boundary of the target image based on elements in the compensation filter matrix; The convolution processing of the block classification matrix based on the compensation convolution kernel and the filter convolution kernel to obtain the compensation filter matrix includes: Determining a non-screening element in the block classification matrix; the block classification result corresponding to the non-screening element is different from the classification result corresponding to the target image; Performing a zero operation on the element values ​​of the non-screening elements to obtain a screening matrix; Performing a two-dimensional convolution process on the screening matrix using the compensating convolution kernel to obtain a first intermediate matrix; Performing compensation processing on the first intermediate matrix based on a compensation threshold to obtain a compensation matrix; Performing two-dimensional convolution processing on the compensation matrix using the filter convolution kernel to obtain a second intermediate matrix; The second intermediate matrix is ​​filtered based on a filtering threshold to obtain the compensation filtering matrix.

8. A device for determining an image boundary, characterized in that: It includes a memory, a processor, a bus and a communication interface; the memory is used to store computer-executable instructions, and the processor is connected to the memory via the bus; When the image boundary determination device is running, the processor executes the computer-executable instructions stored in the memory, so that the image boundary determination device performs the image boundary determination method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when a computer executes the instructions, the computer executes the method for determining an image boundary as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Image processing method and device

    CN109495759A

  • Method for performing filtering at partition boundary of block related to 3D image

    US20180176559A1

  • Convolutional Neural Network-Based Image Processing Method And Image Processing Apparatus

    US20200302265A1

  • System for detecting salient objects in images

    WO2019005255A2