An evaluation method of an image segmentation algorithm, an electronic device and a readable storage medium

Abstract

The present disclosure relates to the field of image segmentation. The evaluation method of the image segmentation algorithm comprises: according to an original image, segmenting the original image by a plurality of to-be-tested algorithms respectively to obtain a plurality of segmentation images; comparing each segmentation image with labeled data to determine the segmentation result of each to-be-tested algorithm; determining the weight value corresponding to each sub-result set; and evaluating the segmentation result of each to-be-tested algorithm according to the segmentation result and the weight value corresponding to each sub-result set to determine an optimal algorithm. The present disclosure can determine a unified evaluation criterion for the image segmentation algorithm, can evaluate a plurality of to-be-tested algorithms simultaneously, has a high evaluation efficiency, has a higher evaluation accuracy and higher reliability by determining four sub-results of the segmentation image of the to-be-tested algorithm. Moreover, the evaluation result has traceability and repeatability, the data used for evaluation can be traced, the original image can be traced back from the evaluation result, and data can be collected to guide the optimization of the image segmentation algorithm.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image segmentation, and in particular, to an evaluation method of an image segmentation algorithm, an electronic device and a readable storage medium. BACKGROUND

[0002] After the ore is segmented, it is particularly important to accurately evaluate the segmentation results generated by the algorithm. However, the traditional manual checking and evaluation method has low checking efficiency, and there are differences in evaluation standards between different personnel, or even the same person may use different standards at different times, which greatly affects the reliability of the evaluation of the image segmentation results. Moreover, it is extremely difficult to trace the source of the data that has been evaluated, including the original source of the segmented data, the identity of the person making the judgment, and the judgment method used, and it is difficult to accurately track and record such information. SUMMARY

[0003] To overcome the problems in the related art, an exemplary embodiment of the present disclosure provides an evaluation method of an image segmentation algorithm, which includes: segmenting an original image by a plurality of to-be-tested algorithms according to the original image, to obtain a plurality of segmented images; comparing each segmented image with labeled data to determine the segmentation result of each to-be-tested algorithm, wherein the segmentation result includes four sub-results: mis-segmentation, missing segmentation, over-segmentation, and normal segmentation; determining the weight value corresponding to each sub-result set; evaluating the segmentation result of each to-be-tested algorithm according to the segmentation result and the weight value corresponding to each sub-result set, to determine the optimal algorithm in the to-be-tested algorithms.

[0004] In some embodiments, the evaluation of the segmentation result of each to-be-tested algorithm according to the segmentation result and the weight value corresponding to each sub-result, to determine the optimal algorithm in the to-be-tested algorithms, includes: determining the optimal data and the worst data in each sub-result according to the segmentation result of each to-be-tested algorithm; determining the relative closeness of the segmentation result of each algorithm to the optimal data and the worst data based on the optimal data and the worst data in each sub-result, and the weight value corresponding to each sub-result, wherein the relative closeness is used to represent the closeness of the segmentation result to the optimal data, and the degree of deviation from the worst data; determining the optimal algorithm in the to-be-tested algorithms according to the relative closeness.

[0005] In some embodiments, each set of the sub-results corresponds to a plurality of sets of the weight values, and the determining the optimal algorithm from the plurality of algorithms comprises: determining, according to each set of the weight values, the algorithm with the maximum relative closeness in each set of the weight values as an optimal sample; determining, according to the optimal sample in each set of the weight values, a frequency of each of the plurality of algorithms being determined as the optimal sample; and determining, according to the frequency of each of the plurality of algorithms being determined as the optimal sample, the algorithm with the highest frequency as the optimal algorithm.

[0006] In some embodiments, the determining the weight value corresponding to each set of the sub-results, wherein the weight value corresponding to each of the mis-segmentation, the under-segmentation, and the over-segmentation is less than the weight value corresponding to the normal segmentation.

[0007] In some embodiments, the weight value corresponding to each of the mis-segmentation, the under-segmentation, and the over-segmentation is greater than or equal to 0 and less than or equal to 0.3, and the weight value corresponding to the normal segmentation is greater than or equal to 0.6 and less than or equal to 1; and the sum of the weight values corresponding to the four sub-results is 1.

[0008] In some embodiments, each of the sub-results can correspond to a plurality of weight values, and each set of the sub-results can determine a plurality of sets of the weight values, wherein an adjustment step size between each set of the weight values is greater than or equal to 0.01 and less than or equal to 0.04.

[0009] In some embodiments, the labeled data is obtained by: pre-processing the original image to obtain a standard image; extracting a contour of a target according to the standard image to determine a standard contour image; extracting information of the standard contour image labeled by a human being as the labeled data according to the standard contour image; and extracting information of the target adhered in the original image as the labeled data.

[0010] In some embodiments, the segmenting the original image by the plurality of algorithms to obtain a plurality of segmented images comprises: pre-processing the original image to obtain a test image; extracting a contour of a target according to the test image to determine a test contour image; determining a registration pair by registering the test contour image with the labeled data, so that the contour images in the registration pair correspond to the same target as the labeled data; determining information of the target according to the test contour image and the test image; and segmenting the test contour image by the test algorithm according to the information of the target to determine the plurality of segmented images.

[0011] In a second aspect, the present disclosure also provides an electronic device, wherein the electronic device comprises a memory, a processor, and a data processing program stored in the memory and executable on the processor, and the data processing program, when executed by the processor, implements the evaluation method of the image segmentation algorithm according to any one of the preceding embodiments.

[0012] In a third aspect, the present disclosure also provides a computer-readable storage medium storing a program for executing the evaluation method of the image segmentation algorithm according to any one of the preceding embodiments.

[0013] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure.

[0014] The evaluation method of the image segmentation algorithm according to the embodiments of the present disclosure can determine a unified evaluation standard for the image segmentation algorithm, has a higher evaluation efficiency, can evaluate multiple to-be-tested algorithms at the same time, and can determine an optimal algorithm among multiple image segmentation algorithms. Four sub-results of the to-be-tested algorithm for segmenting an image are determined, and a corresponding weight value is determined for each sub-result to evaluate, which can have a higher evaluation accuracy in the evaluation of the segmentation algorithm. Moreover, the evaluation method of the image segmentation algorithm according to the present disclosure can evaluate different to-be-tested algorithms based on the same labeled data, the standard can be unified, the obtained evaluation result has a higher accuracy and reliability, and the evaluation result has repeatability. The method according to the present disclosure can avoid evaluation errors caused by manual evaluation, and is convenient for tracing the data of completed evaluation and tracing the data of completed evaluation, so as to trace the original image from the evaluation result, and is convenient for collecting data to guide the optimization of the image segmentation algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0015] The present disclosure can be better understood by describing exemplary embodiments thereof with reference to the accompanying drawings, in which:

[0016] Figure 1 The figure is a flowchart of the evaluation method of the image segmentation algorithm according to an exemplary embodiment of the present disclosure;

[0017] Figure 2 The figure is a flowchart of the evaluation method of the image segmentation algorithm according to another exemplary embodiment of the present disclosure;

[0018] Figure 3 The figure is a flowchart of the evaluation method of the image segmentation algorithm according to another exemplary embodiment of the present disclosure;

[0019] Figure 4 The figure is a flowchart of the evaluation method of the image segmentation algorithm according to another exemplary embodiment of the present disclosure;

[0020] Figure 5 Flowchart of an evaluation method of an image segmentation algorithm according to another exemplary embodiment of the present disclosure;

[0021] Figure 6 Flowchart of an evaluation method of an image segmentation algorithm according to another exemplary embodiment of the present disclosure;

[0022] Figure 7 Flowchart of an evaluation method of an image segmentation algorithm according to another exemplary embodiment of the present disclosure;

[0023] Figure 8 Schematic diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] The specific embodiments of the present application will now be described in detail with specific reference being made to the figures. It is noted that in the description of embodiments below, reference to a particular embodiment is not necessarily to the same embodiment, but rather to a similar or analogous embodiment. In addition, it is noted that the description of the embodiments can not include all possible combinations or features of the application, that variations can be made to the embodiments described, and that use of certain terminology can be interpreted by one skilled in the art in connection with the nature and context of the specific embodiments. It is further noted that the present application encompasses all such variations and alternatives.

[0025] Unless otherwise defined, technical and scientific terms used in the claims and specification have the meaning commonly understood by one of ordinary skill in the art to which this application pertains. The use of the terms "first", "second", and the like do not imply any order, quantity, or importance, but rather are used to distinguish one element from another. The use of the terms "a" and "an" and the like do not exclude the plural, but rather mean "one or more". The use of the terms "including", "containing", and the like do not exclude other elements not explicitly listed. The use of the term "connected" and the like does not exclude the presence of an intermediate element or intervening elements between two elements that are "connected" to each other. The use of the term "coupled" and the like does not exclude the presence of an intermediate element or intervening elements between two elements that are "coupled" to each other.

[0026] The image can be segmented by an image segmentation algorithm, so that the multiple targets in the image can be distinguished from each other, and each target in the image can be independent. For example, the image of the ore is segmented, and the multiple ore blocks that are connected to each other are shown as a color block in the image. In order to distinguish each ore block from each other in the image, the image segmentation algorithm is used to determine the overlapping or connected areas between the ore blocks, and the areas are segmented to make each ore block independent and distinguishable from each other in the image. The image segmentation algorithm can only segment a group of targets that overlap or are connected to each other, or can simultaneously segment multiple groups of different targets in an image. The image segmentation algorithm can produce multiple different segmentation results, including incorrect segmentation, missed segmentation, over-segmentation, and correct segmentation. Incorrect segmentation refers to the fact that the boundaries of the multiple connected targets determined in the segmentation process do not match the actual boundaries, and the segmentation result is different from the actual target. Missed segmentation refers to the fact that part of the targets in the image are missed in the segmentation process, resulting in fewer segmented targets in the image than the actual number of targets in the image. Over-segmentation refers to the fact that a target is incorrectly segmented into multiple parts in the image segmentation process, resulting in more segmented targets in the final image than the actual number of targets in the image. Correct segmentation refers to the fact that each target can be accurately separated in the image segmentation process, so that the number of segmented targets in the final image is consistent with the actual number of targets. The four image segmentation results can be used to determine the quality of the segmentation image output by the image segmentation algorithm.

[0027] To solve the above problems, the present disclosure provides an evaluation method of an image segmentation algorithm, which can include steps S110 to S140 as shown in the following. Figure 1

[0028] In step S110, the original image is segmented by multiple to-be-tested algorithms to obtain multiple segmented images. The original image can be segmented by multiple to-be-tested algorithms, wherein the multiple to-be-tested algorithms can include one or more different to-be-tested algorithms. For each to-be-tested algorithm, a different parameter combination can be set. For an algorithm with a set of parameter combinations, it is determined as a to-be-tested algorithm. For algorithms with different parameter combinations and different types of algorithms, multiple different to-be-tested algorithms can be determined. Each algorithm can segment one or more targets in the original image to obtain multiple different segmented images. Each algorithm can simultaneously segment multiple targets in the image to obtain the segmented image of each target.

[0029] ​Step S120, compare each segmented image with the annotation data to determine the segmentation result of each to-be-tested algorithm, wherein the segmentation result includes the following four sub-results: mis-segmentation, missing segmentation, over-segmentation, and normal segmentation. The segmentation result can be determined according to the segmentation image obtained by the to-be-tested algorithm and the information of the segmentation image, and compared with the annotation data, taking the annotation information as the benchmark, to determine the segmentation result of the segmentation image determined by the to-be-tested algorithm. The judgment of the segmentation result can be determined by the number of segmentation regions, the size of the segmentation regions, and the like obtained by the to-be-tested algorithm for segmenting the image, which can be compared with the information of the number of segmentation regions and the size of the segmentation regions determined by the manual annotation in the annotation data to determine the similarity between the segmentation image and the annotation data, thereby determining the segmentation result of the to-be-tested algorithm. The segmentation result can include mis-segmentation, missing segmentation, over-segmentation, and normal segmentation. In the case that the segmentation image is compared with the annotation data, and the segmentation regions of the segmentation image have a large difference with the manually determined segmentation regions in the annotation data, the overlap degree of the segmentation regions of the segmentation image with the manually determined segmentation regions in the annotation data is very low or no overlap occurs, it can be determined that the segmentation regions determined by the to-be-tested algorithm are inconsistent with the segmentation regions in the annotation data, the segmentation position determined by the segmentation image of the to-be-tested algorithm is different from the segmentation position determined by the manual annotation in the annotation data, it can be determined that the processing of the to-be-tested algorithm is wrong, the result of the image segmentation is inconsistent with the actual result, and it can be determined that the segmentation result is mis-segmentation. In the case that the segmentation image is compared with the annotation data, and the number of regions segmented by the segmentation image is less than the number of manually determined segmentation regions in the annotation data, it indicates that the segmentation of the original image by the algorithm has missing, the segmentation image of the to-be-tested algorithm has regions that are not recognized and segmented, and it can be determined that the segmentation result is missing segmentation. In the case that the segmentation image is compared with the annotation data, and the number of regions segmented by the segmentation image is greater than the number of manually determined segmentation regions in the annotation data, it can be determined that the segmentation result is over-segmentation. In the case that the overlap degree of the segmentation image and the annotation data is high, the overlap degree of the regions segmented in the segmentation image and the manually segmented regions in the annotation data is high, it can be determined that the segmentation result is normal segmentation. Each algorithm can simultaneously segment multiple groups of targets in the image to determine the value of each sub-result of the segmentation image of each to-be-tested algorithm.

[0030] Step S130, determine the weight value corresponding to each set of sub-results. The four sub-results of mis-segmentation, missing segmentation, over-segmentation, and normal segmentation can be determined to have a corresponding weight value, wherein mis-segmentation, missing segmentation, and over-segmentation are all indicators of errors in image segmentation by the to-be-tested algorithm, which are negative indicators, and normal segmentation is an indicator of correct segmentation in image segmentation by the to-be-tested algorithm, which is a positive indicator. Therefore, when determining the size of the weight value of the four sub-results, different weight values can be determined according to the evaluation requirements.

[0031] Step S140, according to the segmentation result and the weight value corresponding to each sub-result set, the segmentation result of each to-be-tested algorithm is evaluated, and the optimal algorithm in the to-be-tested algorithm is determined. According to the segmentation result of each algorithm and the weight value corresponding to each sub-result, the segmentation result of each to-be-tested algorithm can be evaluated, and the segmentation result of each to-be-tested algorithm can be evaluated by a multi-index evaluation method as shown in Figure 6 The segmentation result of all to-be-tested algorithms is calculated by the multi-index evaluation method, and the to-be-tested algorithm with the highest score is determined as the optimal algorithm. The multi-index evaluation method can include analytic hierarchy process, TOPSIS method, fuzzy comprehensive evaluation method, etc., and can evaluate the respective sets of four sub-results of under-segmentation, over-segmentation, missing segmentation and normal segmentation at the same time.

[0032] Through the evaluation method of the image segmentation algorithm of the embodiment, a unified evaluation standard can be determined for the image segmentation algorithm, and the evaluation efficiency is high. Multiple to-be-tested algorithms can be evaluated at the same time, and the optimal algorithm can be determined among multiple image segmentation algorithms. The evaluation efficiency is higher. Through multiple segmentation results, multiple evaluation samples are determined, multiple weight values are determined for each evaluation sample, and the evaluation of the image segmentation algorithm can be more accurately realized. In the embodiment, the image segmentation algorithm is evaluated according to the four sub-results of under-segmentation, missing segmentation, over-segmentation and normal segmentation of each image segmentation algorithm. Multiple indexes can be comprehensively evaluated, and the evaluation of the image segmentation algorithm is more comprehensive through the multi-index evaluation method. Among the four sub-results, under-segmentation, missing segmentation and over-segmentation are negative indexes, and normal segmentation is a positive index. The evaluation method combining positive indexes and negative indexes can improve the accuracy of the evaluation of the image segmentation algorithm and effectively reduce the evaluation deviation. By setting different weight value combinations for a group of sub-results of a to-be-tested algorithm, the adaptability of the evaluation method can be improved to meet various algorithm evaluation requirements. The advantages of each segmentation algorithm can be determined by changing the combination of weight values, or the segmentation algorithm suitable for different image segmentation requirements can be determined according to the algorithm evaluation requirements. Through the evaluation method of the embodiment, the evaluation standard is unified, and evaluation errors caused by manual evaluation can be avoided. It is convenient to trace the data of completed evaluation, and the original image can be traced back from the evaluation result. It is convenient to collect data to guide the optimization of the image segmentation algorithm, and the evaluation result has repeatability. For the same image segmentation algorithm, the evaluation result is consistent. The multiple sub-results of the image segmentation algorithm are evaluated by the multi-index evaluation method, the calculation process is simple, the data quantity is rich, and the evaluation reliability is high.

[0033] In some embodiments, as shown in Figure 2 Step S140, according to the segmentation result and the weight value corresponding to each sub-result set, the segmentation result of each to-be-tested algorithm is evaluated, and the optimal algorithm in the to-be-tested algorithm is determined, which can include steps S141 to S143.

[0034] In step S141, the optimal data and the worst data in each sub result are determined according to the segmentation results of each to-be-tested algorithm. The optimal data can be data in each sub result that can represent the highest image segmentation accuracy, for example, in a series of data of the proportion of mis-segmentation, the data with the smallest proportion of mis-segmentation is the optimal data, and for a series of data of the proportion of normal segmentation, the data with the largest proportion of normal segmentation is the optimal data. The worst data can be data in each sub result that can represent the lowest image segmentation accuracy, for example, in a series of data of the proportion of mis-segmentation, the data with the largest proportion of mis-segmentation is the worst data, and for a series of data of the proportion of normal segmentation, the data with the smallest proportion of normal segmentation is the worst data. The segmentation results of the to-be-tested algorithm can be evaluated by the TOPSIS method. Each to-be-tested algorithm can be used to segment multiple different original images, so that multiple segmented images of a to-be-tested algorithm are determined, the segmentation results of the multiple segmented images determined for the same to-be-tested image are collected and the quantity proportion is determined, and the segmentation results of each to-be-tested algorithm can be integrated into a matrix form to determine a segmentation result matrix:

[0035]

[0036] Wherein, E is the proportion of mis-segmented segmented images in the multiple segmentation results of a to-be-tested algorithm; L is the proportion of missed segmented segmented images in the multiple segmentation results of a to-be-tested algorithm; M is the proportion of over-segmented segmented images in the multiple segmentation results of a to-be-tested algorithm; N is the proportion of normally segmented segmented images in the multiple segmentation results of a to-be-tested algorithm. m is the number of different types of algorithms, and n is the number of different parameter combinations of each algorithm.

[0037] Since the proportion E of mis-segmented segmented images, the proportion L of missed segmented segmented images, and the proportion N of normally segmented segmented images are all negative indicators, that is, indicators for measuring undesirable or negative performance of segmentation results, while the proportion N of normally segmented segmented images is a positive indicator, that is, an indicator for measuring good performance of segmentation results; therefore, the larger E, L and M are, the worse the performance is, and the larger N is, the better the performance is. Since E, L and M are negative indicators, and N is a positive indicator, the evaluation criteria are different, and the negative indicators E, L and M need to be positively transformed to obtain the following positively transformed matrix to unify the evaluation criteria and facilitate scoring.

[0038]

[0039] For convenience of calculation, the positively transformed matrix can be represented by x:

[0040]

[0041] wherein, x mn1 represents the positive index of the false segmentation of the mth algorithm to be tested, x mn2 represents the positive index of the missed segmentation of the mth algorithm to be tested; x mn3 represents the positive index of the over segmentation of the mth algorithm to be tested; x mn4 represents the normal segmentation index of the mth algorithm to be tested. Each data in the positive matrix can be standardized, and the standardized data z of each data in the positive matrix is determined according to the following formula ij :

[0042]

[0043] wherein, i = 1, 2, …, mn; i = 1, 2, 3, 4; so as to obtain the standardized matrix:

[0044]

[0045] Through standardization, the value of each sub result can fluctuate in the same range, so as to effectively avoid the calculation accuracy problem caused by too large or too small value. Thus, the accuracy of the segmentation algorithm evaluation is improved. The dimension and data order of magnitude of each sub result can be different, and through standardization, the difference caused by different dimensions and orders of magnitude can be eliminated, so that all indexes are compared under the same dimension, avoiding affecting the evaluation result, and improving the accuracy and reliability of the segmentation algorithm evaluation. According to the standardized matrix, the optimal data and the worst data of each index can be determined. The maximum value Z + of each column in the standardized matrix is taken, that is, Z + is the maximum value in each sub result, and Z + is determined as the optimal data under each index; the minimum value Z - of each column in the standardized matrix is taken, that is, Z - is the minimum value in each sub result, and Z - is determined as the worst data under each index.

[0046] In step S142, based on the optimal data and the worst data in each sub result, and the weight value corresponding to each sub result, the relative closeness of the segmentation result of each algorithm relative to the optimal data and the worst data is determined, wherein the relative closeness is used to represent the closeness of the segmentation result to the optimal data, and the deviation degree from the worst data. The data of each row in the standardized matrix, that is, the segmentation result of each algorithm to be tested, can be calculated according to the optimal data and the worst data in each sub result, and the weight value corresponding to each sub result, that is, the relative closeness of each segmentation result to the optimal data Z +the distance between each segmentation result and the best data Z - , and then determine the relative closeness of each segmentation result of each algorithm to the best data and the worst data according to the distance. The distance D + between each segmentation result and the best data Z - of its corresponding sub result can be determined according to the following formula:

[0047]

[0048]

[0049] wherein D i + is the distance between the segmentation result of the i-th algorithm to be tested and the best data Z + of its corresponding sub result, D i - is the distance between the segmentation result of the i-th algorithm to be tested and the worst data Z - of its corresponding sub result; Z j + is the best data of the j-th sub result, Z j - is the worst data of the j-th sub result; w j is the weight value of the j-th sub result. Wherein w1 is the weight value corresponding to the mis-segmentation, w2 is the weight value corresponding to the missed segmentation, w3 is the weight value corresponding to the over-segmentation, and w4 is the weight value corresponding to the normal segmentation. For a group of sub results, the distance between each segmentation result and the best data Z + and the worst data Z - of its corresponding sub result can be determined according to a group of weight values, and the distance between each segmentation result and the best data Z + and the worst data Z - of its corresponding sub result can also be determined according to a plurality of different groups of weight values, and a plurality of distance data are determined.

[0050] The relative closeness can be determined according to the distance D + between each segmentation result and the best data Z i + of its corresponding sub result of each algorithm to be tested, and the distance D - between each segmentation result and the worst data Z i - of its corresponding sub result of each algorithm to be tested, according to the following formula:

[0051]

[0052] wherein S iFor relative closeness, the relative closeness of the segmentation result of each to-be-tested algorithm can be determined, and the optimal algorithm in the to-be-tested algorithms can be determined according to the relative closeness.

[0053] In step S143, the optimal algorithm in the to-be-tested algorithms can be determined according to the relative closeness. The relative closeness can represent the closeness of the segmentation result of each to-be-tested algorithm to the optimal data and the deviation of the segmentation result of each to-be-tested algorithm from the worst data. The greater the relative closeness, the closer the segmentation result of the current to-be-tested algorithm to the optimal data and the farther the segmentation result of the current to-be-tested algorithm from the worst data. The smaller the relative closeness of the to-be-tested algorithm, the farther the segmentation result of the current to-be-tested algorithm from the optimal data and the closer the segmentation result of the current to-be-tested algorithm to the worst data. Therefore, it can be determined that the to-be-tested algorithm with the greatest relative closeness can be used as the optimal algorithm.

[0054] By the evaluation method of the image segmentation algorithm of the embodiment, the relative closeness can be determined according to the weight of the segmentation result of each to-be-tested algorithm and the optimal data and the worst data of each sub result, the closeness of the segmentation result of each to-be-tested algorithm to the optimal data can be determined, and the optimal algorithm can be determined. The optimal data and the worst data can be used as one of the evaluation standards of the segmentation algorithm, the evaluation standards of each algorithm are unified, and the accuracy and reliability of the evaluation can be improved by determining one or more groups of different weight values and performing weighted evaluation on the segmentation result. The relative closeness can be determined by determining the optimal data and the worst data in the four sub results and the distance of a group of sub results relative to the optimal data and the worst data, the relative closeness can be comprehensively determined by combining the positive indicators and the negative indicators in the sub results, the accuracy of the evaluation of the image segmentation algorithm is higher, and the evaluation of the to-be-tested algorithm can be more comprehensive by setting a plurality of different weight values, thereby determining the algorithm with the optimal comprehensive performance or adjusting the weight values to adapt to different evaluation requirements and determining the optimal algorithm that best meets the image segmentation requirements, which can effectively reduce the evaluation deviation and improve the evaluation stability of the image segmentation algorithm.

[0055] In some embodiments, each group of sub results corresponds to a plurality of groups of weight values, as shown in Figure 3 In step S143, the optimal algorithm in the to-be-tested algorithms can be determined according to the relative closeness, which can include steps S1431, S1432 and S1433.

[0056] Step S1431, according to each set of weight values, the relative proximity of each set of weight values is the largest to be measured algorithm is determined as the optimal sample. Can be set to have a plurality of different weight values, in the process of determining the relative proximity, by changing the size of each sub result weight, for a to be measured algorithm, can be balanced by a plurality of different weight values for a plurality of sub results, for a to be measured algorithm to determine a plurality of relative proximity. According to a plurality of different weight values, the relative proximity of each to be measured algorithm under different weight values can be determined, and the relative proximity of each to be measured algorithm under each weight can be compared according to each different weight, and the relative proximity of a plurality of to be measured algorithms under the same weight value is determined, and the relative proximity of the largest to be measured algorithm is determined as the optimal sample under the current weight combination, so as to determine the optimal sample under each weight value.

[0057] Step S1432, according to the optimal sample under each weight value, the frequency of each to be measured algorithm is determined as the optimal sample. According to all weight values, the optimal sample under each weight value can be determined, each optimal sample corresponds to a to be measured algorithm, and all optimal samples under all weight value combinations can be determined according to the data of the optimal sample, and the frequency of each to be measured algorithm is determined as the optimal sample.

[0058] Step S1433, according to the frequency of each to be measured algorithm being determined as the optimal sample, the to be measured algorithm with the highest frequency is determined as the optimal algorithm. The frequency of each to be measured algorithm being determined as the optimal sample can be compared, and for the to be measured algorithm with higher frequency, the relative proximity of the to be measured algorithm under a plurality of different weight value combinations can be determined to remain at a higher level, so that the segmentation result of the to be measured algorithm is optimal in most cases, so that the performance of the to be measured algorithm can be determined to be higher. Therefore, the to be measured algorithm with the highest frequency of being determined as the optimal sample is determined as the optimal algorithm.

[0059] Through steps S1431 to S1433, the optimal sample in the to be measured algorithm can be determined under different weight values, and the optimal algorithm can be determined by the frequency of each to be measured algorithm being determined as the optimal sample, which can effectively avoid the influence of single weight value on the evaluation of to be measured algorithm, and can provide more reference samples for the evaluation of to be measured algorithm by changing the weight value, so as to make the evaluation of segmentation algorithm have higher flexibility and comprehensiveness, so as to determine the final optimal algorithm, so as to make the evaluation of segmentation algorithm have higher accuracy and stability.

[0060] In some embodiments, in step S130, among the weight values corresponding to each set of sub-results, the weight value corresponding to each of the mis-segmentation, the under-segmentation, and the over-segmentation is less than the weight value corresponding to the normal segmentation. Each set of sub-results can include four sub-results, i.e., the mis-segmentation, the under-segmentation, the over-segmentation, and the normal segmentation. In the process of evaluating each set of sub-results, the relative closeness of each set of sub-results is determined according to the weight values. The weight values of each sub-result are different. The weight values corresponding to the mis-segmentation, the under-segmentation, and the over-segmentation are all less than the weight value corresponding to the normal segmentation. The normal segmentation is a positive indicator in the sub-results, which can most directly determine the pros and cons of the algorithm, and thus a larger weight value can be set for the normal segmentation. Since the mis-segmentation, the under-segmentation, and the over-segmentation are all negative indicators, which are used to measure the poor or negative performance of the segmentation results, these negative indicators can be normalized in the process of determining the relative closeness. The three sub-results after normalization correspond to the data of the non-mis-segmentation, the non-under-segmentation, and the non-over-segmentation, respectively. There is still a certain negative indicator in the normalized data. In order to ensure the accuracy of the final algorithm evaluation, the method of the present embodiment can effectively reduce the influence of the mis-segmentation, the under-segmentation, and the over-segmentation, which are the sub-results representing the negative performance of the algorithm to be tested, and reduce the negative influence on the final determination of the optimal algorithm. The final optimal algorithm determined is more accurate and reliable. Through the method of the present embodiment, the multi-index decision results according to the four sub-results can be balanced, the influence of the negative indicators on the decision can be avoided, and the stability of the evaluation method can be improved.

[0061] In some embodiments, the weight values corresponding to the mis-segmentation, the under-segmentation, the over-segmentation and the normal segmentation are respectively greater than or equal to 0 and less than or equal to 0.3, the weight value corresponding to the normal segmentation is greater than or equal to 0.6 and less than or equal to 1, and the sum of the weight values corresponding to the four sub-results is 1. The weight values corresponding to each sub-result can be different, where w1 is the weight value corresponding to the mis-segmentation, w2 is the weight value corresponding to the under-segmentation, w3 is the weight value corresponding to the over-segmentation, and w4 is the weight value corresponding to the normal segmentation. For the weight values w1, w2 and w3 corresponding to the mis-segmentation, the under-segmentation and the over-segmentation, the weight value range can be determined to satisfy w1, w2, w3 ∈ [0, 0.3], and for the weight value w4 corresponding to the normal segmentation, w4 can satisfy w4 ∈ [0.6, 1], and each weight value satisfies w1 + w2 + w3 + w4 = 1. The weight values of w1, w2 and w3 can be different. Through the method of the embodiment, different ranges are set for the weight values corresponding to the mis-segmentation, the under-segmentation, the over-segmentation and the normal segmentation, so that the weight values corresponding to the mis-segmentation, the under-segmentation and the over-segmentation are small, the weight value corresponding to the normal segmentation is large, and the sum of the weight values corresponding to each sub-result is 1, which can effectively reduce the negative influence of the three sub-results representing the negative performance of the to-be-tested algorithm, i.e., the mis-segmentation, the under-segmentation and the over-segmentation, on the final determination of the optimal algorithm, and improve the accuracy and reliability of the evaluation method.

[0062] In some embodiments, each sub-result can correspond to multiple weight values, and each set of sub-results can determine multiple sets of weight values, where the adjustment step size between each set of weight values is greater than or equal to 0.01 and less than or equal to 0.04. For the set of sub-results of each to-be-tested algorithm, multiple sets of weight values can be set, and for each set of weight values, the weight value corresponding to each sub-result is adjusted, and the adjustment step size of each adjustment of the weight value corresponding to each sub-result can be greater than or equal to 0.01 and less than or equal to 0.04, where the adjustment step size can be set to 0.02, so that when adjusting the weight value corresponding to each sub-result, each weight value is increased or decreased by 0.02 each time, and the sum of the weight values corresponding to each sub-result in the same set of weight values is 1 and satisfies the range requirement of the weight value in the foregoing embodiment. All possible combinations of weight values in the above value range can be used as multiple sets of weight values corresponding to each set of sub-results according to the adjustment step size, under the condition that w1, w2, w3 ∈ [0, 0.3], w4 ∈ [0.6, 1] and w1 + w2 + w3 + w4 = 1. Through the weight value adjustment method with a small adjustment step size, the weight value corresponding to each sub-result can be more accurately controlled, the evaluation of the image segmentation algorithm can have better flexibility, different scenarios and evaluation requirements can be adapted to, and better evaluation accuracy can be achieved.

[0063] In some embodiments, asFigure 4 As shown, the labeling data can be obtained by the following method, which can include steps S210 to S240.

[0064] In step S210, the original image is preprocessed to obtain a standard image. The original image can be preprocessed by image processing operations such as image enhancement, morphological operation, sharpening, and binarization, to improve the boundary information of the adherent target and the information inside the adherent target in the original image. The linear contrast of the original image can be improved by image enhancement, thereby improving the visibility of the details of the image, making the edges of the adherent target in the original image clearer, and the image information of the adherent target more obvious. The original image can also be denoised by morphological operations such as dilation, erosion, opening operation, or closing operation, thereby reducing the noise of the original image and improving the clarity of the information of the adherent target in the original image. The boundaries of the original image can also be sharpened by sharpening, which can enhance the contrast of the boundaries, thereby making the target and the background more distinct, facilitating the determination of the contour edges of the adherent target. The original image can also be converted to a binary image by binarization, thereby simplifying the information of the original image, distinguishing the target from the background, reducing the data volume, and facilitating subsequent processing and analysis, effectively reducing the data volume and the data calculation amount of subsequent processing.

[0065] In step S220, the contour of the target is extracted according to the standard image to determine a standard contour image. According to the standard image, the contour of the adherent target in the original image can be determined by a contour extraction algorithm, thereby determining the standard contour image. The information of the standard contour image of the image can be obtained by edge detection, contour detection, etc. The contour of the target is the common contour of the adherent target, and does not include the edge contour between the overlapping targets.

[0066] In step S230, the information of the artificially labeled standard contour image is extracted from the standard contour image as labeling data. The information of the artificially labeled standard contour image can be extracted from the standard contour image as a reference, which can include image information separating the multiple adherent targets by artificial labeling, and can be the contour information of each of the multiple adherent targets. In the image information, the target part is white and the background is black, facilitating the distinction between the target and the background. The target information in the artificially labeled image can form mask information and serve as labeling data.

[0067] The step S240 extracts the standard contour image, extracts the information of the standard contour image in which the target is adhered, as the annotation data. The information of the adhered target can be extracted from the standard contour image, wherein the information of the adhered target is displayed as white in the image and the background is black, and the information of the adhered target can be a solid white area image in the image corresponding to the adhered target. The information of the adhered stone can be used as the annotation data, and the information of the artificial annotation standard contour image corresponding to the same target is paired with the information of the standard contour image in which the target is adhered to determine the annotation data.

[0068] Through the steps S210 to S240, the original image can be denoised and the edges can be made more obvious through preprocessing, which facilitates the extraction of more clear and more actual contour image information, and improves the accuracy and completeness of the annotation data. The information of the artificial annotation standard contour image and the information of the standard contour image in which the target is adhered are extracted according to the contour image information, and are determined as the annotation data. The artificial annotation information and the information of the adhered target belonging to the same contour are extracted through the contour image, and are determined as the annotation data. The segmentation results of the to-be-tested algorithms are evaluated based on the annotation data, and the segmentation results of each to-be-tested algorithm are based on the annotation data, so that the evaluation of the algorithm has a unified standard and better reliability. The embodiment can provide annotation data with higher accuracy, thereby improving the accuracy of the evaluation of the to-be-tested algorithm.

[0069] In some embodiments, as shown in FIG. 1, Figure 5 The step S110 includes steps S111 to S115.

[0070] Step S111, the original image is preprocessed to obtain a to-be-tested image. The original image can be preprocessed in the same or similar manner as the standard image to obtain the to-be-tested image. The original image can be preprocessed by image processing operations such as image enhancement, morphological operation, sharpening, and binarization to improve the boundary information of the adhering target and the information inside the adhering target in the original image. The linear contrast of the original image can be improved by image enhancement to improve the visibility of the details of the image, so that the edges of the adhering target in the original image are clearer and the image information of the adhering target is more obvious. The original image can also be denoised by morphological operations such as dilation, erosion, opening operation, or closing operation, so as to reduce the noise of the original image and improve the clarity of the information of the adhering target in the original image. The original image can also be converted into a binary image by binarization. The boundaries of the original image can be sharpened by sharpening, which can improve the contrast of the boundaries, so that the target and the background are more distinct, and the contour edge of the adhering target is easy to determine. The sharpening preset value can be changed according to the image processing requirements of the to-be-tested algorithm to achieve different sharpening effects. The sharpening value of the original image sharpening processing of the labeled data can be taken as a reference, and the original image can be set to be greater than the sharpening value of the labeled data in step S111. As shown in FIG. 8, the sharpening value of the labeled data image sharpening processing is 0, and the sharpening value of the to-be-tested image sharpening processing can be determined as 3, 5, 7, 9, etc. according to the image processing requirements of the to-be-tested algorithm. Figure 7

[0071] Step S112, according to the to-be-tested image, the contour of the target is extracted to determine the to-be-tested contour image. According to the to-be-tested image, the contours of each target in the original image can be determined by a contour extraction algorithm to determine the to-be-tested contour image. The information of the to-be-tested contour image of the image can be obtained by edge detection, contour detection, etc. Among them, for each target adhering to each other, the target contour is the common contour of the adhering target, and does not include the edge contour of the overlapping between the adhering targets.

[0072] Step S113, the to-be-tested contour image is registered with the labeled data to determine a registration pair, so that the contour images in the registration pair correspond to the same target as the labeled data. The to-be-tested contour image can be registered with the standard contour image in the labeled data to form a plurality of registration pairs. Each registration pair can include to-be-tested contour information belonging to the same target and the labeled data. The labeled data and the to-be-tested contour information are registered so that the labeled data and the to-be-tested contour information in the same registration pair correspond to the same region and the same target in the original image.

[0073] ​In step S114, information of the target is determined according to the to-be-tested contour image and the to-be-tested image. The information of the target in the to-be-tested image corresponding to the contour can be determined according to the to-be-tested contour image and the to-be-tested image. The to-be-tested contour image and the to-be-tested image corresponding to the same target in the same region are combined, the contour of the conglutination target in the to-be-tested contour image and the information inside the contour of the conglutination target in the to-be-tested image are determined as the information of the target, and the to-be-tested algorithm is facilitated to segment the information of the target.

[0074] In step S115, a plurality of segmentation images are determined by segmenting the to-be-tested contour image through the to-be-tested algorithm according to the information of the target. Each to-be-tested algorithm is used to segment the to-be-tested contour image according to the information of the target, so that the plurality of segmentation images are determined. Each to-be-tested algorithm is used to segment the same to-be-tested contour image, which facilitates comparison between the segmentation image determined by each to-be-tested algorithm and the labeled information, so that the standard for evaluating the to-be-tested algorithm is unified.

[0075] According to steps S111 to S115, the original image is preprocessed, the contour is extracted, and the image is segmented, so that all to-be-tested algorithms can segment the same original image under the same condition, the objective conditions of all to-be-tested algorithms in the image segmentation process are the same, and the influence of different image qualities or different to-be-segmented targets on the segmentation result is eliminated. Efficient, accurate and standardized evaluation of the image segmentation algorithm can be achieved.

[0076] Based on the same inventive concept, the disclosure also provides an electronic device, wherein the electronic device comprises a memory, a processor, and a data processing program stored in the memory and executable on the processor, and the data processing program is executed by the processor to implement the evaluation method of the image segmentation algorithm according to any one of the preceding embodiments. As shown in the Figure 8 As shown in the figure, one embodiment of the disclosure provides an electronic device 300. The electronic device 300 comprises a memory 310, a processor 320, and an input / output (I / O) interface 330. The memory 310 is configured to store instructions. The processor 320 is configured to invoke the instructions stored in the memory 310 to execute the evaluation method of the image segmentation algorithm according to the embodiments of the disclosure. The processor 320 is connected with the memory 310 and the I / O interface 330, for example, through a bus system and / or other forms of connection mechanism (not shown). The memory 310 can be used to store programs and data, including the program of the evaluation method of the image segmentation algorithm according to the embodiments of the disclosure. The processor 320 executes the program stored in the memory 310, so as to execute various functional applications and data processing of the electronic device 300.

[0077] The processor 320 in the embodiments of the present disclosure can be implemented in at least one of a hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA), and the processor 320 can be a combination of one or several of a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities.

[0078] Based on the same inventive concept, the present disclosure further provides a computer-readable storage medium storing a program for performing the evaluation method of the image segmentation algorithm of any one of the foregoing embodiments. The memory 310 in the embodiments of the present disclosure can include one or more computer program products, and the computer program product can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include a random access memory (RAM), a cache, and / or the like. The non-volatile memory may, for example, include a read-only memory (ROM), a flash memory, a hard disk (HDD), or a solid-state disk (SSD), and the like.

[0079] In the embodiments of the present disclosure, the I / O interface 330 can be used to receive input instructions (such as digital or character information, and generate key signal inputs related to user settings and function control of the electronic device 300, etc.), and can also output various information to the outside (such as images or sounds, etc.). The I / O interface 330 in the embodiments of the present disclosure can include one or more of a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a mouse, a joystick, a trackball, a microphone, a speaker, and a touch panel, etc.

[0080] The present application uses specific words to describe the embodiments of the present application. As "one embodiment", "an embodiment", and / or "some embodiments" means a certain feature, structure or characteristic related to at least one embodiment of the present application. Therefore, it should be emphasized and noted that the "an embodiment" or "one embodiment" or "an alternative embodiment" mentioned in different places in the specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of the present application can be properly combined.

[0081] In the context of the present application, the words "comprise", "comprising", "include", "including" and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense, unless the context clearly indicates otherwise. Generally, the terms "comprise", "comprising", "include", "including" and the like are intended to indicate the inclusion of the stated steps and / or elements but not to those not specifically stated.

[0082] Similarly, it is to be noticed that the term "comprising", used in the context of describing the application, should not be interpreted as implying that the application "comprises only those elements specifically listed, but embraces at least those elements explicitly listed and any equivalent elements. The mere fact that different features are recited in mutually different dependent claims does not indicate that the application requires these different features to each be present.

[0083] Having described basic concepts, it will be obvious to those skilled in the art that various modifications, improvements and changes can be made to the present application without departing from the scope thereof. Such modifications, improvements and changes are therefore intended to be included within the spirit and scope of the application and so long as such modifications, improvements and changes come within the scope of the application are intended to be encompassed by the appended claims.

Claims

1. A method of evaluating an image segmentation algorithm, wherein, The method comprises the following steps: According to the original image, the original image is segmented by a plurality of to-be-tested algorithms to obtain a plurality of segmented images, wherein the original image is a mineral image comprising a plurality of stone blocks adhered to each other, the to-be-tested algorithm is used to determine and segment the overlapping or adhered area between each stone block, so that each stone block is independent, and the to-be-tested algorithm comprises one or more different to-be-tested algorithms, each to-be-tested algorithm is provided with different parameter combinations; Each segmented image is compared with the labeled data to determine the segmentation result of each to-be-tested algorithm, wherein the segmentation result comprises four sub-results: missegmentation, missed segmentation, over-segmentation and normal segmentation; Determine the weight value corresponding to each sub-result set, and each sub-result set corresponds to a plurality of weight values; According to the segmentation result and the weight value corresponding to each sub-result set, the segmentation result of each to-be-tested algorithm is evaluated by a multi-index evaluation method to determine the optimal algorithm in the to-be-tested algorithm for image segmentation of minerals, comprising: determining the optimal sample under each weight value according to all weight values, and each optimal sample corresponds to one to-be-tested algorithm; according to the data of the optimal sample, all optimal samples under all weight value combinations are counted to determine the frequency of each to-be-tested algorithm being determined as the optimal sample; according to the frequency of each to-be-tested algorithm being determined as the optimal sample, the to-be-tested algorithm with the highest frequency is determined as the optimal algorithm, wherein the obtained evaluation result is also used to check the original image to collect data to guide the optimization of the image segmentation algorithm; Wherein, the normal segmentation is a positive index, and the missegmentation, missed segmentation and over-segmentation are negative indexes, and the evaluation of the segmentation result of each to-be-tested algorithm comprises: positive of the missegmentation, missed segmentation and over-segmentation to obtain corresponding non-missegmentation, non-missed segmentation and non-over-segmentation data to balance the multi-index decision result according to the four sub-results.

2. The method of evaluating an image segmentation algorithm according to claim 1, wherein, The evaluation of the segmentation result of each to-be-tested algorithm according to the segmentation result and the weight value corresponding to each sub-result to determine the optimal algorithm in the to-be-tested algorithm comprises: According to the segmentation result of each to-be-tested algorithm, determine the optimal data and the worst data in each sub-result; Based on the optimal data and the worst data in each sub-result and the weight value corresponding to each sub-result, determine the relative closeness of the segmentation result of each algorithm relative to the optimal data and the worst data, wherein the relative closeness represents the closeness of the segmentation result to the optimal data and the degree of deviation from the worst data; According to the relative closeness, determine the optimal algorithm in the to-be-tested algorithm.

3. The method of evaluating an image segmentation algorithm according to claim 2, wherein, Each sub-result set corresponds to a plurality of weight values, and the determination of the optimal algorithm in the to-be-tested algorithm according to the relative closeness comprises: According to each set of weight values, the algorithm to be tested with the maximum relative proximity under each set of weight values is determined as an optimal sample; According to the optimal sample under each set of weight values, the frequency of each algorithm to be tested being determined as the optimal sample is determined; According to the frequency of each algorithm to be tested being determined as the optimal sample, the algorithm to be tested with the highest frequency is determined as the optimal algorithm.

4. The method of evaluating an image segmentation algorithm according to claim 2, said determining a weight value corresponding to each of said sets of sub-results, wherein, In the set of sub-results, the weight values corresponding to the wrong segmentation, the missed segmentation and the over-segmentation are all less than the weight value corresponding to the normal segmentation.

5. The method of evaluating an image segmentation algorithm according to claim 4, wherein, The weight values corresponding to the wrong segmentation, the missed segmentation and the over-segmentation are all greater than or equal to 0 and less than or equal to 0.3, and the weight value corresponding to the normal segmentation is greater than or equal to 0.6 and less than or equal to 1; the sum of the weight values corresponding to the four sub-results is 1.

6. The method of evaluating image segmentation algorithms according to claim 5, each of said sub-results corresponding to a plurality of weight values, each set of said sub-results determining a plurality of said weight values, wherein, The adjustment step between each set of weight values is greater than or equal to 0.01 and less than or equal to 0.

04.

7. The method of evaluating an image segmentation algorithm according to claim 1, wherein, The annotation data is obtained by the following method: The original image is preprocessed to obtain a standard image; According to the standard image, the contour of the target is extracted to determine a standard contour image; According to the standard contour image, the information of the standard contour image annotated by artificial is extracted as the annotation data; The standard contour image is extracted, and the information of the target adhered in the original image is extracted as the annotation data.

8. The method of evaluating an image segmentation algorithm according to claim 1, wherein, The original image is preprocessed to obtain a standard image; According to the standard image, the contour of the target is extracted to determine a standard contour image; The standard contour image is extracted, and the information of the target adhered in the original image is extracted as the annotation data. The original image is preprocessed to obtain a standard image; According to the standard image, the contour of the target is extracted to determine a standard contour image; The standard contour image is extracted, and the information of the target adhered in the original image is extracted as the annotation data.

9. An electronic device, comprising: The original image is preprocessed to obtain a standard image; According to the standard image, the contour of the target is extracted to determine a standard contour image; The standard contour image is extracted, and the information of the target adhered in the original image is extracted as the annotation data. The electronic device includes a memory, a processor, and a data processing program stored on the memory and executable on the processor, and the data processing program is executed by the processor to implement the image segmentation algorithm evaluation method of any one of claims 1-8.

10. A computer readable storage medium storing a program for executing the image segmentation algorithm evaluation method of any one of claims 1-8.

Images