Image segmentation method and device, and three-dimensional reconstruction method and device of image

Abstract

A segmentation method and device of an image and a three-dimensional reconstruction method and device of an image (91) relate to the technical field of computers. The segmentation method comprises: dividing each pixel in an image to be segmented into different pixel sets according to a color gamut range to which a pixel value belongs; determining a matching condition between each pixel in each pixel set according to the pixel value; and performing image segmentation on the image to be segmented according to the matching condition.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an image segmentation method, an image segmentation apparatus, an image three-dimensional reconstruction method, an image three-dimensional reconstruction apparatus, an electronic device, a wearable device, and a non-volatile computer-readable storage medium. Background Technology

[0002] Image segmentation is a fundamental problem in image processing and computer vision because it is a key process in many applications involving the extraction of specific regions.

[0003] In related technologies, most image segmentation methods based on edge detection are adopted. Summary of the Invention

[0004] According to some embodiments of this disclosure, an image segmentation method is provided, comprising: dividing each pixel in the image to be segmented into different pixel sets according to the color gamut range to which the pixel value belongs; determining the matching situation between each pixel in each pixel set according to the pixel value; and performing image segmentation on the image to be segmented according to the matching situation.

[0005] In some embodiments, the method further includes: dividing the color gamut cube composed of the red, green, and blue components into multiple color gamut sub-cubes in a coordinate system with the red, green, and blue components of the pixel values ​​as variables, as each color gamut range.

[0006] In some embodiments, the method further includes: determining one of the vertices of the color gamut cube contained in each color gamut sub-cube, the center point of each color gamut sub-cube, and the mean point of each color gamut sub-cube as the feature pixel value of the corresponding color gamut range; and determining the color gamut range to which the pixel value of each pixel in the image to be segmented belongs based on the feature pixel value.

[0007] In some embodiments, determining the matching status between pixels in each pixel set based on pixel values ​​includes: selecting a pixel in any pixel set as a seed pixel; calculating the difference between the pixel values ​​of other pixels in the pixel set and the pixel value of the seed pixel; and determining whether the other pixels match the seed pixel based on the difference.

[0008] In some embodiments, determining whether the other pixels match the seed pixel based on the difference includes: determining the fuzzy set to which the other pixels belong using a membership function based on the difference; and determining whether the other pixels match the seed pixel based on the determined fuzzy set.

[0009] In some embodiments, the pixel value includes a red component, a green component, and a blue component. The step of determining the fuzzy set to which the other pixels belong based on the difference using a membership function includes: determining the fuzzy sets to which the red, green, and blue components of the other pixels belong based on the differences in the red component, the green component, and the blue component, respectively.

[0010] In some embodiments, selecting a pixel from any set of pixels as a seed pixel includes: sorting the pixels in the set of pixels according to the difference between the pixel value of each pixel in the set and the feature pixel value of the color gamut range to which the set of pixels belongs, wherein the feature pixel value is one of the vertices of the color gamut cube contained in the color gamut sub-cube corresponding to the color gamut range, the center point of the corresponding color gamut sub-cube, and the mean point of the corresponding color gamut sub-cubes; and selecting each pixel in the set of pixels as the seed pixel in turn according to the sorting result.

[0011] In some embodiments, the step of performing image segmentation on the image to be segmented based on the matching conditions includes: generating multiple sub-images based on each pixel and its matching pixels; merging the multiple sub-images based on the overlap between the sub-images; and determining the image segmentation result based on the merging result.

[0012] In some embodiments, merging the multiple sub-images based on the overlap between them includes: calculating the number of pixels contained in the intersection of the first sub-image and the second sub-image; determining an overlap parameter based on the ratio of the number of pixels contained in the intersection to the number of pixels contained in the first sub-image to determine the overlap; and merging the first sub-image and the second sub-image if the overlap parameter is greater than a threshold.

[0013] In some embodiments, the method further includes: determining interfering pixels based on the pixel value distribution of each pixel in the original image; determining matching pixels of the interfering pixels based on the pixel values ​​of each pixel in the original image; and removing the interfering pixels and their matching pixels from the original image to obtain the image to be segmented.

[0014] In some embodiments, the image to be segmented is a two-dimensional image generated based on acquired underwater sonar data.

[0015] According to some other embodiments of this disclosure, a method for three-dimensional reconstruction of an image is provided, comprising: performing segmentation processing on an image to be segmented according to the segmentation method of any of the above embodiments; and performing three-dimensional reconstruction based on the segmentation processing result to obtain a three-dimensional image.

[0016] According to some embodiments of the present disclosure, an image segmentation apparatus is provided, including at least one processor configured to perform the following steps: dividing each pixel in the image to be segmented into different pixel sets according to the color gamut range to which the pixel value belongs; determining the matching situation between each pixel in each pixel set according to the pixel value; and performing image segmentation on the image to be segmented according to the matching situation.

[0017] In some embodiments, the method further includes: dividing the color gamut cube composed of the red, green, and blue components into multiple color gamut sub-cubes in a coordinate system with the red, green, and blue components of the pixel values ​​as variables, as each color gamut range.

[0018] In some embodiments, the processor is further configured to perform the following steps: determining one of the vertices of the color gamut cube contained in each color gamut sub-cube, the center point of each color gamut sub-cube, and the mean point of each color gamut sub-cube as the feature pixel value of the corresponding color gamut range; and determining the color gamut range to which the pixel value of each pixel in the image to be segmented belongs based on the feature pixel value.

[0019] In some embodiments, determining the matching status between pixels in each pixel set based on pixel values ​​includes: selecting a pixel in any pixel set as a seed pixel; calculating the difference between the pixel values ​​of other pixels in the pixel set and the pixel value of the seed pixel; and determining whether the other pixels match the seed pixel based on the difference.

[0020] In some embodiments, determining whether the other pixels match the seed pixel based on the difference includes: determining the fuzzy set to which the other pixels belong using a membership function based on the difference; and determining whether the other pixels match the seed pixel based on the determined fuzzy set.

[0021] In some embodiments, the pixel value includes a red component, a green component, and a blue component. The step of determining the fuzzy set to which the other pixels belong based on the difference using a membership function includes: determining the fuzzy sets to which the red, green, and blue components of the other pixels belong based on the differences in the red component, the green component, and the blue component, respectively.

[0022] In some embodiments, selecting a pixel from any set of pixels as a seed pixel includes: sorting the pixels in the set of pixels according to the difference between the pixel value of each pixel in the set and the feature pixel value of the color gamut range to which the set of pixels belongs, wherein the feature pixel value is one of the vertices of the color gamut cube contained in the color gamut sub-cube corresponding to the color gamut range, the center point of the corresponding color gamut sub-cube, and the mean point of the corresponding color gamut sub-cubes; and selecting each pixel in the set of pixels as the seed pixel in turn according to the sorting result.

[0023] In some embodiments, the step of performing image segmentation on the image to be segmented based on the matching conditions includes: generating multiple sub-images based on each pixel and its matching pixels; merging the multiple sub-images based on the overlap between the sub-images; and determining the image segmentation result based on the merging result.

[0024] In some embodiments, merging the multiple sub-images based on the overlap between them includes: calculating the number of pixels contained in the intersection of the first sub-image and the second sub-image; determining an overlap parameter based on the ratio of the number of pixels contained in the intersection to the number of pixels contained in the first sub-image to determine the overlap; and merging the first sub-image and the second sub-image if the overlap parameter is greater than a threshold.

[0025] In some embodiments, the processor is further configured to perform the following steps: determining interfering pixels based on the pixel value distribution of each pixel in the original image; determining matching pixels of the interfering pixels based on the pixel values ​​of each pixel in the original image; and removing the interfering pixels and their matching pixels from the original image to obtain the image to be segmented.

[0026] In some embodiments, the image to be segmented is a two-dimensional image generated based on acquired underwater sonar data.

[0027] According to further embodiments of the present disclosure, a three-dimensional reconstruction apparatus for an image is provided, including at least one processor configured to perform the following steps: segmenting an image to be segmented according to the segmentation method of any of the above embodiments; and performing three-dimensional reconstruction based on the segmentation result to obtain a three-dimensional image.

[0028] According to further embodiments of this disclosure, a wearable device is provided, comprising: a three-dimensional reconstruction apparatus for images in any of the above embodiments; and a display screen for displaying three-dimensional images acquired by the three-dimensional reconstruction apparatus.

[0029] In some embodiments, the 3D reconstruction apparatus generates an image to be segmented based on the acquired underwater sonar data, and reconstructs the 3D image based on the segmentation result of the image to be segmented.

[0030] According to further embodiments of the present disclosure, an electronic device is provided, including: a memory; and a processor coupled to the memory, the processor being configured to perform an image segmentation method or an image three-dimensional reconstruction method as described in any of the foregoing embodiments based on instructions stored in the memory device.

[0031] According to further embodiments of the present disclosure, a non-volatile computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the image segmentation method or the three-dimensional image reconstruction method described in any of the above embodiments.

[0032] Other features and advantages of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0033] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this application, illustrate exemplary embodiments of this disclosure and are used to explain this disclosure, but do not constitute an undue limitation of this disclosure. In the drawings:

[0034] Figure 1 Flowcharts illustrating some embodiments of the image segmentation method according to this disclosure;

[0035] Figure 2 Flowcharts illustrating other embodiments of the image segmentation method according to this disclosure are shown;

[0036] Figure 3 Schematic diagrams illustrating some embodiments of the image segmentation method according to the present disclosure;

[0037] Figure 4 Show Figure 1 Flowcharts of some embodiments of step 120;

[0038] Figure 5 Schematic diagrams illustrating other embodiments of the image segmentation method according to the present disclosure are shown;

[0039] Figure 6 Showing according to Figure 1 Flowcharts of some embodiments of step 130;

[0040] Figure 7 Schematic diagrams illustrating further embodiments of the image segmentation method according to the present disclosure;

[0041] Figure 8Schematic diagrams illustrating some embodiments of wearable devices according to the present disclosure;

[0042] Figure 9 Block diagrams illustrating some embodiments of wearable devices according to the present disclosure;

[0043] Figure 10 Block diagrams illustrating some embodiments of the electronic devices of this disclosure;

[0044] Figure 11 Block diagrams illustrating other embodiments of the electronic devices disclosed herein are shown. Detailed Implementation

[0045] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0046] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0047] The inventors of this disclosure have discovered the following problems in the aforementioned related technologies. Image segmentation methods using binarization or based on the RGB (Red, Green, Blue) color space are all based on a segmentation principle, namely, each pixel is classified into a unique subset. However, this coarse segmentation principle cannot accurately identify different image regions, resulting in low image segmentation accuracy.

[0048] Moreover, since different observers have different understandings of image colors, this segmentation method cannot accurately reflect human understanding of images, resulting in low accuracy in image segmentation.

[0049] Low image segmentation accuracy can lead to poor results in subsequent processing. For example, the raw underwater sonar data contains information about seawater, seabed, and the target, which have completely different characteristics. In the 3D visualization imaging processing after image segmentation, these three elements also have different opacities and colors. Thus, the segmentation methods described above often result in the target being obscured at the segmentation lines of each region, leading to poor 3D imaging results.

[0050] In view of this, this disclosure proposes an image segmentation technique. This technique groups pixels with the same color gamut value into the same pixel set, and performs pixel value matching within each pixel set to determine the image segmentation result. This refines the color gamut space division, improves the recognition rate of different image regions, and thus improves the accuracy of image segmentation. For example, the technical solution of this disclosure can be implemented through the following embodiments.

[0051] Figure 1 A flowchart illustrating some embodiments of the image segmentation method according to this disclosure is provided.

[0052] like Figure 1 As shown, the method includes: step 110, dividing different pixel sets; step 120, determining the matching situation within the sets; and step 130, segmenting the image according to the matching situation.

[0053] In step 110, pixels in the image to be segmented are divided into different pixel sets according to the color gamut range to which the pixel values ​​belong. For example, the image to be segmented is a two-dimensional image generated based on acquired underwater sonar data.

[0054] For example, the pixel set to which each pixel belongs can be determined based on the difference between the pixel value of each pixel and the feature pixels in each color gamut, thus achieving pixel classification. In this way, similar pixels can be grouped into one category, achieving preliminary image segmentation; further matching of pixel values ​​within each category can improve the accuracy of image segmentation.

[0055] In some embodiments, before performing step 110, the entire color gamut can be modeled, and the color gamut can be divided into multiple color gamut ranges based on this modeling. Based on this, the color gamut range to which the pixel value of each pixel in the image to be segmented belongs can be determined. For example, it can be done by... Figure 2 The embodiments in the text implement color gamut modeling and division.

[0056] Figure 2 Flowcharts illustrating some other embodiments of the image segmentation method according to this disclosure are shown.

[0057] like Figure 2As shown, the method further includes: step 210, modeling a color gamut cube; step 220, dividing the color gamut range; step 230, determining the feature pixel value; and step 240, determining the color gamut range to which the pixel value belongs.

[0058] In step 210, based on the value ranges of the red, green, and blue components of the pixel values, the entire color gamut is modeled as a color gamut cube in a coordinate system with the red, green, and blue components as variables.

[0059] In step 220, the color gamut cube is divided into multiple color gamut sub-cubes, which serve as the color gamut ranges. For example, this can be achieved through... Figure 3 The embodiments in the example model a color gamut cube and divide it into color gamut sub-cubes.

[0060] Figure 3 Schematic diagrams illustrating some embodiments of the image segmentation method according to this disclosure are shown.

[0061] like Figure 3 As shown, the three coordinate values ​​of the coordinate system represent the values ​​of the three components of a pixel: R (red component), G (green component), and B (blue component). In the entire color gamut, the value range of each component is [0, 255]. The pixel value (0, 0, 0) at the origin P5 represents black. The cube with vertices P1 to P8 is the color gamut cube corresponding to the entire color gamut, i.e., the color gamut space.

[0062] In some embodiments, the color gamut cube can be divided into eight color gamut sub-cubes, each containing vertices P1 to P8, along the three component directions at 127 pixel value intervals. For example, vertices P1 to P8 represent blue, pink, white, cyan, black, red, yellow, and green, respectively. Depending on the vertices included, each color gamut sub-cube represents a different color gamut range; that is, pixel values ​​within the same color gamut range have similar color information.

[0063] After modeling the color gamut cube and color gamut sub-cubes, we can continue by... Figure 2 The embodiments in the text determine the color gamut range to which the pixel value of each pixel in the image to be segmented belongs.

[0064] In step 230, the vertices of the color gamut cubes contained in each color gamut sub-cube are determined as the feature pixel values ​​of the corresponding color gamut range. For example, any pixel value in each color gamut sub-cube that can represent the corresponding color gamut range can be determined as a feature pixel value, such as a vertex, center point, mean point, etc.

[0065] In step 240, the color gamut range to which each pixel value in the image to be segmented belongs is determined based on the feature pixel values. For example, the color gamut range to which a pixel belongs can be determined based on the difference between the pixel and each feature pixel. The pixel values ​​of the pixels can be used as multi-dimensional feature vectors, and the differences between pixels can be determined by calculating the similarity between feature vectors (such as Euclidean distance, Minkowski distance, Mahalanobis distance, etc.).

[0066] In some embodiments, in the RGB coordinate system, the distance between the point corresponding to each pixel in the image to be segmented and the eight vertices of the color gamut cube is calculated. For example, this distance is calculated using the following formula:

[0067] d i (n,m)=∑[p(n,m) RGB -v i RGB ] 2 i = 1, 2, ..., 8

[0068] In the formula, p(n,m) RGB It is a point defined in the RGB coordinate system for any pixel in the color image to be segmented, v i RGB The vertices i and d of the color gamut cube i (n,m) is the distance between the point and the vertex.

[0069] For example, eight distances are calculated for each pixel. By comparing the magnitudes of these eight distances, the pixel is assigned to a sub-cube of the color gamut corresponding to the vertex with the shortest distance. Using this method, pixels in the image to be segmented are spatially classified one by one.

[0070] This allows for a clear determination of the number of pixels contained in each subspace. For example, color gamut subcubes that do not contain any pixels are not further processed. Moreover, spatial classification avoids redundant calculations for the same pixels in subsequent processing (such as blurred color extraction).

[0071] After determining the color gamut range to which the pixel value belongs, you can continue by... Figure 1 The embodiments in the text perform image segmentation.

[0072] In step 120, the matching status between pixels within each pixel set is determined based on the pixel values. This approach, compared to aimless matching across the entire color gamut, focuses on matching pixels belonging to the same color gamut range (e.g., belonging to the same color type), improving matching efficiency and accuracy, and thus enhancing the efficiency and accuracy of image segmentation based on the matching results.

[0073] In some embodiments, it can be achieved through Figure 4 The embodiment in the example implements step 120.

[0074] Figure 4 Show Figure 1 A flowchart of some embodiments of step 120.

[0075] like Figure 4 As shown, step 120 includes: step 1210, selecting seed pixels; step 1220, calculating pixel value differences; step 1230, determining the fuzzy set to which the pixel belongs; and step 1240, determining whether the pixels match.

[0076] In step 1210, a pixel is selected from any set of pixels as a seed pixel.

[0077] In some embodiments, the pixels in a pixel set are sorted according to the difference between the pixel value of each pixel in the pixel set and the feature pixel value of the color gamut range to which the pixel set belongs; and according to the sorting result, each pixel in the pixel set is selected as a seed pixel in turn.

[0078] In some embodiments, pixels in each pixel set are sorted in ascending order according to their distance from the vertices of the corresponding color gamut sub-cube. Starting from the nearest point, each pixel is used as a seed pixel for blurred color extraction in a round-robin manner.

[0079] In some embodiments, if a seed pixel cannot find a matching pixel, the seed pixel is discarded, and the next pixel in the corresponding color gamut sub-cube is selected as the seed pixel. This process is repeated, selecting seed pixels sequentially for matching. For example, seed pixel selection and matching can be performed simultaneously in eight color gamut sub-cubes.

[0080] In step 1220, the difference between the pixel value of other pixels in the pixel set and the pixel value of the seed pixel is calculated.

[0081] Based on this difference, it can be determined whether other pixels match the seed pixel. For example, the matching can be determined by performing blurred color extraction in steps 1230 and 1240.

[0082] In step 1230, the fuzzy set to which the difference belongs is determined using membership functions and fuzzy logic.

[0083] In some embodiments, the pixel value includes a red component, a green component, and a blue component, and the fuzzy set to which the red component, green component, and blue component of other pixels belong is determined based on the differences in the red component, the green component, and the blue component, respectively.

[0084] In some embodiments, an FCE (Fuzzy Color Extractor) can be created to extract pixels similar to the seed pixel. For example, for pixel p(n,m), its subpixels have components p(n,m) in the RGB space. R p(n,m) G and p(n,m) B The pixel we want to process is seed, whose RGB components are seed... R seed G and seed B The seed can be selected based on the algorithm's requirements or determined based on the pixels in the image that you want to process.

[0085] The color component difference between any pixel p(n,m) and seed is calculated as follows:

[0086]

[0087] M and N represent the dimensions of the image (positive integers). The fuzzy set to which the color component differences belong can be calculated using a preset membership function based on these differences.

[0088] In some embodiments, it can be achieved through Figure 5 The implementation examples in the document determine the corresponding membership functions for each fuzzy set.

[0089] Figure 5 Schematic diagrams illustrating other embodiments of the image segmentation method according to this disclosure are shown.

[0090] like Figure 5 As shown, the fuzzy sets to which the color component differences belong include the Zero set, the Negative set, and the Positive set. The three function curves correspond to the membership functions of the three fuzzy sets. The vertical axis represents the function value of the membership function, i.e., the degree of membership of the difference to each fuzzy set; the horizontal axis represents the value of the difference. α1 and α2 are adjustable fuzzy thresholds set according to actual conditions and prior knowledge.

[0091] Once the membership function is determined, it can be achieved through... Figure 4 Step 1240 in the process determines the matching status of pixels.

[0092] In step 1240, based on the determined fuzzy set, it is determined whether other pixels match the seed pixel.

[0093] After fuzzy calculation, a fuzzy set of matching and non-matching pixels can be obtained. After defuzzification, the final extracted pixels (i.e., pixels that match the seed) can be obtained.

[0094] In some embodiments, fuzzy logic can be:

[0095] When dif(n,m) R 、dif(n,m)) G and dif(n,m) B When all pixels belong to the Zero set, the pixel p(n,m) is... RGB Matches with seed;

[0096] When dif(n,m) R Or dif(n,m)) G Or dif(n,m) B When it belongs to the Negative or Positive set, p(n,m) RGB It does not match the seed.

[0097] In the above embodiments, fuzzy logic is configured using a linguistic approach, resulting in relatively simple input and output functions that do not require precise mathematical models, thus optimizing computation. Fuzzy matching methods exhibit strong robustness and are suitable for solving nonlinear, strongly coupled, time-varying, and lag-related problems in the classification process, thereby improving the accuracy of image segmentation. Fuzzy matching methods also possess strong fault tolerance, adapting to changes in the characteristics of the controlled object itself and the environmental characteristics.

[0098] Therefore, the fuzzy color extraction algorithm is suitable for image segmentation processing in complex environments (such as underwater sonar data images) and can improve the accuracy of image segmentation.

[0099] Once the matching conditions are determined, it can be done through... Figure 1 Step 130 in the process involves image segmentation.

[0100] In step 130, the image to be segmented is segmented according to the matching results.

[0101] In some embodiments, it can be achieved through Figure 6 The embodiments in the text perform image segmentation.

[0102] Figure 6 Showing according to Figure 1 A flowchart of some embodiments of step 130.

[0103] like Figure 6 As shown, step 130 may include: step 1310, generating multiple sub-images; step 1320, merging the sub-images; and step 1330, determining the segmentation result.

[0104] In step 1310, multiple sub-images are generated based on each pixel and its matching pixels. For example, after each blur color extraction, a sub-image can be obtained based on a seed pixel and its matching pixels.

[0105] Due to the spatial similarity of pixels, sub-images corresponding to seed pixels with similar pixel values ​​generally have overlapping portions. In cases where multiple sub-images share a common color region in the RGB space, it is even possible for one sub-image to completely cover another.

[0106] Therefore, it is necessary to merge the obtained series of sub-images according to a certain method to form the final segmented image.

[0107] In step 1320, multiple sub-images are merged based on the overlap between them. If two sub-images have spatial and color similarity, they are considered to share a common area and can be connected together to form an image partition.

[0108] In some embodiments, the number of pixels contained in the intersection of the first sub-image and the second sub-image is calculated; an overlap parameter is determined as the overlap condition based on the ratio of the number of pixels contained in the intersection to the number of pixels contained in the first sub-image; if the overlap parameter is greater than a threshold, the first sub-image and the second sub-image are merged.

[0109] For example, the two sub-images are I SAMPLE (i) and I SAMPLE (l) The overlap parameter can be determined using the following formula:

[0110] NUM(I SAMPLE (i) ∩I SAMPLE (l) ) / NUM(I SAMPLE (i) )

[0111] NUM() returns the number of pixels within the parentheses. This overlap parameter can be used to detect the size of the common region between two sub-images in RGB space. If the overlap parameter is greater than a threshold, the sub-image I is considered... SAMPLE (i) and I SAMPLE (l) Images that are similar can be merged. For example, a threshold can be set in the algorithm. To make image segmentation more accurate, the threshold can be set to a larger value, such as 90 or 100.

[0112] In step 1330, the image segmentation result is determined based on the merging result. Through merging processing, different regions in an image (such as water bodies and strata in underwater images) can be extracted and segmented.

[0113] In some embodiments, in complex imaging environments, the original image contains many unknown interference factors. For example, the seabed environment is complex, the dynamic range of the sonar data obtained after scanning is very small, and there will be a lot of interference in the underwater sonar image. Therefore, logarithmic transformation can be used to preprocess the image to expand the dynamic range of the data and reduce interference. For example, the original image can also be preprocessed to remove noise and enhance contrast.

[0114] In some embodiments, prior to performing step 110, it can be done through Figure 7 The examples in the document remove interference from images.

[0115] Figure 7 Schematic diagrams illustrating further embodiments of the image segmentation method according to the present disclosure.

[0116] like Figure 7 As shown, the method may further include: step 710, determining the interfering pixel; step 720, determining the matching pixel of the interfering pixel; and step 730, performing interference removal processing.

[0117] In step 710, interfering pixels are determined based on the pixel value distribution of each pixel in the original image. For example, interfering pixels can be selected based on prior knowledge and actual needs.

[0118] In some embodiments, interfering pixels can be selected based on prior knowledge. For example, if the color range of interfering factors in the image (such as the red gamut) has been determined, pixels within that color range can be identified as interfering pixels.

[0119] In some embodiments, the RGB space center point (127, 127, 127) cannot be precisely divided into any color subspace, which will cause significant interference during color extraction. Therefore, the seed pixel seed = (127, 127, 127) can also be selected as the interference pixel for blurred color extraction.

[0120] In step 720, matching pixels for interfering pixels are determined based on the pixel values ​​of each pixel in the original image. For example, matching can be performed using a method from any of the above embodiments (such as blurred color extraction).

[0121] In step 730, interfering pixels and their matching pixels are removed from the original image to obtain the image to be segmented. For example, for the input original image I... SOURCE It is possible to determine the interference image I composed of interfering pixels and matching pixels. INT It can be done through I SOURCE —I INT Remove the color images (matching pixels) that are close to interfering pixels from the original image to obtain the desired segmented image I. SAMPLE .

[0122] In some embodiments, the segmentation method of any of the above embodiments is used to segment the image to be segmented; and three-dimensional reconstruction is performed based on the segmentation result to obtain a three-dimensional image.

[0123] In some embodiments, different regions such as ocean, strata, and targets can be identified in the two-dimensional image obtained after image segmentation of underwater sonar images. Based on the rich information contained in the original underwater sonar data regarding three-dimensional entities, the segmented two-dimensional image can be reconstructed into a three-dimensional structure (e.g., using Unity 3D tools). Furthermore, volume rendering techniques can be used to achieve three-dimensional visualization effects.

[0124] For example, during volume rendering, there is no need to construct intermediate geometric images; only the 3D data volume needs to be processed to display its internal details. Such 3D reconstruction processing is simple to operate and quick to convert.

[0125] In some embodiments, 3D visualization can be achieved using VTK (Visualization Toolkit).

[0126] In the above embodiments, the water body can be separated from the bottom layer more effectively and the target object (such as underwater buried mines, bombs, etc.) can be extracted with high accuracy. In the field of color image segmentation, it can effectively solve the uncertainty and ambiguity in practical applications and adapt to different observers' different emphases on color in different color spaces.

[0127] In the subsequent 3D visualization process, the Unity platform is used to build 3D scenes. Users can construct highly complex 3D images and scenes in a very short time, greatly improving work efficiency.

[0128] Employing a volume rendering method based on the VTK toolkit, the processed 3D data is represented using appropriate geometry, color, and brightness, and then mapped onto a 2D image plane. Ultimately, the 3D image can be rendered in a VR (Virtual Reality) headset, allowing users to view sonar images in a virtual environment, enhancing immersion.

[0129] In some embodiments, the image segmentation apparatus includes at least one processor configured to perform the image segmentation method of any of the above embodiments.

[0130] In some embodiments, the three-dimensional reconstruction apparatus for an image includes at least one processor configured to perform: the segmentation method of any of the above embodiments, performing segmentation processing on the image to be segmented; and performing three-dimensional reconstruction based on the segmentation processing result to obtain a three-dimensional image.

[0131] Figure 8 Schematic diagrams illustrating some embodiments of wearable devices according to this disclosure are shown.

[0132] like Figure 8 As shown, wearable devices can adopt a VR split-type structure, including: a PC (Personal Computer) part (such as an image reconstruction device) and a VR headset part (such as a display screen).

[0133] In some embodiments, image preprocessing, image segmentation, volume rendering, and other processing steps can be completed on the PC part first, and then the obtained 3D image can be rendered to the VR headset part through DP (Display Port).

[0134] For example, image preprocessing may include noise reduction, contrast enhancement, and other processing; image segmentation may include FCE processing of any of the above embodiments; and 3D images and scenes may be constructed using Unity 3D and 3D image visualization processing may be performed using VTK.

[0135] In some embodiments, after image segmentation processing of the sonar data, it can be displayed in a virtual reality headset using volume rendering technology. This allows users to observe three-dimensional images of the water body, strata, and target objects from underwater sonar data within a VR scene.

[0136] Figure 9 Block diagrams illustrating some embodiments of wearable devices according to this disclosure are shown.

[0137] like Figure 9 As shown, the wearable device 9 includes: a three-dimensional reconstruction device 91 for images in any of the above embodiments; and a display screen 92 for displaying the three-dimensional images acquired by the three-dimensional reconstruction device 91.

[0138] In some embodiments, the three-dimensional reconstruction device 91 generates an image to be segmented based on the acquired underwater sonar data, and reconstructs the three-dimensional image based on the segmentation result of the image to be segmented.

[0139] Figure 10 Block diagrams illustrating some embodiments of the electronic devices disclosed herein.

[0140] like Figure 10 As shown, the electronic device 10 of this embodiment includes a memory U11 and a processor U12 coupled to the memory U11. The processor U12 is configured to execute an image segmentation method or an image three-dimensional reconstruction method in any embodiment of this disclosure based on instructions stored in the memory U11.

[0141] The memory U11 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory may store, for example, the operating system, application programs, boot loader, database, and other programs.

[0142] Figure 11 Block diagrams illustrating other embodiments of the electronic devices disclosed herein are shown.

[0143] like Figure 11 As shown, the electronic device 11 of this embodiment includes: a memory U10 and a processor U20 coupled to the memory U10. The processor U20 is configured to execute the image segmentation method or the three-dimensional reconstruction method of the image in any of the foregoing embodiments based on the instructions stored in the memory U10.

[0144] The memory U10 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory may store, for example, the operating system, application programs, a boot loader, and other programs.

[0145] Electronic device 6 may also include input / output interface U30, network interface U40, and storage interface U50. These interfaces U30, U40, U50, as well as the memory U10 and processor U20, can be connected via, for example, a bus U60. Specifically, input / output interface U30 provides a connection interface for input / output devices such as monitors, mice, keyboards, touchscreens, microphones, and speakers. Network interface U40 provides a connection interface for various networked devices. Storage interface U50 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0146] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0147] The present disclosure has now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art will fully understand how to implement the technical solutions disclosed herein based on the above description.

[0148] The methods and systems of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the specific order described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0149] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. An image segmentation method, comprising: Based on the color gamut range to which the pixel value belongs, each pixel in the image to be segmented is divided into different pixel sets. Each color gamut range is a set of multiple color gamut sub-cubes obtained by dividing the color gamut cube composed of the red, green and blue components in a coordinate system with the red, green and blue components of the pixel value as variables. Based on the pixel values, determine the matching status between pixels within each pixel set; Based on the matching results, image segmentation is performed on the image to be segmented. The step of determining the matching status between pixels within each pixel set based on pixel values ​​includes: Select a pixel from any set of pixels as the seed pixel; Calculate the difference between the pixel values ​​of other pixels in the pixel set and the pixel value of the seed pixel; Based on the differences, determine whether the other pixels match the seed pixel. The step of selecting a pixel from any set of pixels as a seed pixel includes: The pixels in any pixel set are sorted according to the distances between the pixel values ​​of each pixel and the feature pixel values ​​of the color gamut range to which the pixel set belongs in the corresponding color gamut sub-cube, from smallest to largest. The feature pixel values ​​are one of the vertices of the color gamut cube contained in the corresponding color gamut sub-cube, the center point of the corresponding color gamut sub-cube, and the mean point of the corresponding color gamut sub-cubes. The corresponding color gamut sub-cube corresponds to the color gamut range to which the pixel set belongs. Based on the sorting results, each pixel in the pixel set is selected as the seed pixel in turn.

2. The segmentation method according to claim 1 further includes: One of the vertices of the color gamut cube contained in each color gamut sub-cube, the center point of each color gamut sub-cube, and the mean point of each color gamut sub-cube is determined as the feature pixel value of the corresponding color gamut range. Based on the feature pixel values, the color gamut range to which each pixel value in the image to be segmented belongs is determined.

3. The segmentation method according to claim 1, wherein, The step of determining whether the other pixels match the seed pixel based on the difference includes: Using membership functions, determine the fuzzy set to which the differences belong; Based on the determined fuzzy set and fuzzy logic, it is determined whether the other pixels match the seed pixel.

4. The segmentation method according to claim 3, wherein, The pixel value includes a red component, a green component, and a blue component. The step of determining the fuzzy set to which the other pixels belong, based on the differences and using a membership function, includes: Based on the differences in the red component, the green component, and the blue component, the fuzzy sets to which the red, green, and blue components of the other pixels belong are determined.

5. The segmentation method according to claim 1, wherein, The step of performing image segmentation on the image to be segmented based on the matching results includes: Based on each pixel and its matching pixel, multiple sub-images are generated; The multiple sub-images are merged based on the overlap between them. Based on the merging results, the image segmentation result is determined.

6. The segmentation method according to claim 5, wherein, The step of merging the multiple sub-images based on the overlap between them includes: Calculate the number of pixels contained in the intersection of the first sub-image and the second sub-image; The overlap parameter is determined based on the ratio of the number of pixels contained in the intersection to the number of pixels contained in the first sub-image to determine the overlap situation; If the overlap parameter is greater than a threshold, the first sub-image and the second sub-image are merged.

7. The segmentation method according to any one of claims 1-6, further comprising: Based on the pixel value distribution of each pixel in the original image, identify the interfering pixels; Based on the pixel values ​​of each pixel in the original image, determine the matching pixel of the interfering pixel; In the original image, the interfering pixels and their matching pixels are removed to obtain the image to be segmented.

8. The segmentation method according to any one of claims 1-6, wherein, The image to be segmented is a two-dimensional image generated based on the acquired underwater sonar data.

9. A method for three-dimensional reconstruction of an image, comprising: The segmentation method according to any one of claims 1-8 performs segmentation processing on the image to be segmented; Based on the segmentation results, 3D reconstruction is performed to obtain a 3D image.

10. An image segmentation apparatus, comprising at least one processor configured to perform the following steps: Based on the color gamut range to which the pixel value belongs, the pixels in the image to be segmented are divided into different pixel sets, where, Each color gamut range is a set of multiple color gamut sub-cubes obtained by dividing the color gamut cube composed of the red, green, and blue components in a coordinate system with the red, green, and blue components of the pixel value as variables. Based on the pixel values, determine the matching status between pixels within each pixel set; Based on the matching results, image segmentation is performed on the image to be segmented. The step of determining the matching status between pixels within each pixel set based on pixel values ​​includes: Select a pixel from any set of pixels as the seed pixel; Calculate the difference between the pixel values ​​of other pixels in the pixel set and the pixel value of the seed pixel; Based on the differences, determine whether the other pixels match the seed pixel. The step of selecting a pixel from any set of pixels as a seed pixel includes: The pixels in any pixel set are sorted according to the distances between the pixel values ​​of each pixel and the feature pixel values ​​of the color gamut range to which the pixel set belongs in the corresponding color gamut sub-cube, from smallest to largest. The feature pixel values ​​are one of the vertices of the color gamut cube contained in the corresponding color gamut sub-cube, the center point of the corresponding color gamut sub-cube, and the mean point of the corresponding color gamut sub-cubes. The corresponding color gamut sub-cube corresponds to the color gamut range to which the pixel set belongs. Based on the sorting results, each pixel in the pixel set is selected as the seed pixel in turn.

11. A three-dimensional reconstruction apparatus for an image, comprising at least one processor configured to perform the following steps: The segmentation method according to any one of claims 1-8 performs segmentation processing on the image to be segmented; Based on the segmentation results, 3D reconstruction is performed to obtain a 3D image.

12. An electronic device, comprising: Memory; and A processor coupled to the memory, the processor being configured to execute the image segmentation method of any one of claims 1-8, or the three-dimensional reconstruction method of the image of claim 9, based on instructions stored in the memory device.

13. A non-volatile computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image segmentation method as described in any one of claims 1-8, or the three-dimensional reconstruction method of the image as described in claim 9.

14. A wearable device, comprising: The three-dimensional reconstruction apparatus for images according to claim 11; and A display screen is used to display the three-dimensional images acquired by the three-dimensional reconstruction device.

15. The wearable device according to claim 14, wherein, The three-dimensional reconstruction device generates an image to be segmented based on the acquired underwater sonar data, and reconstructs the three-dimensional image based on the segmentation results of the image to be segmented.

Images